Skip to main content

Posts

Showing posts with the label Data

Log Sorting with AWS CloudWatch, AWS CloudWatch Insights

 The cool thing is, I was contracted to make these videos in collaboration with CloudAvail Technology Consulting to help people decide which service they wanted to use for their logging - AWS CloudWatch, AWS CloudWatch Insights, DataDog, or New Relic. I'm searching through nginx logs. I have accompanying videos of each service that you can find on the CloudAvail Youtube page; See these links to go to the DataDog and NewRelic posts.   The idea was to be subjective in the videos, but I can be objective on my personal blog.     CloudWatch     The syntax is odd, but easy to grasp. Sort log data by IP addresses, message codes, and status codes. The simplest query system, but not quite robust.   Insights       The syntax has changed - Vastly. I see major SQL influences. You can see that in how the parse function works - in this case, it's often taken pieces of a pre-existing standard - in this case, message - and breaking them into their own categories you can reference.    The funct

Log Sorting With New Relic

 The nifty thing is, I was contracted to make these videos in collaboration with CloudAvail Technology Consulting to help people decide which service they wanted to use for their logging and data visualization - CloudWatch, CloudWatch Insights, DataDog, or New Relic. I'm searching through nginx logs. I have accompanying videos of each service that you can find on the CloudAvail Youtube page; See the following links to go to the CloudWatch , CloudWatch Insights , and DataDog posts.   NewRelic   I expected this to be easier than the previous two and it's rather dense in its information at first glance, although you an adjust it to show real-time analytics.  Documentation is reasonable and concise; One could gather the basics of the syntax quickly. Opening a singular log entry and clicking the elements of it to include or exclude from a new query was very helpful. check out the video for how I did error tracking. The GUI is my favorite; Sleek and modern, and yet it will chan

Log Sorting With DataDog

The nifty thing is, I was contracted to make these videos in collaboration with CloudAvail Technology Consulting to help people decide which service they wanted to use for their logging - CloudWatch, CloudWatch Insights, DataDog, or New Relic. I'm searching through nginx logs. I have accompanying videos of each service that you can find on the CloudAvail Youtube page; See the following links to go to the CloudWatch , CloudWatch Insights , and NewRelic posts.   The idea was to be subjective in the videos, but I can be objective on my personal blog.     ...There's a lot of information.  And a lot of ways to sort it. What's the difference between Patterns and Transactions? The Live Tail setting is pretty cool. Would be very helpful in making you panic if there was some kind of incident and you got a ton of error logs, but then you can sort by patterns and get a bit more insight into what is failing or being attacked. You can see in my video that I had to adjust th

Using AWS Data Migration Service

 Want to share this easily?  Check out the Notion page. By Morgan Lucas (she/her) from this video by Johnny Chivers We use data migration services to, well, migrate data. But why would we want to do this?  Perhaps... We're moving our business to the cloud, and need to shift all of that cold storage we have onsite. We want to use it as a backup in cause our infrastructure is out of commission. We could have information to share with a 3rd party, and instead of giving access to on-site databases, we put it on AWS to share. Nevertheless, let's recap what I've done. Created publicly accessible, password-protected database with Amazon Aurora with PostgreSQL Compatibility to migrate to Amazon Dynamo DB Managed inbound rules of security group to limit access Used open source software HeidiDB to interact with database via a TCP/IP session and specific URL for DB (Not shown here for security)     Connected to Aurora PostgreSQL Database ran queries that deleted and created tables

Winning Multiple Data Scholarships

  In April, I was selected - 1 of 1000, out of 10,000 - by the Blacks in Technology Foundation for Python for Data Science. I used SQL to parse and analyze data. Truth - Python isn't my favorite - but I want to learn to tolerate it - I told them this and was still selected. So glad to see an organization accept honesty! In June, the train kept rolling with Intro to Data Science - Granted to me by Women in Cloud, though a partnership with Microsoft. While I didn't make it to the second round of the BiT scholarship, the first part encouraged me to learn more about data and python on my own. It certainly wasn't a waste. If there are any other data scholarships you know of that are open to people out of school, transitioning industries, or just with a curiosity, please let me know - I'm happy to share them!

A look at Google Colaboratory (While Building a Stock Portfolio)

    After the group effort heard 'round the world in 2021, stocks are on everyone's minds. I didn't have money in GameStop, but I did have money in AMC from 2020. I bought it right as most theaters were closing from the Coronavirus pandemic and it was extremely cheap.   Why? I like the moviegoing experience! As of March 6th 2021, many AMC Theaters in New York City and State are reopening at 25% capacity. I'm still holding onto the stock - It never rose as high as GameStop's - but you won't see me at any theaters so soon. Onto today's topic: Google Colaboratory is a cool tool, allowing you to run Python scripts right in the browser.  You can even install things into the environment! Is it a little container environment running my program?