In this article, we will see an interesting use case or system design where we want to connect AWS SNS to an internal HTTP or HTTPS service. SNS is a public service designed to connect to external public HTTPS endpoints. SNS is purpose-built for external communications and it supports Public[...]
Read more
In this article, we are going to see how to find unused load balancers - based on Cloud Watch usage metrics like request count, active flow count We are going to get the usage metric from CloudWatch API using Python Boto and write it as a CSV report for all[...]
Read more
In this article, we are going to see how to save the SNS messages to S3 using Kinesis Data Firehose The first step of building a data engineering pipeline in your organization is storing the unorganized data with long retention and partitioning. Therefore storing the SNS data with Long-term storage[...]
Read more
Apache Drill is a revolutionary product that identifies itself as a Schema-free SQL Query Engine for Hadoop, NoSQL and Cloud Storage - While it was often represented or perceived as a tool for Data Engineers or Analysts. It is not just for Data Engineers In this article, we are going[...]
Read more
In the world of networking, IP addresses are crucial for devices to communicate with each other over the Internet. However, not all IP addresses are created equal, and some serve unique purposes. One such IP address is 169.254.169.254, which is commonly referred to as a link-local address. In this article,[...]
Read more
Secret Management is an essential part of infrastructure and DevOps. how do you store your Secrets and Environment variables define how safe your infrastructure is As a good engineering practice, most companies guide their developers to not hard code the secrets and environment variable directly but out of convenience or[...]
Read more
Amazon Simple Storage Service (AWS S3) is a highly scalable and cost-effective object storage service provided by Amazon Web Services (AWS). It allows users to store and retrieve vast amounts of data, making it a popular choice for developers, businesses, and organizations worldwide. To interact with S3 buckets and objects[...]
Read more
How do you SSH to an EC2 instance which is on a Private Subnet? You would say either of the following options Create a Bastion Host in Public Subnet then SSH to the Bastion Host and then SSH to the EC2 instance in Private Subnet Setting up EC2 instance Connect[...]
Read more
Relation Database Service (RDS) is a managed database option provided by AWS. It is a great option for those who want to use a relational database without having to manage the underlying infrastructure. RDS provides a number of database engines to choose from including MySQL, PostgreSQL, Oracle, and SQL Server.[...]
Read more
AWS S3 (Simple Storage Service) is a highly scalable and durable object storage service offered by Amazon Web Services (AWS). AWS S3 is one of the most used service offerings of AWS and it is built to provide high-performance and zero-latency object(file) storage with security standards and compliance like HIPAA,[...]
Read more