Be Alerted When an Amazon S3 File is Missing Using Lambda, SNS, and EventBridge

Alarms are very helpful in determining the state of certain processes such as Amazon Web Services (AWS) Glue workflows and jobs as part of an ETL pipeline. Ideally, there would be a team assigned to monitor the AWS Glue workflows regularly, but this is not very practical or feasible as many workflows are scheduled to run overnight or during off-hours. Setting up alarms to ensure these workflows successfully finish and don’t error out is key to troubleshooting issues early and maintaining and smooth-running pipeline.

Amazon Web Services (AWS)Andreea StanoviciFebruary 7, 2022Andreea Stanovici, AWS, Amazon S3 Comments

Passing AWS Secrets to Microsoft PowerShell

Secret managers have become the go-to tool to avoid hard coding credentials in source code. Secret managers provide a centralized place to store sensitive data like database credentials and API keys. In Amazon Web Services (AWS) there are two services for storing secrets: AWS Secrets Manager and the Parameter Store. The key to this workflow is specific PowerShell modules that will allow you to script operations of AWS resources directly from the PowerShell command line.

Amazon Web Services (AWS)Andreea StanoviciDecember 13, 2021Andreea Stanovici, AWS Comments