AWS Config is a service that maintains an inventory of AWS resources. It keeps record of deleted AWS resources as well as the links between resources. This was especially helpful to me the other day during a deployment. Cloudformation would report that it was failing to delete a security group. I was unable to locate any instances or load balancers that were using the security group. Only after using the config service to look it up I was able to see that an AWS Lambda had been manually set up (by myself) a few months ago with that security group attached. This was done as part of the process to allow the Lambda to connect to a Virtual Private Cluster (VPC).
To set up Config go to: https://console.aws.amazon.com/config/home?region=us-east-1#/welcome
And click the blue “Get Started” button.
Next you will want to ensure all resource types are selected as well as all regions.
The Config service sends all change information to an Simple Storage Service (S3) bucket. This is helpful if you are looking to ingest this data to Elasticsearch or some other service. Create a new bucket in this section to enable that.
Once done with the S3 bucket you will then be able to set up an Simple Notification Service (SNS) topic. This is also helpful for ingesting data into Elasticsearch using a Lambda to stream the data into Elasticsearch or some other service.
Lastly you will set up a role. AWS sets up the permissions for you if use click “Create a role”. This enables the Config service to properly and securely access each AWS resource.
On the next page you will be given a chance to set up Config Rules. At this point I am not ready to set these up but they are as follows: I clicked “Skip” but I fully intend to set these up later.
Review the settings and click “Confirm”.
On the next page you will see "Resource inventory". To lookup resources click the dropdown for "Resource type" and select the type of resource you want to lookup.
For testing purposes I have selected "S3: Bucket" and in the optional "Resource identifier" box next to the drop down I entered my bucket name and clicked "Look up".
At this point you will be able to click on the link for the "Config timeline".
The timeline allows you to view the various configurations the resource had over the period of time selected.
AWS Codebuild and Static Sites
Simple play by play of how I deploy this siteread more
More S3 analysis
I wrote a script to do additional analysis on s3 logs via python.read more
S3 Log Analysis
Simple S3 Log analysis using aws cli and bash.read more
Amazon SNS for Nagios monitoring
Amazon SNS for Nagios monitoring, use SNS to notify admins via email, SMS, etc.read more
Amazon Glacier is for long term storage.read more