AWS Resource Cleanup Automation
BACKThe challenge
Some of our customers at TrueAbility, require to provision additional resources on AWS in order to create a viable environment to test the candidate's skills. Due to the inability to automatically delete unused / dangling resources on AWS, we saw a significant and steady increase in our AWS bill. The clean up process was manual, which was time-consuming and tedious. I was tasked with creating an automated solution to fix the problem.
The path to the goal
Collaboration with the HashiCorp dev team to identify the best tools for the task: aws-nuke was the tool picked up for deleting resources on AWS.
Created a microservice with Express.js, serving an API endpoint accepting a payload with an array of AWS account IDs to be cleared up.
Created a worker on Ruby on Rails (on our main app) to collect all nukeable accounts and send them to the Express.js server.
Added calls to the microservice in strategic steps of our provisioning flow to immediately remove unused resources when no longer needed.
Identified a set of resources that are not supported on aws-nuke and implemented a custom solution to delete them with aws-cli from the microservice.
Scaled costs from an average of $300 per day to a few bucks.