Cloud computing, a noticeable paradigm shift in the history of computing offers a very compelling value proposition. But this comes at a growing awareness of the under-pinnings of the variety of costs that one should be mindful of; such as the cost of the underlying technologies, compute cycles, and the costs associated with provisioning virtual resources. In this blog post, we will look at ways that one can develop a good hygiene on automating cloud cleaning process.
In a previous blog post, we had a brief look at my rather untidy AWS Lab and how that escalated into a cost control management issue (whoops!). In this article we are going to look at some tools that we can use to automate the cleaning process to help increase your security posture and add to your cost savings! (…think how many pints you can add to your slosh slush fund!)
Why do I have to tidy it up?
Some of the side effects of having an untidy infrastructure can include:
- decreased productivity due to a cluttered work environment
- difficulty finding important resource information or tools
- increased stress levels
- decreased morale within your development teams
- time spent/lost in perusal of information
It’s funny how the symptoms of a physically messy environment can bleed over to the virtualized Cloud world, it is also very similar when reversed. For example, when you have a tidy cloud infrastructure, you can:
- Leverage the power of automation to manage and maintain your cloud resources which increases productivity
- Optimise performance and improve security which helps locate obscured data or tooling
- Save time and money on administration costs which will alleviate the stress levels of your operations team
- Improve collaboration and communication among team members which in turn will increase the morale of your teams working together.
So let’s have a look at some tools we can use to automate the overall creation and management of a more structured and cleaner environment.
AWS Nuke is a small project that will tear down an AWS Account in its entirety or governed by a YAML file. In the past we have used this project that can be pulled directly as a Go binary from GitHub to be run by a Lambda function to perform the required clean up. In more modern times of course we can use Lambda Layers to ensure the tools we need to do the tidying up are already there which leaves us more options to choose from.
Much like its brother above, AWSweeper is another tool that will tidy up a given account from a Users command line. The additional cool feature with AWSweeper is that you can put some basic logic into your YAML policies.
For example, you could set up your configuration to remove items in relative time to make sure no item in the account is older than 30 days of its creation! This can be a blessing for tidying up a highly used account where everything might not have been removed.
AWS Auto Cleanup
AWS Auto Cleanup is another interesting project in this subject, this tool has broken out the different components of the process and placed them into 3 modules reflecting the Application that does the job, an API to enable remote triggers more easily and integrate with programmatic workflows as well as a web interface to give Administrators a better look at what is going on.
AWS Multi Account Cleanup
AWS Multi Account Cleanup for the huge enterprise clients that want to be running this type of service across multiple accounts from a centralised location. This is the project for you.
Much like its sibling projects, it can be governed by a policy and an array of commands to make it operate over a variety of different accounts as long as the IAM role is there to give it permissions as shown below in the projects Architecture document.
As you can see there is a lot of variety around the tools for different setups and ways of working, from small lab instances all the way to Enterprise grade multi account scenarios. All of which, internally, have their own method for sorting and protecting selected infrastructure that you might not want to remove, like Security Tooling!
The tools mentioned in this article provide no-excuse to not tidy-up lab environments. Start controlling your costs and lowering your spend on your Development Labs Today! Have a look at the tooling and see how you might be able to use them yourself to keep rogue instances and spend under control!
Paul Hardy is a Principal Systems Developer at Cloudreach with a passion for Offensive Security. Having worked with Cloudreach for almost a decade he has built up expertise over a wide range of technologies often finding new and creative ways to improve Cloudreach’s clients Security Posture during engagements.