Building Clouds

I've spent this year building networks using Amazon Web Services and teaching people how to do it. So I'd like to share the code that I've used as teaching examples and as seeds for the creation of some pretty cool environments.

  • AWS PY was my first published attempt at interacting with AWS in python & Puppet to instantiate, provison and control EC2 instances, as well as the seed for an incredibly cool project at the start of this year.
  • AWS RB followed on to duplicate the instantiation and provisioning of EC2 instances using Amazon's Ruby APIs and Chef Solo. This was initially done as an itch that I had to scratch but since then it has been used as the seed for some of my paid AWS work.
  • AWS VPC once again uses Amazon's Ruby APIs and Chef Solo, this time to provision a Virtual Private Cloud. Amazon provides excellent documentation on what a VPC is and how to provision one using its web-based admin console, but I wanted to create a cloud from the ground up using repeatable scripts with no admin console interaction. The only pre-requisite is that you have an AWS account and have API keys (and have provided all the necessary details to Amazon so that they will allow you to create EC2 instances).
  • AWS VPC PY is my latest example and still a work in progress. It is designed to showcase how to create an AWS VPC using python, Boto and Fabric rather than Amazon's Ruby APIs. I'm not wedded to either approach so it is interesting to try out which one works & feels better in different contexts.

May you find these as useful as I have, and still do.

Tags: , , , , , , , ,

How to use rsync on OSX

I don’t really want to copy dot files (eg. .DS_Store), and I want to avoid the bug that rsync exhibits with time-capsule where it loops creating multiple ..DS_Store.xxxx files.

rsync -vrW --ignore-existing --exclude ".*" --progress ~/Movies/ /Volumes/Backup/Movies/

Tags: , , ,

How to delete all zero length files in a directory tree

find . -type f -size 0 -print0 | xargs -0 rm -f

Tags: , , , , ,