Archive for category Software Development

Building Clouds

I've spent this year building networks using Amazon Web Services and teaching people how to do it. So I'd like to share the code that I've used as teaching examples and as seeds for the creation of some pretty cool environments.

  • AWS PY was my first published attempt at interacting with AWS in python & Puppet to instantiate, provison and control EC2 instances, as well as the seed for an incredibly cool project at the start of this year.
  • AWS RB followed on to duplicate the instantiation and provisioning of EC2 instances using Amazon's Ruby APIs and Chef Solo. This was initially done as an itch that I had to scratch but since then it has been used as the seed for some of my paid AWS work.
  • AWS VPC once again uses Amazon's Ruby APIs and Chef Solo, this time to provision a Virtual Private Cloud. Amazon provides excellent documentation on what a VPC is and how to provision one using its web-based admin console, but I wanted to create a cloud from the ground up using repeatable scripts with no admin console interaction. The only pre-requisite is that you have an AWS account and have API keys (and have provided all the necessary details to Amazon so that they will allow you to create EC2 instances).
  • AWS VPC PY is my latest example and still a work in progress. It is designed to showcase how to create an AWS VPC using python, Boto and Fabric rather than Amazon's Ruby APIs. I'm not wedded to either approach so it is interesting to try out which one works & feels better in different contexts.

May you find these as useful as I have, and still do.

Tags: , , , , , , , ,

Default HTML-escape using Freemarker

Most java developers have at least heard of Freemarker.

FreeMarker is a “template engine”; a generic tool to generate text output (anything from HTML to autogenerated source code) based on templates. It’s a Java package, a class library for Java programmers. It’s not an application for end-users in itself, but something that programmers can embed into their products.

It is the “generic” nature of Freemarker that trips up java web developers. Freemarker by default does not provide any facilities to allow default HTML-escaping of content – a necessity if you want to attempt to prevent Cross-Site Scripting attacks on your web applications. Yes I know that it has the ?html built-in, and that you can wrap blocks of text in <#escape x as x?html> directives, but you have to remember to do that on each page.

What if there was another way?

The class below is a Freemarker TemplateLoader that automatically wraps each loaded template with the HTML-escape directive. Now there is no need to remember to do that in your templates. You can find it being used in my example project on GitHub.

import freemarker.cache.TemplateLoader;
import org.apache.commons.io.IOUtils;

import java.io.IOException;
import java.io.Reader;
import java.io.StringReader;

public class HtmlTemplateLoader implements TemplateLoader {

    public static final String ESCAPE_PREFIX = "<#ftl strip_whitespace=true><#escape x as x?html>";
    public static final String ESCAPE_SUFFIX = "</#escape>";

    private final TemplateLoader delegate;

    public HtmlTemplateLoader(TemplateLoader delegate) {
        this.delegate = delegate;
    }

    @Override
    public Object findTemplateSource(String name) throws IOException {
        return delegate.findTemplateSource(name);
    }

    @Override
    public long getLastModified(Object templateSource) {
        return delegate.getLastModified(templateSource);
    }

    @Override
    public Reader getReader(Object templateSource, String encoding) throws IOException {
        Reader reader = delegate.getReader(templateSource, encoding);
        try {
            String templateText = IOUtils.toString(reader);
            return new StringReader(ESCAPE_PREFIX + templateText + ESCAPE_SUFFIX);
        } finally {
            IOUtils.closeQuietly(reader);
        }
    }

    @Override
    public void closeTemplateSource(Object templateSource) throws IOException {
        delegate.closeTemplateSource(templateSource);
    }
}

To wire this up using SpringFramework’s Freemarker support you do have to take another step and extend its FreeMarkerConfigurer to register the HtmlTemplateLoader as the one to use for view resolution and rendering. If on the other hand you don’t use Spring then you have one less bit of code to maintain.

import freemarker.cache.TemplateLoader;
import org.springframework.web.servlet.view.freemarker.FreeMarkerConfigurer;

import java.util.List;

public class HtmlFreeMarkerConfigurer extends FreeMarkerConfigurer {

    @Override
    protected TemplateLoader getAggregateTemplateLoader(List<TemplateLoader> templateLoaders) {
        logger.info("Using HtmlTemplateLoader to enforce HTML-safe content");
        return new HtmlTemplateLoader(super.getAggregateTemplateLoader(templateLoaders));
    }
}

Tags: , , , , , ,

Provision EC2 instance using boto

Sam Newman recently published a very interesting blog entry on using fabric to apply puppet scripts on remote machines. He left the provision_using_boto() method as an exercise to the reader. That just sounded tempting enough to be a challenge since I hadn’t gotten around to looking at boto. You can find the result of my attempt on GitHub. To be precise aws.py implements the provisioning using boto and fabfile.py drives fabric and puppet. Hope you find it as useful as I have.

Tags: , , , , , , ,