in which the crew rocks out before the house opens

and also some stuff about security


Building a Robust etcd cluster in AWS

Consensus based directories are the core of many distributed systems. People use tools like zookeeper, etcd and consul to manage distributed state, elect leaders, and discover services. Building a robust cluster of these services in a chaotic environment was trickier than I thought, so I’m documenting what I figured out here. The source for all this is available on github. Goals Use cloudformation to establish a three node autoscaling group of etcd instances.


Gerrit Code Review

At work, we’ve been long-time users of Gerrit for code review. We recently started a new project with a new team and had an opportunity to reevaluate that choice against other workflows, particularly Github Pull Requests. Collectivist Approach I’m a strong believer in a collective approach to team decision making (you could call it the anarchist (with a capital “A”), but that freaks people out). Particularly around tooling, leaders should not be perscriptive.


Command line helpers for Gerrit

At work we use Gerrit for pre-commit code review. I described the reasons previously. Here are some handy shell helpers to make working with gerrit reviews a bit quicker. The full script is here Assumptions: These examples assume you have a code review server at, although obviously yours will be somewhere else. These examples assume you have a git remote named review that defines your code review server. A partial .git/config might look like: [remote "review"] url = ssh:[email protected]/project fetch = +refs/heads/*:refs/remotes/review/* push = HEAD:refs/for/master You have git (obviously), ssh, and jq (from here) in your path.


Yubikey for Local Authentication on Mac

How I set up OS X to require a Yubikey for local login. (Obsolete)


Tiny Docker + Go Pattern

Docker is a handy way to deploy applications, and Go is handy way to build them. Here is how we build Docker containers for small apps.


Docker build environments

Like a lot of folks, we’ve found that Docker is a handy way handy way to avoid having long complex build environment setup instructions. Instead, we define a docker container and assume that the build always runs there.



Withebs runs a command with the specified EBS volume attached to the currently running EC2 instance. The source is available on Github. Usage: withebs --volume=$VOLUME_ID docker run -v /ebs/$VOLUME_ID:/data training/webapp The volume is mounted at /ebs/$VOLUME_ID. If the volume does not contain a recognized filesystem, it is formatted with mkfs before mounting. Options: -volume - which volume to mount. -attach-timeout - how long to wait for the EBS volume to successfully attach to the instance.


IP_PKTINFO and removing network devices (Updated)

If you use the IP_PKTINFO flag on your UDP listener, everything breaks when network devices change


Using Google Apps to log in to AWS

In which we point that maintaining user accounts sucks, avoiding it is hard, and share a tool to make it a tiny bit easier (I hope).


Whats in a (Windows) name?

In which we introduce a forensically sound encoding scheme for Windows names.


Ephemeral Encryption in AWS

How we use volume encryption in our AWS ephemeral disks


Random notes that might be useful, but probably are not.