Link to wealthfront.com

Fork me on GitHub

Friday, December 10, 2010

IRC bots for Laziness, Impatience, and Continuous Deployment

1988 was an epic year. Not only were the movies "Die Hard", "Coming to America" and "The Naked Gun" released, but according to Wikipedia, IRC was invented. A tried-and-true technology, constantly reincarnated in similar (perhaps inferior?) forms, IRC is a superb communications hub for a company because it affords so many things:
  • Everyone can chat about coding, operational concerns, and the weird things non-engineers do.
  • Folks can be connected if working remotely.
  • You can store a record of those conversations, events and actions to create an "institutional memory".
  • Bots (computer programs) can join the chat rooms to do our bidding.
The last point is sometimes under-appreciated, because IRC can then become a "networked command line". To that end we've hooked up our epic Deployment Manager and a few other services as IRC bots. These bots keep us informed about our continuous deployments and the state of the production systems, and allow us to quickly and easily react to any situation. Writing IRC bots is a virtuous activity.

Our bot lineup includes:
  • "DM": (Deployment Manager) Build breakage/fixes, deployments, roll backs, service instance bounces and leadership changes.
  • "RS": (Report Service) Notifies us of operational reports that need our attention.
  • "ESP": (Esper Service) Exception alerting and clearing, live site stats.