DevOps at Wealthfront

November 12, 2013

A DevOps engineer starting a new position fully expects to be handed over keys to a custom model helicopter (aka company’s infrastructure) and encouraged to take a daring flight on day one. No two helicopters are quite the same, and flying a different model is not quite as intuitive as riding a new custom bike. However, with enough experience, DevOps engineers have few problems quickly coming up to speed in a new environment.

I started as a DevOps engineer at Wealthfront two weeks ago. In the two weeks, I did production pushes, service and data migrations, updates to configuration management system, almost entire server migration into the cloud, and major Jira and Confluence upgrades. The opportunities to make immediate and significant impact are limitless at Wealthfront.

From the DevOps perspective, three traits set the Wealthfront environment apart from other organizations. First, unlike organizations that rebranded build engineers with deployment responsibilities as DevOps, Wealthfront understands what DevOps actually is. At Wealthfront, nobody needs to be convinced of the values of Configuration Management, Infrastructure as Code, Continuous Integration, and Automation. Second, unlike environments that hypothesize about continuous production deploys, Wealthfront has a robust continuous production deployment system fully implemented and actively used. Third, unlike companies that have multi-layered development/operations/support organizations with elaborate communication channels between teams, at Wealthfront, Development and DevOps are closely integrated. Developers deploy to production multiple times a day, and DevOps commit code to Development repositories and use the same Development workflow.

My first significant undertaking at Wealthfront is developing an Amazon Elastic MapReduce (AWS EMR) monitoring service. EMR is a third-party service running outside of Wealthfront infrastructure controlled through APIs. In the current setup, the interaction with EMR is managed by scripts. Amazon APIs provide full state information of EMR jobs. In order to efficiently monitor the state of EMR workflows, the state information needs to be brought in from Amazon into Wealthfront platform, contextualized according to Wealthfront implementation, and exposed through APIs that can be consumed by platform services.

I chose to implement the monitoring service using JRuby, Grape, and AWS-SDK. Checkout my boilerplate project for implementing API service with Grape and JRuby. Grape/JRuby is, in my opinion, one of the most scalable, testable, modular, flexible, rapid ways to implement API services.

Using Google Guice bindings from JRuby my service seamlessly integrates into Wealthfront platform. Running a Wealthfront Zookeeper client is done with just a couple of lines thanks to JRuby/Guice.

I have never had an easier time integrating Operations services with the Application SOA Platform.