At Wealthfront everything we build is designed to move quickly. For example, it is common for new hires to deploy new code to production on their first day. Our continuous integration and deployment environment are designed so that if the tests pass we ship. This, of course, is actually quite normal for a Silicon Valley startup with a dedicated team of talented engineers. We believe that’s just how things should work.
Unfortunately the same cannot be said for iOS development in general where that cadence is, strictly speaking, atypical. App Store friction aside most organizations are challenged to ship an update to their iOS app every month, let alone every week or every day. Yet that’s exactly what we can do here. In the time it took Apple to approve our first release, we shipped four significant updates to the app internally.
Let’s take a look at how we’ve achieved such rapid cadence with Wealthfront for iOS.
The Wealthfront Way
Our dedication to effective continuous integration, test driven development and painless deployment were a big factor in my decision to join the company. I was actually quite excited to apply the same paradigms and methodology to our mobile initiatives. Using the existing infrastructure as a model I identified several important goals:
- High quality tests should be easy to write
- Tests should fail when something breaks, and pass when nothing is broken
- Continuous Integration should create “production” ready builds
- Establish a “master” stable development cycle where we can release immediately without periods of “stabilization” or “convergence”
Write Great Tests
This might sound obvious, the better the tests are the better the final product will be. However in a high velocity development environment the tests have to be as easy as possible to write. At Wealthfront we use a number of tools to facilitate rapid test iteration like OCMock and CoreData. Both provide elegant and scalable solutions to the core challenges of creating a scalable test framework.
OCMock is the Objective C equivalent of JMock. It’s a mock object framework that allows engineers to separate the concerns of individual tests and ensure that each test is only executing the specific code under test. Leveraging a framework like OCMock in our continuous integration test suite greatly reduces the number of spurious or accidental failures by ensuring that tests are very tightly scoped. The natural corollary is that test results are consistent and “believable”, so the build server’s emails are never ignored or written off as “just having a bad day”.
Our app uses Core Data under the covers to cache and organize data from our API server. Electing to use Core Data was a conscious decision based partly on its ability to be used in a test framework. At Wealthfront we believe that data driven testing is essential to our ability to validate our code. Therefore whatever persistence framework we chose had to integrate seamlessly with our test suite.
With Core Data data driven tests are actually easier to write than many alternative solutions because the test can simply configure a managed object context with the desired data and pass that context to the object under test. This type of fixture facilitates easy case enumeration (from common to boundary) with reliable and consistent results.
Continuous integration with iOS has, until recently, been rather challenging. The Internet is full of workarounds and scripts for making solutions like Jenkins play nicely with Xcode and the iOS Simulator. But with the release of iOS 7 and OSX 10.9 Apple included a native way to support continuous integration, Xcode Server. We use Xcode server to pull changes from our Git repository and run them against the simulator and real hardware.
As with the other projects at Wealthfront our build server is at the heart of our continuous integration suite. After a pull request is approved and merged to the master branch of our git repository Xcode Server runs four different bots:
- Our unit test suite against the app store build
- Our unit test suite against the internal build
- A device ready app store build
- A device ready internal build
The unit test bots give us coverage over our production and internal builds while the other two build production ready archives that can be deployed to devices.
Stable Continuous Deployment
With this infrastructure in place we have been able to reliably hit a steady one week cadence, where improvements and even smaller new features are released at the end of each week to our employees. At the time of this writing our test framework has ~400 unit tests which evaluate everything from our parsing / caching code to custom implementations of -drawRect:. The tests take ~6-7 seconds to run per instance; the entire suite takes less than a minute on all the flavors of the iOS simulator and a few choice bits of real hardware we keep hooked up to the build server.
We’ve also been able to maintain a very high quality bar. For example our first internal beta had zero reported bugs, an achievement we have since repeated many times. Of course that doesn’t mean there aren’t any bugs in our code, they were just very difficult to find and reproduce, as they should be with well tested code.
Similar to our other projects we’ve achieved a master stable release cycle where changes that are successfully built by the build server after being merged to master can be shipped to the App Store. Of course, that doesn’t mean we ship every passing build to the App Store. But we have the infrastructure and tools in place to know that we could if we had to.
In all we are quite happy with we have achieved thus far and we’ll be sharing a lot more about our test infrastructure and iOS development at Wealthfront in general in the coming weeks. Until then please let us know your thoughts and questions in the comments.