A Fix-It Day with Cyclomatic Complexity

November 01, 2010
It is here that the stereotype of the programmer, sitting in a dim room growling from behind Coke cans, has its origins. The disorder of the desk, the floor; the yellow Post-It notes everywhere; the whiteboards covered with scrawl: all this is the outward manifestation of the messiness of human thought.
The messiness cannot go into the program; it piles up around the programmer.
—Ullman, Ellen. Close to the machine : technophilia and its discontents : a memoir / by Ellen Ullman. City Lights Books, San Francisco : 1997
Wealthfront Engineering is a truly test-driven organization—for example, see our extreme testing practices, our continuous deployment infrastructure, and our previous attempts at pizza-driven development. From time to time, despite our best efforts, messiness creeps into our software. It may be due to business priorities, lack of foresight, or a simple miscalculation during your Ballmer Peak experiments, but the net effect is always equally grim: crucial methods are left untested. Your code evolves, ages, and, without tests as documentation, your code falls out of sync with your hand-written documentation. You must act often and decisively.
Accordingly, here at Wealthfront, we try to hold a Fix-It Day once every couple of months. During a Fix-It Day, it’s all hands on deck. We pause the world and we dedicate ourselves to fixing a well-scoped problem in our codebase.
RSpec’ing our Front-End. During out last Fix-It Day, the goal for our front-end team (with only one pair of Hugh and Tony) was to convert all of our (J)Ruby unit and functional tests from Test::Unit to RSpec. By the time they were done, our 6381 lines of code of Test::Unit had been eradicated, and, along with pre-existing RSpec tests, we ended up having 8415 lines of code of RSpec extraordinaire. Is there even a buzzword for that kind of turn-key live-sync active-hot over-the-air cloud-based transition to a new technology?
Cyclomatic-Based Goal for Backend Team. The goal for our backend team was to test as many Queries as possible, starting with the most complex ones. (A Query is our internal interface that exposes a unit of work to our RPC ecosystem.) To estimate the complexity (therefore likelihood of failure) of queries, we used cyclomatic complexity (Wikipedia)—although consensus has not been reached in terms of the metric’s correlation to number of defects, it does directly measure the number of possible execution paths. A lot of branching and conditional logic is usually related to delicate business logic. We used CyVis to procure the measurements for our untested queries, some Unix magic to process the results, analyze and sort them.
Goals. Despite our QueriesAreTestedTest which ensures that all queries are tested (allowing for exceptions), when we started, amongst our 845 queries, 49.11% were untested (that’s 415 untested queries). About 50 of theses queries had a high cyclomatic complexity of over 5. We managed to test nearly 95% of those untested queries with high cyclomatic complexity over the course of a day!
Pair Programming. To maximize our throughput, we paired up everyone for the day! Each pair was picked as to maximize mixing of experience levels and sitting groups. (The optimal arrangement would be such an arrangement in which each pair has the most experienced unpaired person and the most unexperienced unpaired person that sit in different sitting groups.)

Results & Cost. With 4 pairs in the backend team (one pair excused due to prior conflicts), we managed to get our untested queries down to 42.05% in exchange for 5 hours of coding among 4 pairs and $60 worth of pizza. We now have 837 queries in our system (we grasped the opportunity to get rid of some deprecated ones) and only 352 of them are untested (about 90% of those have a very low cyclomatic complexity (less than 3).

Conclusion. With a quick day of painless team effort, the quality of our codebase has dramatically increased in the places that needed the most attention. We believe Fix-It Days are an easy way to ban the messiness that will inevitably creep into your codebase, as soon as it occurs, with might and decisiveness.
Let us know what your experiences are with organizing a Fix-It Day at your organization, or let us know if you have any questions, suggestions, or thoughts.