Menu

Intelligent Metrics Monitoring

August 12, 2016

Here at Wealthfront we have many offline computations running in Spark. In some cases, small changes have caused a job to slow down dramatically, and other times, the size of the input may be growing and causing the job runtime to quickly increase. We normally check pipeline runtimes manually to make sure jobs are running… Read more

Testing Cascading applications

October 15, 2013

This post explores how we apply our test-driven-development philosophy to analytics problems. In particular, it shows how use test-driven development with┬áCascading, which we’ve recently started using to drive analytics at Wealthfront. Cascading let’s us specify complicated analytics pipelines in Java. It works well for problems that would normally require multiple MapReduce jobs to get a… Read more