Integrating Apache Spark into your existing Hadoop system – Part I

June 22, 2016

As evidenced by our previous blog post, Statistics is Eating the World, data is at the very center of Wealthfront’s values. At Wealthfront, fields ranging from research, analytics to marketing, client services, human resources and even employee productivity all heavily rely on data in their decision makings. Such a variety of data sources and requirements need… Read more

Connecting to an FTPS Server with SSL Session Reuse in Java 7 and 8

June 10, 2016

“Good programmers write good code… Great programmers reuse great code.”  Or so I told myself as I snagged an Apache Commons class to connect to a new vendor’s FTPS server.  Several hours of debugging later, however, I realized to my dismay that the omnipotent Apache Commons did not support a major security feature required by most modern FTPS servers.  This post outlines my process… Read more

How to Make Your Persistent Queues Run Faster Safely

May 04, 2016

What is a Persistent Queue? A persistent queue is a list of objects that persist in the database waiting to be polled and processed in some way. Usually it is a table that has columns for data, timestamp of when the object persisted, and timestamp of when the object is polled. As opposed to an… Read more

Identifying Non-Heap Class Leaks

April 18, 2016

At Wealthfront, a significant portion of our backend applications are written in Java. The Java Virtual Machine (JVM) uses Garbage Collection (GC) for memory management, this forces us to pay close attention to its characteristics and behavior. Every JVM service we run accumulates JVM and application-level statistics, spools these metrics into a statsd server, and… Read more

Building Mobile A/B Testing Infrastructure

April 11, 2016

Twenty years ago, one would have hardly heard the phrase “A/B testing”. In 2000, Google served its first A/B test to determine the optimal number of search results to display. In 2011 alone, they ran 7000 A/B tests across their platforms. A/B testing has become the bread and butter of product decision-making for Silicon Valley… Read more

Build System Integration Testing with Chef

April 05, 2016

At Wealthfront, we’re constantly pushing to use cutting edge technologies to take advantage of new language features, improved compilers, new libraries, and the like. Downtime in our build system would be a big problem with how often we like pushing new code to production, so we want a process that would allow us to upgrade parts… Read more