Menu

Continuous Deployment with Spark Streaming (Part II)

April 16, 2018

In Part 1 of this series, we gave a brief look into the data platform at Wealthfront, focusing on how we minimize data processing downtime while redeploying our Spark Streaming jobs. In this final part of this series, we will describe how we coordinate the data in our serving layer and wrap the blog post… Read more

Continuous Deployment with Spark Streaming (Part I)

April 06, 2018

Here at Wealthfront, our mission is to democratize access to sophisticated financial advice. One of the ways we accomplish that is by applying data processing technologies to financial services. In this two-part blog post we will describe how our near-real time data platform is built to support continuous deployment in order to allow us to… Read more

A specific trick and a general habit for achieving fast computations

December 15, 2016

I recently worked on a computation involved in a client-facing projection graph. In order to delight our users, we wanted it to respond instantly to user interaction. However, there are two forces at play that make the calculation complicated: the consistent contributions of recurring deposits and the erratic motion of stock prices. We needed to… Read more

Connecting to an FTPS Server with SSL Session Reuse in Java 7 and 8

June 10, 2016

“Good programmers write good code… Great programmers reuse great code.”  Or so I told myself as I snagged an Apache Commons class to connect to a new vendor’s FTPS server.  Several hours of debugging later, however, I realized to my dismay that the omnipotent Apache Commons did not support a major security feature required by most modern FTPS servers.  This post outlines my process… Read more

Lessons Learned when Scaling our Data Analytics Platform

August 20, 2014

Over the past year, we’ve gone from a single Java server running all analytics to a multi-node data pipeline. Along the way, we’ve refined our metrics for all parts of the business: from web and mobile analytics, to investment research, to trading and operations. We build metrics dashboards for everything we do, empowering us to… Read more