Tagged In data :
Continuous Deployment with Spark Streaming (Part II)
In Part 1 of this series, we gave a brief look into the data platform at Wealthfront, focusing on how we minimize data processing downtime while redeploying our Spark Streaming jobs. In this final part of this series, we will describe how we coordinate the data in our serving layer and wrap the blog post… Read more
Continuous Deployment with Spark Streaming (Part I)
Here at Wealthfront, our mission is to democratize access to sophisticated financial advice. One of the ways we accomplish that is by applying data processing technologies to financial services. In this two-part blog post we will describe how our near-real time data platform is built to support continuous deployment in order to allow us to… Read more
A specific trick and a general habit for achieving fast computations
I recently worked on a computation involved in a client-facing projection graph. In order to delight our users, we wanted it to respond instantly to user interaction. However, there are two forces at play that make the calculation complicated: the consistent contributions of recurring deposits and the erratic motion of stock prices. We needed to… Read more
Connecting to an FTPS Server with SSL Session Reuse in Java 7 and 8
“Good programmers write good code… Great programmers reuse great code.” Or so I told myself as I snagged an Apache Commons class to connect to a new vendor’s FTPS server. Several hours of debugging later, however, I realized to my dismay that the omnipotent Apache Commons did not support a major security feature required by most modern FTPS servers. This post outlines my process… Read more
Lessons Learned when Scaling our Data Analytics Platform
Over the past year, we’ve gone from a single Java server running all analytics to a multi-node data pipeline. Along the way, we’ve refined our metrics for all parts of the business: from web and mobile analytics, to investment research, to trading and operations. We build metrics dashboards for everything we do, empowering us to… Read more