Link to wealthfront.com



Like what you see here? Join the Wealthfront engineering team! Fork me on GitHub

Thursday, May 31, 2012

Do They Really, Really Like You?

For decades, companies have been trying to figure out if you like them using various surveys — long and short, direct and not — as the primary tool to get you to talk about your relationship with them. And, for decades, you've mostly ignored them, responded sometimes, and largely avoided talking about your relationship.

As with all relationships, there's one thing that really matters:
will they tell their friends about you? Word of mouth is the most powerful promotional tool you've got as a company. People who are fans of your product or service — really devoted to you and your company — will put their reputation on the line by sharing you with their colleagues and friends.

Enter the net promoter score (NPS), the simplest measure of satisfaction available. Introduced in 2003, NPS is quite simply a ratio of people who would recommend you to someone they know versus those who wouldn't, based on the answer to a single question: How likely are you to recommend this to your colleagues or friends? At Wealthfront, we're big fans of being direct and getting right down to it, so you know this clicked with us immediately.

Gathering the data is really easy: ask people to answer your question on a scale from 1 to 5, where 1 is “not at all likely” and 5 is “extremely likely.” The top 20% (the 5s) are your active promoters; the next 20% (the 4s) are passive but satisfied, unlikely to recommend but they'll say good things if asked; and the rest are detractors who not only won't risk their reputation by recommending you to others, they'll probably give at best luke-warm reviews when asked. You could use a scale from 1-10 or even 1-100; since the metric divides into quintiles, we think the nuance offered by larger scales is lost and that a larger scale introduces risks paralyzing them with too many choices in the name of false precision.

Let's apply this to a practical, if somewhat absurdist, example: the annual deluge of April Fools' Day pages on teh internets. Our own John Hitchings made an April Fools' Ranking app (now closed) to collect some pages and votes. Let's look at data for the 45 pages submitted:


Pagescore(1)score(2)score(3)score(4)score(5)
AdBlock: Inturdusing CatBlock009012
Box.net: Discover a New Dimension of Collaboration010016
ByteScout Socks Reader Beta101130
Introducing reddit timeline019030
Introducing the Roku Shake Remote111822
Collective Idea: Walken on Rails0110053
CNET UK: Pirate Bay to launch own sub100917
Sony USA: The VAIO Q, World's Smallest Ultrabook1102212
Google App Engine: Cloud API App101110
Google Australia: Google Street Roo000111
Google Racing0104316
Google App Engine Blog: Cloud API112100
Google Voice expands to new markets010160
MailBait: Fill Your INBOX0014821
Google Maps000877
CNet News: Google Tap tries [...] Gmail a la Morse code101121
Slashdot: News for nerds, stuff that matters107101
TechCrunch: Foxconn Plans New Iowa Plant121101
TechCrunch: Google's Sergey Brin To Retire800112
Introducing Shapes by Toshiba0120101
Google AdWords: Click-to-Teleport001131
Google Apps for Biz: Jargon-Bot100121
Google Fiber & Kansas City101010
Google Racing106101
KODAK: Print Kittens At Home0001218
Bunker LDLC0211112
MapsTD010022
PashionSense.com, Fashionably lonely?719117
Mind-boggling XKCD April Fools comic010120
ThinkGeek: Star Trek Inflatable Captain's Chair101920
ThinkGeek: Minecraft Marshmallow Creeps101010
ThinkGeek: Skyrim Electronic Dragon Shout Hoodie001010
ThinkGeek: Star Wars Admiral Ackbar Singing Bass0101021
ThinkGeek: Technomancer Digital Wizard Hoodie011120
ThinkGeek: Keurig K-cup 5-Star Meals111101
ThinkGeek: Barbie Digital Fashion Styling Head for iPad70101
ThinkGeek: Electronic Hungry Hungry Hippos for iPad000201
ThinkGeek: Game of Thrones Fire and Blood Perfume1001416
Twilio Telegram API1010122
Ubuntu Eyewear0012015
Virgin Volcanic: Journeys to the centre of the Earth117923
The YouTube Collection11112130
Bitbucket - Spooning0001210
What is Skype for String?0011230
Chrome Multitask Mode1001010

Calculating NPS is as straightforward as the question itself. Just subtract the percentage of detractors from the percentage of promoters (both as compared to total respondents).


Let's see how our merry pranksters did. Here's the top 10:

PagePromotersDetractorsCountNPS
MapsTD2212391.30
Google Maps7708590.59
Box.net: Discover a New Dimension of Collaboration1611788.24
Mind-boggling XKCD April Fools comic2012286.36
ThinkGeek: Skyrim Electronic Dragon Shout Hoodie1011181.82
What is Skype for String?3014367.44
Collective Idea: Walken on Rails53116465.62
ThinkGeek: Star Wars Admiral Ackbar Singing Bass2113262.50
KODAK: Print Kittens At Home1803060.00
CNET UK: Pirate Bay to launch own sub1712759.26


And the bottom 10:
PagePromotersDetractorsCountNPS
Bunker LDLC122235-28.57
Google Racing1718-33.33
Slashdot: News for nerds, stuff that matters1819-36.84
Introducing Shapes by Toshiba11223-47.83
ThinkGeek: Barbie Digital Fashion Styling Head for iPad189-77.78
TechCrunch: Foxconn Plans New Iowa Plant11415-86.67
ThinkGeek: Minecraft Marshmallow Creeps01112-91.67
Google Fiber & Kansas City01112-91.67
Google App Engine: Cloud API App01213-92.31
Google App Engine Blog: Cloud API01414-100.00


NPS can range from -100 (no one would tell a soul) to +100 (everyone can't wait to post it on their wall). Ranking above 30% is good; ranking above 75-80% is exceptional. And, of course, negative scores mean you may want to reconsider your hygiene regimen.

So there you have it. Now, go get undeniable proof that they like you, because they really, really do.

Wednesday, May 2, 2012

Belt and Suspenders: Safety and Trust in a Distributed System

Wealthfront is built on a traditional service-oriented architecture: requests from web browsers are dispatched to a cluster of Rails instances, which will in turn query back-end services. As a single page load might trigger tens of internal remote calls, it quickly becomes difficult to understand why a back-end service was called, who initiated the call, and where was the time spent.

As described last year on this blog, we built a simple tracing system to address this issue. As our services use HTTP to communicate, HTTP headers are used as a side channel to request and collect tracing information. If X-WF-Trace: true is sent with the request when querying a service, tracing is enabled and a header of the form X-WF-Trace: (local/q=GetAccount,200,406,((local/q=GetPrices,200,254,()))) is returned with the response. In this example, we learn that the queried service called GetAccount which returned a 200 OK status in 406 milliseconds. We also learn that GetAccount called GetPrices to help it accomplish that.

In this blog post, I'll describe how we built a similar system to verify that a request was initiated from a specific logged in user. The goal of this system is twofold. First, we want to prevent the possibility of information leakage such as displaying a client account to a different client because some identifiers were mingled somewhere in the flow of queries. Second, we want to generate a detailed audit trail when someone acts on behalf of somebody else.

The key idea behind the system we built is what we call a request authorization token. The token is a JSON array encrypted and authenticated. The array contains the authorized user, the underlying user and the expiration time. The underlying user is a free-form string identifying the person who requested the token. In most cases, it is the same as the authorized user.

Our Rails instances get a fresh request authorization token from our user manager on each page load. The token is passed using the X-WF-Authorization header to our back-end services with all remote calls made while rendering the page. Back-end services then rely on the UserVerifier to verify that the query is properly authorized. Of course, the token is propagated if a back-end service calls another back-end service.

The UserVerifier has two modes: enforcing and permissive (stolen from SELinux.) In enforcing mode, if a request does not include a proper authorization token, a 401 Unauthorized status is returned. In permissive mode, the lack of authorization is logged to disk and published to an internal dashboard. The default mode is enforcing, but the permissive mode can be used when introducing verification to a new query.

Unfortunately, using the UserVerifier is cumbersome. We strongly believe that unless this system is trivial to use, no one will use it and its entire purpose would be defeated. Consequently, we came up with an alternate, declarative solution. Instead of using the UserVerifier directly, query arguments can be annotated with the @Owned annotation.

Our framework takes care of the rest for you. Behind the scenes, we rely on what we call OwnerExtractors.

Implementing an OwnerExtractor is not simple. However, they only have to be written once and are thus highly leveraged. Our entire system only relies on a dozen extractors.

Last but not least, we also improved our internal tools to deal with request authorization tokens. Our simple wrapper around curl is called ikq. If we try to invoke the GetAccount query using ikq, it will fail with a 401.

To generate and send a request authorization token, we can use the -a flag. Before invoking GetAccount, ikq will first contact our user manager to obtain a new token. The user manager will record the UNIX user name of the engineer who invoked the ikq command as part of the audit trail.

Wednesday, April 11, 2012

Get thee to a nullery: self-aware nulls for testing

Pryce and Freeman's Growing object-oriented software is a good practical guide to test driven development, as you'd expect from the authors of jMock. It isn't necessarily easy to adapt piecemeal -- introducing end-to-end tests after the fact would require a lot of rewriting and probably introduce more problems than it would be worth assuming you have good partial integration tests, but there is a ton of great information in there. What makes it great is dealing with the hard-to-test pieces like a remote XMPP server and external data types from the very beginning. It's a welcome break from the tendency in some TDD and BDD writing that seems to assume the world consists entirely of business logic on objects that come from nowhere and disappear after they have had the proper calculation performed on them.

One thing that seems obvious in retrospect is their suggestion to explicitly name your nulls. We've all seen tests with such clear code as:

thingifier.broadcast(null, null, amount, null, false);

Naming the nulls can turn this into:

thingifier.broadcast(NO_ACCOUNT, NO_TRANSACTION, account, NO_NOTIFIER, NO_EMAIL);

This is more verbose once you include the definitions, which often means repetition and doing it wrong, naming those nulls makes it much easier for the next person to read the code. As a side effect, the nulls are now typed, so you can't mix them up with the parameters you care about, and there's a handy list of all collaborators, used or not, at the top of your test.

I started thinking what if we can take this a step further, because one of the problems I run into with dependencies in tests is I may not know that the Thingifier actually uses the AccountNotifier which I was passing in as null, or conversely, when I go to look at a stack trace, a stack trace pointing me to this line is still ambiguous:

notifier.updateAccountInfo(transaction.getAccount(), customers.get(customerId));

What I'm playing with is using smarter nulls, that is, an object that throws its own NullPointerException with an actual helpful stack trace when any method is called.

There are kinks that haven't been worked out. The stack trace is useful, but has some extra frames related to the imposteriser. Here's an example of the test, and the stack trace it generates:

Et voila, a stack trace that tells me which dependencies I assumed were unused that in fact were.

java.lang.NullPointerException: Called invokable#invoke
at wlth.experimental.NulleryTest$Invoker.process(NulleryTest.java:22)
at wlth.experimental.NulleryTest.shouldThrowNpeWithGoodMessage(NulleryTest.java:31)

The actual code is short, because I delegate all the tricky parts to piece of jMock.

If I'm talking about jMock, you might wonder why I don't simply mock the type and use a never expectation. It would accomplish exactly the same thing, but it's a bit misleading, because it appears as though I'm saying "this object must not be used" when in fact all I'm saying is "I don't think I need this object."

This is currently sitting in our experimental package. I'm trying it out to see whether it actually makes my tests easier to debug and more readable, and, of course, whether it's still clear when that code gets to other engineers.

Update: Jared points out that this isn't really all that much like null anymore and suggests renaming Nullery#unused to Unused#no, so with import static you'd use it like this:

Invoker invoker = new Invoker(no(Invokable.class))

This is more readable, but then I'd lose the catchy title for this post.

Sunday, April 1, 2012

Rating The April Fools' Day Shenanigans

April Fools Rankings!

Hey guys, I made a quick April Fools Ranking webapp on Google App Engine. It allows you to submit great April Fools' sites (while you aren't watching the new season of Game of Thrones) and rate them, and shows the leaderboards for best and worst pages.

The ratings are based on a simple question, how likely you are recommend this to a friend or colleague? Next week after the data's been collected, I'll do a follow up post explaining how the scoring mechanism works, and how to use it to track your customers' satisfaction with your products. Maybe we'll even talk a little code.

Good luck. The Internets can get a little weird today. Check it out.

Thursday, March 22, 2012

Scrolling the Z-axis with CSS 3D Transforms

We recently decided to make a gimmicky appeal to designers in the hopes that they’d come talk to us. We knew we wanted to do something a little flashy and cutting edge, so CSS 3D Transforms seemed like a good place to start.

Ultimately I decided I wanted a page that would allow users to scroll along the Z-axis instead of the Y-axis we’re all used to. Instead of simulating movement with scaling I wanted to use the translateZ() transform that’s supported by the latest-and-greatest browsers.

In this article we'll be recreating the z-scroll effect step-by-step. This gist has the final, complete example.

A quick refresh on CSS transforms

From the W3C CSS Transform spec:

The CSS visual formatting model describes a coordinate system within which each element is positioned. Positions and sizes in this coordinate space can be thought of as being expressed in pixels, starting in the upper left corner of the parent with positive values proceeding to the right and down.

This coordinate space can be modified with the ‘transform’ property. Using transform, elements can be translated, rotated and scaled in two dimensional space.

The 3D transform module extends 2D CSS Transforms to include the Z-axis, allowing for 3D transformations of DOM elements.

Getting Perspective

Before you can transform an element in 3D space you have to set a perspective. The element with the perspective property becomes the “viewport” for any child elements you intend to transform. In this article we’ll be using the -webkit- prefix, but you can use the prefix for browser of your choice (update: cers has forked the gist for mozilla). Here's what our viewport will look like:

 #viewport {
   -webkit-perspective:100px;
   border: 2px solid black;
   height: 400px;
   width: 400px;
   left:50%;
   margin-left:-200px;
 }
 ...

 <div id="viewport"></div>

The perspective value is the depth between you and the plane at Z=0. Let’s look at an excerpt from the spec first which describes this in more detail:

perspective(depth) specifies a perspective projection matrix. This matrix maps a viewing cube onto a pyramid whose base is infinitely far away from the viewer and whose peak represents the viewer’s position.

The viewable area is the region bounded by the four edges of the viewport (the portion of the browser window used for rendering the webpage between the viewer’s position and a point at a distance of infinity from the viewer). The depth, given as the parameter to the function, represents the distance of the Z=0 plane from the viewer.

Lower values give a more flattened pyramid and therefore a more pronounced perspective effect.

A quick sketch can help us understand what they mean. This is a 2D simplification, showing the pyramid from above:

You, the viewer, are at the peak of the pyramid. The volume of the pyramid defines your field of view.

As you can see, a lower perspective value means the rate at which things “fill up” the field of view is faster, making 3D effects more dramatic:

If the concept is still unclear, Ryan Collins has an excellent visual demonstration of the perspective pyramid in CSS 3D, and how depth effects rendering.

The Purpose of perspective-origin

Sometimes you also want to set the perspective-origin property. This sets the x and y position from which you are viewing the elements in the viewport. Again, a visual example helps us illustrate the point. Below we have the view of the perspective pyramid with an origin shifted on the x-axis:

For our purposes we’ll keep the perspective-origin at the center of the viewport.

#viewport {
  -webkit-perspective-origin:50% 50%;
...

Creating Our Layers

Now that we have our perspective set up we can start creating our layers. We’ll create 3 child div tags that we’ll plot along the Z-axis, these will be the boxes that we scroll through:

 <div id="viewport">
   <div class="box" id="a"></div>
   <div class="box" id="b"></div>
   <div class="box" id="c"></div>
 </div>

Our box class will define some basics - our divs will be 100x100, and centered within the viewport. We use position:absolute so that they’ll disregard the standard flow and overlap each other.

 .box {
   position:absolute;
   width:100px;
   height:100px;
   left:50%;
   top:50%;
   margin-left:-50px;
   margin-top:-50px;
 }

Next, our per-box styles translate each div by an offset along the Z-axis to create the layer effect. Each box is separated by -50px with “C” starting at Z=0.

 #a {
   background-color:rgba(255,0,0,.5);
   border:2px solid red;
   -webkit-transform:translateZ(-100px)

 }
 #b {
   background-color:rgba(0,255,0,.5);
   border:2px solid green;
   -webkit-transform:translateZ(-50px)
 }
 #c {
  background-color:rgba(0,0,255,.5);
  border:2px solid blue;
  -webkit-transform:translateZ(0px)
 }

You can play with the translateZ values in your browser’s inspector to see how the divs move along the Z-axis.

The Camera

You can think of our point-of-view (POV) in the browser window as the “camera” in our 3D scene. To create the effect of moving through space we won’t actually be moving our POV like you would a video camera, instead we’ll be moving the elements in our scene to create the effect that our POV is changing while we actually remain stationary.

First, we’ll set our viewport to position:fixed so we can keep it fixed during scrolling. This lets us capture the scrolling event and use it for our own purposes:

#viewport {
   position:fixed; 
...

We must also set the height of our body so we can have a certain amount of scroll room to allow us to move along the Z-axis. Since this is a toy example, let’s assume your browser window is 400px high (the height of the viewport). Now, we’ll need enough pixels past the window’s height to scroll through our three divs.

The initial distance we must cover along the Z-axis is the distance betwen us and Z=0, as you recall this is defined by our perspective value of 100px. Next we’ll need to scroll 50px between box C and box B, and another 50px between B and A. That gives us a final height of 400 + 100 + 50 + 50 = 600px.

body {height:600px;}

Now that our CSS is in order we can actually start coding. Remember, our goal is to allow the user to scroll along this z-axis, so we must appear to move along it as the user scrolls.

The first thing we’ll need to know is how far the user has scrolled. The difference between the last scroll position and the scroll position at the time a scroll event is fired represents the distance we’re going to cover along the Z-axis. We can get the delta like this:

var scrollPosition = document.body.scrollTop;
function scrollDelta() {
  var newScrollPosition = document.body.scrollTop,
      delta = newScrollPosition - scrollPosition;
  scrollPosition = document.body.scrollTop;
  return delta;
}

Next we’ll use the scroll event to know when the scroll wheel is moving and actually move each div based on the delta. Here’s our moveCamera function:

var boxPositions = [-100, -50, 0];
function moveCamera() {
  var boxes = document.getElementsByClassName("box"),
      delta = scrollDelta();
  for (var i=0,l=boxes.length;i<l;i++) {
    boxPositions[i] += delta;
    boxes[i].style["-webkit-transform"] = "translateZ("+boxPositions[i]+"px)";
  }
}
window.addEventListener("scroll", moveCamera, false);

First, we start with an array of the initial Z positions of each box, in DOM order, so that we can know their initial positions. This could also be initialized by parsing the computed style of each div to extract the translation value we’ve set in CSS.

Our first step in the actual moveCamera function is to get the boxes by their class name, as well as the current scroll delta. Next, we iterate through each of the boxes; in this loop we first increment the Z position of the box by the delta, then we set the -webkit-transform style attribute on the element to translate the element by our new Z value. This will effectively increment or decrement the Z-position of each element by the number of pixels we’ve scrolled. Finally, we attach the event handler.

As we scroll the elements will move toward the viewer along the z-axis, making it appear as if we’re moving through the scene! Necessary? Who knows. Awesome? Undoubtedly. And don’t forget — none of this should be limited to the Z-axis. CSS 3D Transforms let you specify any 3D transformation matrix. So what’s next? The DivDoom engine?

Friday, February 17, 2012

Interesting Reads

Have a great president's day weekend!


Friday, February 10, 2012

Taco Friday

Catered lunch? Too easy.

Ten foot six inch taco truck under a ten foot five inch roof? Challenge accepted.