Interesting Reads

Have a great president's day weekend!

• Gotta love R and ggplot2
  http://serialized.net/2012/02/using-r-and-ggplot2-to-contemplate-relocation/ via @jbarratt


• Wealthfront Eng is hitting the road - If you're in the Portland area and are interested in a continuous deployment tech talk, let us know!


• "What you think of oracle is even truer than you think it is" Awesome work coming out of the Illumos project.
  http://www.youtube.com/watch?feature=player_embedded&v=-zRN7XLCRhc#t=1981s


• Meetings: Where Work Goes to Die.
  http://www.codinghorror.com/blog/2012/02/meetings-where-work-goes-to-die.html
  Reminds me of this classic @ericries presentation from 2010:
  http://www.slideshare.net/wealthfront/lean-startup-for-geeks-with-eric-ries


• Interesting SSL traffic analysis. Always mind your side channels.
  http://blog.ioactive.com/2012/02/ssl-traffic-analysis-on-google-maps.html


• Food for thought - Why Concatenative Programming Matters:
  http://evincarofautumn.blogspot.com/2012/02/why-concatenative-programming-matters.html


• Registration's open for the Feb. 28 webinar on stock options and taxes by Wealthfront and Mohler, Nixon & Williams.
  https://www4.gotomeeting.com/register/560747735.


• Clojure in Python or PyPy, monads, techno and much more! http://disclojure.org/2012/02/12/intertweets via @disclojure


• The Idaho Loophole - article about a 50-square-mile swath of Idaho in which one can commit felonies with impunity.
  http://papers.ssrn.com/sol3/papers.cfm?abstract_id=691642

Taco Friday

Catered lunch? Too easy.

Ten foot six inch taco truck under a ten foot five inch roof? Challenge accepted.

Boots-n-Cats: Every Techno Song Ever In Clojure

I've wanted to do a post about programming music for quite a while, but honestly there aren't very many opportunities for an online financial advisor to do very much research into the breadth of music libraries (even for a sophisticated, yet simple one, like us). Regardless, I chose to disregard these realities when I discovered a neat little library called Overtone. Here's the technical summary for the project:

Overtone is a musical programming library written in Clojure which uses the SuperCollider audio engine and synthesis server under the covers. We're essentially marrying an awesome live-synthesis server with an insanely cool state-of-the-art lisp to create a glorious union that only the Gods can dream about.

Seriously, that's the kind of bravado that's going to propel us to rock stardom in no time (leather pants optional). But what shall we create? Well, it also happens that a friend of mine sent me this video, and I instantly thought, I can do that!



Now, let's see how.

It's really easy actually if you're on a mac, but first, you'll probably want to install Leiningen, the Clojure project tool. Follow the instructions on their page to get it set up. Then you can type:

$ lein new boots-and-cats
$ cd boots-and-cats

Then edit the project.clj file to contain:

(defproject boots-and-cats "1.0"
  :dependencies [[org.clojure/clojure "1.3.0"]
                 [overtone "0.6.0"]])

At this point, I'll let you know that you should also just clone my repo at GitHub. It already contains the samples, but you can also just go there and download them directly.

git clone git://github.com/hitch17/boots-n-cats.git

Then you can use lein to grab all the dependencies and be you on your way. If you're on linux, you'll also want to follow these instructions for getting your audio set up.

$ lein deps # Note: this downloads the entire internet.

Now let's get this thing started. You'll also want to turn your speakers to be sufficiently loud. I can guarantee that all the co-workers/family/roommates in your vicinity don't want to miss out on hearing the birth of a legend.

$ lein repl
REPL started; server listening on localhost port 48182
user=> (use 'overtone.live)
[... prints a bunch of stuff talking about your audio system ...]
          _____                 __
         / __  /_  _____  _____/ /_____  ____  ___
        / / / / | / / _ \/ ___/ __/ __ \/ __ \/ _ \
       / /_/ /| |/ /  __/ /  / /_/ /_/ / / / /  __/
       \____/ |___/\___/_/   \__/\____/_/ /_/\___/


                          Programmable Music. v0.6
[...]

You should get some sweet Ascii Art and then the fun begins. Load some samples and play them:

user=> (def boot (sample "boot.wav"))
#'user/boot
user=> (boot)
28
user=> (def cat (sample "cat.wav"))
#'user/cat
user=> (cat)
28

Let's add a little reverb to the samples in one ear and get a little bit of a panning effect.

user=> (def boot-sample (load-sample "boot.wav"))
#'user/boot-sample
user=> (defsynth reverb-boot []
  (let [dry (play-buf 1 boot-sample)
        wet (free-verb dry 0.6)]
      (out 0 [wet dry])))
#<synth: reverb-boot>
user=> (reverb-boot)
28
user=> (def cat-sample (load-sample "cat.wav"))
#'user/cat-sample
user=> (defsynth reverb-cat []
  (let [dry (play-buf 1 cat-sample)
        wet (free-verb dry 0.6)]
    (out 0 [dry wet])))
#<synth: reverb-cat>
user=> (reverb-cat)

Pretty cool and shockingly easy. Let's load our other samples.

user=> (def bees (sample "bees.wav"))
#'user/bees
user=> (def kneehigh (sample "kneehigh.wav"))
#'user/kneehigh

Let's also set up a beat with our boots and cats samples using the metronome function which helps the library keep everything in time and create a player for our beat.

user=> (def metro (metronome 60))
#'user/metro
user=> (defn player [beat]
  (at (metro beat) (reverb-boot))
  (at (metro (+ 0.5 beat)) (reverb-cat))
  (apply-at (metro (inc beat)) #'player (inc beat) []))
#'user/player
user=> (player (metro))

And when you're done, call (stop):

user=> (stop)

You can also record your masterpiece and when you're finished call (recording-stop).

user=> (recording-start "awesome.wav")
:recording-started
[... play something awesome ...]
user=> (recording-stop)

With a little practice and decent timing (copy-and-paste skills help), you should be able to create a pretty decent techno song like this one: Bootz-and-Catz. Well, okay, maybe making something really awesome takes a little more work, but hopefully by now you have a good idea of where to get started.

Now go forth and unleash your inner Techno-Viking!

Interesting Reads

It's Friday and you didn't want to work anyway. Here's some things to check out while you wait for the weekend to arrive.

• Forensic scientists know too much about the cases they investigate


• Visualize Facebook IPO prospectus - Social-networking darling Facebook yesterday filed an epic initial public offering seeking to raise $5B with the company valued between $75B to $100B. Qian did a quick visualization of what their IPO prospectus (aka "Form S-1") says.


• The Sketchbook of Susan Kare, the Artist Who Gave Computing a Human Face - Loved seeing the original sketches of those little icons that shaped my early computer experiences.


• Geeks and repetitive tasks - So true.


• Marek Potociar's slides at Devoxx on JAX-RS 2.0


• Pre-commit Tests - Keeping the trunk stable matters a lot at Wealthfront. Eyal wrote a post on running Pre-commit Tests before committing to your revision control.

Visualize Facebook's IPO prospectus

It's here. Finally. Social-networking darling Facebook yesterday filed an epic initial public offering seeking to raise $5B with the company valued between $75B to $100B. The IPO prospectus (also known as "Form S-1") filed with the SEC is a massive registration document describing the details of the offering, the company's business model and preliminary financial results. But who has the time to read the lengthy document in details? I know you don't. You probably don't want to either, especially having read tons of news articles about it already. So I did some simple text analysis in R to visualize the prospectus such that you can take a quick peek.

The following graph shows the word cloud of Facebook's IPO prospectus, i.e. the most frequent words in the document. Not surprisingly, majority of the words are related to IPO: stock, class, shares, common, prospectus, stockholders, financial, initial, public, offering, capital, equity, and so on. The other words are related to Facebook's business: mobile, advertisers, users, platform, social, data, awards and so on.


I'm actually more interested in what the prospectus tells about Facebook's business, by somehow removing most IPO-related words. I choose Facebook's archrival -- Google. Even Facebook lists Google as its "significant competitor" in the prospectus. I compare Facebook's prospectus with Google's filed back in 2004, and plot the comparison word cloud between the two in the following graph. The top part of the graph shows the words that are frequent in Facebook's prospectus but infrequent in Google's; the bottom part shows the words that are frequent in Google's but infrequent in Facebook's. This exercise (hopefully) would remove most of the IPO-reated words because they are frequent in both documents, and focus more on the two companies' businesses. Facebook-specific words are: facebook, social, users, friends, apps, mobile, platform, developers, engagement, etc. Google-specific words are: google, search, adsense, adwords, services, auction, technology, etc. Interestingly, company executives' names show up in both documents: "mark" and "zuckerberg" for Facebook; "eric" and "sergey" for Google. Maybe even more interestingly, the word "revenues" appears very prominent in Google's cloud.


What about the other social-networking site LinkedIn? This graph shows the comparison between Facebook's prospectus and LinkedIn's filed last year. LinkedIn-specific words are: solutions, professional, linkedin, hiring, talent, profile, job, marketing, subscriptions, premium, etc. LinkedIn's CEO "weiner" also shows up in the graph. The company name "linkedin" is much less prominent compared to "facebook" and "google".


If you are interested doing similar analysis, here is how.

The prospectuses are downloaded from SEC's website. Data links --
- Facebook's S-1 (2012)
- LinkedIn's S-1 (2011)
- Google's S-1 (2004)

The "tm" and "wordcloud" packages in R are used to generate the graphs. The "tm" package in R is a text mining library. I use it to build a corpus by reading the prospectuses from text files, and perform the standard drills to pre-process text data, such as removing numbers, punctuations and stop words. I use the "wordcloud" package in R to plot the word cloud graphs, both the stand-alone graph and the comparison graph. Special thanks to the contributors of "tm" and "wordcloud", which makes text analysis and visualization easy and fun. In the end I'm including some R code --

require(tm)
require(wordcloud)

# Build corpus and pre-process data
c = Corpus(DirSource("/home/qian/Work/IPO_S1"), readerControl=list(language="en")) # put the prospectuses in the directory; build a corpus by reading the files
c = tm_map(c, removeNumbers) # remove numbers

c = tm_map(c, removePunctuation) # remove punctuations

c = tm_map(c, removeWords, stopwords("en")) # remove stop words

c = tm_map(c, tolower) # to lower case

f = c[1] # facebook

g = c[2] # google

l = c[3] # linkedin




# plot the facebook word cloud

tdm = TermDocumentMatrix(f)

m = as.matrix(tdm)

freq = rowSums(m)

words = rownames(m)

wordcloud(words,freq,min.freq=100,col='blue')




# plot the facebook-google comparison word cloud

tdm = TermDocumentMatrix(c(f,g))

m = as.matrix(tdm)

comparison.cloud(m,max.words=100)