Best Things This Year (2013)

Anecdotally, it seems like a lot of people shook up their lives in 2013. I certainly did. Here are the best things that happened to me in 2013.

1. RJMetrics – In March I started working at RJMetrics, an e-commerce data analytics firm in center city Philadelphia. Leaving Garvey Corp was a difficult decision, but being a developer at of the best SaaS data visualization companies in the world has been amazing.

RJMetrics
RJMetrics

2. The Bulldog Budget – I worked with Philadelphia City Controller candidate Brett Mandel to implement his vision for the city’s open data future. We built a visualization tool using D3 and MySQL that gives both a high level view of the General Fund budget, but still allows you to drill down to individual transactions. A lot of people got excited about it and I think it made an impact in Philadelphia. It also influenced similar projects in Italy and Oakland, California.

Treemap of the Philadelphia General Budget
Treemap of the Philadelphia General Budget

3. Coffeescript – I was skeptical at first whether Coffeescript was a worthwhile abstraction from Javascript. After 9 months of using it at RJMetrics, I’m a fan. Here’s why:

  • Cleaner syntax: No parenthesis, braces, or semi colons. The time I save writing console.log instead of console.log(); has been worth the switch.
  • Improved workflow: Continuously running the Coffeescript to Javascript compiler alerts me of stupid mistakes (ie. ones that won’t even compile) faster than finding them after I’ve loaded the browser.
  • Existential operator: I can’t count the number of bugs I’ve fixed with one character are due to Coffeescript’s great ? operator, which checks to see if it’s null or undefined before proceeding. For example, if in javascript you previously did this:

    if (player != null) {
    player.levelUp();
    }

    In Coffeescript you just write:

    player?.levelUp()

  • Comprehensions: The Coffescript.org docs say you almost never have to write a multiline for loop and they can be replaced by comprehensions. For example:

    for (player in players) {
    if (player.health < 0) { player.kill(); } }

    In Coffeescript you can write:

    player.kill() for player in players when player.health < 0
  • I'm looking forward to getting better at Coffeescript in 2014.

4. AngularJS - I don't want to develop another interactive UI without AngularJS.

5. Bought this swingset from craigslist - With the help of my friend Mike and my father in law, we disassembled, packed it up and a U Haul, and reassembled it in my back yard. I'm amazed it went back together so well.

swingset
swingset

6. Read 13 Books - My morning commute afforded me more reading time. Here's what I did with it.

  • Bonfire of the Vanities by Tom Wolfe
  • Ready Player One by Ernest Cline
  • Look at the Birdie by Kurt Vonnegut
  • The Trial by Franz Kafka
  • A Beautiful Mind by Sylvia Nassar
  • Boys from Brazil by Ira Levin
  • Game of Thones (books 1-3) by George RR Martin
  • Life of Pi by Yann Martel
  • Timequake by Kurt Vonnegut
  • How to Win Friends and Influence People by Dale Carnegie
  • Thinking Fast and Slow by Daniel Kahneman

7. Public Speaking - I got way out of my comfort zone this year and did some public speaking at Ignite Philly and Technically Philly's Civic Hacking Demo Night.

8. Built the Gonginator

9. Spark Program - Some coworkers and I participated in an apprenticeship program for Philadelphia school kids where we spent 2 hours a week with 8th graders interested in programming and computers. Together we built a game!

That's as much as I could remember from 2013. Check out my lists from 2012 and 2011.

My Ignite Philly 11 Presentation on Data Visualization

Update: Here’s the video of my presentation

Last Thursday I gave a talk on Data Visualization at Ignite Philly 11. I was nervous as hell, but the encouragement you get from that crowd is amazing. The organizers (David, Geoff and Adam) did a great job and it could not have gone smoother.

Me speaking at Ignite Philly 11
Photo by Kara LaFleur

Here are my slides with some additional commentary that doesn’t fit into 15 seconds.

First, a few fun facts about my talk:

  1. I never tested it in Powerpoint. I meant to, but I didn’t.  I just wrote it in Keynote, exported it, and prayed it would look ok.
  2. The next day, James Miller complained on twitter how much he would have liked to have been there and didn’t realize there was a woman there chanting his name.
  3. I tried to put on one of Brett Mandel’s campaign tattoos before the presentation, but it wouldn’t stick, so I just used one of his giant stickers instead.

Why data visualization works and how it can save the world
I tried to go for big problems in the talk. Earlier versions included a number of interesting sports visualizations, but in the end I felt they detracted from my overall thesis, but I snuck one into my opening slide which shows NL East games above 0.500 for 2007. See how far the Phillies were out of the race at the end of the season but ended up tied for first on the last day.   They beat the Nationals that day 6-1 and the Mets fell to 2nd place.  I use a version of this on my example page for evidensity.js.

primitive societies don't have math
The info about pre-math societies came from the excellent book Here’s Coming at Euclid by Alex Bellos. In chapter 0 he talks about how primitive societies make decisions without mathematics and why we eventually needed math. This slide also gave me a chance to post a picture of Phil Hartman’s Unfrozen Caveman Lawyer. RIP.

ignite-philly-slides.003 Making snap, life and death decisions based on ratiosThis was my least favorite slide and of course, it’s the one shown in most of the pics I’ve seen of my talk. It’s just a tree with fruit and a lego guy about to get jumped. I used it to represent why we need the ability to make quick decisions in the wild and I struggled with what to use here.

ignite-philly-slides.004 the brain eye system
The 10-20Mbs figure comes from Ed Tufte and a UPenn study. Originally my talked was called, “How Data Visualization Works” and then I realized I didn’t have any idea of HOW it actually works. I only know WHY, because we’ve evolved over millions of years to make life and death decisions based on the ratios of visualized objects. So I changed it.

ignite-philly-slides.005
The numbers on this page represent the area in square pixels on the next slide.  I used it to show how much faster you can pick out the largest and smallest values when they are visualized as shapes.

ignite-philly-slides.006I missed out on a good eye-chart sobriety test joke here.

ignite-philly-slides.007
Hopefully at this point I’ve convinced you of the plausibility of why dataviz works from an evolutionary standpoint, and that we’ve been given this great gift of receiving and processing visual data. This slide is call to action. It says, “You have a superpower and you don’t even realize it. Let’s use it for good.”

ignite-philly-slides.008
My whole talk was a just a way to show this slide to anyone who never heard of Minard’s masterpiece, Napoleon’s March. Seeing it inspired me to think more about what data visualization could be.

ignite-philly-slides.009
I needed more than 15 seconds so I took Pam Selle’s advice and doubled up on the slide.

ignite-philly-slides.010
I probably should have disclosed that I’m working for RJMetrics now, but I only had 15 seconds to get through each slide! In fact, I actually have to start talking about this slide about 3 seconds before it shows up in order to make it through. It’s a picture of our awesome new UI we’ve been working on which should be out soon.

ignite-philly-slides.011
This is a screenshot from my macbook air’s terminal window. It only shows 0.0026% of the data in the Philadelphia General Fund Budget.

ignite-philly-slides.012
This is an exploded view of Mayor Nutter’s budget summary. I think he deserves a lot of credit (along with Mark Headd) for opening up the city’s data, but I wonder how many people read this summary.

ignite-philly-slides.013
The infamous Bulldog Budget, powered by d3.js and a bit of Ruby and mysql. If you’re an advocate for open government and open data, you should really consider voting for Brett Mandel in the Democratic primary in May. Read what Technically Philly wrote about the Bulldog Budget in January.

ignite-philly-slides.014
“Yeah James Miller!” I loved this map from the 2000 census and had no idea until recently it was by James Miller, brother of my friend and journalist Jen Miller. I think the woman chanting his name was just swept away by the enthusiasm of the night, but who knows?

ignite-philly-slides.015
On August 14th, I stopped by Azavea to hear some guy named Mark Headd give a talk on open data. The same night, Casey Thomas, now at Axis Philly gave a presentation on an app that tracked voting records and lobbying expenses together. But it was Tamar Manik-Perlman’s map showing the impacts of Pennsylvania’s voter ID laws on poor Philadelphia neighborhoods that blew my mind. Here was the perfect case for how data visualization can help illuminate a problem. People like me and most of you reading this live in a world where getting an ID is not a problem. It’s difficult to imagine life being any other way. You can concede that maybe there would be some people affected, but seeing on a map that in some neighborhoods over 60% of voters could be impacted was shocking. I don’t know how much of an influence it had on the eventual delaying of the law, but it had a huge one on me.

ignite-philly-slides.016
I rehearsed this slide many times and just could not fit it into 15 seconds, so it got double duty as well.

ignite-philly-slides.017
I originally had this image here, but I had to purge the sports visualizations. This slope graph is literally sickening. We spend twice as much for middle-of-the-pack results. The only downside was that I’m sure it was impossible to read on stage at Johnny Brenda’s. If you like it, check out this great article on slopegraphs.

ignite-philly-slides.018
I gave my conclusion over the next two slides and didn’t mention the content at all.

Seeing pictures of a devastated city make us more likely to help others and my conclusion was that data visualization taps into those same parts of the brain that motivate us. They create the will and courage to act. This is a tool we all posses no matter where you come from or what your education level is.

ignite-philly-slides.019
Admittedly, the last two charts could have looked a little better and I wish I had more time to either get better ones or recreate my own with the data, but I ran out of time.

It felt more powerful to just talk and let them speak for themselves and I think that was the right call.

I had an absolute blast and loved doing IgnitePhilly 11. The other speakers were fantastic so check out Technically Philly’s recap.

Media Roundup for Mandel Bulldog Budget

The reaction to the Mandel for Controller Bulldog Budget has been amazing! Here’s a list of all the places it was mentioned online.

Technically Philly – “Use the coolest, most comprehensive City of Philadelphia budget visualization you’ve ever seen”

Made it to story #4 on Hacker News with over 60 comments

Tons of activity on twitter

Reddit Discussion

Flying Kite Media

Axis Philly video tutorial

PhillyMag.com

Newsworks

Philly.com

City Paper

And a big thanks to Indyhall for hosting the launch event!

Philadelphia Budget Data Visualized

For the past few months I’ve been working with Brett Mandel on creating a data visualization of Philadelphia’s General budget. We used a treepmap from D3 to show expenditures categorized by department, but lets you drill down to al 217,450 expenditures.

 

Treemap of the Philadelphia General Budget
Treemap of the Philadelphia General Budget

 

We’re launching it today, so check it out and see what you can find. If you see anything interesting or find any errors, email me.

Best Things This Year (2012)

Last year I did a loosely types list of all the best things from 2011, and here’s my list for 2012. Not all are new (even to me), but they stood out in 2012.

11. Solved a Family Mystery

10. Travel – Saw some great places this year including the Alamo, Niagara Falls, a Phillies spring training game in Clearwater, Spokane, Longwood Gardens, and more.

Sasha looking out over Niagara Falls
Sasha looking out over Niagara Falls

9. Reddit.com – I’ve known about reddit for a while, but I’ve grew to appreciate it this year as a general resource for anything. It’s like an improved version of usenet.

8. Found my sunglasses after losing them for months!

sunglasses
Found my sunglasses

7. Highest Trafficked Blog PostThis post about the 8-bit Google Maps got 82,000 page views.

6. Dark Sky (iOS App) – Killer weather app for iOS that alerts you when it’s about to start raining and tells you when it will stop.

5. Nate Silver – I saw James Carville along with Dennis Miller give a talk last fall and Carville said this is the year that we’ll see who is better at predicting close elections, traditional polling or the geeks. Nate Silver of 538 predicted just about every election correctly and solidified the position of the newer statistical models.

4. Ruby – Perl was one of my first programming loves, but I’ve used Ruby for 3 or 4 projects this year and have become a convert.

3. AndyD – Hey Tina is my new favorite song. I saw these guys randomly in Jacksonville, FL and they are exactly as the appear in the video.

2. D3.js – I’ve pretty much put on hold my javascript data visualization library, evidensity.js, once I found this amazing library. Mike Bostock does some of the best data visualizations for the New York Times and he open sourced the tools he uses to create them. Amazing stuff.

1. Minecraft – I’ve been meaning to try out Minecraft since it was released as an independent project in 2009, but never got around to it until now. It’s the most innovative game I’ve seen in 20 years and Sasha and I are hooked. It’s a sim / dungeon crawler / FPS shooter / adventure game with retro graphics and high tech terrain generation. The thing that impressed me the most was that if you ventured out to the limits of each generated world it would be about 9.3 million times the surface area of Earth!

Minecraft!
Minecraft!