behind the gist

thoughts on code, analysis and craft

Fun with pandas and bikes

I’ve been slowly migrating from R to Python for new data analysis tasks. In particular, learning pandas and ŷhat’s port of ggplot to Python. I’m coming a little late to the party, but the Divvy Data Challenge completed recently to award prizes for visually compelling analysis of anonymized data from Chicago’s new bike share service. New York’s new bike share program has been in the news recently, highlighting ‘financial and operational challenges’ and implying the problem stems from low purchases of daily pass sales. The chart the Journal uses to support that claim is almost a textbook case of making the chart fit the story you want to tell rather than making a chart reflect the data. They plot weekly pass purchases vs. weekly cumulative membership numbers to make an apples-to-oranges comparison instead of something more sensible.

The story got me thinking about what real operational differences might exist between New York and Chicago. Digging into that was interesting in its own right, and it helped me better understand pandas compared to R.

Goodbye, Motorola

Driving in to work, I remembered thinking we’re going to launch today. I had thought the same thing the day before as well, but that was scrubbed. When I got to the support room, I did a quick hand-off with the people on the previous shift, but I noticed something wrong after they left. I had set things up to monitor all the important telemetry, but someone had changed it while I was off shift. I yelled, likely using a swear word or two, and ran off to get the guys. It’s strange, but that outburst is how I started working at Motorola.

I had been on-site for a few months to prepare for launch and had seen they were looking for an Orbit Services tech lead. I had seen first-hand what hell the orbit analysts had been going through with the existing system and was really excited about the possibility of helping them. But I found out from a senior manager who interviewed me that it was my outburst that day that really convinced him to hire me. I’m pretty even-keeled, not prone to outbursts, but he liked to see that I could get riled up.

So that is how I started my career with Motorola. Now, after over 16 years, I’m moving on.

Simulating point clouds

I’m working on a project to monitor trailer loading using 3D sensors. The data is really interesting and somewhat challenging. PrimeSense has some amazing technology for low-cost depth sensing. But at large distances (and possibly because of reflections and our unique geometry), quantization and other errors give examples like below. This image is looking head-on at a flat wall, but the point cloud is anything but flat. The points are spread out over a few feet in sheets perpendicular to the camera sensing axis.

To make sure we’re not developing algorithms “tuned” to this specific error model, I wanted to generate some “truth” point clouds as references. I ran across a couple different ways to do this, including ROS Gazebo plugins. But they all required building a world model which seemed quite a bit of effort for my simple case. So I spent some time to make a simulator that lets me generate point clouds during an idealized loading process. It runs so fast that I can generate a separate point cloud for each loaded package. A whole load profile consists of hundreds of point clouds, but only takes a few seconds to generate.

Introductory Java tutorial

My son took an introductory object-oriented programming class in high school this last semester. He kept having to ask the teacher for other project ideas because he was finishing the assignments so quickly. At the time, they were doing simple graphical stuff (like making bouncing balls) and some Project Euler kind of stuff.

I was trying to find some examples that would be a little more like real programming work, in particular, something that was more project-based and test-driven. In the end, I built my own tutorial for him to try.

What excites you about technology today?

I interview people for permanent or intern positions in our research labs and CTO office and one of the things I always ask is what excites them about technology today. If the position is in a particular area (e.g. computer vision) I’ll ask specifically about that area and adjacent areas (e.g. machine learning).

It astounds me how many people can’t answer the question. There are no right or wrong answers, I’m not saying they come up with things that aren’t actually exciting. I’m saying they don’t express an opinion and they don’t convey real enthusiasm for the examples they come up with. In many cases, we’re talking about PhD candidates who should have some knowledge of what is cutting edge right now and what possibilities exist in the near future.

I don’t intend it to be a challenging question. I ask it to set up a foundation for a broader discussion about our customers and the problems we need to solve for them. But I’ve found it to be very effective at uncovering how thoughtful, motivated and inquisitive they are.