retroactive quality metrics with git

The Background

For a recent project retrospective we wanted to chart some metrics over the course of the entire project. Things like number of unit tests, test coverage, how long builds took to run, number of failing tests, etc. Taken in isolation these metrics aren't incredibly exciting, but when you plot them over time and hold that against other metrics like team morale, story point velocity, open defects, etc then often some interesting correlations can emerge.

The Challenge

So, we wanted metrics for internal quality, but we actually hadn't done the best job at collecting those metrics, particularly at the start of the project. So under pressure to get ready for our project retro we decided that we'd have to pass on graphing the internal quality metrics.

After the retrospective - which was a very valuable exercise regardless - I decided to figure out a way to capture this kind of data retroactively. I reasoned that if we had a script which generated a metric for the codebase at an instance in time then we can easily leverage The Awesome Power Of Git™ to collect that same metric over an entire set of commits.

Find someone else who's solved 80% of your problem

I came across a script in Gary Bernhardt's dotfiles repo on github called run-command-on-git-revisions. Sounds promising, doesn't it? This script will take a given range of git commits and run whatever command you provide on each commit in the range one by one. Just what I needed.

Solve the remaining 20%

I just needed a few additional tweaks. First I created a little 5 line ruby script which ran some test coverage metrics on our codebase, outputting a single line of comma-separated data. Then I created a modified version of Gary's script. The modified script only outputted raw CSV data, and tacked on the timestamp of each git commit as it processed it. Once that was done all I had to do was kick off the script with the appropriate range of commits, asking it to run my metric reporter script on each commit. I took the output and pulled it into graphing software and now have a pretty graph of how our test code has trended over time.

Inspect the state of your running iOS app's UI with Symbiote

What's Symbiote?

Frank comes with a useful little tool called Symbiote. It's a little web app which is embedded inside your native iOS application. Its purpose is to let you inspect the current state of your app's UI, and to test the UIQuery selectors which Frank uses to help it automate your app. Essentially Symbiote is Firebug for your native iOS app.

Recently I've added some improvements to Symbiote, adding some new features and making it easier to use. This screencast demonstrates most of the new-and-improved Symbiote.

Main features

View hierarchy

Symbiote provides you with a tree describing the current state of the view hierarchy of your running app. This gives you an overview of the general structure of your app's UI, and also helps you write UIQuery selectors which drill down from high level UI features to individual UI elements. For example, you might want to find a specific UITableView, and from there drill down to a specific row within that table view.

View properties

After selecting a specific view in the view hierarchy you can see details of that view in a properties tab. Inside this tab is a table describing all the properties exposed by the specific view.

UI Snapshot, with view locator

The View Locator tab shows a snapshot of the current app UI. As you mouse over views in the view hierarchy that view will be highlighted in the view locator, making it easy to visualize which parts of the view tree map to which parts of the app's UI.

Selector testing

Frank uses UIQuery selectors during testing to select a specific view or set of views. This is very similar to how Selenium uses XPath or CSS selectors. In order to test these, Symbiote allows you to provide free-form UIQuery selectors and then have Frank flash any matching views in your app's live UI. This makes it a lot easier to test selectors when writing your automated tests.

Accessible elements

Frank makes use of accessibility labels to make it easier for it to find elements in the UI. To that end, Symbiote provides a list of all the views in your current UI which have accessibility labels. You can click on these elements and see the corresponding UI element flash in your app's running UI, using the same mechanism as the free-from selector input field.

Cool, where do I get it?

That's easy. Just get your app Frankified, and you're good to go!

Gracias!

We wouldn't have what we have today without the original UI cleanup of Symbiote by Cory Smith, and the UX and CSS help of Mike Long.