trac Stats Gathering with Storm

One of the things we've wanted to do recently at Divmod is track development status of milestones as well as gain some perspective on repository history/trends in activity. The first thing I did? Looked at SQL for about 2 and a half minutes. The second thing I did? Fired up a python instance and imported storm.locals :-)

With a few minutes of typing and looking stuff up (e.g., how to define compound keys for an already extant schema/db), I was up and running and was able to concentrate fully on the problem at hand (reports) and how to represent data visually (matplotlib). Now that's how an ORM is supposed to work :-)

Here are the schemas I defined in python:

class Revision(object):
   """
   A storm ORM object representing a row in the trac 'revsion' SQLite table.
   """
   __storm_table__ = 'revision'
   rev = Unicode(primary=True)
   time = Int()
   author = Unicode()
   message = Unicode()

   def getDateTime(self):
       try:
           self.dt
       except AttributeError:
           self.dt = convertEpoch2DT(self.time)
       return self.dt

class NodeChange(object):
   """
   A storm ORM object representing a row in the trac 'node_change' SQLite
   table.
   """
   __storm_table__ = 'node_change'
   __storm_primary__ = ('rev', 'path', 'change_type')
   rev = Unicode()
   path = Unicode()
   node_type = Unicode()
   change_type = Unicode()
   base_path = Unicode()
   base_rev = Unicode()

Revision.nodeChanges = ReferenceSet(Revision.rev, NodeChange.rev)

class TicketChange(object):
   """
   A storm ORM object representing a row in the trac 'ticket_change' SQLite
   table.
   """
   __storm_table__ = 'ticket_change'
   __storm_primary__ = ('ticket', 'time', 'field')
   ticket = Unicode()
   time = Int()
   author = Unicode()
   field = Unicode()
   oldvalue = Unicode()
   newvalue = Unicode()

Note the 1-to-m relationship of revision to nodes changed -- so easy :-)

This is exactly what a (good) tool is supposed to let you do: focus on and solve the larger problem at hand, not get lost fixing tools. With the schemas defined, the queries took literal seconds to write, I had my data, and was able to start generating textual summaries as well as processing the queries for graphs.

Wanna see some graphs? Below is a slide show of some graphs (just a few of the hundreds that are generated) representing the work of the Divmod dev team over various points in time on a secondary trac instance (not our main, Divmod trac instance). Though I haven't included them in the slide show, the code also generates graphs per-user, per-time-period.


Created with Admarket's flickrSLiDR.

Update: Thanks Nafai for poking me about the oversight of not giving shouts out for the graphing tool: matplotlib (of which I have been a huge fan of since I first heard about it years ago, having suffered through proprietary and kludgey Matlab as a physics student).

Technorati Tags: howto, matplotlib, python, software, storm, work

rst2a

I've been meaning to blog about this for a couple weeks: rst2a. I think this is a hot little site for the following reasons, in order of what my brain experiences when visiting the site:

• Visually, this is a very "clean" (and good looking) application; in a world of over-featured web junk, this is most refreshing.
• Similarly, the functionality of the application is represented excellently by the clear design and layout: I always know where to go and what button to click.
• I use ReStructured Text all the time; any notes I take for everything, I write using rst (and I have already used rst2a a great deal).
• This applications is most useful: it can output my rst text to styled HTML or PDFs, it handles unicode well, and new styles can be defined and uploaded.

Finding a nice web application that does one thing useful and doesn't try to be everything to everyone, looks good, and actually freakin' works feels like a one in a million occurrence. Having a web tool that converts your rst may not seem very "big" or sexy, but try it -- I think you'll find it surprisingly useful. And the way that they have designed this thing, you will enjoy the actual usage experience in addition to the result (or at the very least, won't have the pain associated with poorly- or over-designed apps).

Now, to encourage them to commit a crime which I have just decried, perhaps they could a feature I would use: slide show output. This would increase the output formats from two to three: HTML, HTML Slideshow, and PDF. That's not too bad, is it?

Technorati Tags: web, python, software

Twisted Horse

I've loved riding horses since I was a kid. I have no riding skillz, but I've been on horses (irregularly, and sometimes with large gaps of time in between) since I was a little boy. But *damn* it's amazing riding a horse.

So imagine my joy when

1. my girlfriend tells me she wants to rescue two trail horses she's known for a couple years and is able to get a really good deal on;
2. they are larger horses (quarter horses), so they will fit me; and
3. the larger of the two is named "Twist".

Well, they arrived today, brought up from Colorado by our friend Lindsey. They were treated poorly by someone (riders and/or handlers) this summer, and so are in pretty rough shape, but Marjorie has already started working on them and is putting their recovery plan into effect immediately. As soon as they get fattened and happied back up, I'll be learning to ride bareback! Woo-hoo! Or perhaps "yee-haw," in the parlance of our locale.

I've got some initial pics up here, taken after we got them off the trailer and into the arena. Twist is the lighter colored, larger horse. I can tell you what I'll be doing on weekends, now! When I'm not traveling to Denver. Or hacking python. Or...

But, man! I can't believe that I've got a horse named Twist, of all things! How sweet is that synchronicity?

Technorati Tags: personal, twisted

Chicago

I was in Chicago very briefly this weekend for the Chicago Marathon; several of my friends from the 311th MI Bn (101st Airborne Division) were running in the event and others of us from the same unit were there supporting them. For us, the whole event was dedicated to our dear friend Todd Drobnick who was killed in Iraq several years ago. It was really great seeing some old friends, savoring some memories, making some new ones and even planning the next get-together. I put some pics up here.

For the weekend, we ended up renting this great little place (two places in fact: 2nd and 3rd stories) that was right on Clark Ave. at mile marker 9: a perfect location. I also ended up bringing my dog with me, and as a result, did lots of walking in and around the area.

I am not a city person; I generally am fairly averse to them. Cities with character, however, I find most appealing. Boston, San Francisco and New Orleans come at the top of that list. But I tell ya, I gotta admit that Chicago rocketed through the ranks this last weekend. I've been there before, but only downtown. Staying for a day and a half in Chicago's Mid-North District made me want to come back right away! Here are some pics I took (with my cell phone; not the best light control) of some of the places that caught my eye.

Sadly, PyCon 2008 won't be held in Mid-North. However, 1) I look forward to exploring the area near the conference, and 2) Mid-North is only going to be about 13 miles from the hotel, so I can always talk some of you into catching the 'L' with me and riding over for dinner and a walk :-)

Technorati Tags: personal, pycon