Skater Heaven

So I completely flipped out this morning before breakfast: I looked out the bedroom window after a morning conference call, and there are skaters slaloming down the road behind the house. I've got a 3' Sector9 bamboo longboard and love to get me some slalom action. But I suck. These guys were awesome.

I grabbed Sengey (my girlfriend's Tibetan mastif) and took him out for a walk up the hill. The skaters were amazingly friendly and totally mellow; they jumped at a chance to tell someone about their event. Turns out today is day one of the Seismic Nationals which is being held in my back yard!

I took my phone with me and snapped some shots of the skaters, and they are here in this flickr set. Enjoy!

We've been renting this house month-to-month while waiting to move into the house in Valentine, NE. It's a McMansion and not cozy at all, so this skating event has made the past two months here totally worth while.

Update: a sweet video of the same event last year.

Technorati Tags: skating, colorado

One Number to Rule Them All and in the Phone Book Bind Them

Thanks to Grand Central, I've got a new perma-number. This service completely rocks. I've already got my current temporary house number connected to it, and when I move in a week, I'll be updating it with that number.

Here are the features I love:

• being able to chose you number
• no more hated phone voice mail -- I can check all voice mail on line
• I get an email for every new voicemail
• I can share voice mails with friends with a link

Yeah, I have voice mail issues...

I believe registration is open now, but ping me if you need an invite. Don't for get to use this once you've got your new number.

Ubuntu 7.04 on iMac Core 2 Duo, 2G

Well, I busted me arse this afternoon and evening trying to get Ubuntu set up on my new iMac. It installed without issue (using the "alternate" x386 iso), but has the following critical issues:

1. No wireless
2. No widescreen support
3. No sound
4. Doesn't recognize CD/DVD drive after install and reboot

I'm kind of bummed, since I'd planned on doing dev work on this box (can we even call the iMacs "boxes"?) in native Ubuntu, and now I'll have to use parallels until the driver support is developed. Or until someone sends me a link that I've managed to miss despite incessant googling today...

Once I'm moved into the new place, I'll have a GB switch in the office and will be able to work in Ubuntu over copper. The resolution bugs me more than the lack of sound and DVD, but I can live with those for now. I guess.

But, hey -- it plays WoW and UT2004 in OS X like a champ :-)

Technorati Tags: apple, ubuntu, mac os x, linux

publish-bot with logging capability

Last year sometime, I received several emails from people wanting me to release the supybot IRC plugins I wrote back in 2005 for svn and RSS updates. I'd actually written for several different IRC bots (both twisted-based, and non-twisted bots) but the code was particularly messy for supybot. During one conversation, I said that I'd really done it all wrong, and the last thing I wanted to do was clean up wrong code. If I got a chance, I would do it right:

• create an IRC client for communicating to servers+channels
• create a message server running on local host
• process/parse data from the local server to the IRC client (i.e., appropriate IRC channel)

Such a flexible design would suit every need I have for writing data to an IRC channel.

Turns out radix did just so earlier this year for Canonical, as he relates in this blog post. This is exactly what I needed this weekend while migrating my old IRC bot to a new server. Well, almost all I needed. There were a couple features I wanted to add...

Not having worked with bzr or launchpad much, I asked radix about branching his code, since I heard that Canonical actually provides developers a way of doing that without leaving the launchpad environment of trunk. His instructions were quite simple, and I summarize them here for the benefit of others:

• find the project you want to branch on launchpad
• checkout trunk (if that's what you're branching)
• commit (a local action) with a message about this being a new branch
• make your changes (with any number of commits)
• push all your commits to your branch on launchpad

The steps above assume that you have bzr installed and have uploaded your SSH keys to launchpad.

Here are the bzr commands that I used:

bzr branch http://bazaar.launchpad.net/~radix/publish-bot/trunk publish-logger-bot
cd publish-logger-bot
bzr commit --unchanged -m "Created branch of radix's publish-bot."
bzr push sftp://oubiwann@bazaar.launchpad.net/~oubiwann/publish-bot/publish-logger-bot

The changes I committed:

• enabled logging
• added log-rotation
• added an HTTP service for browsing the log files

The code is available here:
https://code.launchpad.net/~oubiwann/publish-bot/publish-logger-bot

Forthcoming features:

• a search form for the log files
• a parser for bot commands, thus enabling human -> bot communications (will probably dust off my old pyparsing code for this)
• arbitrary time log rotation (currently it's by day, ad midnight)

And for dessert:

[18-Aug-2007 17:56:50] [connected at Sat Aug 18 17:56:50 2007]
[18-Aug-2007 17:57:00] [wallflower (logger bot) has joined #supersecretchannel]
[18-Aug-2007 18:00:38] <keturn> what's a wallflower?
[18-Aug-2007 18:20:09] <PenguinOfDoom> It's a flower that grows out of a wall
[18-Aug-2007 18:20:52] <keturn> and has an irc client?
[18-Aug-2007 18:22:49] <keturn> is it related to an oubiwann?
[18-Aug-2007 18:34:00] <exarkun> I reckon it is

Technorati Tags: bzr, canonical, irc, python, software, twisted, web

Export iTunes Playlists as .m3u Files

I got really pissed at iTunes today: it crashed and then wouldn't start back up, throwing the infamous 'locked file' error. In exasperation, I tried Songbird. It looks promising, but still needs lots of work and didn't play my music-over-Samba very well. I then got the latest Cog release.

In my experience, Cog is simple, straight-forward, performs no voodoo and just works (Troll: "Stay away from the voodoo!"). I really like the fact that I can browse the file system in Cog, and drag the albums into the list pane, but I didn't want to start years worth of playlists all over again...

At which point I remembered that a couple of years ago I blogged about parsing iTunes playlists with ElemetTree. It looks like the effbot archives no longer point to the link I updated that post with, but google cache served me well and I found Fredrik Lundh's code, repasted here:

try:
    from cElementTree import iterparse
except ImportError:
    from elementtree.ElementTree import iterparse
import base64, datetime, re

unmarshallers = {

    # collections
    "array": lambda x: [v.text for v in x],
    "dict": lambda x:
        dict((x[i].text, x[i+1].text) for i in range(0, len(x), 2)),
    "key": lambda x: x.text or "",

    # simple types
    "string": lambda x: x.text or "",
    "data": lambda x: base64.decodestring(x.text or ""),
    "date": lambda x:
        datetime.datetime(*map(int, re.findall("\d+", x.text))),
    "true": lambda x: True,
    "false": lambda x: False,
    "real": lambda x: float(x.text),
    "integer": lambda x: int(x.text),

}

def load(file):
    parser = iterparse(file)
    for action, elem in parser:
        unmarshal = unmarshallers.get(elem.tag)
        if unmarshal:
            data = unmarshal(elem)
            elem.clear()
            elem.text = data
        elif elem.tag != "plist":
            raise IOError("unknown plist type: %r" % elem.tag)
    return data

Which was a great start, but I needed .m3u output. After reading how simple the format was, I was off and running. The end result was all of my iTunes playlists playable by Cog, which I am using now -- as I type -- to enjoy my music, free of pain. One thing worth exploring would be how to preserve the ordering of iTunes' playlist items.

Here's the code I used to "export" the iTunes playlists as .m3u:

import re

m3uList = "#EXTM3U\n%s\n"
m3uEntry = "#EXTINF:%(length)s,"
m3uEntry += "%(artist)s - %(album)s - %(song)s\n%(filename)s\n"

def phraseUnicode2ASCII(message):
    """
    Works around the built-in function str(message) which aborts when non-ASCII
    unicode characters are given to it.

    Modified from http://mail.python.org/pipermail/python-list/2002-June/150077.html
    """
    try:
        newMsg = message.encode('ascii')
    except (UnicodeDecodeError, UnicodeEncodeError):
       chars=[]
       for uc in message:
          try:
             char = uc.encode('ascii')
             chars.append(char)
          except (UnicodeDecodeError, UnicodeEncodeError):
             pass
       newMsg = ''.join(chars)
    return newMsg.strip()

class Playlists(object):

    def __init__(self, filename=None, destDir=None):
        self.lib = None
        if filename:
            self.lib = load(filename)
        if not destDir:
            destDir = './'
        self.destDir = destDir

    def processTrack(self, trackData):
        length = trackData.get('Total Time') or 300000
        song = trackData.get('Name') or 'Unknown'
        artist = trackData.get('Artist') or 'Unknown'
        album = trackData.get('Album') or 'Unknown'
        data = {
            'filename': trackData['Location'],
            'length': int(length)  / 1000 + 1,
            'song': phraseUnicode2ASCII(song),
            'artist': phraseUnicode2ASCII(artist),
            'album': phraseUnicode2ASCII(album),
        }
        return m3uEntry % data

    def processTrackIDs(self, ids):
        output = ''
        for id in ids:
            try:
                trackData = self.lib['Tracks'][str(id)]
                output += self.processTrack(trackData)
            except KeyError:
                print "Could not find track %i; skipping ..." % id
        return output

    def cleanName(self, unclean):
        clean = re.sub('[^\w]', '_', unclean)
        clean = re.sub('_{1,}', '_', clean)
        return clean

    def exportPlaylists(self):
        for playlist in self.lib['Playlists']:
            playlistName = self.cleanName(playlist['Name'])
            try:
                items = playlist['Playlist Items']
            except KeyError:
                print "Playlist seems to be empty; skipping ..."
                continue
            trackIDs = [x['Track ID'] for x in items]
            data = m3uList % self.processTrackIDs(trackIDs)
            fh = open("%s/%s.m3u" % (self.destDir, playlistName), 'w+')
            fh.write(data)
            fh.close()

def exportPlaylists(filename, dest=None):
    pls = Playlists(filename, dest)
    pls.exportPlaylists()

With usage like the following:

>>> from iTunesExport import exportPlaylists
>>> BASE = "/Volumes/itunes/__Playlists__"
>>> exportPlaylists('%s/Library.xml' % BASE, BASE)

Update: I've tweaked the code in this post a little bit, due to a reader's questions. To run this, copy both code blocks into a single file you should be good to go.

OpenOffice, Python and Plone... and Java

The last big project I'm working on for Zenoss is an OpenOffice-to-Plone publisher. Specifically, for the Zenoss Guide (combined admin and user manuals). The Guide is maintained in OpenOffice format, but in addition to .sxw/.doc and .pdf formats, they want to publish as HTML, where each section of the doc gets its own page in the Zenoss Community portal and users have the ability to comment on each section.
There are several things I'm using to implement this:

• Zope/Plone and Five
• OOoPy (for processing sections and creating .sxw files)
• lxml.etree (for preserving the original XML namespace names)
• writer2LaTeX (for converting generated .sxw files to HTML)

Sadly, I spent a great deal of time trying to figure out why w2l (writer2LaTeX) wasn't converting sections with images; I won't relay the horrors of debugging. Finally, I tossed in the towel and emailed the author, Henrik Just. He was phenomenally helpful... and he uncovered the underlying issue, one that I consider to be a dirty little secret (though perhaps those familiar with the combinations of Zip files, python and Java consider an openly acknowledged issue). From Henrik:

A little debugging showed that the java's zip classes causes the problem. Unfortunately they are not very tolerant with variations of the structure of zip files. With java 1.4, I cannot open the file at all. This, it turns out, is due to a bug that was fixed in java 1.4.2. But even with java 1.5 I can only read 5 of the 12 files in the zip file, which causes the odd behaviour you have seen.

I don't know who to blame here: Python or Java. Regardless, doesn't this seem absurd? That the standard compression format used to create .zip files is not implemented completely either in Python, Java or both? Though not completely surprised, I was surely flabbergasted. Despite my frustration, Henrik was helpful in offering some alternatives, as well as using w2l with an option for splitting a doc by headings.

Yet another example of the phenomenal goodness that is the community of open source developers (if not the languages used in that community...). Thanks Henrik!

Technorati Tags: community, five, openoffice, plone, java, python, zope

Twisted JSON-RPC on Google Code/Projects

I've been getting lots of emails from folks asking for the Twisted JSON-RPC code that I had put up a while ago. They weren't able to find it due to my having to bring down the trac instances I was running. Good news: I've moved the repository up to Google:

http://code.google.com/p/twisted-jsonrpc/

The front page has a link to the wiki which in turn has all the usage info. You can get recent eggs here:

http://twisted-jsonrpc.googlecode.com/svn/releases/

If you want to contribute to this code base and bring it up to date, email me.

Enjoy!

Update: Primary development has moved to Launchpad. Documentation is maintained on the Ubuntu.com wiki page here.

Farewell, Zenoss

Well, python community, I am back on the market.

After a year and half of consulting for Zenoss, where I contributed to the Zenoss code base and was lead dev on the community development effort, I am on my own again.

It's been pretty amazing having a front-row seat on the growth of Zenoss as a company. When I fist started consulting for them, there were only three employees and the general view was that most of the employees would end up being remote developers (this didn't end up being the case, as they found a fair number of local python developers). I was actually on-site when they interviewed Drew "the Docs guy", who ended up being employee #4. Since then, the number of consultants and employees has increased as greater and greater focus has been placed on non-dev business like marketing, sales and customer support. It was kind of strange being there this week... while at a party, I remarked to Erik Dahl that I barely recognized the place anymore.

I do feel badly that I have poured so much time and work into the recent Zenoss community effort and not had the time to contribute to the Twisted and Divmod code bases (as well as other projects) like I had wanted to this Summer. Hopefully my next gig will provide for more support in that arena. At the very least, I could very well have some spare time in the next few months to tune back in and maybe even crank out some tickets :-)

Well, I'm off to pick up some red curry or pad thai at the Lemon Grass!

Technorati Tags: community, life, open source, python

Open Source Community Interaction: Is More Less?

I have a friend who needs our help. In fact, there are potentially disturbing ramifications in the trend mentioned below, so *we* may need our help.

Here's the deal: my friend is a research assistant at McGill University in Quebec. He is working with his professor with regard to the following topic:

"Open source software (OSS) developers who interact MORE with leaders (administrators) apparently contribute LESS (e.g., write less codes). We need more insights into this finding. Basically, we want to know why this is happening. We have speculated that open source software (OSS) communities are public, voluntary-based and self-organized communities. Therefore, if OSS leaders are too controlling, then developers don't contribute as much. This was one of our speculations. I am wondering if any OSS developers agree with this."

Sadly, very few of the developers their research team has contacted have given feedback on this matter. Below is the short list of questions they are asking developers. Note that though the example of contribution was code in the above statement, these questions assume contribution to take any of the following forms:

• source code
• documentation
• answering questions and providing assistance to other users via mail lists, IRC channels, forums, etc.
• grass-roots marketing
• other forms of community involvement that contribute positively to an open source project

Okay, so here are the questions:

1. Does the open source project's leader/lead developer have a huge impact on your contribution?
2. What reasons would make developers who interact more with core project contributors apparently contribute less to the open source project?
3. How would you feel if the leaders of the open source projects to which you contributed became too controlling?

I responded when Meral sent me his email with the two examples of Twisted and Python communities, but these guys need many more points of view than a few... and they aren't getting much traction. I feel that this research is important, and when published, could affect how open source projects and contributions to said projects are viewed by a wider population.

If this is also important to you, I encourage you to send an email to meral dot hussein at mail dot mcgill dot ca. Give it the subject of "McGill University Survey" and let 'er rip. The sooner you can do this, the better -- the deadline for the first round data is July 13th.

Technorati Tags: community, open source, surveys, python, research, software, twisted

It's Been a Long Time

To use the words of Rakim (track 4), though without their import: it's been a long time. There's so much that's going on in my non-technical life, just thinking about blogging has seized up all my joints and limbs. Until now, that is.

To summarize:

• the good folks at Zenoss keep me mercilessly busy where our community efforts are release-a-rific
• I'm no longer living in Loveland, CO, but leasing month-to-month in other parts of this great State... and wondering where home base will end up being (I'm thinking of buying my first-ever actual property)
• I've got a girlfriend for the first time in 4 years; her magical qualities defy enumeration, though I will say that she is a highly accomplished practitioner of the healing art of Jin Shin Jyutsu (studied with Mary Burmeister herself) as well as being a fellow meditator
• I'm preparing to support two people at an up-coming Lakota Sundance in South Dakota
• In addition, I'm preparing for my first shedra classes this fall (specifically, Madhyamaka)

Since I now need a nap after deciding which (of the millions of) things to recount/summarize, the rest of this post will be dedicated to game music, in particular System Shock 2 :-)

Check out this thread. In particular, scroll down to "Reply #6 on: 23. April 2007, 13:05:18" and witness the goodness:

"These are the real arrangements created by Eric Brosius himself, authored as full soundtrack songs and burned as an album. Crafted from the original audio, unlike the 22khz audio files from the shipped game, this release is CD quality."

Enjoy!

Technorati Tags: blogging, entertainment, games, life