Tuesday, July 03, 2007

It's Been a Long Time

To use the words of Rakim (track 4), though without their import: it's been a long time. There's so much that's going on in my non-technical life, just thinking about blogging has seized up all my joints and limbs. Until now, that is.

To summarize:
  • the good folks at Zenoss keep me mercilessly busy where our community efforts are release-a-rific
  • I'm no longer living in Loveland, CO, but leasing month-to-month in other parts of this great State... and wondering where home base will end up being (I'm thinking of buying my first-ever actual property)
  • I've got a girlfriend for the first time in 4 years; her magical qualities defy enumeration, though I will say that she is a highly accomplished practitioner of the healing art of Jin Shin Jyutsu (studied with Mary Burmeister herself) as well as being a fellow meditator
  • I'm preparing to support two people at an up-coming Lakota Sundance in South Dakota
  • In addition, I'm preparing for my first shedra classes this fall (specifically, Madhyamaka)
Since I now need a nap after deciding which (of the millions of) things to recount/summarize, the rest of this post will be dedicated to game music, in particular System Shock 2 :-)

Check out this thread. In particular, scroll down to "Reply #6 on: 23. April 2007, 13:05:18" and witness the goodness:

"These are the real arrangements created by Eric Brosius himself, authored as full soundtrack songs and burned as an album. Crafted from the original audio, unlike the 22khz audio files from the shipped game, this release is CD quality."
Enjoy!

Technorati Tags: , , ,

Sunday, March 25, 2007

Genshi Templates in Nevow

Update: When done reading through this example and code, please see the most recent blog post about this that addresses the dynamic rendering of Genshi templates in Nevow.

I have been exploring the extensibility of Nevow, and I'm finding it surprisingly easy to integrate other tools. I've only used Genshi for recent Trac hacking, but I hear more and more about it and it seems to be gaining a little momentum as one of the preferred template systems for python web application developers.

Using the code from the blog post Object Publishing with Nevow, I have created an example of how to integrate Genshi with Nevow. I will provide a link to the source code at the end of the post.

Integrating another templating system into Nevow is actually a no-brainer: you just need to have the "alien" template parsed by its own parsers prior to Nevow performing its parsing. This impacts only one module: nevow.loaders.

Before we dig any deeper, how about some eye-candy? Then a quick review, and finally the code. Here are some screenshots of the Genshi templates running along-side native Nevow templates:


434358390 65710Bc3A0 O
434358446 Df7816377A O

The second image is the page in the little demo that showcases all the functionality of Genshi as documented on this page.

Traditionally with Nevow, Page classes contain data_*() and render_*() methods. These Page classes are "web resources" whose locateChild() method is called after an HTTP GET. Each page class that will have an HTML view needs to set a docFactory attribute; this is the loader instance I referred to above.

Here is a toy example of a page class from a previous blog post:
from nevow import rend
from nevow import loaders

class SiteRoot(rend.Page):

docFactory = loaders.xmlstr('''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"
xmlns:nevow="http://nevow.com/ns/nevow/0.1">
</html>''')
What we need is to be able to do something link this:
from nevow import rend

import myloaders

class GenshiStuffResource(rend.Page):

docFactory = myloaders.genshistr('''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"
xmlns:nevow="http://nevow.com/ns/nevow/0.1"
xmlns:genshi="http://genshi.edgewall.org/">
<h1>Genshi Test</h1>
<div>
<span genshi:with="y=7; x=y**2; z=x+10">$x $y $z</span>
</div>
<div>
<span>Your name is ${username}</span>
</div>
</html>''')
To make this work, all we have to do is create myloaders.py with the following content:

from genshi.template import MarkupTemplate

from nevow import inevow
from nevow.loaders import xmlstr

class genshistr(xmlstr):

def load(self, ctx=None, preprocessors=()):
attrs = inevow.IRenderer(ctx).__dict__
tmpl = MarkupTemplate(self.template)
xmlStr = tmpl.generate(**attrs).render('xhtml')
self.template = xmlStr
return super(genshistr, self).load(ctx=ctx,
preprocessors=preprocessors)
There is a minimal amount of magic happening here: MarkupTemplate.generate() takes keywords (or a dict, with extended call syntax) and those key/value pairs are available to the template that is rendered. We are using our class's __dict__ for this; so if you want data in your template, you need to make it a class attribute. There are many ways to do this, and you can choose the one that is safest, cleanest, and least consumptive of resources for your application.

This is an example using a string as a template. The source code link below includes a folder where the templates are broken out into files and with a myloaders.genshifile class defined. The screenshots above were taken from a running instance of that example.

Browse the example source code here: nevow/templating/genshi.

Technorati Tags: , , , ,

Wednesday, March 21, 2007

Front Range Pythoneers Meeting

Sean Reifschneider of tummy.com (Twisted and PyCon sponsor) started up Front Range Pythoneers here in Colorado about a million years ago. Last year at PyCon 2006 Jim Baker talked about breathing life back into it... and he has. Today was not only proof of this (and my first meeting) but yet more validation of the local python community: enthusiastic, motivated and highly intelligent . Sean presented his PyCon 2007 Vim+python talk and Matt Boersma gave his wxPython and XRC talk.

Afterwards, I talked with Fernando Perez about IPython and its distributed computing branch, lead by Brian Granger and Benjamin Kelly (yet another physics guy/python coder). They are doing interesting things with Twisted and IPython, stuff that was brought up at the Twisted BoFs at PyCon this year (and in which radix and I took an interest). I look forward to attending the sprint in April and looking over Brian's shoulder, getting up to speed on how they are using Twisted and what they want to be doing (possibly exploring the use of twisted.conch.insults in IPython).

Technorati Tags: , , , ,

Wednesday, March 14, 2007

Zope 3 Surprise

I haven't blogged about Zope 3 in a while. Haven't had much of a chance to use it, really. I have had a bunch of people contact me recently about z3 work that I've done in the past, and it's been kind of nice refreshing my memory.

More importantly, though, one of my oldest clients asked for a site update and new set of features for a custom-built z3 CMS solution I started work on about 2 years ago. It only had one update in the past: from Zope 3X to Zope 3.1.0. Tonight, a year and a half later, I performed the second upgrade, and I'll be damned if it didn't take more than 30 minutes to "migrate" form 3.1 to 3.3.1. I use the quotes because the codebase actually ran without any problems on 3.3.1; all I did was make changes to a few ZCML files and update some import statements in order to remove the deprecation warnings. Unbelievable.

This really needs to be emphasized: the upgrade of Zope 3 was freaking amazingly painless. There aren't too many projects that can make that kind of boast. In fact, I know only one: Twisted. When I first started at Zenoss, I wrote some code for them that required deferredSemaphore, but sadly they were still running on Twisted 1.3. Eric Newton upgraded to Twisted 2.4, with only a few minor fixes required. I don't think it took him longer than an hour or two.

Projects that can do this are definitely doing something right. There's a good chance that their software development practices have a lot to teach other open source projects. And hell, closed source projects, for that matter.

Technorati Tags: , ,

Monday, March 12, 2007

Bouncing Balls

As mentioned earlier, there was a surprise Twisted contingent at PyCon this year, and I had the opportunity to hang out with both those of the gang that I'd already met and done some hanging with, as well as those that I had not. Brian Warner was one that I hadn't had a chance to chat with much until PyCon, and it was amazing. I plan on interviewing him in the future regarding Foolscap, his thoughts on E and python, as well as the innovative business he's working for right now.

But what the hell, you ask, does this have to do with "Bouncing Balls"? Well, in addition to going (or not going) to talks, attending BoFs, engaging in extensive business networking, and eating/drinking copious quantities food/drink stuffs, we also had time for good old-fashioned geek fun: watching movies. I do use the term loosely, though, indicating essentially anything *.mov, *.avi, *.wmv, etc. Brian shared with a whole gaggle of us the joys of The IT Crowd (all six episodes), Stalking Santa (preview), George Lucas in Love (which I had never seen), and the apparently famous (and rightfully so) Bravia commercials. The paint "fireworks" advert filmed in Scotland was just brilliant and hilarious (be sure to watch the making of), but I left my heart in a superball, bouncing down the streets of San Francisco. See if the same doesn't happen to you when you watch the Bouncing Balls.

That video simply mesmerizes me. I've watched it tens of times and it still makes me feel like a joyful little kid. Just amazing. Side by side with the insanely awesome cinematography and the astounding colors, the music makes this a stunning classic. The song has been haunting me since PyCon and last night I finally had a chance to sit down and search for the artist. I didn't have to search long, though, as the Bravia and last.fm pages share all. Turns out the song is "Heartbeats" from José González's album Veneer, a cover from eccentric Swedish band The Knife. After reading more about Veneer, I bought it immediately. I highly recommend it.

And, no, I'd never see this ad before; I don't have TV service. I am of the "pull" school for entertainment, versus "push" (or "always on").

Technorati Tags: , , ,

Friday, March 09, 2007

Django-esque URL Resolution in Nevow

So, from the previous post on object publishing in Nevow, we had the following resources:
  • http://localhost:8080/
  • http://localhost:8080/mystuff
  • http://localhost:8080/yourstuff
We added child_* attributes and methods to the "root" class, thus defining the "mystuff" and "yourstuff" resources. But what if we want to do it like Django does it? In other words, how do we map paths in the URL to resources via regular expressions?

It's actually fairly straight-forward. All we need to do is the following:
  1. Create a tuple of patterns.
  2. Override the locateChild() method in our "root" class.
Working with and extending the examples from the previous post, we might create a patterns tuple like this:

urlPatterns = (
(r'/mystuff(.*)', MyStuffResource),
(r'/yourstuff/blog(.*)', BlogResource),
(r'/yourstuff(.*)', YourStuffResource),
)
Then, in SiteRoot, we could do something like the following:

    def locateChild(self, context, segments):
path = '/'.join(('',) + segments)
for regex, resource in urlPatterns:
match = re.match(regex, path)
if match:
newPath = match.groups()[0]
r = resource()
if newPath in ['', '/']:
return r, ()
else:
newSegments = newPath.split('/')[1:]
return r.locateChild(context, newSegments)
return super(SiteRoot, self).locateChild(context, segments)
What we're doing here is interrupting the "natural" flow of Nevow's path processing. If there are more segments once we've found a match, we pass the additional segments on to the child resource's locateChild() method. If not, we have a final destination and can return the resource itself.

Here are some screenshots of this in action:


415352821 6C5Cd31031 O
415352839 64Bee68A6A O
415352871 267C96D7A0 O
415352893 73D1095E30 O

You can browse the code for this at the following links:

Technorati Tags: , , , ,

Friday, March 02, 2007

Python Will Rule the World

Before I tell you why, how about a story first?

When I was a kid, my parents bought a KayPro II so that they could use the spreadsheet program for financials and my dad could write a medical text book. I, of course, discovered the BSD games (BASIC clones) that came with it almost immediately. All of my friends had TSR-80s and C-64s, and it seemed that we all discovered BASIC and the hackability of games simultaneously. We would gather around the school computer and take turns making changes to the available games and then playing each others' variants. It was a time of bliss that was only surpassed 15 years later when I discovered Linux and began contributing to and writing open source software.

Most programmers I talk to in my generation have similar stories to share: we got completely hooked on the industry (long before we knew there was an industry) solely on the merits of editable source code. My parents hated when I played "too much" and they never really "got" that I was actually only playing a fraction of the time. They thought I was getting wound up on high scores and grue evasion... but they were missing the bigger picture. I spent far more than half the time changing the games, adding levels, extending star fields... enriching my gameplay. The high that I had discovered and was totally addicted to was the mutability of my universe.

Enter OLPC. The One Laptop Per Child project is absolutely phenomenal, far more so than I had ever anticipated. After Ivan Krstić's keynote at PyCon this year, everyone I talked to had the same impression that I had: Python Will Rule the World. And here's why:

Thousands, hundreds of thousands and possibly millions of children around the world will be using these innovative little machines. Children are amazing, curious, and fantastically brilliant sentient beings. Highly sentient. And into these little genius hands will be placed laptops that connect them to their friends, their teachers, and the rest of the world. Their interface to these laptops will be written in python. There is even a button on the laptops to "show source." Just as a small portion of my generation became addicted to the mutability of our universe, an insanely larger number of kids will have this same experience thanks to OLPC and python.

Look at what we've done so far. It's been amazing. And look how few of us it took. But what's going to happen when thousands and thousands of eager, curious, children from all over the world discover that, through the power of changing software, they can make changes to their world?

They will learn python. And with that, they will make this world theirs.

Technorati Tags: , , , , ,

Tuesday, February 27, 2007

Object Publishing with Nevow

Update: Code for the "Nevow Object Series" of blog posts is available here. Note that the examples can be viewed and run as single files or "project style" with code broken out into separate files.

Object publishing is a very simple concept: making code objects accessible over some medium. The medium most often discussed, due to the prevalence of web applications is HTTP/S. For our purposes, the published objects we will discuss are instantiations of python classes written to provide resources over the network.

In the python community, the most famous example of object publishing is the Zope application server. Zope exposes object written in python or with various Zope scripts/products/etc. In an old Zope 2 Book, Zope's object publishing is broken down very simply:
  1. The client sends a request to the server
  2. The server locates the object using the request URL as a map
  3. The server calls the published object with arguments from the request.
  4. The server interprets and returns the results to the client
What we have here is essentially object look-ups by URL. Given the web context, this makes obvious sense.

With such a simple definition of "object publishing", we can begin seeing "object publishing" pretty much anywhere python is used to deliver content or other resources on the web. For instance, Django qualifies as having an object publisher by means of the configuration mechanism whereby one can use regular expressions to map URLs to its "view" objects. We'll come back to Django in a bit.

Twisted's web "framework" Nevow has a object-publishing capabilities that, due to it being controlled by python classes is limited only by the python code you can write. There are four primary mechanisms by which one can publish objects in Nevow:
  • child attributes on a resource
  • child methods on a resource, or
  • dynamic children using the childFactory() method
  • manipulating the traversal of objects via locateChild() method
I will be focusing on the first two, however, before showing examples of these let's cover some minimal background. Twisted services are commonly run using what's called a .tac file ("Twisted Application Configuration"). Here is a trivial example of this:
from twisted.application import service
from twisted.application import strports

from nevow import appserver

import stuff

# instantiate the application
application = service.Application('Site')

# set up the main resource
root = mystuff.SiteRoot()
site = appserver.NevowSite(root)

# setup the web server
server = strports.service('tcp:8080', site)
server.setServiceParent(application)
Give than this file was saved as site.tac (and that there was a sibling file named stuff.py with the resource SiteRoot defined), you would enter twistd -noy site.tac at the system prompt to run this. This would start a server listening on port 8080. But what is SiteRoot? It is a resource we are making available at the root of our application, "publishing" it at http://localhost:8080/. And what would it look like, you ask? Well, a silly example of stuff.py that define SiteRoot might be something like the following:
from nevow import rend
from nevow import loaders

class SiteRoot(rend.Page):

docFactory = loaders.xmlstr('''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"
xmlns:nevow="http://nevow.com/ns/nevow/0.1">
</html>''')
One reason that this is a silly example is that it does nothing other than declare that the published resource will be an XHTML file with a Nevow namespace, providing no methods (and little content). Additionally, it uses the XML "string" loader, instead of using stan or the XML file loader (for using templates from disk). Regardless, if we add a child attribute that points to another class, we can start to see the object publishing behavior of Nevow:
from nevow import rend
from nevow import loaders

class MyStuffResource(rend.Page):

docFactory = loaders.xmlstr('''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"
xmlns:nevow="http://nevow.com/ns/nevow/0.1">
<h1>My Stuff</h1>
</html>''')

class SiteRoot(rend.Page):

docFactory = loaders.xmlstr('''<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0
Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"
xmlns:nevow="http://nevow.com/ns/nevow/0.1">
</html>''')

child_mystuff = MyStuffResource()
Starting up our application with these changes will allow us to access the root and child object via the following URLs, respectively:
  • http://localhost:8080/
  • http://localhost:8080/mystuff
This demonstrates how to publish objects via the child_ attribute. child_*() methods do the same thing, while offering all the flexibility that comes with methods. For instance, we could define the method child_yourstuff(self, context) in SiteRoot that would check for your username in the application cookie and pass that info when instantiating a UserStuffResource class. Then the URL http://localhost:8080/yourstuff could publish data specific to whatever was available in a data source associated with the username. The child_*() methods do need to return a resource object, though.

This has been a brief run-down on how you can use child_*() methods and child_ attributes to publish objects. In the next installment, I will show how to customize URL traversal for accessing these resources via the locateChild() method. In particular, I will show how to duplicate the Django URL conf functionality in Nevow.

Technorati Tags: , , ,

Saturday, February 24, 2007

Twisted BoF, Part Deux

Yes, we had another BoF today. How could we not? "But why?" you ask. "How could it get any better?" Well, if it wasn't better, it was just as insanely awesome as last night's. Only completely different.



I don't want to give away too much... there are many others who have a much greater right to share (and celebrate!) this news with you. So I will tease and tantalize instead: exciting things are afoot in the Twisted community. If you ever were curious about diving into Twisted and decided not to, now is definitely the time to reconsider, start checking it out, learning, and getting involved. You won't want to miss out on this.



"Do you know what I'm talking about?"

"The Matrix?"

"Do you want to know what - it - is?"




Technorati Tags: , , , ,

Friday, February 23, 2007

The Twisted Conference

I just got up to my hotel room from the BEST freakin' BoF tonight. Not only were there way more people than any of us expected, but most hung in there like CHAMPS: we started at 8:30pm and didn't get out until almost midnight! Okay, that's a little exaggeration... it was more like 11:30pm. Those present represented the spectrum of new users to intermediate through "founding members", and it was a damned trip. Just tons of amazing energy and latent excitement. And the love, baby -- you could just feel the love :-)



This came as a surprise to many of us, considering that 1) like last year, there are no Twisted talks and 2) most of the Twisted folks didn't know who all was showing up and we didn't think there'd more than a few. At last count, there are eight of us here now, and when Itamar shows up tomorrow, that will make nine.



But back to the BoF: the energy was truly palpable and the room was pretty packed. The topics covered were diverse and very interesting. The questions were stimulating, the answers insightful (and often hilarious). It was, in essence a conference within a conference :-) I believe over half stayed for the duration, and not only did it end with a standing-O (well, we were standing to leave...), but there were cheers and suggestions to do it again tomorrow. An hour later, I'm still pumped and totally incapable of considering sleep. If the enthusiasm of the BoF continues through the rest of the conference, we could end up with a couple sprints...

Technorati Tags: , , ,