Divmod Tech: Making the "Next Gen" Grade

Last night, after I already posted the latest Twisted in the News, I came across another post that would have made the list had I found it sooner. However, this is a good opportunity to give it a little extra attention.

The title of the post is "Next Gen Web Dev: Playing with Python Twisted/Nevow/Athena" and I gotta say, that made my day :-) Between that post and Colin Alston's post that I mentioned in the News, Nevow had a good week. And people are appreciating it for the right reasons. It may not be the easiest web framework to use and certainly not the best documented, but when you need the flexibility to interact with your (Twisted) web server in particular ways as well as benefit from the functionality that COMET provides, Nevow comes out shining.

It's also refreshing to see new developers entering the community who not only see the potential of these tools (designed with that potential in mind) but are capable of taking advantage of it immediately. If nothing else, the author of that post has motivated me to finally merge the Athena tutorial to trunk in order to bring the publicly available and published content in sync with the new code that's in the branch.

Update: Along similar lines, but with more details, Tristan has provided an excellent write-up for this motivation to use Twisted/Nevow/Axiom/Mantissa. Be sure to check it out!

The Problem with and Solution to Google's App Engine

I know everyone is all aglow with the new web development offering from Google, but let me do the unpopular thing and put some things into perspective: there are limitations.

In fact, the limitations that exist will prevent me from using App Engine with all of my projects, save one (that one being a very simple web site). First, the limitations that prevent me from using App Engine (from one of their FAQs):

• Sockets are disabled with Google App Engine
• The system does not allow you to invoke subprocesses, as a result some os module methods are disabled
• Threading is not available

This means that I can't write a deferred wrapper for their data layer, I can't use Twisted for such things as XML-RPC or AMP-based communications, and I can't use an async templating system (like Nevow). I'm stuck with CGI and blocking code. And for all but the simplest projects, that's a big "No Thank You" from me.

This doesn't mean that I won't use it -- I will. I have one project that this will be perfect for... but it's for someone else, not me.

However, these limitations are actually good news :-) Here's the silver lining:

As Glyph as alluded to in his recent blog post (and in our tweets), we've recently completed a massive week-long BizDev Divmod sprint in Boston. One of the results of this is based on community feedback we've had over the last year, and which culminated at PyCon 2008 in Chicago with multiple requests for particular services from The Twisted Company. That result is a set of tools, features, and management options folks will be able to use with our software (app server, smart object db, network services, etc.). People really want to start using our stuff in cloud/grid computing environments. They need support for multiple and diverse network services, inter-store communications, massive deployments, etc. Two months before PyCon, we started working on tickets to support this, and we're making excellent progress toward providing the requested features.

We're still unclear as to which parts of this will be open source, as that will be driven by a combination of business and community demand. Regardless, Google's lack of support for this stuff has (for now) left the field wide open for us. And that, folks, is a big "Thank You Google!" :-)

Nevow: URLdb

Well, I am really, really impressed with Nevow. It really grows on you as you code. I've been posting little tidbits that I learn on the Nevow wiki, the kinds of things that beginners should have easy access to.

Today, I dug really deep into the nevow libs and followed the path of form posts (configurables, validation, etc.). Very interesting stuff. Not only do I like the way Nevow behaves and handles for developers, I like its guts, too. That's actually fairly uncommon when it comes to web app frameworks.

Anyway, today was very successful: my TinyURL knockoff (URLdb) is "fully" functional. It has a form interface for posting/creating shorter URLs as well as a URL you can post to via javascript, etc. The reason that I put "fully" in quotes, is that there is no persistent backend storage yet: it's all in memory. This was for simplifying the development/learning process only, and I have already started on writing the Atop classes.

I look forward to using Nevow in the future -- immediately, on the PyKon project; but in general, for any web app that doesn't require the pre-built functionality of a full-blown CMS.