Saturday, March 26, 2005

Pete Yorn

music :: leisure



I am really behind the times. I discovered Pete Yorn only yesterday. I
was watching "Me, Myself, & Irene" with my mother last night. We were
both in a punchy mood, just having watched Bruce Campbell's "Alien
Invasion", so we laughed ourselves silly at Jim Carey's schizoid
antics. At some point in the movie near where they met Whitey, the
coolest song started playing. I've owned this movie since it came out,
and had never heard this song before. Well, it turned out it was the
song "Strange Condition" by Pete Yorn.


So I got a couple of his CDs, and holy crap -- this guy's good! He
reminds me of Elliot Smith, some classic REM, Counting Crows and a
little Grant Lee Buffalo. And sometimes you can hear a little Robert
Smith (The Cure). I feel like one of those dorks in "High Fidelity"
now... "She's kind of a
Sheryl-Crow-meets-a-post-Partridge-Family-pre-LA
Law-Susan-Day...only...black."


Thursday, March 24, 2005

Nevow: URLdb

Well, I am really, really impressed with Nevow. It really grows on you as you code. I've been posting little tidbits that I learn on the Nevow wiki, the kinds of things that beginners should have easy access to.

Today, I dug really deep into the nevow libs and followed the path of form posts (configurables, validation, etc.). Very interesting stuff. Not only do I like the way Nevow behaves and handles for developers, I like its guts, too. That's actually fairly uncommon when it comes to web app frameworks.

Anyway, today was very successful: my TinyURL knockoff (URLdb) is "fully" functional. It has a form interface for posting/creating shorter URLs as well as a URL you can post to via javascript, etc. The reason that I put "fully" in quotes, is that there is no persistent backend storage yet: it's all in memory. This was for simplifying the development/learning process only, and I have already started on writing the Atop classes.

I look forward to using Nevow in the future -- immediately, on the PyKon project; but in general, for any web app that doesn't require the pre-built functionality of a full-blown CMS.


Wednesday, March 23, 2005

Ruby on Rails vs Nevow

nevow :: web :: programming :: python :: ruby



I've just read the post at Blue Sky on Mars, and
I've got to agree 100%. In fact, earlier today, I posted a couple small
projects I am working on in the Nevow
wiki
, and mentioned a personal preference for Nevow over Ruby on
Rails. Personally, I believe one has more power and capabilities at
one's finger tips with Nevow. And this in no way takes away from the
simplicity and elegance of Nevow.


First, let me say that I am a huge Zope/Plone, twisted/Nevow,
and PEAK fan. Each has its strengths. But as far as general web
development frameworks go in python, nothing compares to Nevow for
building custom applications from the ground up. I know there are lots
of tools to choose from (Quixote, WebWare, among others), but the
simplest, cleanest -- and in my view -- the most powerful, is Nevow.

Nevow
has a thriving, active community, so I propose we give David
Heinemeier Hansson
and Ruby on Rails a run for their money. Nevow
and it's community can beat them hands-down. Let's get some amazing and
easy-to-read docs up. Let's get some videos. Let's show the world that
python has even more to offer than Ruby, and a clean web app dev
framework in Nevow is one of them.

Here are some Nevow resources:

nevow.org

Nevow Manual


Nevow at Divmod

Nevow Wiki

Code Repository

Nevow mail list archives

Nevow at del.icio.us

Nevow at PyWebOff

Nevow examples - these are
the examples that come with the Nevow distribution


Python Eggs

python :: programming



Okay, two developers that I highly regard are working on
something very, very cool: a package/utility that works around
distutils limitations and will be able to provide the same
functionality as java .jar files do, but for python. And then some.
They are called python .egg files, and you can read about them here:


Python Eggs

DirtSimple


Tuesday, March 22, 2005

New Projects

As if things haven't been busy enough, with work, etc., I've got a couple python/twisted/Nevow projects in the works.

The first one is PyKon (Python Ikon), a gallery application for managing galleries, photo albums, and images. It's just getting started, so there's nothing to show yet, but it uses Nevow and the Python Imaging Library. This project actually comes at the request of many friends who have been bugging me about this for quite some time. You know who you are ;-) This will be the first non-technical, user-oriented app I have built with Nevow, and am therefore quite looking forward to it. I will also be delighted to not have to install, run, and manage any more PHP-based galleries.

The second project is a complete whim: I want to be able to provide fellow developers on mail lists and irc channels with a means of using something like TinyURL, but with the additional ability of being able to add an optional, distinguishing string at the end. This will make it more clear when reading a post to a mail list or seeing a bunch of URLs pop up on IRC. I'm calling it URLdb. I started by copying the Nevow pastebin example app (which totally rocks, btw). The code is here.


supybot IRC Channel Bot and Subversion Commits

programming :: python :: internet



Last night at the Hacking Society meeting in Fort Collins, I finished
my latest utility. I use supybot on our dev channel and the other day,
I learned how to write plugins for it. After writing one, I realized
that wasn't exactly what I needed. What I really needed was to add my
own daemon to run inside supybot's daemon. Then I learned the complete
internals of supybot. Quite an interesting experience. I wrote a
daemon/notification server last year that monitors arbitrary
directories and files for changes to contents, time stamps, etc. I
wanted my daemon to monitor a directory where our subversion hooks post
updates, see which files (updates) have been added, and then parse the
updates for information that is useful to post on an IRC channel. *And*
I wanted this daemon to run inside the supybot daemon. In less than 24
hours, supybot was learned and my daemon was buried inside of it nicely
;-)


At the Hacking Society meeting, I finished up the parsing for the
subversion commit updates... today, the results were viewable as
everyone made their code changes. Ah, data is wonderful.


Sunday, March 20, 2005

Living in an Epic Tolkien Setting

leisure :: nature



We worked pretty hard at Adytum last week, so Friday and the weekend
were filled with little rewards... like pizza, going to the movies, and
most importantly, hiking in the Rockies. I have been in the Alps;I have
been in the Himalayas. Each are impressive and beautiful. The Rockies,
though... mmm. That's home.


I am now the proud owner of a season pass to the US National Parks.
This sponsorship ranks among the highest causes I contribute to -- just
as important as preserving indigenous cultures and human rights.
Environment, ecologies, biosystems -- these are insanely precious
resources. And, you know, one of the most potent ways to recharge,
de-stress, and get reconnected to the stuff that's really important in
life: living.

We drove up into Rocky Mountain National Park, just West of Estes Park,
CO. It was less than an hour from my front door, a fact I am
continually in awe of and grateful for. (Yes, those're *two* dangling
prepositions.) Driving down Routes 34 and 36, seeing these Immense
Mountains and then More Immensity behind those Immensities was just
extraordinary. I was just giggling all yesterday. My mind simply went
on overload and I reverted to the state of a 4-year-old who is fresh to
the world, has absorbed all he can handle, and simply babbles in
synaptic twitching.

The Rockies. This is my home, now. I just can't get over it. Whenever I
am hiking or driving or bicycling in those mountains, I feel like those
enormous, snow-capped peaks are my big brothers, or uncles... like
they're sheltering and taking care of me. It feels like an excerpt from
the Silmarillion or The Two Towers, where -- like the Ents -- the
mountains will, at any moment, begin speaking in deep, booming voices.

I simply couldn't be happier with living in Colorado. I see now that
I've been waiting to live here all my life. I'm home. And I'll let you
know if the mountains start talking about little people or feisty
rings.


Friday, March 18, 2005

Constantine

entertainment



I haven't seen any reviews, and I don't want to. I just got back from
the movies where I saw this little gem, and I've got to say that it was
awesome! I had such a great time watching it. I laughed at all the
jokes, enjoyed the fun occult references, and thought that Keanu did a
great job. My biggest fear was that this was going to be a poor attempt
at Matrixhood (like the first half of Underworld), and it totally
wasn't. There were a couple humorous nods to the Matrix, but the feel
was entirely different. Not having read the original comic, I have no
idea how true they stayed to the story, but whatever the case may be,
they told a great story ;-)


Very good fun.


iTunes Playtlist with cElementTree

python :: programming :: music



When we moved to Colorado from Maryland in January, a Tibetan friend of
ours who is a monk helped us pack, drive, unpack, and setup house when
we got here. During the drive, we spent many hours practicing English,
Tibetan, and listening to "injee" music on my iPod. There were certain
songs that he could listen to more easily than others. It was sometimes
surprising to me the songs where lyrics were enunciated well enough to
be mimicked without understanding the content. The one that made me
laugh the hardest was "Cold As Ice" by Foreigner... there is something
totally hilarious about driving cross-country with a Tibetan monk, both
of you singing "She's as cold as ice!" at the top of your lungs. It's
one of those moments you know will stick with you forever. Another good
one was KC & the Sunshine Band's "Shake Your Booty" -- it turns out
English "shake" is cognate with a Tibetan colloquial word of similar
pronunciation and meaning.


Anyway, I was supposed to build CDs of photos and music from our
several weeks together and send them back home to Lama Gyaltsen. iTunes
exports playlists in several formats, including XML. This was a perfect
excuse to play with ElementTree some more, while the CDs burned (that's
another cool thing about iTunes: got a play list that's 3 and a half
hours worth of music and you want to burn to CD? No problem! It splits
them up for you. Save your brain cells for the important stuff like
playing with python).

Here's the code:

import cElementTree as et
filename = '/Users/oubiwann/Desktop/Mix - Lama Gyaltsen.xml'
tree = et.parser(filename)
root = tree.getroot()
template = '%s: %s'
for i in root.find('./*/dict').getchildren():
if i.tag == 'dict':
data = i.getchildren()
for j in range(0,len(data)):
if data[j].tag == 'key':
if data[j].text == 'Name':
print template % ('Song Name', data[j+1].text)
if data[j].text == 'Artist':
print template % ('Band Name', data[j+1].text)
if data[j].text == 'Total Time':
converted = '%s:%.2s' %
divmod(int(data[j+1].text), 60000)
print template % ('Length', converted)


Update: Fredrik Lundh has just blogged about this :-) He has a
added simpler loader for for iTunes playlists to ElementTree. Read
about it here.


Nevow + Twisted, or mod_python?

python :: web :: programming



To twist or not to twist, this is always the question.


Or, rather, I always want to use twisted, but can I produce the numbers
and the tests for customers that show it performs as well as
mod_python?

Or, even better: how does one architect a killer twisted web app server
that performs insanely well with high concurrency and blinding speed?
Hmm, It's been a while since I checked out the Nevow code base, so I
just did an svn update, and I see there there is now a wsgi cgi for
mod_python in the examples directory. This is cool, I shall have to
test... Perhaps I can have my cake and eat it too.

Well, I just installed it on a server of mine and have been running
apache bench against the following:

nevow wsgi mod_python code
a nevow example running on the twisted daemon
the AdytumSolutions web page (custom mod_python/publisher mini-cms)
a tomcat example jsp page.

I ran 5000 requests, 20 concurrent requests for each. Here are some
results:

nevow wsgi - Requests per second: 48.96 [#/sec] (mean)
nevow twisted - Requests per second: 64.85 [#/sec] (mean)
simple mod_python app - Requests per second: 824.22 [#/sec] (mean)
tomcat - Requests per second: 288.01 [#/sec] (mean)

I've just spent about an hour reviewing the Nevow example apps (started
up the examples by cd'ing to the examples directory and
running 'twistd -noy examples.tac'). I must say: Donavan,
Matt, and company have done an amazing job in the last year. Truly
impressive. Lots of really good examples that get the creative juices
flowing and jump start one right into seeing how to solve all kinds of
problems with Nevow. In fact, I am so impressed that, if I can learn
how to get as much performance out of Nevow, I would gladly sacrifice
some raw running/processing speed for ease of development and
maintenance. And extensibility. And joy.

I wonder if I can get my clients to see things this way?

Perhaps it's time to hop back on #twisted.web and start asking
questions I'm sure they are quite tired of answering...


Tuesday, March 15, 2005

Advogato Diary Archiver

web :: python :: programming



Last night I finished the script to pull (XML-RPC) my diary entries on
Advogato into a custom archive format. I had to use an ugly sys.stdout
hack to use the cElementTree.dump() method... but discouraging it's
misuse was probably a motivating factor for Fredrik Lundh to send its
output to stdout in the first place. I just wanted the entire contents
of a particular div, and I didn't want to have to do all that text/tail
crap.


The really nice thing thing about this, though, is piece of mind: a
single server crash anywhere will no longer result in catastrophic loss
of blog entries. Because, you know, blog entries are so damned
important ;-)


Update: Duh. cElementTree.tostring(aElement). Say no more.



Sunday, March 13, 2005

America Beyond Capitalism -- Posters!

economics :: governance :: politics



Today, I got an email from Ted Howard (director of the Democracy
Collaborative
) letting me know that there are posters based on the
book America Beyond
Capitalism
available for download here. Very cool stuff. Some very good facts to know and
be aware of. One that really pisses me off is in 1945 corporate
taxes provided 35.4% of federal revenues and now corps are only paying
7.4%. That shit just ain't right. I also like this one -- and in to answer their question,
no! it's not what I want. In fact, since working for myself, I have
changed my work week to less than 40 hours a week. I strive for 30, but
would like to get it down to 25 hours/week. With more time off, I get
to spend my time relaxing, hiking, volunteering for projects I care
about -- improving my quality of life. As a result, during my work
hours, I am much more productive and energetic, happy about life and
working.


BTW, if you haven't read the book America Beyond
Capitalism
, you really ought to check it out. It's a fast read with
some intense (but accessible) data. And it's not a doom and gloom thing,
either - there is all kinds of information in it about what various
groups, researchers, corporations, communities, etc., are doing to
change the immense imbalance of power and wealth.


Thursday, March 10, 2005

Advogato and Bad HTML

web :: programming :: python :: html



I don't know if I'm the only one who's trying to use Advogato in
non-ordinary ways (and I'd guess that at least Titus Brown is), but
I'm having a hell of a time getting mechanize to
work with the site. I love vi and I won't apologize for it ;-) I love
using it as my IDE as well as my CMS. I want to be able to edit
valuable information in a vi/vim terminal window. Sean Reifschneider wrote a nifty little python
script for editing messages in vi (I forget which RFC to which they
were compliant) and posting to websites. Alex Levy did something related, where you
could edit pickled data files via command line editors. I just totally
dig that stuff. I've written three mini-CMSs in PHP, perl, and python
(over the past 5 years) that use the command line as the primary means
of adding and editing data. Now, I want to do the same thing with my
Advogato account.


Alas, this is proving very difficult so far. Advogato does not seem to
be using proper html! So far, none of the text input fields in the
forms I have looked at are labled type="text" -- they are missing the
"type" attribute all together. As a result (at least, I think the lack
of "type" is the cause), mechanize does not see these inputs as
controls, and I can therefore not manipulate them. This means that I
cannot login and I cannot update my account settings. Very troublesome.
Additionally, it would be wonderful if the forms could be given
name attributes for easy access. I would love for someone to point out
a way around this or my misunderstanding of mechanize.

Of course, there is always the hope that the Advogato XML-RPC API will
be extended to allow for the managing of account data as well as for
publishing articles. Then I wouldn't care how the html gets abused ;-)


Update:
It seems that I can, in fact, get controls for all the account form
elements in mechanize, even though the text boxes are missing the
"type" attribute. However, I am still unable to process the login form
via mechanize. I am able to login, but that requires passing the
passphrase for the account in a GET, the consequence being that the
account password will turn up in log files.




Wednesday, March 09, 2005

pymon

python :: programming :: networking



Well, after several months, I've finally had a chance to sit down and
work on pymon again. We're plugging
away, slowly but surely, on the next alpha release. Every time I dig
into twisted code, I am just delighted. Those guys just rock. The
flexibility it provides in writing network-oriented apps is just
amazing. I'm really excited to see how this project matures.


However, I'm getting tons of Zope/Plone work now, and that's stuff's
currently paying the bills, so pymon might have to wait for a few more
weeks. Until I get crazy with working with Zope/Plone and need a real
coding break ;-)


Sunday, March 06, 2005

Jefferson in "The Economy of Ideas"

history :: open source



I was checking out Ian Bicking's recent del.icio.us additions (that's
not stalking!), and I got high on hypertext crack... yeah, I couldn't
stop. Finally, I hit the wall here:


The Economy of Ideas.

And the intro to the article is a thought-provoking quote by Thomas
Jefferson. It's been ages since I read this, and its eloquence (though
the phrasing is dated, i.e., not PC) is still enough to take the breath
away:


"If nature has made any one thing less susceptible than all others of
exclusive property, it is the action of the thinking power called an
idea, which an individual may exclusively possess as long as he keeps
it to himself; but the moment it is divulged, it forces itself into the
possession of everyone, and the receiver cannot dispossess himself of
it. Its peculiar character, too, is that no one possesses the less,
because every other possesses the whole of it. He who receives an idea
from me, receives instruction himself without lessening mine; as he who
lights his taper at mine, receives light without darkening me. That
ideas should freely spread from one to another over the globe, for the
moral and mutual instruction of man, and improvement of his condition,
seems to have been peculiarly and benevolently designed by nature, when
she made them, like fire, expansible over all space, without lessening
their density at any point, and like the air in which we breathe, move,
and have our physical being, incapable of confinement or exclusive
appropriation. Inventions then cannot, in nature, be a subject of
property."


- Thomas Jefferson

I have been and continue to be involved in many open source projects;
as developer, contributer, user, etc. I have also been involved in many
closed-source, proprietary projects. There are trade secrets and code I
will take to the grave with me, but this is from a sense of honor and
ethics; of agreements between entities. Given my preference, though, I
would love for all software to be open source... the idealist beauty of
Jefferson's words makes promises too sweet to resist.


Saturday, March 05, 2005

Experiment in Community

community :: society :: governance



I live with three very different people; that's three, including
myself. We are all different, and we decided to try living together as
a social experiment. Personally, I am interested in decentralized
social and governmental systems: distributed power, distributed
decision making, etc. I am an optimist in that I believe that a
"sufficiently large" collection of people "educated" on a particular
topic will collectively make statistically wise decisions on that
topic, no matter the variations in individual decisions. To me, this
seems one (of possibly uncountably many) functional definitions of
democracy. It may come off as elitist (and to some, that's okay) due to
the use of the term "educated" but this in no way refers to social,
economic, or any other form of privilege. I use the term very generally
to connote experience, knowledge, and understanding of a certain topic.
Of course, there is that ambiguity "sufficiently large"...


Anyway, I digress. Social experiment. Understanding of social systems
via practical experience, with willing participants that are aware of
the process, the goals, etc. We have hit the "growth" stage now, i.e.,
problems. And in a discussion with one of the members today, I was
asked how I define "community." This was an interesting question... I'm
always talking about this concept, this abstraction; I'm always
listening to others talk about it. We all seem to agree that we're
talking about the same thing, but here was my chance to see, in fact,
if that's the case. I paced around the room for a while, like an
autistic child about to make a break through. Here's how I defined it:


A community is a group of entities that interact for the common benefit
of all entities in the group through exchange of information between
entities and the compensation for deficiencies and/or efficiencies of
individual entities.

I'd never thought of it like this before, and it really took me by
surprise: community dynamics is an engineering problem! Of course, I
say this with all the humor it deserves ;-) And this revealed the
nature of the problem between two of the individuals in our social
experiment: two of us are unwilling to compensate for each other's lack
for the common benefit of the group. We are therefore failing as a
community, in that we cannot satisfy the definition. This awareness was
a painful moment for me, as I have been greatly looking forward to this
adventure. It's early still, so there is hope. The biggest difficulty
we may encounter is motivation: if we don't feel in our bones that we
have to work with these differences in order to survive, we may
not put in the required effort.

Community interaction is not an endeavor of self. In the "West", we've
got plenty of that! We are all about the individual. I find this
particularly true of Americans, and have been guilty of such for much
of my life. We feel that many (all!) sacrifices somehow lessen "the
self" and are therefore tantamount to the complete violation of our
most precious principles. Practical experience in communities that meet
the requirements of the functional definition above show us something
quite the reverse: personal sacrifice for the benefit of all improves
the quality of life for all members of the group, and sometimes
especially for the one who sacrificed.


In these "modern", post-tribal times and for the most part, we are no
longer bound to each other by the glue of survival needs, forced to
live in communities and tribes, getting along out of necessity. As
such, it may be that Quality of Life has become the new definition of
Survival. And in fact, our survival as a sane and happy society may
require that quality of life be perceived as such.

Curious now, I have looked up the definitions of community on WordNet:


    * community -- (a group of people living in a particular local
    area; "the team is drawn from all parts of the community")

    * community -- (a group of people having ethnic or cultural or
    religious characteristics in common; "the Christian community of the
    apostolic age"; "he was well known throughout the Catholic community")

    * community -- (common ownership; "they shared a community of
    possessions")

    * community -- (a group of nations having common interests; "they
    hoped to join the NATO community")

    * profession, community -- (the body of people in a learned
    occupation; "the news spread rapidly through the medical community")

    * community, community of interests -- (agreement as to goals;
    "the preachers and the bootleggers found they had a community of
    interests")

    * residential district, residential area, community -- (a district
    where people live; occupied primarily by private residences)

    * community, biotic community -- ((ecology) a group of
    interdependent organisms inhabiting the same region and interacting
    with each other)


I particularly like the last one, from the ecological point of view.
Being an adherent to a Tibetan Buddhist school of philosophy, I enjoy
the point of interdependence.


Friday, March 04, 2005

Generic Functions and Rule-Based Systems

Well, I've been pretty bummed that business is keeping me too occupied to attend PyCon 2005 this year. Last year's was so great, and I had a chance to sprint with Chris McDonough and Donovan Preston and learned a great deal from them both.

In fact, I was decompressing the data I took in during the python sprints for the following 6 months. I guess it shows what a geek I am too, that I bemoan too much business, as it's preventing me attending this year.

I was checking out the presentations here, and I came across Phillip Eby's presentation here. This really hit home, since one of the projects I am working on is for processing *massive* amounts of data produced while testing equipment at manufacturing plants. There are sets of sets of sets of ... tests. Thousands of the buggers. Many of them are very similar in nature, and they all have associated rules for pass and failure.

I can't wait to tryout Phillip's generic functions in PyProtocol's... in fact, I haven't waited ;-) I'm already writing some prototype code for it...

Python rocks.