Friday, March 18, 2005

Constantine

entertainment



I haven't seen any reviews, and I don't want to. I just got back from
the movies where I saw this little gem, and I've got to say that it was
awesome! I had such a great time watching it. I laughed at all the
jokes, enjoyed the fun occult references, and thought that Keanu did a
great job. My biggest fear was that this was going to be a poor attempt
at Matrixhood (like the first half of Underworld), and it totally
wasn't. There were a couple humorous nods to the Matrix, but the feel
was entirely different. Not having read the original comic, I have no
idea how true they stayed to the story, but whatever the case may be,
they told a great story ;-)


Very good fun.


iTunes Playtlist with cElementTree

python :: programming :: music



When we moved to Colorado from Maryland in January, a Tibetan friend of
ours who is a monk helped us pack, drive, unpack, and setup house when
we got here. During the drive, we spent many hours practicing English,
Tibetan, and listening to "injee" music on my iPod. There were certain
songs that he could listen to more easily than others. It was sometimes
surprising to me the songs where lyrics were enunciated well enough to
be mimicked without understanding the content. The one that made me
laugh the hardest was "Cold As Ice" by Foreigner... there is something
totally hilarious about driving cross-country with a Tibetan monk, both
of you singing "She's as cold as ice!" at the top of your lungs. It's
one of those moments you know will stick with you forever. Another good
one was KC & the Sunshine Band's "Shake Your Booty" -- it turns out
English "shake" is cognate with a Tibetan colloquial word of similar
pronunciation and meaning.


Anyway, I was supposed to build CDs of photos and music from our
several weeks together and send them back home to Lama Gyaltsen. iTunes
exports playlists in several formats, including XML. This was a perfect
excuse to play with ElementTree some more, while the CDs burned (that's
another cool thing about iTunes: got a play list that's 3 and a half
hours worth of music and you want to burn to CD? No problem! It splits
them up for you. Save your brain cells for the important stuff like
playing with python).

Here's the code:

import cElementTree as et
filename = '/Users/oubiwann/Desktop/Mix - Lama Gyaltsen.xml'
tree = et.parser(filename)
root = tree.getroot()
template = '%s: %s'
for i in root.find('./*/dict').getchildren():
if i.tag == 'dict':
data = i.getchildren()
for j in range(0,len(data)):
if data[j].tag == 'key':
if data[j].text == 'Name':
print template % ('Song Name', data[j+1].text)
if data[j].text == 'Artist':
print template % ('Band Name', data[j+1].text)
if data[j].text == 'Total Time':
converted = '%s:%.2s' %
divmod(int(data[j+1].text), 60000)
print template % ('Length', converted)


Update: Fredrik Lundh has just blogged about this :-) He has a
added simpler loader for for iTunes playlists to ElementTree. Read
about it here.


Nevow + Twisted, or mod_python?

python :: web :: programming



To twist or not to twist, this is always the question.


Or, rather, I always want to use twisted, but can I produce the numbers
and the tests for customers that show it performs as well as
mod_python?

Or, even better: how does one architect a killer twisted web app server
that performs insanely well with high concurrency and blinding speed?
Hmm, It's been a while since I checked out the Nevow code base, so I
just did an svn update, and I see there there is now a wsgi cgi for
mod_python in the examples directory. This is cool, I shall have to
test... Perhaps I can have my cake and eat it too.

Well, I just installed it on a server of mine and have been running
apache bench against the following:

nevow wsgi mod_python code
a nevow example running on the twisted daemon
the AdytumSolutions web page (custom mod_python/publisher mini-cms)
a tomcat example jsp page.

I ran 5000 requests, 20 concurrent requests for each. Here are some
results:

nevow wsgi - Requests per second: 48.96 [#/sec] (mean)
nevow twisted - Requests per second: 64.85 [#/sec] (mean)
simple mod_python app - Requests per second: 824.22 [#/sec] (mean)
tomcat - Requests per second: 288.01 [#/sec] (mean)

I've just spent about an hour reviewing the Nevow example apps (started
up the examples by cd'ing to the examples directory and
running 'twistd -noy examples.tac'). I must say: Donavan,
Matt, and company have done an amazing job in the last year. Truly
impressive. Lots of really good examples that get the creative juices
flowing and jump start one right into seeing how to solve all kinds of
problems with Nevow. In fact, I am so impressed that, if I can learn
how to get as much performance out of Nevow, I would gladly sacrifice
some raw running/processing speed for ease of development and
maintenance. And extensibility. And joy.

I wonder if I can get my clients to see things this way?

Perhaps it's time to hop back on #twisted.web and start asking
questions I'm sure they are quite tired of answering...


Tuesday, March 15, 2005

Advogato Diary Archiver

web :: python :: programming



Last night I finished the script to pull (XML-RPC) my diary entries on
Advogato into a custom archive format. I had to use an ugly sys.stdout
hack to use the cElementTree.dump() method... but discouraging it's
misuse was probably a motivating factor for Fredrik Lundh to send its
output to stdout in the first place. I just wanted the entire contents
of a particular div, and I didn't want to have to do all that text/tail
crap.


The really nice thing thing about this, though, is piece of mind: a
single server crash anywhere will no longer result in catastrophic loss
of blog entries. Because, you know, blog entries are so damned
important ;-)


Update: Duh. cElementTree.tostring(aElement). Say no more.



Sunday, March 13, 2005

America Beyond Capitalism -- Posters!

economics :: governance :: politics



Today, I got an email from Ted Howard (director of the Democracy
Collaborative
) letting me know that there are posters based on the
book America Beyond
Capitalism
available for download here. Very cool stuff. Some very good facts to know and
be aware of. One that really pisses me off is in 1945 corporate
taxes provided 35.4% of federal revenues and now corps are only paying
7.4%. That shit just ain't right. I also like this one -- and in to answer their question,
no! it's not what I want. In fact, since working for myself, I have
changed my work week to less than 40 hours a week. I strive for 30, but
would like to get it down to 25 hours/week. With more time off, I get
to spend my time relaxing, hiking, volunteering for projects I care
about -- improving my quality of life. As a result, during my work
hours, I am much more productive and energetic, happy about life and
working.


BTW, if you haven't read the book America Beyond
Capitalism
, you really ought to check it out. It's a fast read with
some intense (but accessible) data. And it's not a doom and gloom thing,
either - there is all kinds of information in it about what various
groups, researchers, corporations, communities, etc., are doing to
change the immense imbalance of power and wealth.


Thursday, March 10, 2005

Advogato and Bad HTML

web :: programming :: python :: html



I don't know if I'm the only one who's trying to use Advogato in
non-ordinary ways (and I'd guess that at least Titus Brown is), but
I'm having a hell of a time getting mechanize to
work with the site. I love vi and I won't apologize for it ;-) I love
using it as my IDE as well as my CMS. I want to be able to edit
valuable information in a vi/vim terminal window. Sean Reifschneider wrote a nifty little python
script for editing messages in vi (I forget which RFC to which they
were compliant) and posting to websites. Alex Levy did something related, where you
could edit pickled data files via command line editors. I just totally
dig that stuff. I've written three mini-CMSs in PHP, perl, and python
(over the past 5 years) that use the command line as the primary means
of adding and editing data. Now, I want to do the same thing with my
Advogato account.


Alas, this is proving very difficult so far. Advogato does not seem to
be using proper html! So far, none of the text input fields in the
forms I have looked at are labled type="text" -- they are missing the
"type" attribute all together. As a result (at least, I think the lack
of "type" is the cause), mechanize does not see these inputs as
controls, and I can therefore not manipulate them. This means that I
cannot login and I cannot update my account settings. Very troublesome.
Additionally, it would be wonderful if the forms could be given
name attributes for easy access. I would love for someone to point out
a way around this or my misunderstanding of mechanize.

Of course, there is always the hope that the Advogato XML-RPC API will
be extended to allow for the managing of account data as well as for
publishing articles. Then I wouldn't care how the html gets abused ;-)


Update:
It seems that I can, in fact, get controls for all the account form
elements in mechanize, even though the text boxes are missing the
"type" attribute. However, I am still unable to process the login form
via mechanize. I am able to login, but that requires passing the
passphrase for the account in a GET, the consequence being that the
account password will turn up in log files.




Wednesday, March 09, 2005

pymon

python :: programming :: networking



Well, after several months, I've finally had a chance to sit down and
work on pymon again. We're plugging
away, slowly but surely, on the next alpha release. Every time I dig
into twisted code, I am just delighted. Those guys just rock. The
flexibility it provides in writing network-oriented apps is just
amazing. I'm really excited to see how this project matures.


However, I'm getting tons of Zope/Plone work now, and that's stuff's
currently paying the bills, so pymon might have to wait for a few more
weeks. Until I get crazy with working with Zope/Plone and need a real
coding break ;-)


Sunday, March 06, 2005

Jefferson in "The Economy of Ideas"

history :: open source



I was checking out Ian Bicking's recent del.icio.us additions (that's
not stalking!), and I got high on hypertext crack... yeah, I couldn't
stop. Finally, I hit the wall here:


The Economy of Ideas.

And the intro to the article is a thought-provoking quote by Thomas
Jefferson. It's been ages since I read this, and its eloquence (though
the phrasing is dated, i.e., not PC) is still enough to take the breath
away:


"If nature has made any one thing less susceptible than all others of
exclusive property, it is the action of the thinking power called an
idea, which an individual may exclusively possess as long as he keeps
it to himself; but the moment it is divulged, it forces itself into the
possession of everyone, and the receiver cannot dispossess himself of
it. Its peculiar character, too, is that no one possesses the less,
because every other possesses the whole of it. He who receives an idea
from me, receives instruction himself without lessening mine; as he who
lights his taper at mine, receives light without darkening me. That
ideas should freely spread from one to another over the globe, for the
moral and mutual instruction of man, and improvement of his condition,
seems to have been peculiarly and benevolently designed by nature, when
she made them, like fire, expansible over all space, without lessening
their density at any point, and like the air in which we breathe, move,
and have our physical being, incapable of confinement or exclusive
appropriation. Inventions then cannot, in nature, be a subject of
property."


- Thomas Jefferson

I have been and continue to be involved in many open source projects;
as developer, contributer, user, etc. I have also been involved in many
closed-source, proprietary projects. There are trade secrets and code I
will take to the grave with me, but this is from a sense of honor and
ethics; of agreements between entities. Given my preference, though, I
would love for all software to be open source... the idealist beauty of
Jefferson's words makes promises too sweet to resist.


Saturday, March 05, 2005

Experiment in Community

community :: society :: governance



I live with three very different people; that's three, including
myself. We are all different, and we decided to try living together as
a social experiment. Personally, I am interested in decentralized
social and governmental systems: distributed power, distributed
decision making, etc. I am an optimist in that I believe that a
"sufficiently large" collection of people "educated" on a particular
topic will collectively make statistically wise decisions on that
topic, no matter the variations in individual decisions. To me, this
seems one (of possibly uncountably many) functional definitions of
democracy. It may come off as elitist (and to some, that's okay) due to
the use of the term "educated" but this in no way refers to social,
economic, or any other form of privilege. I use the term very generally
to connote experience, knowledge, and understanding of a certain topic.
Of course, there is that ambiguity "sufficiently large"...


Anyway, I digress. Social experiment. Understanding of social systems
via practical experience, with willing participants that are aware of
the process, the goals, etc. We have hit the "growth" stage now, i.e.,
problems. And in a discussion with one of the members today, I was
asked how I define "community." This was an interesting question... I'm
always talking about this concept, this abstraction; I'm always
listening to others talk about it. We all seem to agree that we're
talking about the same thing, but here was my chance to see, in fact,
if that's the case. I paced around the room for a while, like an
autistic child about to make a break through. Here's how I defined it:


A community is a group of entities that interact for the common benefit
of all entities in the group through exchange of information between
entities and the compensation for deficiencies and/or efficiencies of
individual entities.

I'd never thought of it like this before, and it really took me by
surprise: community dynamics is an engineering problem! Of course, I
say this with all the humor it deserves ;-) And this revealed the
nature of the problem between two of the individuals in our social
experiment: two of us are unwilling to compensate for each other's lack
for the common benefit of the group. We are therefore failing as a
community, in that we cannot satisfy the definition. This awareness was
a painful moment for me, as I have been greatly looking forward to this
adventure. It's early still, so there is hope. The biggest difficulty
we may encounter is motivation: if we don't feel in our bones that we
have to work with these differences in order to survive, we may
not put in the required effort.

Community interaction is not an endeavor of self. In the "West", we've
got plenty of that! We are all about the individual. I find this
particularly true of Americans, and have been guilty of such for much
of my life. We feel that many (all!) sacrifices somehow lessen "the
self" and are therefore tantamount to the complete violation of our
most precious principles. Practical experience in communities that meet
the requirements of the functional definition above show us something
quite the reverse: personal sacrifice for the benefit of all improves
the quality of life for all members of the group, and sometimes
especially for the one who sacrificed.


In these "modern", post-tribal times and for the most part, we are no
longer bound to each other by the glue of survival needs, forced to
live in communities and tribes, getting along out of necessity. As
such, it may be that Quality of Life has become the new definition of
Survival. And in fact, our survival as a sane and happy society may
require that quality of life be perceived as such.

Curious now, I have looked up the definitions of community on WordNet:


    * community -- (a group of people living in a particular local
    area; "the team is drawn from all parts of the community")

    * community -- (a group of people having ethnic or cultural or
    religious characteristics in common; "the Christian community of the
    apostolic age"; "he was well known throughout the Catholic community")

    * community -- (common ownership; "they shared a community of
    possessions")

    * community -- (a group of nations having common interests; "they
    hoped to join the NATO community")

    * profession, community -- (the body of people in a learned
    occupation; "the news spread rapidly through the medical community")

    * community, community of interests -- (agreement as to goals;
    "the preachers and the bootleggers found they had a community of
    interests")

    * residential district, residential area, community -- (a district
    where people live; occupied primarily by private residences)

    * community, biotic community -- ((ecology) a group of
    interdependent organisms inhabiting the same region and interacting
    with each other)


I particularly like the last one, from the ecological point of view.
Being an adherent to a Tibetan Buddhist school of philosophy, I enjoy
the point of interdependence.


Friday, March 04, 2005

Generic Functions and Rule-Based Systems

Well, I've been pretty bummed that business is keeping me too occupied to attend PyCon 2005 this year. Last year's was so great, and I had a chance to sprint with Chris McDonough and Donovan Preston and learned a great deal from them both.

In fact, I was decompressing the data I took in during the python sprints for the following 6 months. I guess it shows what a geek I am too, that I bemoan too much business, as it's preventing me attending this year.

I was checking out the presentations here, and I came across Phillip Eby's presentation here. This really hit home, since one of the projects I am working on is for processing *massive* amounts of data produced while testing equipment at manufacturing plants. There are sets of sets of sets of ... tests. Thousands of the buggers. Many of them are very similar in nature, and they all have associated rules for pass and failure.

I can't wait to tryout Phillip's generic functions in PyProtocol's... in fact, I haven't waited ;-) I'm already writing some prototype code for it...

Python rocks.