Phillip Eby has changed my life... again.

python :: programming

First it was PEAK, then it was PyProtocols/generic functions (which has
now been split from PyProtocols in to its own package called
RuleDispatch). Now, it's EasyInstall. Everyone has been raving about
it, but I hadn't tried it out until tonight. Holy crap. This is crack
for python programmers. I built my first python egg when Bob Ippolito
first started writing about the alpha code he and Phillip Eby were
working on. Pretty cool stuff, but it didn't really hit me as to how
great eggs were until I actually saw it in action today with my upgrade
of setuptools, my first use of EasyInstall, and my upgrade of
PyProto/RuleDispatch (each creates an egg and dumps the file in
site-packages).

It blew my mind when I saw these two luminaries getting together. I
knew that it didn't matter what the hell these guys did together, it
would rock the world. Awesome. And thanks :-)

Solaris Returns?

operating systems :: solaris

Since moving to Colorado, I have had many friends visit... and now
family members are signing up. This is actually very nice for me, as I
have always been the one visiting. People *like* coming to Colorado,
hiking in the mountains, breathing the incredible air at high
altitudes. However, every single commodity-based workstation I have
(and I have over 10 of these boxes) has failed. These two topics are
actually related.

Most of the x86 machines I have, I did not purchase new. All of them
have had replacement parts, after the originals had burned out or
become obsolete. And all have since died, in some fashion or another.
In fact, the only exception to this are two machines I have: a Gateway
PC from 1997 with most of it's original hardware running FreeBSD (with
3 extra NICs) as my firewall/router and a Gateway laptop from 1998.

As a result of all these dead machines, there are no machines available
for use by all the people visiting -- there are two Mac OS X laptops
for use, but I am on my 17" almost 90% of the time I am awake, and
there are often more than 3 people in the house. This means that 2 or
more people have to juggle one 15" PowerBook. I have considered buying
a Mac mini, but with the changes of architecture at Apple, I am
exploring other options (I *will* be buying a 17" x86 PowerBook).

One of those is Sun. In a past life, I was a systems administrator and
worked a great deal with Sun hardware and was pretty familiar with
SunOS and intimately familiar with Solaris 2.5 and up. Last night, I
discovered (a month late) that Solaris has been open sourced. I
remember discussing this possibility with fellow admins and developers
in the late 90s, where a few of us optimists were called crazy for
thinking this was a possibility. I had since given up hope, and haven't
worked with Solaris in about two years, and then only in passing.

I spent several hours yesterday reading developer blogs and browsing
the site
(http://www.opensolaris.org/os/),
and I'm getting pretty
excited about the potential of this move by Sun. It is very likely that
this could move me back into not only using Solaris machines again, but
brushing up on my Solaris sysadmin skills, particularly catching up
with all the work they've done on virtual server technology. Running
python application servers in Solaris virtual machines will provide
many excellent solutions for application architectures.

Anyway, Sun is offering a "free" Opteron Sun box, with a 3-year support
subscription for about $30/month, and I'm heavily leaning towards
purchasing this box to fill the machine gap in the house:

http://tinyurl.com/cf4j2

If that goes well, and I like it, I may get another one for a local
subversion repository/backup development server... If their new
hardware is as solid and reliable as their old stuff (I have a
still-working Sparc server from 1995 that I bought from a DiGEX
employee -- it was the original web server for Dole.com), then I will
be most happy and they will have a new hardware customer for developer
platforms.

Tibetan Village Project

tibet :: culture :: society :: internet

A good friend of mine, Tamdin Wangdu, started and currently manages a
non-profit called the
Tibetan Village
Project. I have talked with Tibetan friends and Westerners (Injees)
about a concept that is tangentially related: the preservation of
Tibetan culture in the West by means of establishing a functional,
thriving, "organic" Tibetan village. The architecture we are planning
is 100% authentic traditional. Tibetan monks and laity are providing
time, drawings, ideas, planning, and resources. It's getting exciting.

Tremendous resources would have to be poured into this. We're looking
at all kinds of options, from forming several companies that would
employ members of the village and whose profits would be put into
village infrastructure and support, to artisans and farmers.

The village would be comprised of Tibetans and Injees, both. Monks and
laity. Babies growing up, learning Tibetan and English fluently; the
culture and history of Tibet and the West being taught simultaneously.
The idea being that Tibetans would evolve from
refugees to a people with a new home, an old, preserved culture, and a
new integration.

But a group of people living together tends to have limited wide-spread
impact. We want this to be something that can be easily duplicated; we
want this idea to catch hold in the hearts and minds of others. We are
therefore going to publish ALL documents related to this project and
organize them for easy navigation and quick access of particulars.

This is a start... but we need to actively encourage and participate in
the wide-spread preservation of Tibetan culture. The first idea I had
regarding this was rather silly (but I still like it): to provide
streaming images and an application that could make these streamed
images the desktop background image on Mac OS X, Linux/*BSD, or even
*shudders* Windows workstations. Similarly for screensavers. The
streamed images would be taken from massive Tibetan digital archives
that detail every aspect of Tibetan life and culture.

The second idea I had is 100% inspired by BBC Radio Scotland. And yes,
we Scots are refugees ourselves -- just ask us! (Especially the
MacGregors...). BBC Radio Scotland is an amazing online resource for
both real-time and pre-recorded Scottish programs. Very well organized,
very simple to navigate, and a wide selection of modern Scottish
culture. It provides a perfect template or starting point for doing
this with any culture.

Imagine a Tibetan village in rural, mountain America. A village where
several clever and community-owned companies employ villagers; where
there is a focus on sustainability and cooperation; where there is a
very good broadband connection, with regular shows broadcast on the
internet live and archived for later listening. Shows of traditional
Tibetan music, world, national, local, and village news in Tibetan;
Tibetan news in Tibetan and English; shows discussing the cutting edge
politics, political research, self-governance, sustainable living, and
the Tibetan Village project; history shows; fictional story shows
presenting Tibetan folklore; various dharma talks and instruction by
monks and nuns; Tibetan language classes that are broadcast, as well as
English-for-Tibetans classes. Every member of the village would be
actively involved in preserving culture and spreading it; greatly
decreasing any possibility of the current trend: its complete
annihilation.

The list could go on and on...

And this is technically and exciting project: I see visions of twisted
python streaming media code :-) Twisted Oog Radio player anyone? Raise
your hand if you're sick of RealMedia and WMP? Yes... this could be
very cool.

Update

I didn't make it to pycon this year, and I just found some good links
on streaming media with twisted, including a presentation at PyCon2005:

http://twistedmatrix.com/pipermail/twisted-python/2005-March/009730.html ([Twisted-Python] streaming producer)

http://twistedmatrix.com/pipermail/twisted-python/2004-October/008846.html
([Twisted-Python] First public release of Flumotion Streaming
MediaServer)


http://www.flumotion.net/


http://www.flumotion.net/doc/flumotion/presentation/pycon2005/html/img0.html

Thinking in Twisted

I've been emailing with a fellow developer and friend about learning to write applications and parts of applications using the Twisted networking framework. Even though Twisted is written in Python, conceptually -- and from an abstract point of view -- it is like a language of its own, with a foundation, syntax, and grammar that is Python.

My friend was trying to make use of basic Twisted stuff like deferreds, combine that with other parts of Twisted, but traverse the problem-solving process from an essentially ground-up, Python approach. I had tried to provide the shift in perspective needed in order to write Twisted apps with much less effort than he was exerting, but I fear that I failed.

Then, this morning, I thought of the perfect analogy. Actually, it's more than an ananlog -- it is the thing itself. Since really starting to learn Twisted, my programming has changed. Well, that and since absorbing such books as Patterns of Enterprise Application Architecture and Refactoring to Patterns. The deal with writing applications in Twisted is that you're no longer programming in the same language (Python); you're learning a new language. A more abstract one.

When we learn to think in Twisted and write applications in Twisted, our solutions will be elegant and compact. Before that starts to happen, we try to use our "old" ways of Python programming mixed in with the new one we are learning... and that, of course, defeats the purpose of the new "language."

The tricky thing about Twisted is that its patterns are an integral part of its "language" (perhaps it would be better to stop using quotes, and just call it a meta-language...). This is not true of most other languages -- though I would venture to say it is true of good frameworks in general. You don't need to know the patterns to use the grammar/syntax/control structures of most languages. Patterns in a language are usually optional.

With Twisted, if you aren't using the patterns, you aren't really using Twisted.

I am midway through this learning process. Therefore, I can't point to the solutions for many Twisted problems immediately or directly. I have, however, learned to see most of the critical components of a Twisted implementation. I still miss a few, though, and I have to keep reviewing code until I catch them all. Seeing these components clearly is what leads to the solution. As the time between "component vision" and "solution" gets shorter and shorter, we come closer and closer to thinking in Twisted.

pymon Nearing 0.3.3 Release

software

pymon is nearing its second public release (but first packaged
release). We've spent the past couple months doing a refactor by
examining the patters we were using or not using. In the process, we
have made MUCH better use of twisted's internals and are increasingly
more impressed with it as a networking application development
framework. To be honest, I would be very happy doing full-time twisted
development now.

Email alerts are turned on; ping hosts are running; HTTP status, HTTP
text checks, DNS dig checks, and MySQL admin queries are currently
being developed. We're going to try to knock out FTP and SMTP in the
next couple weeks, as well -- and then we'll bundle it up and release
it :-)

A tentative road map is up on the wiki, and we've got the beginnings of
developer docs on how to write "monitors" (like ping checks, HTTP
status checks, etc.).

http://pymon.sf.net

Vacation

leisure :: nature

So, even though it's been a week, I still haven't blogged about my
mini-vacation. I didn't go anywhere and I still had to do a little
work, but I did get to relax and hang out with a really good friend who
was visiting from out of town. She had her Spring break from
acupuncture school, and for the second half she came out to Colorado to
chill in the Rockies. We had the absolute best time; a much needed
vacation for both of us.

I swear, I will never get tired of the Rocky Mountains. The most
amazing vistas, trails, weather, etc. Just incredible. We went
horseback riding through a place in Estes Park, CO that does rides 365
days a year. It was such a blast. We were out in the afternoon sun on a
gorgeous day for over two hours. The best views I have seen since I've
been here (with the possible exception of a couple amazing sights inside
Rocky Mountain National Park).

For most of the trip, we vegged, watched movies, went window shopping
in Boulder, listened to music and talked. A genuine vacation. Not one
of those "run around and do all you can do till you pass out and need
another vacation" vacations. Rather, a time to really relax and
breathe. Also, her last night here, I introduced her to my Tibetan
friends at the weekly momo-making party. She had the absolute best
time. All of us laughed, ate, drank, and laughed some more. In fact, we
laughed so hard we were actually in pain. She got to listen to lots of
Tibetan and build up her ear for it. I taught her a little and
translated a little, but my understanding is so limited that it's not
much use. On the ride back to Loveland, we chatted away like a couple
prairie dogs about the Tibetan culture, their deep understanding of and
commitment to community, about how this impacts our own experiences,
understanding, and limitations, etc. It was an awesome time :-)

Accounting

accounting :: software :: society

So, I'm preparing for taxes (after my extension, of course). I've been
using this open source software called SQL-Ledger (written in perl,
unfortunately) to do my books this year; it's a double entry accounting
package. It's ugly as hell (I've made improvements -- css and python),
but I love it. I emailed my accountant with some double entry questions
(checking to see if the lights really are dawning), and I decided to do
some checking on the internet, where I found this gem I thought I'd
share ;-)

This is the first thing I've ever read that has made me not only feel
good about accounting, but want to participate in the process :-)
We'll, the second thing, actually. The first thing was starting to use
SQL-Ledger to it's capacity.

It's kinda fun:

3.1 The history of double entry accounting

People had been tracking money (to pay bills, collect taxes) for
thousands of years, but it wasn't till the 1400's that the Italians
invented double entry accounting. This made banking reliable, enhanced
trade and commerce and very quickly Italy became the banking capital
and wealthiest country in Europe. It was the 1400's equivalent of the
invention of the internet and the dotcom boom. They had buckets of
money to spend. They spent it on paintings of Adam talking to God done
on black velvet. Well they would have but velvet hadn't been invented
yet. They made do with what they had, which was plaster ceilings. They
spent money on painters, sculptors and people like Leonardo da Vinci.

In grade school this period is called the Renaissance, a flowering of
art and intellect, that appeared in Italy for no obvious reason and
then spread to the rest of Europe.

Quite why all these geniuses suddenly appear without any warning is not
explained by your history teachers (who spend their life pondering deep
questions like this), but Jared Diamond (Guns, Germs and Steel, Pub:
Norton 1997) is happy to tell you. He says that geniuses like
Michealangelo and Einstein are rather commonplace. Most of them are
oppressed and are living in abject poverty, and are busy surviving if
they even do that. Give them a good feed, treat them well, put them in
the company of peers and pretty soon they'll be coming up with all
sorts of things you hadn't dreamed of.

The Renaissance then was a result of the invention of double entry
accounting, not (as we've been told) a flowering of intellect and art
that happened for no reason at all.

Reference:
http://www.austintek.com/gnucash/ncsa-
gnucash-talk-3.html

pymon

python :: programming :: software

As so often happens, when working/playing on one thing, the miraculous
inner lights of the frail but mysterious human mind shed their
brilliance on another. Okay, so perhaps in this instance the areas are
not so far separated, nor the lumens so brilliant as those in a dying
plastic camping flashlight. Nonetheless, the fancy is tickled, and
besides actual development, that's what matters!

So what the hell am I talking about? APIs. I was working on the PyKon
Nevow code the other day, and some things I had written and prototyped
fermented in a corner of my brain, and popped into a completely new
perspective today, one that I could use with
pymon. I am addicted to good APIs. I
study twisted's internals as well as that of Phillip Eby's PEAK
framework(s). I hunger for elegant, efficient, and clever design
strategies. It's very Newtonian of me, but I love well-ordered systems.
The discussions that have been underway on the pymon maillist boiled
down (very quickly) to the following:

* The need for several mini-APIs to be used internally by pymon


* The need to define user- and/or enternal-APIs


* The need to get rid of a little cruft that had accumulated while in
branch (both old an new)


* The need to utilize twisted component-based approach (adaptors) to
accomplish these goals

We're really excited about this, because we're trying to be true to the
art of programming. QualityRUs, not time-contraint, budget-driven,
happenstancery. I wrote some more about it on the 2005.04.06 news item
at http://pymon.sf.net. It's going to
be fun writing all the adaptors we will need :-)

PyKon

I got some more work done on PyKon last night after finishing a deadline for a client. I'm excited about having a modern, functional, python-based image gallery application. The first version will allow both:

• manual image uploads (ftp/scp, i.e., changes you would make on the file system with a shell account) and the daemon that runs the web user interface will also regularly check for gallery/album/image changes and process added/changed files), as well as
• through-the-web uploads.

For the second release, I want to enable some work we've been doing on an object registry. This would essentially make PyKon like iPhoto for Mac OS X: a global repository for images, and the capability to create albums with multiple overlapping images (duplicates) with no extra space taken up by the images (as they would only be references/links to the image tagged in repository).

We're hoping to make this better than IDS (written in perl), Gallery (in PHP), and JAlbum (in java). We really want this thing to kick ass and be very easy to use. We're a ways off, but progress so far is exciting.

Oh, and Nevow (for Twisted) rocks.

Pete Yorn

music :: leisure

I am really behind the times. I discovered Pete Yorn only yesterday. I
was watching "Me, Myself, & Irene" with my mother last night. We were
both in a punchy mood, just having watched Bruce Campbell's "Alien
Invasion", so we laughed ourselves silly at Jim Carey's schizoid
antics. At some point in the movie near where they met Whitey, the
coolest song started playing. I've owned this movie since it came out,
and had never heard this song before. Well, it turned out it was the
song "Strange Condition" by Pete Yorn.

So I got a couple of his CDs, and holy crap -- this guy's good! He
reminds me of Elliot Smith, some classic REM, Counting Crows and a
little Grant Lee Buffalo. And sometimes you can hear a little Robert
Smith (The Cure). I feel like one of those dorks in "High Fidelity"
now... "She's kind of a
Sheryl-Crow-meets-a-post-Partridge-Family-pre-LA
Law-Susan-Day...only...black."