Tuesday, November 11, 2008

Python and Lisp... Again

It seems that Lisp continually comes up in various conversations (virtual and otherwise) in the context of Python. In fact, maybe we could even call such occurances the Python-Church conversations. Well, here it is again.

Earlier this year I started working on a new project: an object-oriented genetic programming library. I had a bunch of experiments I wanted to do, but I needed to assemble parts of programs in order to do it. I had hoped to use Python, but inspecting Python's AST ended up being too much of a hassle. I wanted to distribute, process, and manage evolutionary algorithms/programs across multiple remote Twisted servers, and manipulating permutations of partial programs would be much easier to integrate with Twisted (the target "platform") if the programs themselves could be evaluated and introspected easily with Python.

After some digging around, I eventually settled on using PyLisp, mostly for the simplicity of the code and the fact that it was just a single file. Since it hadn't been maintained since 2002, I decided to roll the original file into the genetic programming code and then apply any changes as-needed, over time.

More recently, I've wanted to use this modified PyLisp on other projects and as a result, I have split it out into it's own project: pyLisp-NG. This naturally led to further code break-out, for a total of three projects:
  • pyLisp-NG - the functional programming (and introspection) component of the original project
  • Evolver - the code that allows one to do Python-based evolutionary programming (string-based as well as source code tree node optimization/search solution discovery)
  • txEvolver - will enable users to distribute genetic programming operations (such as merging parallel generations of computations)
pyLisp-NG was released earlier today and is available for download on PyPI.


Anonymous said...

Hi Duncan!

That's all very interesting. My phd was on evolutionary algorithms :-)

I hope one day we'll reach the point where we can use some of that in what we're building. Mainly in running offline optimizations to decide where to locate resources. But, as with many optimization problems, it's likely that a cheap/simple/fast/greedy algorithm will get us most of the way. But it's fun to find out in any case.


Duncan McGreggor said...

Oh, yeah! I remember you mentioning that, now. How cool! Had the concept of schemata gained wide enough interest to come up in your studies? I'm looking forward to working more with them at a practical level.

I love this suff, man. I can't get enough of Koza. I'm anticipating the day when mutation and crossover for arbitrary Lisp programs is working beautifully... 'cause then I'm going to move back to coding on txEvolver, remotely distributing population generations and cross-breeding between them :-)