OpenStack Developer Summit: Heat Followup

Folks are finally starting to recover from the OpenStack Developer Summit that was held in Portland, Oregon recently. All reports indicate that it was a truly phenomenal experience, record-breaking in many ways, and something that has inspired incredible enthusiasm within the community. And that's great news, since there's an enormous amount of work to be done this release ;-)

Of particular importance to many in the community is the work around maturing the autoscaling feature in OpenStack Heat. There was a fantastic session at the summit, facilitated by the bow-tied and most dapper Ken Wronkiewicz (his notes from the Summit were published on the Rackspace blog).

In preparation for the session, the following resources were created:

• https://blueprints.launchpad.net/heat/+spec/heat-autoscaling
• https://etherpad.openstack.org/heat-autoscaling
• http://summit.openstack.org/cfp/details/172

That one in the middle is important, as it is also where notes were taken during the actual session itself (see the section entitled "ODS Session Notes"). Devs at Rackspace have started going through the notes from the session and started planning work around this -- all of which will be carried on in the open, on the OpenStack mail list (tagged with "[Heat]"), on Freenode, and on github/gerrit.

The discussion at the Summit indicated strong interest in building a REST API for the existing autoscaling feature. Needless to say, there is a lot involved in this, touching upon significant OpenStack components like Quantum, LBaaS, and Ceilometer. Once the appropriate code is in place, a REST API will need to be created, features will need to be expanded/added, etc., and we'll be off and running :-)

Lots to do, and lots of great energy and excitement around this to keep us all chugging through this cycle.

On that note, we'd like to send out a special "thanks" to all the countless folks who worked so hard to make ODS happen. This event anchors us in a most excellent way, providing the insight and fuel that supports future development work so well!

Tea Time at Rackspace SF

One of the Rackspace teams here in the SF office doesn't to standups; it does "tea time" instead. A delightful change, to be sure. One of my coworkers published a Rackspace blog post about this today.

More than a gratutious reblog, I wanted to highlight this bit of lovely coincidence I came across while reading Turing's Cathedral last night:

"Afternoon tea— a ritual introduced at Fine Hall by Oswald Veblen, who, according to Herman Goldstine, “tried awfully hard to be an Englishman”— was served on real china daily at exactly three o’clock. According to Oppenheimer, “tea is where we explain to each other what we do not understand.”

Dyson, George (2012-03-06). Turing's Cathedral: The Origins of the Digital Universe (p. 90). Knopf Doubleday Publishing Group. Kindle Edition. 

The part of the book where this quote occurred was discussing Fine Hall which was built for Von Neumann's computer team at the Princeton Institute for Advanced Studies.

And, yes, tea time is a great time for our team mates to explain to each other what they don't understand.

In case you missed it above, here's the link again:

http://www.rackspace.com/blog/tea-time-because-we-wont-stand-for-a-standup/

Erlang Meetup at Rackspace, San Francisco

I'm very pleased to announce that Rackspace's San Francisco office will be hosting an Erlang Meetup immediately after the closing session of last day of SF Erlang Factory on 22 March.

We are honored to announce that both Robert Virding and Francesco Cesarini will be present for the event. The night will open up at 6:30pm with Mediterranean food, refreshments (and I hear there will be a keg). Around 7:15pm we will gather for a presentation by the very talented folks at Boundary which will start at 7:30pm.

The Boundary presentation will be followed by one from Robert Virding. We'll finish the night with a door prize give-away of two copies of Francesco and Simon's book Erlang Programming: A Concurrent Approach to Software Development. Perhaps the winners might be able to talk Francesco into signing their copies? ;-)

Last but not least, some of Rackspace's own Erlangers will be on hand for conversations about how we're using Erlang/OTP to deliver Fanatical Support in in our Data Center and Cloud Ops groups.

We're going to start with a cap of 60 people for the event, but if there's high demand, we have room to adjust this. Due to some communications hiccups, we're going to do the planning for this event the old-fashioned way: send an email to duncan.mcgreggor@rackspace.com, and let me know that you'd like to attend. Your name will be added to the list in the order I receive emails. The list of attendees and the waiting list are published here and will be refreshed with each new addition.

Update! The event is now published on meetup.com -- Thanks for all your help, Andra!

As special thanks go out to Robert Virding, Phil Toland, and Brian Troutwine -- conversations with them were the inspiration for putting this event together. Thanks guys! (Note that I've put that background info into its own blog post, so as not to confuse the event announcement ... to much ;-) ).

See you at Erlang Factory SF and then on Friday at the Rackspace office!

Lisp Flavored Erlang

I've flirted with Lisp since the 90s, really started getting into it around 2008 when I started playing with genetic programming, and more recently investigated Common Lisp, Scheme (Gambit and Chicken), and Clojure with the intent of discovering the best way to write distributed programs in Lisp. (I even seriously explored implementing chunks of Twisted in Common Lisp. If only I had more time...)

Needless to say, I kept coming back to Erlang as it is a natural choice for the sort of concurrent programming I was interested in doing. On several occasions, I'd run across Robert Virding's Lisp-2 that he had written on top of the Erlang VM. At first blush, this appeared quite promising. Yet faced with the perceived burden of learning Erlang, I consistently set it aside for a future date.

"Excuse me, could I have some Erlang, please? Yes, just a cup. Oh, and can I get that Lisp-flavored? Thanks so much."

After bouncing between Clojure and CL, after running into difficulties with Termite on Chicken Scheme, and finally, after being quite impressed with the efforts made by the Elixir folks (who I believe took inspiration from LFE!), I decided to give LFE a chance. Within minutes of that decision, I came to two conclusions:

1. LFE is brilliant.
2. LFE needs docs like Elixir... and tutorials... and exposure! Why haven't I been using LFE all along?!

At which point, I started hacking on some docs to see if I could stick with it. When, after a few days, I proved to myself that I could, I contacted Robert and let him know not only how much I adored his masterpiece, but that I really wanted to write tons and tons of docs for it so that anyone could pick it up and start using it right away. I wanted to write docs for an audience like me, that didn't know Erlang, who weren't Lisp gurus.

This seemed like a doable goal, since I had about 5 minutes' worth of Erlang experience at the time I was having these conversations with Robert. I was learning Erlang at a rapid pace simply by virtue of the Lisp hook that LFE provided.

Our interactions led to the publicizing of the new docs site for Lisp Flavored Erlang on the LFE google groups list. We also created a Twitter account (we both have full access to it, but I tend to maintain it) whose sole purpose is to bring LFE to more people, keep the news around LFE fresh, etc.

"I could have sworn you just said 'Lisp'..."

A side note about Lisp: S-expressions are concise and elegant. Code and data using the same form is quite powerful. I do believe that the technology industry has the capacity to realize that old biases against Lisp are just that: old and outdated. The many dialects of Lisp are anything but. Clojure and (I believe) LFE are perfect examples of this. Whole new generations of programmers are delighting in the expressive power of a language whose roots can be traced back to actual manipulations of memory registers.

To resume the narrative: in the course of various efforts focused on documenting LFE, asking questions on the mail list, and having various other discussions, Robert pointed out that some of my coworkers at Rackspace had been working on Erlang projects. I subsequently reached out to Phil Toland. Then, within minutes of this (and entirely coincidentally), Kai Janson emailed a group of us about Erlang Factory SF and his desire to provide Erlang workshops for engineers at Rackspace.

This led to further conversations with Robert, then with Francesco, with several Rackers signing up for Erlang Factory this year, and finally, with me volunteering to put a Meetup together afterwards, hosted at Rackspace's SF office (more on that in a few hours).

For the curious, I do continue to work in Python and Twisted; I am excited about the new async support that Guido is spearheading for Python 3 and which has electrified so many hard-core Python hackers. Similarly, I continue to hack on Common Lisp projects. However, I am quite delighted that I have found a way to interface with Erlang which matches how I think, matches my aesthics. And finally, I look forward to many fruitful years of LFE in my life :-)

Thanks Joe! Thanks Mike! Thanks Robert!

Async in Python 3

Update: Guido has been working on PEP 3156; check on it regularly for the latest! (In the last two hours I've seen it updated with three big content changes.)

The buzz has died down a bit now, but the mellowing of the roaring flames has resulted in some nice embers in which an async for Python 3 is being forged. This is an exciting time for those of us who 1) love Python and 2) can't get us enough async.

I wanted to take the time to record some of the goodness here before I forgot or got too busy working on something else. So here goes:

The latest bout of Python async fever started in September of 2012 in this message when Christian M. Amsüss emailed the Python-ideas mail list about the state of async in Python and the hopes that a roadmap could be decided upon for Python 3. Note that this is the latest (re)incarnation of conversations that have been going on for some time and for which there is even a PEP (with related work on github).

After a few tens of messages were exchanged, Guido shared his thoughts, starting with:

This is an incredibly important discussion.

This seemed to really heat things up, eventually with core Twisted and Tornado folks chiming in. I learned a tremendous amount from the discussions that took place. There's probably a book deal in all that for a motivated archivist/interviewer...

After this went on for chunks of September and October, Guido stated that he'd like to break the discussion up into various sub-topics:

• reactors
• protocol implementations
• Twisted (esp. Deferred)
• Tornado
• yield from vs. Futures

This was done in order to prevent the original thread from going over 100 messages and to better organize the discussion... but wow, things completely exploded after that (in good ways. mostly). It was async open season, and the ringing of shots in the air seemed continuous. If you scroll to about the half-way point of the October  archive page, you will see the first of these new threads ([Python-ideas] The async API of the future: Reactors). These messages essentially dominate the rest of the October archives. It's probably not unexpected that this continued into November. A related thread was started on Python-dev and it seemed to revive an old thread this month (on the same list).

All of this got mentioned on Reddit, too. It inspired at least two blog posts of which I am aware:  one post by Steve Dower, and another by Allen Short. Even better, though, Guido started an exploratory project called Tulip to test out some of these ideas in actual running code. As he mentions in the README, a tutorial by Greg Ewing was influential in the initial implementation of Tulip and initial design notes were made in the message [Python-ideas] Async API: some code to review.

Shortly after that, some of the Twisted devs local to Guido met with him at his former office in San Francisco. This went amazingly well and revolved mostly around the pros and cons of separating the protocol and transport functionality. Guido started experimenting with that in Tulip on December 6th. Yesterday, a followup meeting took place at the Rackspace office, this time with notes.

There's a long way to go still, but I find myself compulsively checking the commit log for Tulip now :-) It's exciting to imagine a future where Twisted and Tornado could easily interoperate with async support in Python 3 with a minimum of fuss. In fact, Glyph has already sketched out two classes which might be all that's needed for 2-way interoperation between Twisted and Python 3.

Here's to the future!