Saturday, March 25, 2006

Twisted JSON-RPC

My clients/partner companies use web services a great deal. To be honest, WS have made my life easier in many respects... but they can be a real pain. And implementations can be pretty lacking...

For reasons I won't get into (business, politics, and AJAX), I discovered today that I needed to convert an XML-RPC server to a JSON-RPC server. I looked at several implementations and they were either not general enough for my use, or they were horrible.

So, I hacked a copy of twisted.web.xmlrpc and turned it into jsonrpc using the simplejson library. Right now, it's doing JSON over HTTP, but I fully intend to write a TCP implementation as well. The problem, though, is this: as I was putting this together today, all I could think about was ways to make the code general enough to provide a common basis for use in implementing *-RPC. Ah, down that path lies madness. And it's one of those things you just can't avoid thinking about...

I'm currently writing some twisted.trial tests for it, but I also need to add some more stuff to make this generally useful (not to mention easier to debug). Hmmm, I'm actually really looking forward to doing a twisted TCP implementation of JSON-RPC. That should be fairly fast. And clean. Maybe.

And, of course, I'm sure I'll do a twisted.web2 implementation as well.

I have an amazing headache now, and need to get some food.

Update: All twisted.trial tests are passing and I am running a twisted JSON-RPC server now.

Update: For those wanting to use this code, please read the comments on this entry (dated May 5, 2006) and then see this post.

Technorati Tags: , , , , ,

3 comments:

  1. If you check the trunk of simplejson, I anticipated this use case a couple days ago and added some support that will make the class hinting on decode a lot more efficient (since you get a hook on object-decode, rather than having to walk the data structure afterwards).

    If you want to release the code, bug me and I'll make sure to bag and tag a simplejson release containing that feature if I haven't gotten around to it already.

    ReplyDelete
  2. Have you posted this code somewhere or has it gotten into the Twisted code base? I'm looking for a simple JSON-RPC and/or XMLHttpRequest implementation for Twisted but am not using Nevow.

    ReplyDelete
  3. Ron, let's see if I can answer your question...

    About web services and twisted in general: web services are not viewed in an entirely positive light by the twisted community ;-)

    About JSON-RPC: there's really no spec. There's a guy who has provided one, but parts of it are dependent upon the solution he has in mind (and has impemented). You simply don't need to do some of the things he's defined in the spec when you use a different appraoch (a more sensible approach, in my mind). So, there's no real spec yet, and there's no equivalent of python's stdlib xmlrpclib (which twisted.web.xmlrpc depends upon).

    I would only feel comfortable submitting the JSON-RPC code for inclusion in twisted if there was a standard or widely-used library upon which it was based.

    The JSON-RPC server library I wrote simply passes {'method': method_name, 'params': parameters} These's a simple check for errors, but that's about it. If I have the time, I will flesh out this nascent jsonrpclib. If someone else does, and does a good job, I will use that. twisted.web.xmlrpc imports and makes use of xmlrpclib. My twisted jsonrpc code would similarly make use of such a library, swaping my current simplisitc json data and checks with those of a more robust jsonrpc implementation. What I'm trying to say is that the twisted code is good and solid -- for JSON-RPC over HTTP, it shouldn't need to change at all. But we all need a jsonrpclib for this to really be robust.

    Oh, and another complication with inclusion into twisted: what about JSON-RPC over TCP? There are a whole set of twisted project (code, goals, standards, structure, etc.) qustions that this raises and would not be easily addressed. What's more, it's actually trivial to implement whatever XML-RPC analog one might need in twisted. (I've recently added a highly-customized RPC functionality to a nevow-based project as a child of the main nevow "page.") So, from the eyss of the core twisted crew, it might not seem such a pressing thing to include, since it's so easy to integrate as-needed. With increased popularity and use cases, however, the likelihood of inclusion increases.

    At any rate, I will post an update to this blog entry with a URL for downloading an .egg I built.

    ReplyDelete