Cruising HTTP with LFE

In the last post, you learned how to get LFE running on Ubuntu. This one will give you some insight into how LFE can be used in something approaching real-world problems. In the next post, we're going to jump back into the lambda calculus, and we'll see some more LFE shortly after that.

Because Lisp Flavored Erlang is 100% compatible with Erlang Core, it has access to all the Erlang libraries, OTP, and many third-party modules, etc.  Naturally, this includes the Erlang HTTP client, httpc. Today we're going to be taking a look at how to use httpc from LFE. Do note, however that this post is only going to provide a taste, just enough to give you a sense of the flavor, as it were.

If you would like more details, be sure to not only give the official docs a thorough reading, but to take a look at the HTTP Client section of the inets Reference Manual.

Note that for the returned values below, I elide large data structures. If you run them in the LFE REPL yourself, you can view them in all of the line-consuming glory.

Synchronous GET

Let's get started with a simple example. The first thing we need to do is start the inets application. With that done, we'll then be able to make client requests:

	> (: inets start)
	>

view raw http-sync-1.lfe.lisp hosted with ❤ by GitHub

Now we can perform an HTTP GET:

	> (set
	(tuple 'ok result)
	(: httpc request '"http://lfe.github.io"))
	#(ok
	#(#("HTTP...
	>

view raw http-sync-2.lfe.lisp hosted with ❤ by GitHub

This just makes a straight-forward HTTP request (defaults to GET) and returns a bunch of associated data:

• HTTP version
• status code
• reason phrase
• headers
• body

All of that data is dumped into our result variable. Here's the same GET but with pattern matching set up so that we can easily access all that data:

	> (set
	(tuple 'ok
	(tuple (tuple http-version status-code reason-phrase)
	headers body))
	(: httpc request '"http://lfe.github.io"))
	...
	>

view raw http-sync-3.lfe.lisp hosted with ❤ by GitHub

For those not familiar with Erlang patterns, we've just told LFE the following:

• the return value of the function we're going to call is going to be a tuple composed of an atom ('ok) and another tuple
• the nested tuple is going to be composed of a tuple, some headers, and a body
• the next nested tuple is going to be composed of the HTTP version, status code, and status code phrase

If you'd like to learn more about using patterns in LFE, be sure to view the patterns page of the LFE User Guide.

Once the request returns, we can check out the variables we set in the pattern:

	> http-version
	"HTTP/1.1"
	> status-code
	200
	> reason-phrase
	"OK"
	> headers
	(#("cache-control" "max-age=600")
	#("connection" "keep-alive")
	#("date" "Sun, 14 Apr 2013 03:43:17 GMT")
	#("accept-ranges" "bytes")
	#("server" "GitHub.com")
	#("vary" "Accept-Encoding")
	#("content-length" "13430")
	#("content-type" "text/html")
	#("expires" "Sun, 14 Apr 2013 03:53:17 GMT")
	#("last-modified" "Sat, 13 Apr 2013 18:28:49 GMT"))
	> (: string len body)
	13430
	>

view raw http-sync-4.lfe.lisp hosted with ❤ by GitHub

That's great if everything goes as expected and we get a response from the server. What happens if we don't?

Well, errors don't have the same nested data structure that the non-error results have, so we're going to have to make some changes to our pattern if we want to extract parts of the error reason. Pattern matching for just the 'error atom and the error reason, we can get a sense of what that data structure looks like:

	> (set
	(tuple 'error reason)
	(: httpc request '"http://localhost:8000"))
	#(error
	...
	>

view raw http-sync-5.lfe.lisp hosted with ❤ by GitHub

Looking at just the data stored in the reason variable, we see:

	> reason
	#(failed_connect
	(#(to_address #("localhost" 8000)) #(inet (inet) econnrefused)))
	>

view raw http-sync-6.lfe.lisp hosted with ❤ by GitHub

If you check out the docs for httpc request and look under "Types", you will see that the error returned can be one of three things:

• a tuple of connect_failed and additional data
• a tuple of send_failed and additional data
• or just unspecified additional data

In our example our additional data is a tuple of the address we were trying to connect to and the specific error type that for our failed connection.


Async GET

Now that we've taken a quick look at the synchronous example, let's make a foray into async. We'll still be using httpc's request function, but we'll need to use one of the longer forms were extra options need to be passed, since that's how you tell the request function to perform the request asynchronously and not synchronously.

For clarity of introducing the additional options, we're going to define some variables first:

	> (set method 'get)
	> (set url '"http://lfe.github.io")
	> (set headers ())
	> (set request-data (tuple url headers))
	> (set http-options ())
	> (set request-options (list (tuple 'sync 'false)))
	>

view raw http-async-1.lfe.lisp hosted with ❤ by GitHub

You can read more about the options in the httpc docs.

With the variables defined, let's make our async call:

	> (set (tuple 'ok request-id-sender)
	(: httpc request method request-data http-options request-options))
	#(ok #Ref<0.0.0.230>)
	> request-id-sender
	#Ref<0.0.0.230>
	>

view raw http-async-2.lfe.lisp hosted with ❤ by GitHub

The sender receives the results, and since we sent from the LFE REPL, that's the process that will receive the data. Let's keep our pattern simple at first -- just the request id and the result data:

	> (receive ((tuple 'http (tuple request-id-receiver result))
	(: io format '"data: ~p~n" (list result))))
	data: {{"HTTP/1.1",200,"OK"},
	...
	>

view raw http-async-3.lfe.lisp hosted with ❤ by GitHub

Needless to say, parsing the returned data is a waste of Erlang's pattern matching, so let's go back and do that again, this time with a nice pattern to capture the results. We'll need to do another request, though, so that something gets sent to the shell:

	> (set (tuple 'ok request-id-sender)
	(: httpc request method request-data http-options request-options))
	#(ok #Ref<0.0.0.242>)
	>

view raw http-async-4.lfe.lisp hosted with ❤ by GitHub

Now we can set up a pattern that will allow us to extract and print just the bits that we're looking for. The thing to keep in mind here is that the scope for the variables is within the receive call, so we'll need to display the values within that scope:

	> (receive ((tuple 'http
	(tuple request-id-receiver
	(tuple (tuple http-version status-code reason-phrase)
	headers body)))
	(: io format '"data: ~p ~p ~p ~n"
	(list http-version status-code reason-phrase))))
	data: "HTTP/1.1" 200 "OK"
	ok
	>

view raw http-async-5.lfe.lisp hosted with ❤ by GitHub

This should demonstrate the slight differences in usage and result patterns between the sync and async modes.

Well, that about sums it up for an intro to the HTTP client in LFE! But one last thing, for the sake of completeness. Once we're done, we can shut down inets:

	> (: inets stop)
	ok
	>

view raw http-async-6.lfe.lisp hosted with ❤ by GitHub