Eventlet A better way to write network applications

One of the simple user errors that keeps on cropping up is accidentally having multiple greenthreads reading from the same socket at the same time.  It’s a simple thing to accidentally do; just create a shared resource that contains a socket and spawn at least two greenthreads to use it:

 import eventlet
 httplib2 = eventlet.import_patched('httplib2')
 shared_resource = httplib2.Http()
 def get_url():
     resp, content = shared_resource.request("http://eventlet.net")
     return content
 p = eventlet.GreenPile()
 p.spawn(get_url)
 p.spawn(get_url)
 results = list(p)
 assert results[0] == results[1]

Running this with Eventlet 0.9.7 results in an httplib.IncompleteRead exception being raised. It’s because both calls to get_url are divvying up the data from the socket between them, and neither is getting the full picture.  The IncompleteRead error is pretty hard to debug — you’ll have no idea why it’s doing that, and you’ll be frustrated.

What’s new in the tip of Eventlet’s trunk is that Eventlet itself will warn you with a clear error message when you try to do this. If you run the above code with development Eventlet (see sidebar for instructions on how to get it) you now get this error instead:

RuntimeError: Second simultaneous read on fileno 3 detected.  Unless
 you really know what you're doing, make sure that only one greenthread
 can read any particular socket.  Consider using a pools.Pool. If you do know
 what you're doing and want to disable this error, call
 eventlet.debug.hub_multiple_reader_prevention(False)

Cool, huh? A little clearer about what exactly is going wrong here. And if you really want to do multiple readers or multiple writers on the same socket simultaneously, there’s a way to disable the protection.

Of course, the fix for this particular toy example is to have a single instance of Http() for every greenthread:

 import eventlet
 httplib2 = eventlet.import_patched('httplib2')
 def get_url():
     resp, content = httplib2.Http().request("http://eventlet.net")
     return content
 p = eventlet.GreenPile()
 p.spawn(get_url)
 p.spawn(get_url)
 results = list(p)
 assert results[0] == results[1]

But you probably created that shared_resource because you wanted to reuse Http() instances between requests. So you need some other way to sharing connections. This is what pools.Pool objects are for! Use them like this:

 from __future__ import with_statement
 import eventlet
 from eventlet import pools
 httplib2 = eventlet.import_patched('httplib2')

 httppool = pools.Pool()
 httppool.create = httplib2.Http

 def get_url():
     with httppool.item() as http:
         resp, content = http.request("http://eventlet.net")
         return content

 p = eventlet.GreenPile()
 p.spawn(get_url)
 p.spawn(get_url)
 results = list(p)
 assert results[0] == results[1]

The Pool class will guarantee that the Http instances are reused if possible, and that only one greenthread can access each at a time. If you’re looking for somewhat more advanced usage of this design pattern, take a look at the source code to Heroshi, a concurrent web crawler written on top of Eventlet.

One of the rough edges of the tiny port forwarder app that I wrote as an example a while back was that it involved too much socket fiddling.  Making things less complicated is Eventlet’s reason for existence, and it is undeniably difficult to get from “I want to do this” to “here’s the functions to call” with the standard socket interface, plus a lot of the defaults are not what you want.  These are some of the reasons we added convenience functions back to Eventlet.

So here’s the port forwarder example cleaned up with those convenience functions.  It forwards connections received on port 7000 to port 22.  Test it out by running ssh -p 7000 localhost and logging in to your own machine.  It’s very fast; there’s no perceptible difference between using the forwarder and connecting directly.

import eventlet
def closed_callback():
    print "called back"

def forward(source, dest, cb = lambda: None):
    while True:
        d = source.recv(8096)
        if d == '':
            cb()
            break
        dest.sendall(d)

listener = eventlet.listen(('localhost', 7000))
while True:
    client, addr = listener.accept()
    server = eventlet.connect(('localhost', 22))
    eventlet.spawn_n(forward, client, server, closed_callback)
    eventlet.spawn_n(forward, server, client)

When using the forwarder to scp a large file around on localhost, I observed nearly the same transfer rates with the forwarder as without: 22 versus 23 MB/s. It consumed very little CPU, as well.

I’m not putting this forward as a production solution; there are clearly far better port forwarders out there, and better solutions to the original problem of needing a callback when the connection closes.  The point is to illustrate how you can bang together a pretty damn decent special-purpose application with an astonishingly small amount of work.  It’s the duct tape of network programming!