Beautiful-er Eventlet

One of the rough edges of the tiny port forwarder app that I wrote as an example a while back was that it involved too much socket fiddling.  Making things less complicated is Eventlet’s reason for existence, and it is undeniably difficult to get from “I want to do this” to “here’s the functions to call” with the standard socket interface, plus a lot of the defaults are not what you want.  These are some of the reasons we added convenience functions back to Eventlet.

So here’s the port forwarder example cleaned up with those convenience functions.  It forwards connections received on port 7000 to port 22.  Test it out by running ssh -p 7000 localhost and logging in to your own machine.  It’s very fast; there’s no perceptible difference between using the forwarder and connecting directly.

import eventlet
def closed_callback():
    print "called back"

def forward(source, dest, cb = lambda: None):
    while True:
        d = source.recv(8096)
        if d == '':
            cb()
            break
        dest.sendall(d)

listener = eventlet.listen(('localhost', 7000))
while True:
    client, addr = listener.accept()
    server = eventlet.connect(('localhost', 22))
    eventlet.spawn_n(forward, client, server, closed_callback)
    eventlet.spawn_n(forward, server, client)

When using the forwarder to scp a large file around on localhost, I observed nearly the same transfer rates with the forwarder as without: 22 versus 23 MB/s. It consumed very little CPU, as well.

I’m not putting this forward as a production solution; there are clearly far better port forwarders out there, and better solutions to the original problem of needing a callback when the connection closes.  The point is to illustrate how you can bang together a pretty damn decent special-purpose application with an astonishingly small amount of work.  It’s the duct tape of network programming!

0.9.6 In The House

Eventlet 0.9.6 has gone gold, and you can pick it up from your favorite PyPI.

This release is unique, because I feel that it’s the most community-driven one: the majority of code came from the sprinters in Hanover C.  And, man, is this a good batch of contributions, and there are more in the pipeline.  If you haven’t heard of Pycon and its sprints, they’re where a bunch of nerds get together and sit in a windowless room with occasional beer injections, coding and sharing ideas and being productive without being too serious.

We spent our time doing everything from improving PEP-8 compliance to refactoring the way timers are handled in the hubs.  Mad props go out to the following sprinters who I had the pleasure of meeting in person (in no particular order, and let me know if I totally forgot you): Chris AtLee, Eugene Oden, Ada Majorek, Chuck Thier, Mike Barton, Andrew Svetlov, Tavis Rudd, Brantley Harris, and Donovan Preston.

Here’s the changelog for 0.9.6:

  • new EVENTLET_HUB environment variable allows you to select a hub without code
  • improved GreenSocket and GreenPipe compatibility with stdlib
  • bugfixes on GreenSocket and GreenPipe objects
  • code coverage increased across the board
  • Queue resizing
  • internal DeprecationWarnings largely eliminated
  • tpool is now reentrant (i.e., can call tpool.execute(tpool.execute(foo)))
  • more reliable access to unpatched modules reduces some race conditions when monkeypatching
  • completely threading-compatible corolocal implementation, plus tests and enthusiastic production adoption
  • tests stomp on each others’ toes less
  • performance improvements in timers, hubs, greenpool
  • Greenlet-aware profile module courtesy of CCP
  • support for select26 module’s epoll
  • better PEP-8 compliance and import cleanup
  • new eventlet.serve convenience function for easy TCP servers

Remember that this is intended to a stabilization release, so mostly stabilization-related stuff made it into this.  Some of these stemmed from people testing out existing apps with Eventlet and finding a bug, writing a test, and committing fixes.   That’s exactly the way it’s supposed to be!

Interesting images after the jump.

Continue reading

0.9.6 almost ready!

Man, Pycon has been exhilarating! We got a lot of changes from new contributors that fixed bugs, increased coverage, and improved the code. I’m not really doing it justice — I gotta look over these changelogs and generate release notes to fully remember.

I’m finishing up the final touches on eventlet.serve on the plane, and writing up the release notes, and then I’ll release 0.9.6 when I get home! Thanks for all the great work, everybody!

0.9.5 out!

I’ve been so caught up in this Pycon insanity that I forgot to officially announce it! Here’s the changelog:

0.9.5
=====
* support psycopg in db_pool
* smart patcher that does the right patching when importing without needing to understand plumbing of patched module * patcher.monkey_patch() method replacing util.wrap_*
* monkeypatch threading support
* deprecated api.named
* imported timeout module from gevent, replace exc_after and with_timeout()
* replace call_after with spawn_after; this is so that users don’t see the Timer class
* added cancel() method to GreenThread to support the semantic of “abort if not already in the middle of something” * eventlet.green.os with patched read() and write(), etc
* moved stuff from wrap_pipes_with_coroutine_pipe into green.os * eventlet.green.subprocess instead of eventlet.processes
* improve patching docs, explaining more about patcher and why you’d use eventlet.green * better documentation on greenpiles
* deprecate api.py completely
* deprecate util.py completely
* deprecate saranwrap
* performance improvements in the hubs
* much better documentation overall
* new convenience functions: eventlet.connect and eventlet.listen. Thanks, Sergey!

0.9.5 Almost Done

Wow, I just took a look at the roadmap and deciding what to do about api.named was one of two remaining tasks before releasing 0.9.5! Crazy!

I really don’t want to do any more deprecation after this release, so I’ll also be doing an API review to make sure that there’s no warts or anything still out there. I’ll let you know if I find anything.

After 1.0, the strategy ought to be: anything that is in the documentation should not have any breaking changes made to it. We’ll probably not be able to be perfect about this, so we’ll be as conservative as possible about migration. Anything that’s not in the documentation, or is explicitly declared internal, is internal and might change at any time. Bewarez!

PEP-387 is a good basis, but I think we can be more consistent about maintaining compatibility only with documented interfaces, simply because we have such comprehensive documentation already.

Anyway, all that is to say, we’re gonna try real hard to keep things non-broken for everyone. Namaste!

The Fate of api.named

In case you hadn’t heard of it, there’s a little function in the api module of eventlet called “named”. It does import-related stuff, and I’ve been trying to find a place where something like that even makes sense in the API, but haven’t been able to. I strongly suspect that the only user of this function is Spawning, so I’ve asked R. Tyler to just migrate it over there, and I’ll be deprecating and then removing it from Eventlet.

Sound good? Folks are jake with this development?

Rollback

(This is a cross-post to the mailing list; seemed like it should go here, too.)

It turns out that people largely dislike the deprecation of the Eventlet convenience functions (tcp_listener and friends). The elimination of redundant boilerplate code is a pretty important part of Eventlet. In the past few weeks since I’ve deprecated them and tried to force myself to use the standard socket interface in all my coding, I’ve come to agree.

So, convenience functions are back on the menu! This is an opportunity to do them up right, however. The old names were confusing, and the old functions could have been a bit more flexible. Sergey and I talked this over in #eventlet, and came up with the following four functions. These would be accessible as eventlet.connect, etc.

def connect(addr, family=socket.AF_INET, bind=None):
    """Convenience function for opening client sockets.

    :param addr: Address of the server to connect to.  For TCP sockets,
    this is a (host, port) tuple.
    :param family: Socket family, optional.  See :mod:`socket`
    documentation for available families.
    :param bind: Local address to bind to, optional.
    :return The connected green socket object.
    """
    pass

def listen(addr, family=socket.AF_INET, backlog=50):
    """Convenience function for opening server sockets.  This
    socket can be used as the argument to :func:`serve`, or
    directly by setting up an ``accept()`` loop.

    Sets SO_REUSEADDR on the socket to save on annoyance.

    :param addr: Address to listen on.  For TCP sockets, this is a
    (host, port)  tuple.
    :param family: Socket family, optional.  See :mod:`socket`
    documentation for available families.
    :param backlog: The maximum number of queued connections. Should be
    at least 1; the maximum value is system-dependent.
    :return The listening green socket object.
    """
    pass

def wrap_ssl(sock, keyfile=None, certfile=None, server_side=False,
    cert_reqs=None, ssl_version=None, ca_certs=None,
    do_handshake_on_connect=True, suppress_ragged_eofs=True):
    """Convenience function for converting a regular socket into an SSL
    socket.  Has the same interface as :func:`ssl.wrap_socket`, but
    works on 2.5 or earlier, using PyOpenSSL.

    The preferred idiom is to call wrap_ssl directly on the creation
    method, e.g., ``wrap_ssl(connect(addr))`` or
    ``wrap_ssl(listen(addr), server_side=True)``. This way there is
    no "naked" socket sitting around to accidentally corrupt the SSL
    session.

    :return Green SSL object.
    """
    pass

def serve(sock, handle, concurrency=1000):
    """Runs a server on the supplied socket.  Calls the function
    *handle* in a separate greenthread for every incoming request.
    This function blocks the calling greenthread; it won't return until
    the server completes.  If you desire an immediate return,
    spawn a new greenthread for :func:`serve`.

    The *handle* function must raise an EndServerException to
    gracefully terminate the server -- that's the only way to get the
    server() function to return.  Any other uncaught exceptions raised
    in *handle* are raised as exceptions from :func:`serve`, so be
    sure to do a good job catching exceptions that your application
    raises.  The return value of *handle* is ignored.

    The value in *concurrency* controls the maximum number of
    greenthreads that will be open at any time handling requests.  When
    the server hits the concurrency limit, it stops accepting new
    connections until the existing ones complete.
    """
    pass

You read that last one correctly — the ghost of tcp_server rises! But I think that we’ve been able to deal with the issues faced by the original tcp_server and that serve is something that we can continue to support for the future.

Tell me what you think!

Changes to WSGI to Support WebSockets

In my earlier post, I referred to some minor changes that I had to make to the wsgi server to seamlessly support a websocket client.  I think these changes are generic and simple enough that they, or something similar in spirit, should make it into the WSGI standard someday.  Being able to support websockets from within WSGI applications is just too powerful of a use case to ignore.

Get the Socket Out of the Environment

The first change I made was to add a get_socket() method to the file-like object that appears in 'wsgi.input'.  This method returns the underlying socket object directly, which is then used in the WebSocket object to handle message transfer.

One reason that this might not be the most ideal of implementations is buffering.  It happens that eventlet.wsgi is implemented such that when calling the wsgi application, the very next byte to be read off the socket is the first byte of the first websocket messsage.  It seems plausible that other WSGI servers might not have this property.

Cancel Post-Application Processing

The second change is that when eventlet.wsgi sees that the application returned a special flag value ALREADY_HANDLED, it aborts all post-application processing and skips straight to closing the socket.  Normally it would write the headers and return value to the socket, but since the application has already handled that stuff, we don’t want the WSGI server throwing junk on the line (or complaining that the app didn’t call start_response).

The downside of this approach is that most middleware would have to be changed (slightly) to support ALREADY_HANDLED.  Raising a special exception seems to have the same problem.  But then again, changing the behavior of middleware is kinda the point.

So, those are my two changes.  I’m certain that there is a way to do this that is even more seamless with the intent of the WSGI standard, but these changes required three lines of code to implement, and work today.  :)

Nice Words

Ted Dziuba has some nice things to say about Eventlet.  Thanks, Ted!  The rest of his blog is pretty entertaining to read, while you’re over there.  :)

Scalable, WSGI-compatible Websockets

I wanted to mess around with websockets, and create something cool and Eventlet-y for using them.  This was my first experience using websockets, and I found these sites helpful in learning about it.

One of the design patterns that I observed was that people were running their websocket server as a separate process on a separate port, because there are just enough differences between the websocket protocol and HTTP that it’s difficult to get them into a normal web server.  That’s crap; if websockets are going to replace xmlhttprequest, it has to be just as easy to write the server part of web sockets as it is to write a web page.

So here’s a websocket implementation that works inside (a slight modification to) WSGI.  It requires the latest development version of Eventlet (see sidebar to the right for instructions on getting that). It’s very very rough at this point, but it’s powerful enough that it seemed worth sharing this initial version.

Now, here’s the code for a simple echo server:

def handle(ws):
    while True:
        m = ws.wait()
        if m is None:
            break
        ws.send(m)

It shows the basics in a nutshell, which are incredibly simple:

  • call wait() to receive a message from the browser
  • call send() to send a message to the browser
  • wait() returns None when the browser closes the socket
  • return from the function to close the socket from the server side

You’re free to spawn off new greenthreads to do complex stuff with reading from and writing to the ws object concurrently; it will just work.

Here’s how you set up the server:

from eventlet.green import socket
from eventlet import wsgi
listener = socket.socket()
listener.bind(('localhost', 7000))
listener.listen(500)
wsgi.server(listener, WebSocketWSGI(handle, 'http://localhost:7000'))

That’s it! Most of that is socket creation boilerplate; the WebSocketWSGI class handles the work of converting an incoming WSGI request to a websocket call. The “http://localhost:7000/” is the websocket ORIGIN field, which needs to be hardcoded to some degree in order to prevent XSS attacks.

So, that’s how the code looks, but you want to run something right away. In your copy of the eventlet tree, run this command:

PYTHONPATH=. python examples/websocket.py

Now, fire up Google Chrome and navigate to http://localhost:7000.  It should show you a page that looks like this:

Google Chrome showing a graph of random values between 0 and 1It’s a graph that fills in dynamically based on values that it gets from the websocket.  The cool thing about this is that the HTML page and the websocket are both served from the same port, and the same WSGI application.  It simply dispatches to the websocket handler like any other url-path-based dispatcher.  It’s flashy, self-contained, and scalable, and all it took was ~30 lines of code.

Take a look at the code!  Have fun with it!  I’ll make a post shortly about the slight modifications I had to make to eventlet.wsgi to get this working seamlessly.