I'm trying to use twisted with greenlets, so I can write synchronous looking code in twisted without using inlineCallbacks.
Here is my code:
import time, functools from twisted.internet import reactor, threads from twisted.internet.defer import Deferred from functools import wraps import greenlet def make_async(func): @wraps(func) def wrapper(*pos, **kwds): d = Deferred() def greenlet_func(): try: rc = func(*pos, **kwds) d.callback(rc) except Exception, ex: print ex d.errback(ex) g = greenlet.greenlet(greenlet_func) g.switch() return d return wrapper def sleep(t): print "sleep(): greenelet:", greenlet.getcurrent() g = greenlet.getcurrent() reactor.callLater(t, g.switch) g.parent.switch() def wait_one(d): print "wait_one(): greenelet:", greenlet.getcurrent() g = greenlet.getcurrent() active = True def callback(result): if not active: g.switch(result) else: reactor.callLater(0, g.switch, result) def errback(failure): if not active: g.throw(failure) else: reactor.callLater(0, g.throw, failure) d.addCallback(callback) d.addErrback(errback) active = False rc = g.parent.switch() return rc @make_async def inner(): print "inner(): greenelet:", greenlet.getcurrent() import random, time interval = random.random() print "Sleeping for %s seconds..." % interval sleep(interval) print "done" return interval @make_async def outer(): print "outer(): greenelet:", greenlet.getcurrent() print wait_one(inner()) print "Here" reactor.callLater(0, outer) reactor.run()
There are 5 main parts:
- A sleep function, that starts a timer, then switches back to the parent greenlet. When the timer goes off, it switches back to the greenlet that is sleeping.
- A make_async decorator. This takes some synchronous looking code and runs it in a greenlet. IT also returns a deferred so the caller can register callbacks when the code completes.
- A wait_one function, which blocks the greenlet until the deferred being waited on resolves.
- The inner function, which (when wrapped) returns a deferred, sleeps for a random time, and then passes the time it slept for to the deferred.
- The outer function, which calls inner(), waits for it to return, then prints the return value.
When I run this code I get this output (Note the error on the last two lines):
outer(): greenelet: <greenlet.greenlet object at 0xb729cc5c> inner(): greenelet: <greenlet.greenlet object at 0xb729ce3c> Sleeping for 0.545666723422 seconds... sleep(): greenelet: <greenlet.greenlet object at 0xb729ce3c> wait_one(): greenelet: <greenlet.greenlet object at 0xb729cc5c> done 0.545666723422 Here Exception twisted.python.failure.Failure: <twisted.python.failure.Failure <class 'greenlet.GreenletExit'>> in <greenlet.greenlet object at 0xb729ce3c> ignored GreenletExit did not kill <greenlet.greenlet object at 0xb729ce3c>
Doing a bit of research I've found that:
- The last line is logged by greenlet.c
- The previous line is logged by python itself, as it's ignoring an exception raised in a del method.
I'm having real trouble debugging this as I can't access the
twisted.python.failure.Failure exceptions to get their stack traces.
Does anyone have any ideas what I'm doing wrong, or how I get debug the exceptions that are being thrown?
One other data point: If I hack wait_one() to just return immediately (and not to register anything on the deferred it is passed), the errors go away. :-/