Twisted for RPyMostat¶
This documents my initial tests for using Twisted as the framework for the main/hub process.
First, a really good basic doc on Deferreds and a not-so-short introduction to Asynchronous programming.
Twisted Basics¶
- Reactors
- Spawning Processes
- Deferreds; Deferred Reference
- Writing Servers
- Scheduling - “run in X seconds” or “run every N seconds”
- Threading - “most code in Twisted is not thread-safe”
- Application Framework - seems like this might be too baisc for my needs?
- Logging using the logging module, and capturing Twisted’s internal messages; note this can block logging
The more I read about this, the more I think Twisted is probably not the solution I need (I seem to need real threading or multiprocessing, not just async network IO). i.e. see this really important FAQ, Why does it take a long time for data I send with transport.write to arrive at the other side of the connection?.
Third-Party Twisted Modules¶
- Paisley CouchDB client
- sAsync Async SQLAlchemy
- TxScheduling cron-like scheduling
- txAMQP
- txrdq resizable dispatch queue
ReST API¶
- we need to serve it nicely (not a horrible hack)
- read/write from database used by other threads (DB tech still unknown; maybe flat files for now?)
- read/write to some shared global memory (or main thread)
Links¶
Klein¶
Signals¶
- Signals or some other sort of notification mechanism
Links¶
- helga uses smokesignal quite nicely
Scheduling¶
- Scheduled tasks
Testing¶
- test-ability (i.e. pytest, possibly something else to test the threading/network)
Links¶
- pytest-twisted
- Twisted TDD/Trial docs
- TwistedTrial
- Unit Tests in Twisted (internal to Twisted itself)
- some notes on using nose to test Twisted
- Random selection of GitHub projects using pytest-twisted: scrapy, pyrake, snappy, pokerthproto, mcloud
- Interestingly, I can’t find anything on GitHub that uses
pytest-twisted’s
pytest.blockon.pytest.inlineCallbacksis used by a number of mcloud tests, jukebox, webmonitor and spiral