.svg)
.svg)
The web is a big place now. We need to support thousands of clients at a time, and here comes Tornado. Tornado is a Python web framework and asynchronous network library, originally developed at FriendFreed.
Tornado uses non-blocking network-io. Due to this, it can handle thousands of active server connections. It is a saviour for applications where long polling and a large number of active connections are maintained.
Tornado is not like most Python frameworks. It's not based on WSGI, while it supports some features of WSGI using module `tornado.wsgi`. It uses an event loop design that makes Tornado request execution faster.
A function blocks, performs its computation, and returns, once done . A function may block for many reasons: network I/O, disk I/O, mutexes, etc.
Application performance depends on how efficiently application uses CPU cycles, that's why blocking statements/calls must be taken seriously. Consider password hashing functions like bcrypt, which by design use hundreds of milliseconds of CPU time, far more than a typical network or disk access. As the CPU is not idle, there is no need to go for asynchronous functions.
A function can be blocking in one, and non-blocking in others. In the context of Tornado, we generally consider blocking due to network I/O and disk, although all kinds of blocking need to be minimized.
1) Single-threaded architecture:
Means, it can't do computation-centric tasks parallely.
2) I/O concurrency:
It can handover IO tasks to the operating system and continue to the next task to achieve parallelism.
3) epoll/ kqueue:
Underline system-related construct that allows an application to get events on a file descriptor or I/O specific tasks.
4) Event loop:
It uses epoll or kqueue to check if any event has happened, and executes callback that is waiting for those network events.
In case of synchronous model, each request or task is transferred to thread or routing, and as it finishes, the result is handed over to the caller. Here, managing things are easy, but creating new threads is too much overhead.
On the other hand, in Asynchronous framework, like Node.js, there is a single-threaded model, so very less overhead, but it has complexity.
Let's imagine thousands of requests coming through and a server uses event loop and callback. Now, until request gets processed, it has to efficiently store and manage the state of that request to map callback result to the actual client.
Most of these comparison points are tied to actual programming language and not the framework:
| import tornado.ioloop | |
| import tornado.web | |
| class MainHandler(tornado.web.RequestHandler): | |
| def get(self): | |
| self.write("Hello, world") | |
| def make_app(): | |
| return tornado.web.Application([ | |
| (r"/", MainHandler), | |
| ]) | |
| if __name__ == "__main__": | |
| app = make_app() | |
| app.listen(8888) | |
| tornado.ioloop.IOLoop.current().start() |
Note: This example does not use any asynchronous feature.
Using AsyncHTTPClient module, we can do REST call asynchronously.
| from tornado.httpclient import AsyncHTTPClient | |
| from tornado import gen | |
| @gen.coroutine | |
| def async_fetch_gen(url): | |
| http_client = AsyncHTTPClient() | |
| response = yield http_client.fetch(url) | |
| raise gen.Return(response.body) |
As you can see `yield http_client.fetch(url)` will run as a coroutine.
Please have a look at Asynchronous Request handler.
Tornado has built-in package for WebSockets that can be easily used with coroutines to achieve concurrency, here is one example:
| import logging | |
| import tornado.escape | |
| import tornado.ioloop | |
| import tornado.options | |
| import tornado.web | |
| import tornado.websocket | |
| from tornado.options import define, options | |
| from tornado.httpserver import HTTPServer | |
| define("port", default=8888, help="run on the given port", type=int) | |
| # queue_size = 1 | |
| # producer_num_items = 5 | |
| # q = queues.Queue(queue_size) | |
| def isPrime(num): | |
| """ | |
| Simple worker but mostly IO/network call | |
| """ | |
| if num > 1: | |
| for i in range(2, num // 2): | |
| if (num % i) == 0: | |
| return ("is not a prime number") | |
| else: | |
| return("is a prime number") | |
| else: | |
| return ("is not a prime number") | |
| class Application(tornado.web.Application): | |
| def __init__(self): | |
| handlers = [(r"/chatsocket", TornadoWebSocket)] | |
| super(Application, self).__init__(handlers) | |
| class TornadoWebSocket(tornado.websocket.WebSocketHandler): | |
| clients = set() | |
| # enable cross domain origin | |
| def check_origin(self, origin): | |
| return True | |
| def open(self): | |
| TornadoWebSocket.clients.add(self) | |
| # when client closes connection | |
| def on_close(self): | |
| TornadoWebSocket.clients.remove(self) | |
| @classmethod | |
| def send_updates(cls, producer, result): | |
| for client in cls.clients: | |
| # check if result is mapped to correct sender | |
| if client == producer: | |
| try: | |
| client.write_message(result) | |
| except: | |
| logging.error("Error sending message", exc_info=True) | |
| def on_message(self, message): | |
| try: | |
| num = int(message) | |
| except ValueError: | |
| TornadoWebSocket.send_updates(self, "Invalid input") | |
| return | |
| TornadoWebSocket.send_updates(self, isPrime(num)) | |
| def start_websockets(): | |
| tornado.options.parse_command_line() | |
| app = Application() | |
| server = HTTPServer(app) | |
| server.listen(options.port) | |
| tornado.ioloop.IOLoop.current().start() | |
| if __name__ == "__main__": | |
| start_websockets() |
One can use a WebSocket client application to connect to the server, message can be any integer. After processing, the client receives the result if the integer is prime or not.
Here is one more example of actual async features of Tornado. Many will find it similar to Golang’s Goroutine and channels.
In this example, we can start worker(s) and they will listen to the 'tornado.queue'. This queue is asynchronous and very similar to the asyncio package.
| # Example 1 | |
| from tornado import gen, queues | |
| from tornado.ioloop import IOLoop | |
| @gen.coroutine | |
| def consumer(queue, num_expected): | |
| for _ in range(num_expected): | |
| # heavy I/O or network task | |
| print('got: %s' % (yield queue.get())) | |
| @gen.coroutine | |
| def producer(queue, num_items): | |
| for i in range(num_items): | |
| print('putting %s' % i) | |
| yield queue.put(i) | |
| @gen.coroutine | |
| def main(): | |
| """ | |
| Starts producer and consumer and wait till they finish | |
| """ | |
| yield [producer(q, producer_num_items), consumer(q, producer_num_items)] | |
| queue_size = 1 | |
| producer_num_items = 5 | |
| q = queues.Queue(queue_size) | |
| results = IOLoop.current().run_sync(main) | |
| # Output: | |
| # putting 0 | |
| # putting 1 | |
| # got: 0 | |
| # got: 1 | |
| # putting 2 | |
| # putting 3 | |
| # putting 4 | |
| # got: 2 | |
| # got: 3 | |
| # got: 4 | |
| # Example 2 | |
| # Condition | |
| # A condition allows one or more coroutines to wait until notified. | |
| from tornado import gen | |
| from tornado.ioloop import IOLoop | |
| from tornado.locks import Condition | |
| my_condition = Condition() | |
| @gen.coroutine | |
| def waiter(): | |
| print("I'll wait right here") | |
| yield my_condition.wait() | |
| print("Received notification now doing my things") | |
| @gen.coroutine | |
| def notifier(): | |
| yield gen.sleep(60) | |
| print("About to notify") | |
| my_condition.notify() | |
| print("Done notifying") | |
| @gen.coroutine | |
| def runner(): | |
| # Wait for waiter() and notifier() in parallel | |
| yield([waiter(), notifier()]) | |
| results = IOLoop.current().run_sync(runner) | |
| # output: | |
| # I'll wait right here | |
| # About to notify | |
| # Done notifying | |
| # Received notification now doing my things |
1) Asynchronous frameworks are not much of use when most of the computations are CPU centric and not I/O.
2) Due to a single thread per core model and event loop, it can manage thousands of active client connections.
3) Many say Django is too big, Flask is too small, and Tornado is just right:)
Did you like the blog? If yes, we're sure you'll also like to work with the people who write them - our best-in-class engineering team.
We're looking for talented developers who are passionate about new emerging technologies. If that's you, get in touch with us.
The web is a big place now. We need to support thousands of clients at a time, and here comes Tornado. Tornado is a Python web framework and asynchronous network library, originally developed at FriendFreed.
Tornado uses non-blocking network-io. Due to this, it can handle thousands of active server connections. It is a saviour for applications where long polling and a large number of active connections are maintained.
Tornado is not like most Python frameworks. It's not based on WSGI, while it supports some features of WSGI using module `tornado.wsgi`. It uses an event loop design that makes Tornado request execution faster.
A function blocks, performs its computation, and returns, once done . A function may block for many reasons: network I/O, disk I/O, mutexes, etc.
Application performance depends on how efficiently application uses CPU cycles, that's why blocking statements/calls must be taken seriously. Consider password hashing functions like bcrypt, which by design use hundreds of milliseconds of CPU time, far more than a typical network or disk access. As the CPU is not idle, there is no need to go for asynchronous functions.
A function can be blocking in one, and non-blocking in others. In the context of Tornado, we generally consider blocking due to network I/O and disk, although all kinds of blocking need to be minimized.
1) Single-threaded architecture:
Means, it can't do computation-centric tasks parallely.
2) I/O concurrency:
It can handover IO tasks to the operating system and continue to the next task to achieve parallelism.
3) epoll/ kqueue:
Underline system-related construct that allows an application to get events on a file descriptor or I/O specific tasks.
4) Event loop:
It uses epoll or kqueue to check if any event has happened, and executes callback that is waiting for those network events.
In case of synchronous model, each request or task is transferred to thread or routing, and as it finishes, the result is handed over to the caller. Here, managing things are easy, but creating new threads is too much overhead.
On the other hand, in Asynchronous framework, like Node.js, there is a single-threaded model, so very less overhead, but it has complexity.
Let's imagine thousands of requests coming through and a server uses event loop and callback. Now, until request gets processed, it has to efficiently store and manage the state of that request to map callback result to the actual client.
Most of these comparison points are tied to actual programming language and not the framework:
| import tornado.ioloop | |
| import tornado.web | |
| class MainHandler(tornado.web.RequestHandler): | |
| def get(self): | |
| self.write("Hello, world") | |
| def make_app(): | |
| return tornado.web.Application([ | |
| (r"/", MainHandler), | |
| ]) | |
| if __name__ == "__main__": | |
| app = make_app() | |
| app.listen(8888) | |
| tornado.ioloop.IOLoop.current().start() |
Note: This example does not use any asynchronous feature.
Using AsyncHTTPClient module, we can do REST call asynchronously.
| from tornado.httpclient import AsyncHTTPClient | |
| from tornado import gen | |
| @gen.coroutine | |
| def async_fetch_gen(url): | |
| http_client = AsyncHTTPClient() | |
| response = yield http_client.fetch(url) | |
| raise gen.Return(response.body) |
As you can see `yield http_client.fetch(url)` will run as a coroutine.
Please have a look at Asynchronous Request handler.
Tornado has built-in package for WebSockets that can be easily used with coroutines to achieve concurrency, here is one example:
| import logging | |
| import tornado.escape | |
| import tornado.ioloop | |
| import tornado.options | |
| import tornado.web | |
| import tornado.websocket | |
| from tornado.options import define, options | |
| from tornado.httpserver import HTTPServer | |
| define("port", default=8888, help="run on the given port", type=int) | |
| # queue_size = 1 | |
| # producer_num_items = 5 | |
| # q = queues.Queue(queue_size) | |
| def isPrime(num): | |
| """ | |
| Simple worker but mostly IO/network call | |
| """ | |
| if num > 1: | |
| for i in range(2, num // 2): | |
| if (num % i) == 0: | |
| return ("is not a prime number") | |
| else: | |
| return("is a prime number") | |
| else: | |
| return ("is not a prime number") | |
| class Application(tornado.web.Application): | |
| def __init__(self): | |
| handlers = [(r"/chatsocket", TornadoWebSocket)] | |
| super(Application, self).__init__(handlers) | |
| class TornadoWebSocket(tornado.websocket.WebSocketHandler): | |
| clients = set() | |
| # enable cross domain origin | |
| def check_origin(self, origin): | |
| return True | |
| def open(self): | |
| TornadoWebSocket.clients.add(self) | |
| # when client closes connection | |
| def on_close(self): | |
| TornadoWebSocket.clients.remove(self) | |
| @classmethod | |
| def send_updates(cls, producer, result): | |
| for client in cls.clients: | |
| # check if result is mapped to correct sender | |
| if client == producer: | |
| try: | |
| client.write_message(result) | |
| except: | |
| logging.error("Error sending message", exc_info=True) | |
| def on_message(self, message): | |
| try: | |
| num = int(message) | |
| except ValueError: | |
| TornadoWebSocket.send_updates(self, "Invalid input") | |
| return | |
| TornadoWebSocket.send_updates(self, isPrime(num)) | |
| def start_websockets(): | |
| tornado.options.parse_command_line() | |
| app = Application() | |
| server = HTTPServer(app) | |
| server.listen(options.port) | |
| tornado.ioloop.IOLoop.current().start() | |
| if __name__ == "__main__": | |
| start_websockets() |
One can use a WebSocket client application to connect to the server, message can be any integer. After processing, the client receives the result if the integer is prime or not.
Here is one more example of actual async features of Tornado. Many will find it similar to Golang’s Goroutine and channels.
In this example, we can start worker(s) and they will listen to the 'tornado.queue'. This queue is asynchronous and very similar to the asyncio package.
| # Example 1 | |
| from tornado import gen, queues | |
| from tornado.ioloop import IOLoop | |
| @gen.coroutine | |
| def consumer(queue, num_expected): | |
| for _ in range(num_expected): | |
| # heavy I/O or network task | |
| print('got: %s' % (yield queue.get())) | |
| @gen.coroutine | |
| def producer(queue, num_items): | |
| for i in range(num_items): | |
| print('putting %s' % i) | |
| yield queue.put(i) | |
| @gen.coroutine | |
| def main(): | |
| """ | |
| Starts producer and consumer and wait till they finish | |
| """ | |
| yield [producer(q, producer_num_items), consumer(q, producer_num_items)] | |
| queue_size = 1 | |
| producer_num_items = 5 | |
| q = queues.Queue(queue_size) | |
| results = IOLoop.current().run_sync(main) | |
| # Output: | |
| # putting 0 | |
| # putting 1 | |
| # got: 0 | |
| # got: 1 | |
| # putting 2 | |
| # putting 3 | |
| # putting 4 | |
| # got: 2 | |
| # got: 3 | |
| # got: 4 | |
| # Example 2 | |
| # Condition | |
| # A condition allows one or more coroutines to wait until notified. | |
| from tornado import gen | |
| from tornado.ioloop import IOLoop | |
| from tornado.locks import Condition | |
| my_condition = Condition() | |
| @gen.coroutine | |
| def waiter(): | |
| print("I'll wait right here") | |
| yield my_condition.wait() | |
| print("Received notification now doing my things") | |
| @gen.coroutine | |
| def notifier(): | |
| yield gen.sleep(60) | |
| print("About to notify") | |
| my_condition.notify() | |
| print("Done notifying") | |
| @gen.coroutine | |
| def runner(): | |
| # Wait for waiter() and notifier() in parallel | |
| yield([waiter(), notifier()]) | |
| results = IOLoop.current().run_sync(runner) | |
| # output: | |
| # I'll wait right here | |
| # About to notify | |
| # Done notifying | |
| # Received notification now doing my things |
1) Asynchronous frameworks are not much of use when most of the computations are CPU centric and not I/O.
2) Due to a single thread per core model and event loop, it can manage thousands of active client connections.
3) Many say Django is too big, Flask is too small, and Tornado is just right:)
.svg)
.svg)
