tornado.iostream — Convenient wrappers for non-blocking sockets

Utility classes to write to and read from non-blocking files and sockets.

Contents:

• BaseIOStream: Generic interface for reading and writing.
• IOStream: Implementation of BaseIOStream using non-blocking sockets.
• SSLIOStream: SSL-aware version of IOStream.
• PipeIOStream: Pipe-based IOStream implementation.

Base class

class tornado.iostream.BaseIOStream(io_loop=None, max_buffer_size=None, read_chunk_size=None)

A utility class to write to and read from a non-blocking file or socket.

We support a non-blocking write() and a family of read_*() methods. All of the methods take an optional callback argument and return a Future only if no callback is given. When the operation completes, the callback will be run or the Future will resolve with the data read (or None for write()). All outstanding Futures will resolve with a StreamClosedError when the stream is closed; users of the callback interface will be notified via BaseIOStream.set_close_callback instead.

When a stream is closed due to an error, the IOStream’s error attribute contains the exception object.

Subclasses must implement fileno, close_fd, write_to_fd, read_from_fd, and optionally get_fd_error.

Main interface

BaseIOStream.write(data, callback=None)

Asynchronously write the given data to this stream.

If callback is given, we call it when all of the buffered write data has been successfully written to the stream. If there was previously buffered write data and an old write callback, that callback is simply overwritten with this new callback.

If no callback is given, this method returns a Future that resolves (with a result of None) when the write has been completed. If write is called again before that Future has resolved, the previous future will be orphaned and will never resolve.

Changed in version 3.3: Now returns a Future if no callback is given.

BaseIOStream.read_bytes(num_bytes, callback=None, streaming_callback=None, partial=False)

Asynchronously read a number of bytes.

If a streaming_callback is given, it will be called with chunks of data as they become available, and the final result will be empty. Otherwise, the result is all the data that was read. If a callback is given, it will be run with the data as an argument; if not, this method returns a Future.

If partial is true, the callback is run as soon as we have any bytes to return (but never more than num_bytes)

Changed in version 3.3: Added the partial argument. The callback argument is now optional and a Future will be returned if it is omitted.

BaseIOStream.read_until(delimiter, callback=None, max_bytes=None)

Asynchronously read until we have found the given delimiter.

The result includes all the data read including the delimiter. If a callback is given, it will be run with the data as an argument; if not, this method returns a Future.

If max_bytes is not None, the connection will be closed if more than max_bytes bytes have been read and the delimiter is not found.

Changed in version 3.3: Added the max_bytes argument. The callback argument is now optional and a Future will be returned if it is omitted.

BaseIOStream.read_until_regex(regex, callback=None, max_bytes=None)

Asynchronously read until we have matched the given regex.

The result includes the data that matches the regex and anything that came before it. If a callback is given, it will be run with the data as an argument; if not, this method returns a Future.

If max_bytes is not None, the connection will be closed if more than max_bytes bytes have been read and the regex is not satisfied.

Changed in version 3.3: Added the max_bytes argument. The callback argument is now optional and a Future will be returned if it is omitted.

BaseIOStream.read_until_close(callback=None, streaming_callback=None)

Asynchronously reads all data from the socket until it is closed.

If a streaming_callback is given, it will be called with chunks of data as they become available, and the final result will be empty. Otherwise, the result is all the data that was read. If a callback is given, it will be run with the data as an argument; if not, this method returns a Future.

Changed in version 3.3: The callback argument is now optional and a Future will be returned if it is omitted.

BaseIOStream.close(exc_info=False)

Close this stream.

If exc_info is true, set the error attribute to the current exception from sys.exc_info (or if exc_info is a tuple, use that instead of sys.exc_info).

BaseIOStream.set_close_callback(callback)

Call the given callback when the stream is closed.

This is not necessary for applications that use the Future interface; all outstanding Futures will resolve with a StreamClosedError when the stream is closed.

BaseIOStream.closed()

Returns true if the stream has been closed.

BaseIOStream.reading()

Returns true if we are currently reading from the stream.

BaseIOStream.writing()

Returns true if we are currently writing to the stream.

BaseIOStream.set_nodelay(value)

Sets the no-delay flag for this stream.

By default, data written to TCP streams may be held for a time to make the most efficient use of bandwidth (according to Nagle’s algorithm). The no-delay flag requests that data be written as soon as possible, even if doing so would consume additional bandwidth.

This flag is currently defined only for TCP-based IOStreams.

New in version 3.1.

Methods for subclasses

BaseIOStream.fileno()

Returns the file descriptor for this stream.

BaseIOStream.close_fd()

Closes the file underlying this stream.

close_fd is called by BaseIOStream and should not be called elsewhere; other users should call close instead.

BaseIOStream.write_to_fd(data)

Attempts to write data to the underlying file.

Returns the number of bytes written.

BaseIOStream.read_from_fd()

Attempts to read from the underlying file.

Returns None if there was nothing to read (the socket returned EWOULDBLOCK or equivalent), otherwise returns the data. When possible, should return no more than self.read_chunk_size bytes at a time.

BaseIOStream.get_fd_error()

Returns information about any error on the underlying file.

This method is called after the IOLoop has signaled an error on the file descriptor, and should return an Exception (such as socket.error with additional information, or None if no such information is available.

Implementations

class tornado.iostream.IOStream(socket, *args, **kwargs)

Socket-based IOStream implementation.

This class supports the read and write methods from BaseIOStream plus a connect method.

The socket parameter may either be connected or unconnected. For server operations the socket is the result of calling socket.accept. For client operations the socket is created with socket.socket, and may either be connected before passing it to the IOStream or connected with IOStream.connect.

A very simple (and broken) HTTP client using this class:

import tornado.ioloop
import tornado.iostream
import socket

def send_request():
    stream.write(b"GET / HTTP/1.0\r\nHost: friendfeed.com\r\n\r\n")
    stream.read_until(b"\r\n\r\n", on_headers)

def on_headers(data):
    headers = {}
    for line in data.split(b"\r\n"):
       parts = line.split(b":")
       if len(parts) == 2:
           headers[parts[0].strip()] = parts[1].strip()
    stream.read_bytes(int(headers[b"Content-Length"]), on_body)

def on_body(data):
    print data
    stream.close()
    tornado.ioloop.IOLoop.instance().stop()

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)
stream = tornado.iostream.IOStream(s)
stream.connect(("friendfeed.com", 80), send_request)
tornado.ioloop.IOLoop.instance().start()

connect(address, callback=None, server_hostname=None)

Connects the socket to a remote address without blocking.

May only be called if the socket passed to the constructor was not previously connected. The address parameter is in the same format as for socket.connect, i.e. a (host, port) tuple. If callback is specified, it will be called when the connection is completed; if not this method returns a Future.

If specified, the server_hostname parameter will be used in SSL connections for certificate validation (if requested in the ssl_options) and SNI (if supported; requires Python 3.2+).

Note that it is safe to call IOStream.write while the connection is pending, in which case the data will be written as soon as the connection is ready. Calling IOStream read methods before the socket is connected works on some platforms but is non-portable.

Changed in version 3.3: If no callback is given, returns a Future.

class tornado.iostream.SSLIOStream(*args, **kwargs)

A utility class to write to and read from a non-blocking SSL socket.

If the socket passed to the constructor is already connected, it should be wrapped with:

ssl.wrap_socket(sock, do_handshake_on_connect=False, **kwargs)

before constructing the SSLIOStream. Unconnected sockets will be wrapped when IOStream.connect is finished.

The ssl_options keyword argument may either be a dictionary of keywords arguments for ssl.wrap_socket, or an ssl.SSLContext object.

class tornado.iostream.PipeIOStream(fd, *args, **kwargs)

Pipe-based IOStream implementation.

The constructor takes an integer file descriptor (such as one returned by os.pipe) rather than an open file object. Pipes are generally one-way, so a PipeIOStream can be used for reading or writing but not both.

Exceptions

exception tornado.iostream.StreamClosedError

Exception raised by IOStream methods when the stream is closed.

Note that the close callback is scheduled to run after other callbacks on the stream (to allow for buffered data to be processed), so you may see this error before you see the close callback.

exception tornado.iostream.UnsatisfiableReadError

Exception raised when a read cannot be satisfied.

Raised by read_until and read_until_regex with a max_bytes argument.