gevent.fileobject
– Wrappers to make file-like objects cooperative¶FileObject
¶The main entry point to the file-like gevent-compatible behaviour. It will be defined to be the best available implementation.
There are two main implementations of FileObject
. On all systems,
there is FileObjectThread
which uses the built-in native
threadpool to avoid blocking the entire interpreter. On UNIX systems
(those that support the fcntl
module), there is also
FileObjectPosix
which uses native non-blocking semantics.
A third class, FileObjectBlock
, is simply a wrapper that executes everything
synchronously (and so is not gevent-compatible). It is provided for testing and debugging
purposes.
You may change the default value for FileObject
using the
GEVENT_FILE
environment variable. Set it to posix
, thread
,
or block
to choose from FileObjectPosix
,
FileObjectThread
and FileObjectBlock
, respectively.
You may also set it to the fully qualified class name of another
object that implements the file interface to use one of your own
objects.
Note
The environment variable must be set at the time this module is first imported.
FileObjectPosix
(fobj, mode='rb', bufsize=-1, close=True)¶Bases: gevent._fileobjectcommon.FileObjectBase
A file-like object that operates on non-blocking files but provides a synchronous, cooperative interface.
Caution
This object is only effective wrapping files that can be used meaningfully
with select.select()
such as sockets and pipes.
In general, on most platforms, operations on regular files
(e.g., open('a_file.txt')
) are considered non-blocking
already, even though they can take some time to complete as
data is copied to the kernel and flushed to disk: this time
is relatively bounded compared to sockets or pipes, though.
A read()
or write()
call on such a file
will still effectively block for some small period of time.
Therefore, wrapping this class around a regular file is
unlikely to make IO gevent-friendly: reading or writing large
amounts of data could still block the event loop.
If you’ll be working with regular files and doing IO in large
chunks, you may consider using
FileObjectThread
or
tp_read()
and tp_write()
to bypass this
concern.
Note
Random read/write (e.g., mode='rwb'
) is not supported.
For that, use io.BufferedRWPair
around two instance of this
class.
Tip
Although this object provides a fileno()
method and so
can itself be passed to fcntl.fcntl()
, setting the
os.O_NONBLOCK
flag will have no effect (reads will
still block the greenlet, although other greenlets can run).
However, removing that flag will cause this object to no
longer be cooperative (other greenlets will no longer run).
You can use the internal fileio
attribute of this object
(a io.RawIOBase
) to perform non-blocking byte reads.
Note, however, that once you begin directly using this
attribute, the results from using methods of this object
are undefined, especially in text mode. (See issue #222.)
Changed in version 1.1: Now uses the io
package internally. Under Python 2, previously
used the undocumented class socket._fileobject
. This provides
better file-like semantics (and portability to Python 3).
Changed in version 1.2a1: Document the fileio
attribute for non-blocking reads.
Parameters: |
|
---|
Changed in version 1.2a1: A bufsize of 0 in write mode is no longer forced to be 1. Instead, the underlying buffer is flushed after every write operation to simulate a bufsize of 0. In gevent 1.0, a bufsize of 0 was flushed when a newline was written, while in gevent 1.1 it was flushed when more than one byte was written. Note that this may have performance impacts.
FileObjectThread
(fobj, mode=None, bufsize=-1, close=True, threadpool=None, lock=True)¶Bases: gevent._fileobjectcommon.FileObjectBase
A file-like object wrapping another file-like object, performing all blocking operations on that object in a background thread.
Caution
Attempting to change the threadpool or lock of an existing FileObjectThread has undefined consequences.
Changed in version 1.1b1: The file object is closed using the threadpool. Note that whether or not this action is synchronous or asynchronous is not documented.
Parameters: |
|
---|
FileObject
alias of FileObjectPosix
Next page: gevent.subprocess
– Cooperative subprocess
module