1.2.2.6. keepalive – Handler for HTTP Keep-Alive Headers

1.2.2.6.1. keepalive

class lib.browser.plugins.keepalive.KeepAlivePlugin

Adds a urllib handler to make urllib and lib.browser.Browser utilize HTTP’s Keep-Alive header, making muliple page loads from the same server significantly faster.

1.2.2.6.2. keepalive.handler

An HTTP handler for urllib2 that supports HTTP 1.1 and keepalive.

This is keepalive.py from the urlgrabber project, available under the GNU LGPL ported to python 3 (with help from 2to3).

>>> import urllib2
>>> from keepalive import HTTPHandler
>>> keepalive_handler = HTTPHandler()
>>> opener = urllib2.build_opener(keepalive_handler)
>>> urllib2.install_opener(opener)
>>> 
>>> fo = urllib2.urlopen('http://www.python.org')

To remove the handler, simply re-run build_opener with no arguments, and install that opener.

You can explicitly close connections by using the close_connection() method of the returned file-like object (described below) or you can use the handler methods:

close_connection(host) close_all() open_connections()

>>> keepalive_handler.close_all()

EXTRA ATTRIBUTES AND METHODS

Upon a status of 200, the object returned has a few additional attributes and methods, which should not be used if you want to remain consistent with the normal urllib2-returned objects:

close_connection() - close the connection to the host readlines() - you know, readlines() status - the return status (ie 404) reason - english translation of status (ie ‘File not found’)

If you want the best of both worlds, use this inside an AttributeError-catching try:

>>> try: status = fo.status
>>> except AttributeError: status = None

Unfortunately, these are ONLY there if status == 200, so it’s not easy to distinguish between non-200 responses. The reason is that urllib2 tries to do clever things with error codes 301, 302, 401, and 407, and it wraps the object upon return.

You can optionally set the module-level global HANDLE_ERRORS to 0, in which case the handler will always return the object directly. If you like the fancy handling of errors, don’t do this. If you prefer to see your error codes, then do.

class lib.browser.plugins.keepalive.handler.HTTPConnection(host, port=None, strict=<object object at 0x29ce730>, timeout=<object object at 0x368ba30>, source_address=None)

response_class

alias of HTTPResponse

class lib.browser.plugins.keepalive.handler.HTTPHandler

close_all()

close all open connections

close_connection(host)

close connection to <host> host is the host:port spec, as in ‘www.cnn.com:8080’ as passed in. no error occurs if there is no connection to that host.

do_open(http_class, req)

http_open(req)

open_connections()

return a list of connected hosts

class lib.browser.plugins.keepalive.handler.HTTPResponse(sock, debuglevel=0, strict=0, method=None)

close_connection()

geturl()

info()

read(amt=None)

readline(limit=-1)

readlines(sizehint=0)

lib.browser.plugins.keepalive.handler.comp(N, url)

lib.browser.plugins.keepalive.handler.continuity(url)

lib.browser.plugins.keepalive.handler.error_handler(url)

lib.browser.plugins.keepalive.handler.fetch(N, url, delay=0)

lib.browser.plugins.keepalive.handler.test(url, N=10)