Python: sending a multipart/form-data request with urllib2

18 May 2010

Earlier this week I found a nice snippet of code that lets you send a multipart/form-data request with urllib2. To use it, just download this small file: MultipartPostHandler.py. I made a couple small modifications, though:

1. In the params array, if the keys are unicode, it can throw an annoying error like utf8 codec can't decode bytes in position .... This is a result of concatenating a UTF-8 encoded unicode string, and a binary string which contains random data that confuses the unicode decoder. Yes, Python’s string representation is weird. The solution is to convert the keys to regular python strings.
2. A StringIO buffer is used to build the request. This is a small performance improvement.

Using it is simple:

import MultipartPostHandler
import urllib2
...
url = 'some url that accepts http multipart/form-data post requests'
path = 'some filepath'
params = { 'file1' : file(path, 'rb'), 'param1':'hello' }
opener = urllib2.build_opener(MultipartPostHandler.MultipartPostHandler)
return opener.open(url, params).read()

As you can see, the parameters are passed as a dictionary. Values that are file objects are written to the multipart stream.