Django Doctest Tips
In addition to metadata everywhere, I'm a big fan of testing
I suspect that as the month progresses, Eric will provide quite a bit of information of the unittest-style of testing, so I'll leave that to him and instead cover a few points on doctests.
In my opinion, neither unittest nor doctest are the better testing tool. Each shine in their own way and actually play very nicely together. For intensive testing of low-level functionality, it's hard to beat the organization and customizability that unittests bring to the table. But where doctests shine is in function/integration testing, where you can simulate how the lower-level bits will be used in practice. And in the context of Django, doctests evaluate much quicker, which is a big deal when you're testing a large suite or continuously testing in a TDD manner.
Using doctests do come with their own set of issues. In particular, when a failure occurs, it is sometimes difficult to track down where in the tests it occurred. Failures also prevent further execution, so an early error can cause many more to follow, incorrectly representing the number of failures in the test. Here are some ways to deal with these shortcomings.
1. Locating A Failure
A very common pattern is to make a request with the test client then check to see what the HTTP response code would have been. Unfortunately, this code has a habit of looking very similar over time. For example:
>>> from django.test import Client
>>> c = Client()
>>> r = c.get('/')
>>> r.status_code
200
>>> r = c.get('/blog/')
>>> r.status_code
200
>>> r = c.get('/blog/2008/')
>>> r.status_code
200
A failure on any of the status code checks will result in printing out only the line that failed (i.e. r.status_code). A common way I deal with this is to repeat the URL I'm requesting as a comment on the r.status_code line. So the example would become:
>>> from django.test import Client
>>> c = Client()
>>> r = c.get('/')
>>> r.status_code # /
200
>>> r = c.get('/blog/')
>>> r.status_code # /blog/
200
>>> r = c.get('/blog/2008/')
>>> r.status_code # /blog/2008/
200
Now, when the failure occurs, it's obvious (or more obvious) where it stems from.
2. Use Conditionals
Another common error that can crop up is when a failure occurs when processing a form. The normal pattern in the view would be to check if the form is valid, save then redirect. But an error will fall through, presenting the failures to the user. This will cause multiple failures in the doctest if the form's processing is incorrect. Example:
>>> from django.test import Client
>>> c = Client()
>>> r = c.get('/wall/add/')
>>> r.status_code # /wall/add/
200
>>> r = c.post('/wall/add/', {'name': 'Daniel', 'shout': ''})
>>> r.status_code # /wall/add/
302
>>> r['Location']
'http://testserver/wall/'
A failure in the form will cause both the r.status_code AND the r['Location'] lines to fail, when really there is only one failure causing the problem.
Rather than having to resort to testing this page in the browser, we can provide the programmer with more information to make debugging a failing test here go quickly. We'll conditionally check the r.status_code and supply conditional blocks that make sense based on it.
>>> from django.test import Client
>>> c = Client()
>>> r = c.get('/wall/add/')
>>> r.status_code # /wall/add/
200
>>> r = c.post('/wall/add/', {'name': 'Daniel', 'shout': ''})
>>> r.status_code # /wall/add/
302
>>> r['Location']
'http://testserver/wall/'
>>> if r.status_code != 302:
... r['context'][-1]['form'].errors # Or r['context'][0]['form'].errors if you're not using template inheritance...
Now, if the form's processing fails (no redirect, so hence no 302), the tests will also output the errors from the form. This occurs because we've introduced a test that we know will fail (no output from the form's errors). This makes debugging the form much easier and faster.
As a warning, conditionals like this are slightly fragile, especially if there are other things in your view that could cause a failure. The point is more to introduce the idea of leveraging conditionals on a case by case basis. I also only use this technique when posting data, as that's when the more difficult errors seem to creep in.
3. Checking Context Variables
A great way to sanity-check what's happened in your views is to check the values that have been put in your context. Some people prefer to use tests or assertions here, like so:
>>> from django.test import Client
>>> c = Client()
>>> r = c.get('/acronyms/')
>>> r.status_code # /acronyms/
200
>>> len(r['context'][-1]['acronym_list']) == 5
True
>>> r['context'][-1]['acronym_list'] == ['Ajax', 'ORM', 'MVC', 'TDD', 'PEBKAC']
True
>>> isinstance(r['context'][-1]['form'], SearchForm)
True
Personally, I prefer to avoid boolean tests and instead output the context variable itself. So this would become:
>>> from django.test import Client
>>> c = Client()
>>> r = c.get('/acronyms/')
>>> r.status_code # /acronyms/
200
>>> len(r['context'][-1]['acronym_list'])
5
>>> r['context'][-1]['acronym_list']
['Ajax', 'ORM', 'MVC', 'TDD', 'PEBKAC']
>>> type(r['context'][-1]['form'])
While this doesn't represent a major difference in the amount of typing when creating the test, this saves a ton of time when running the tests, as the doctest runner will provide what it got instead of what you were expecting, further reducing the amount of time you spend debugging. If left as it was originally, you only know that the test failed (got False instead of True) and would have to dig in further yourself to find out what was actually returned and how.
In combination with fixtures, checking for correct output is a relatively simple, straightforward task.
4. Content Type Relations
One final failing point during testing (which can equally affect both unittests and doctests) is the use of content types when relating two models together. Because of the way things are handled as it stands in Django, the order of content types in testing can be different from that of development (or worse, from developer machine to developer machine). This can also happen when using generic relations (because it uses content types and foreign keys). Assume for this example that your User model has a relation to the Favorite model, and that using this, you recently favorited a Friend model.
>>> from django.test import Client
>>> c = Client()
>>> from django.core.management import call_command
>>> call_command('loaddata', 'myapp_testdata.yaml') #doctest: +ELLIPSIS
Installing yaml fixture 'myapp_testdata' ...
Installed 4 object(s) from 1 fixture(s)
>>> r = c.get('/favorites/')
>>> r.status_code # /favorites/
200
>>> r['context'][-1]['most_recent_favorite']
The way to handle this rather-sticky and sometimes sneaky problem is simple. At run time, simply load the correct content type and reprocess your fixture data, correcting the content type as you go.
>>> from django.test import Client
>>> c = Client()
>>> from django.core.management import call_command
>>> call_command('loaddata', 'myapp_testdata.yaml') #doctest: +ELLIPSIS
Installing yaml fixture 'myapp_testdata' ...
Installed 4 object(s) from 1 fixture(s)
# Fix the CTs.
>>> from django.contrib.contenttypes.models import ContentType
>>> from myapp.models import Favorite
>>> friend_ct = ContentType.objects.get(app_label='myapp', model='Friend')
>>> for fav in Favorite.objects.all():
... # We thought the CT id was 3, but at run-time it is 5...
... if fav.content_type_id == 3:
... fav.content_type = friend_ct.id
... fav.save()
>>> r = c.get('/favorites/')
>>> r.status_code # /favorites/
200
>>> r['context'][-1]['most_recent_favorite']
Now your tests will pass, regardless of what order apps/models were installed in on the machine running the tests.
Conclusion
Hopefully this gives you some ways to manage complex doctests and to speed up the debugging process when using doctests. For further reading, I highly recommend Django's doctest documentation as well as Python's doctest documentation. There's lots more that can be done with this flexible, simple tool.
1 - The name of my site/company/whatever is actually a play on Test Driven Development. I like it that much.