Edited on May 29 : After writing this article, Christian Heimes pointed out to me that the debug method on unttest.TestCase allows you to run tests interactively. The idiot in me didn’t do enough research before writing this rant. I’m leaving this post up to provide some context for future visitors/googlers who are looking for a quick way debug their test cases interactively.
One lesson I’ve learned over many years of observation and programming in Python is that the easier I make writing and running tests for my programs, the faster I can produce bug-free software. That is why I start most projects nowadays with two pre-baked files: main.py and tests.py.
With this setup and tools like Nose, testing code in Python for me is nearly pain free. Recently however, I notice one aspect of the unittest package in Python’s standard libary drives me absolutely crazy:
Why is there no simple method to run a single test case from within an interactive python shell?
As developers, this is important because if we are going to make writing tests an integral part of
our development workflow, one needs some way to actually run the damn test without dropping out of
our Python sessions and breaking out of our mental flow.
Being forced to run tests solely from the commandline also means we can’t take advantage of features like ipython’s ability to automatically drop into a pdb debugging session when an error occurs. This is extremely useful for when you want to introspect one or two variables to determine why a test failure occurred.
In general, it seems like the unittest library’s lack of a simple function to run single test cases in an interactive shell dis-incentivizes programmers from writing tests and detracts from python’s whole “rapid iteration” ethos.
After googling around and failing to find any good alternatives, I’ve finally settled on writing my own utility function to run single test cases:
def runtest(testCase, methodName): """ Runs a test case from within interactive shell """ tc = testCase(methodName) getattr(tc, "setUp", lambda:None)() try: getattr(tc, methodName)() finally: getattr(tc, "tearDown", lambda:None)()
I add this function to every single project that I create now, and the qualitative feel of writing tests just feels so much more natural now. I also find myself writing more tests and running them more often with this function in my projects.
Give it a try, and let me know whether this has any effect for you. I’d love to hear your feedback on how it affects your development workflow.
10 responses so far
You might want to use an exception handler to ensure that tearDown gets called, even if the test method raises an exception.
Have you looked at unittest2? It has that functionality AFAIK.
Hey, dumb question, why lambda: None, instead of just None ?
@tito, the lambda:None default is so that I can call as a function the return value of that getattr regardless of whether the method actually exists or not. If I returned just None, then the program would throw an error when the method name does not exist.
@EricH Thanks , that’s a good point. I’ve changed it now so that it will always call the tearDown method.
Have you looked at Trial from Twisted? With Trial you can specify a module, a class or a method/function to be tested. This way, you can easily test a single method or all methods of a TestCase-class. It has some nice other features, like integrated monkey patching and the ability to Skip tests. And of course, Twisted support
The interface to run a test case is just the run method. You don’t need to call setUp and tearDown yourself, and there are potentially many other things that the test case will want to do aside from those that your code is missing.
def runtest(testCase, methodName): results = TextTestResult(sys.stdout, True, 1) testCase(methodName).run(results) results.printErrors()Read up on the unittest interfaces, then you can write tools that properly cooperate with the rest of the unittest ecosystem.
Python’s unit test system already has the functionality for a long, long time:
MyTestClass(“test_method”).debug()
or
suite = unittest.TestSuite()
suite.addTest(MyTestClass(“test_method”))
suite.addTest(MyTestClass(“test_method2″))
suite.addTest(unittest.defaultTestLoader.loadTestsFromTestCase(MyOtherTestClass))
unittest.TextTestRunner(verbosity=2).run(suite)
Christian,
I originally used your second method to run tests in ipython, but found that the testrunner caught all the exceptions so that I was not able to debug them interactively in PDB.
MyTestClass(“test_method”).debug()is exactly what I was looking for. I feel like a major idiot for not seeing this before. Thanks for showing the way.
Have you ever tried py.test?
import pytest
pytest.main(‘-k package.module.func’)
http://doc.pytest.org/en/latest/usage.html#calling-pytest-from-python-code