As an OpenStack Swift dev I obviously write a lot of Python. Further Swift is cluster and so it has a bunch of moving pieces. So debugging is very important. Most the time I use pudb and then jump into the PyCharms debugger if get really stuck.
Pudb is curses based version of pdb, and I find it pretty awesome and you can use it while ssh’d somewhere. So I thought I’d write a tips that I use. Mainly so I don’t forget 🙂
The first and easiest way to run pudb is use pudb as the python runner.. i.e:
pudb <python script>
On first run, it’ll start with the preferences window up. If you want to change preferences you can just hit ‘<ctrl>+p’. However you don’t need to remember that, as hitting ‘?’ will give you a nice help screen.
I prefer to see line numbers, I like the dark vim theme and best part of all, I prefer my interactive python shell to be ipython.
While your debugging, like in pdb, there are some simple commands:
- n – step over (“next”)
- s – step into
- c – continue
- r/f – finish current function
- t – run to cursor
- o – show console/output screen
- b – toggle breakpoint
- m – open module
- ! – Jump into interactive shell (most useful)
- / – text search
There are obviously more then that, but they are what I mostly use. The open module is great if you need to set a breakpoint somewhere deeper in the code base, so you can open it, set a breakpoint and then happily press ‘c’ to continue until it hits. The ‘!’ is the most useful, it’ll jump you into an interactive python shell in the exact point the debugger is at. So you can jump around, check/change settings and poke in areas to see whats happening.
As with pdb you can also use code to insert a breakpoint so pudb will be triggered rather then having to start a script with pudb. I give an example of how in the nosetest section below.
nosetests + pudb
Sometimes the best way to use pudb is to debug unit tests, or even write a unit (or functaional or probe) test to get you into an area you want to test. You can use pudb to debug these too. And there are 2 ways to do it.
The first way is by installing the ‘nose-pudb’ pip package:
pip install nose-pudb
Now when you run nosetests you can add the –pudb option and it’ll break into pudb if there is an error, so you go poke around in ‘post-mortem’ mode. This is really useful, but doesn’t allow you to actually trace the tests as they run.
So the other way of using pudb in nosetests is actually insert some code in the test that will trigger as a breakpoint and start up pudb. To do so is exactly how you would with pdb, except substitute for pudb. So just add the following line of code to your test where you want to drop into pudb:
import pudb; pudb.set_trace()
And that’s it.. well mostly, because pudb is command line you need to tell nosetests to not capture stdout with the ‘-s’ flag:
nosetests -s test/unit/common/middleware/test_cname_lookup.py
testr + pudb
Not problem here, it uses the same approach as above. Where you programmatically set a trace, as you would for pdb. Just follow the ‘Debugging (pdb) Tests’ section on this page (except substitute pdb for pudb)
Update – run_until_failure.sh
I’ve been trying to find some intermittent unit test failures recently. So I whipped up a quick bash script that I run in a tmux session that really helps find and deal with them, I thought I’d add to this post as I then can add nose-pudb to make it pretty useful.
#!/bin/bash n=0 while [ True ] do clear $@ if [ $? -gt 0 ] then echo 'ERROR' echo "number " $n break fi let "n=n+1" sleep 1 done
With this I can simply:
run_until_failure.sh tox -epy27
It’ll stop looping once the command passed returns something other then 0.
Once I have an error, I have then been focusing in on the area it happens (to speed up the search a bit), I can also use nose-pudb to drop me into post-mortem mode so I can poke around in ipython, for example, I’m currently running:
run_until_failure.sh nosetests --pudb test/unit/proxy/test_server.py
Then I can come back to the tmux session, if I’m dropped in a pudb interface, I can go poke around.