Had a rough day today. I just wanted a log. Not just any log mind you, but one that could handle writes from multiple processes running at the same time. A naive me would have put python’s basic log handler within each process and then watch as each of the processes crash and burn because of conflicting disk write access.

But I’ve learned from my past lessons - now I use the python SocketHandler for my logging needs. Here’s a basic snippet below to get you started.

# ============== Socket Logger =============== #
import logging
import logging.handlers # you NEED this line
logger = logging.getLogger("%s_%s" % (os.getpid(), sys.argv[1]) )
logger.setLevel(logging.DEBUG)
socketHandler = logging.handlers.SocketHandler('localhost',
                    logging.handlers.DEFAULT_TCP_LOGGING_PORT)
logger.addHandler(socketHandler)
# ============== Socket Logger =============== #

In a nutshell, it gets a logger instance, sets the logging level to DEBUG, and then attaches a SocketHandler to itself. This means that whenever any piece of your code calls logger.debug(“this is my log message”) , it will send that message through the pipes to your server sitting on the other end which will handle writing your log messages to disk. On most machines the DEFAULT_TCP_LOGGING_PORT will be 9020 and the server will be sitting on localhost:9020 .

The Log Server

I was surprised that python didn’t already have a canonical implementation of a logging server to interface “SocketHandler” available in the standard library. Luckily there’s a pretty neat project that does just that: python-loggingserver. As a bonus it comes with a web interface to view your logs. On most systems, you can access the website by going to http://localhost:9021/ once you’ve started the server.

It requires the twisted networking library so if you haven’t had that installed do a

sudo easy_install twisted

Then svn checkout the project:

svn checkout http://python-loggingserver.googlecode.com/svn/trunk/ logserver

Start the server:

cd logserver
twistd --pidfile=loggingserver.pid --logfile=logginserver.log --python=loggingserver.py

Now you’re ready to log. You can now use the code snippet posted at the beginning of this article or just use the prepackaged testing script to send messages to the server:

python loggingtest.py this_is_one_process

View the results at http://localhost:9021

Happy logging!