Logging is one of those things that, being orthogonal to actually getting something done, many developers fail to learn the nuances of. So I want to go over a few things I had to learn the hard way:
We are blessed in the python community because we have the wonderful ‘logging’ module in our standard library, so there is no barrier to entry or excuse to not use proper logging mechanisms. There are often reasons to roll your own of something, that something will probably never be logging. Don’t do it (this goes for all major languages).
The logging module is incredibly flexible. The ‘handlers’ are the key to leveraging the power of the logging module. Handlers can do pretty much whatever you want them to do. Once you get past the most basic logging, you should start reading up on Handlers. Understanding handlers is the key to understanding logging, in my experience.
Root-level configuration should generally only be done by the application, not any library modules. Ie, ‘logging.basicConfig’ should only be (and usually can only be) called very early on. Examples of root-level configuration are setting the format of the logs, setting the logs to print to stdout/stderr, etc. Anything that has to do with global state (and streams are examples of global state), should be handled by the application, never by a library. Rarely should you add a StreamHandler. A FileHandler for a single logger can be useful in some cases (like, if you have a server that is part of a larger application) but should generally be avoided.
If you have multiple classes in a file, give them each their own logger. Do not use a single module logger for many classes. Identify the logger by the class name so you know what logger produced what log.
self.logger = logging.getLogger(type(self).__name__) on a base class is a good way to get a unique logger for each subclass, without each subclass having to set up their own logger.
logger.<methodname>('spam, eggs, and %s', myvar) should be used instead of
logger.<methodname>('spam, eggs, and %s' % myvar), as it saves a string formatting.
Make a module with your commonly used log format strings, so each developer doesn’t have to come up with their own, and you achieve some standardization.
Almost never use printing. Use logging, and set your logger(s) up to log to stdout with a StreamHandler while you are debugging. Then you can leave your ‘prints’ in, which will make life easier when you need to go back in to find bugs.
You almost never want to catch, log, and re-raise. Let the caller be responsible for logging and handling the error, at the level it can be handled properly. Imagine if at every level, every exception was logged and re-raised. Your log would be a mess!
I consider the levels are follows- DEBUG only for developers, INFO for general internal usage, WARNING for deployment (I don’t know why you’d have your log level set higher than WARNING). Another way of thinking about them is, DEBUG has all information which only developers care about, INFO has little enough information that the stuff there is relevant and enough that problems can be diagnosed by a technical person, and WARNING will just tell you when something goes wrong. I wouldn’t make any more fine-grained levels than this, but it is up to you and your team to figure out where to use what. For example, do you log every server and client send/recv as DEBUG or INFO? It depends, of course.
The more library-like your code, the less you generally log. Your library should be clear, working, and throw meaningful exceptions, so generally your real library-libraries shouldn’t even need to log.
Logging is not a replacement for raising exceptions. Logging is not a way to deal with exceptions, either.
Remember these are guidelines only (and my guidelines). There are always exceptions to these rules (no pun intended).
I have a feeling those of you writing web/server apps are more familiar with logging best practices than those of us writing code in client apps. But these are all things I’ve seen in the real world so I thought them worth giving my two cents about them. What are your logging guidelines?
Also published on Medium.