Logging

Sprinkling print statements throughout your code is a quick and dirty way of tracking code execution when debugging, but it’s time-consuming to keep on adding and removing them. It also isn’t sustainable for larger or more complex pieces of software.

Instead, Python’s logging module provides a cleaner and easier method.

Essentially, you add log statements throughout your code, set to different levels. Then, when running your code, you can set the log level, which specifies which subsets of these statements should actually run. The outputs can be piped to the console, to a text file, or somewhere else.

Quick Start

We can set up a logger with logging.basicConfig, which automatically:

• sets the output to the console
• defines the default log message format

From there, whenever you want to log a message, you can use one of the following methods:

• debug() for detailed debugging info
• info() for general operating information
• warning() for a current or impending issue or error, but where the software will continue to run
• error() for a serious problem which breaks functionality
• critical() for a serious problem which may stop the program running altogether

Then, at any point in the code, you can set the log level, using e.g. logger.setLevel(logger.WARNING). This will output only logs of severity WARNING or above, for example.

Details

If not using basicConfig, or if you want more granular control, use the following options.

Logger Definition

The name of the logger can be specified, but you have to be careful to avoid namespace collisions. The best way to set the name is to use __name__, for example

import logging

logger = logging.getLogger(__name__)

Output

The default output is sys.stderr. Normally, if logging, you want to pipe output to the console, which is done with

stream_handler = logging.StreamHandler(sys.stdout)
logger.addHandler(stream_handler)

If you instead (or also) want to save the logs in a file, use logging.FileHandler(filename.log) and add as a handler. Every handler will run, so to save to multiple files, add multiple handlers!

Formatting

The format of the log message must be specified for each output stream. The formatter is specified with

formatter = logging.Formatter('<FORMAT>')

And then bound to an output handler with

stream_handler.setFormatter(formatter)

The default formatting is %(levelname)s:%(name)s:%(message)s:

• %(levelname)s is a string for log level
• %(name)s is the logger name
• %(message)s is the logged message

Formatstring can also include

• %(asctime)s for the timestamp as a string
• %(lineno)d for the line number as a digit

Example

import logging

logging.basicConfig(level=logging.DEBUG)

logging.debug("Starting main file")

def divide(a:float, b:float):
    logging.debug("Check if b is zero")
    if b==0:
        logging.error("b cannot be zero!")
        return
    else:
        return a/b

logging.debug("Testing")

assert divide(10, 2) == 5
assert divide(10, 0) == None