The popular tools and techniques used for maintaining cross-version compatibility

Maintaining compatibility between versions of Python is a challenge. It may add a lot of additional work depending on the size of the project, but is definitely doable and worth doing. For packages that are meant to be reused in many environments it is absolutely a must-have. Open source packages without well-defined and tested compatibility bounds are very unlikely to become popular, but closed third-party code that never leaves the company network can also greatly benefit from being tested in different environments.

It should be noted here that, while this part focuses mainly on compatibility between various versions of Python, these approaches apply for maintaining compatibility with external dependencies such as different package versions, binary libraries, systems, or external services.

The whole process can be divided into three main areas, ordered by their importance:

• Defining and documenting target compatibility bounds and how they will be managed
• Testing in every environment and with every dependency version declared as compatible
• Implementing actual compatibility code

Declaration of what is considered compatible is the most important part of the whole process because it gives your code users (developers) the ability to have expectations and make assumptions on how it works and how it can change in the future. Our code can be used as a dependency in different projects that may also strive to manage compatibility, so the ability to reason how it behaves is crucial.

While this book tries to always give a few choices and not to give absolute recommendations on specific options, here is one of the few exceptions. The best way to define how compatibility may change in the future is by using proper approach to versioning numbers using Semantic Versioning (semver) (http://semver.org/). It describes a broadly accepted standard for marking scope of changes in code by the version specifier, consisting only of three numbers. It also gives some advice on how to handle deprecation policies. Here is an excerpt from its summary (licensed under Creative Commons - CC BY 3.0):

When it comes to testing the sad truth, that is, to be sure that code is compatible with every declared dependency version and in every environment (here Python version), it must be tested in every combination of these. This, of course, may not be possible when the project has a lot of dependencies, because the number of combinations grows rapidly with every new dependency version. So, typically some trade-off needs to be made so that running full compatibility tests does not need to take ages. The selection of tools that help testing in so-called matrixes is presented in Chapter 12, Test-Driven Development, which discusses testing in general.

The implementation of the compatibility layer is the last, and also the least important, step of the process if the bounds of that compatibility are well-defined and rigorously tested. Still, there are some tools and techniques that every programmer interested in such a topic should know.

The most basic is Python's __future__ module. It backports some features from newer Python releases back into the older ones and takes the form of an import statement:

from __future__ import <feature>

Features provided by the future statements are syntax-related elements that cannot be easily handled by different means. This statement affects only the module where it was used. Here is an example of a Python 2.7 interactive session that brings Unicode literals from Python 3.0:

Python 2.7.10 (default, May 23 2015, 09:40:32) [MSC v.1500 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more 
information.
>>> type("foo")  # old literals
<type 'str'>
>>> from __future__ import unicode_literals
>>> type("foo")  # now is Unicode
<type 'unicode'>

Here is a list of all the available __future__ statement options that developers concerned with two-thirds compatibility should know:

• division: This adds a Python 3 division operator (PEP 238)
• absolute_import: This makes every form of an import statement not starting from dot character be interpreted as absolute imports (PEP 328)
• print_function: This changes a print statement into a function call so that parentheses around print become mandatory (PEP 3112)
• unicode_literals: This makes every string literal be interpreted as Unicode literals (PEP 3112)

A list of the __future__ statement options is very short, and it covers only a few syntax features. The other things that have changed, such as the metaclass syntax (which is an advanced feature that's covered in Chapter 5, Elements of Metaprogramming), are a lot harder to maintain. Reliable handling of multiple standard library reorganizations also cannot be solved by the future statements. Fortunately, there are some tools that aim to provide a consistent layer of ready-to-use compatibility code. The most well-known of these is Six (https://pypi.python.org/pypi/six/), which provides a whole common two-thirds compatibility boilerplate as a single module. The other promising, but slightly less popular, tool is the future module (http://python-future.org/).

In some situations, developers may not want to include additional dependencies in some small packages. A common practice is the additional module that gathers all the compatibility code, usually named compat.py. Here is an example of such compat modules taken from the python-gmaps project (https://github.com/swistakm/python-gmaps):

# -*- coding: utf-8 -*-
"""This module provides compatibility layer for
selected things that have changed across Python versions.
"""
import sys 
 
if sys.version_info < (3, 0, 0): 
    import urlparse  # noqa 
 
    def is_string(s):
        """Return True if given value is considered string"""
        return isinstance(s, basestring) 
 
else:
    # note: urlparse was moved to urllib.parse in Python 3
    from urllib import parse as urlparse  # noqa 
 
    def is_string(s):
        """Return True if given value is considered string"""
        return isinstance(s, str)

Such compat.py modules are popular, even in projects that depend on Six for two-thirds compatibility, because it is a very convenient way to store code that handles compatibility with different versions of packages being used as dependencies.

In the next section, we'll take a look at what CPython is.