• SSC Lunch Time Python
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14

Lunch Time Python¶

Lunch 4: pytest¶

No description has been provided for this image

pytest is a widely used Python test framework, which makes it easy to write small and readable tests, and also offers more advanced features such as fixtures and mocks. There is also a large ecosystem of plugins providing additional functionality.

Press Spacebar to go to the next slide (or ? to see all navigation shortcuts)

Lunch Time Python, Scientific Software Center, Heidelberg University

Why write tests?¶

  • ensure correctness
  • maintain correctness
  • find bugs earlier and more easily
  • allow refactoring without fear
  • allow others to contribute without unknowingly breaking stuff
  • can complement documentation as examples of use
  • give others confidence in your code

pytest installation¶

  • Conda: conda install pytest
  • Pip: python -m pip install pytest

First steps¶

Create a test¶

  1. create a file that starts with test_, e.g. test_math.py
  2. add a function to it that starts with test_ and asserts things, e.g.
# in file: test_math.py
def test_add():
        assert 1 + 1 == 2

Run the tests¶

  1. run pytest -v or python -m pytest -v
================ test session starts =======================
platform linux -- Python 3.10.2, pytest-7.0.0, pluggy-1.0.0
rootdir: /home/liam/test
plugins: anyio-3.5.0
collected 1 item                                                                                                                                                                                  

test_math.py::test_add PASSED  
                                                       [100%]

====================== 1 passed in 0.00s ====================

What just happened?¶

  • pytest looks for all files that start with test_
  • it collects all functions in these files that start with test_
  • it runs them all, and reports PASS/FAIL for each assertion in each function

Some things we didn't do¶

  • import a test library
  • inherit from some base test class
  • use some special assertEqual function
  • register our test file or test cases

Simple pytest test strategy¶

  • for each file abc.py, add a test_abc.py
  • for each function foo() in abc.py, add a test_foo() to test_abc.py
  • inside test_foo() assert things involving foo() that should be true
  • that's it

ipytest¶

disclaimer¶

  • For demonstration purposes, these tests will be written inside this jupyter notebook
  • We'll use a helper library ipytest to call pytest on them from inside the notebook
  • But generally I would recommend putting functions and tests into files and running pytest directly
In [1]:
if "google.colab" in str(get_ipython()):
    !pip install ipytest -qqq
import ipytest
import pytest

ipytest.autoconfig()
In [2]:
%%ipytest -vv


def test_math():
    assert 1 + 1 == 2
======================================= test session starts ========================================
platform linux -- Python 3.11.11, pytest-8.3.4, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.11.11/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/lunch-time-python/lunch-time-python
configfile: pytest.ini
plugins: anyio-4.6.2.post1, dash-2.18.2
collecting ... 
collected 1 item

t_e84a8210ab5448f490f08e002ec41699.py::test_math 
PASSED                                      [100%]

======================================== 1 passed in 0.01s =========================================

Failing tests¶

In [3]:
%%ipytest -q


def f(x):
    return 2 * x


def g(x):
    return f(x) + 3


def test_math():
    a = 2
    b = 3
    assert f(a) * g(b) == 36
.
                                                                                            [100%]

In [4]:
%%ipytest -q


def test_list():
    a = [1, 2, 5, 8]
    b = [1, 2, 5, 8]
    assert a == b
.
                                                                                            [100%]

Exceptions¶

In [5]:
%%ipytest -q


def test_exception():
    my_list = [1, 2, 3]
    with pytest.raises(IndexError):
        my_list[5]
.
                                                                                            [100%]

In [6]:
%%ipytest -q


def test_exception():
    my_list = [1, 2, 3]
    with pytest.raises(Exception) as e:
        my_list[5]
    assert e.type == IndexError
    assert "out of range" in str(e.value)
.
                                                                                            [100%]

Temporary files¶

  • often need to write to a temporary file in a test
  • simply add tmp_path as an argument to your test function
  • pytest will provide a unique temporary path object for each test
  • this is an example of a fixture
In [7]:
%%ipytest -qs


def test_write(tmp_path):
    print(tmp_path)
    assert str(tmp_path) != ""
/tmp/pytest-of-runner/pytest-0/test_write0
.

Monkey-patching¶

  • a fixture to temporarily modify an object, dict or environment variable
  • all modifications are undone after the test is finished
  • add monkeypatch as an argument to your test function
  • provides various methods, e.g.
    • monkeypatch.setattr(obj, name, value)
    • monkeypatch.setenv(name, value)
    • monkeypatch.syspath_prepend(path)
    • monkeypatch.chdir(path)
In [8]:
%%ipytest -qs

import os


def test_env(monkeypatch):
    assert os.getenv("TEST_API_KEY") == None
    monkeypatch.setenv("TEST_API_KEY", "abc123")
    assert os.getenv("TEST_API_KEY") == "abc123"
.

Fixtures¶

  • a way to provide context (e.g. data or environment) to a test
  • test "requests" a fixture by declaring it as an argument
  • various built-in fixtures (tmp_path, monkeypatch, ...)
  • you can create your own with the @pytest.fixture decorator
  • for each test function argument, pytest looks for a fixture with the same name
  • fixtures can themselves request other fixtures
In [9]:
%%ipytest -vv

# a fixture to provide some data to a test
@pytest.fixture
def colours():
    return ["red", "green", "blue"]


def test_colours(colours):
    assert colours[0] == "red"
======================================= test session starts ========================================
platform linux -- Python 3.11.11, pytest-8.3.4, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.11.11/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/lunch-time-python/lunch-time-python
configfile: pytest.ini
plugins: anyio-4.6.2.post1, dash-2.18.2
collecting ... 
collected 1 item

t_e84a8210ab5448f490f08e002ec41699.py::test_colours 
PASSED                                   [100%]

======================================== 1 passed in 0.01s =========================================
In [10]:
%%ipytest -vv


@pytest.fixture
def colours():
    return ["red", "green", "blue"]


# a fixture that itself requests another fixture
@pytest.fixture
def sorted_colours(colours):
    return sorted(colours)


def test_colours(sorted_colours):
    assert sorted_colours[0] == "blue"
======================================= test session starts ========================================
platform linux -- Python 3.11.11, pytest-8.3.4, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.11.11/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/lunch-time-python/lunch-time-python
configfile: pytest.ini
plugins: anyio-4.6.2.post1, dash-2.18.2
collecting ... 
collected 1 item

t_e84a8210ab5448f490f08e002ec41699.py::test_colours 
PASSED                                   [100%]

======================================== 1 passed in 0.01s =========================================
In [11]:
%%ipytest -vv

# a fixture that uses monkeypatch to set an environment variable
@pytest.fixture
def api_key(monkeypatch):
    monkeypatch.setenv("TEST_API_KEY", "abc123")


def test_missing_api_key():
    assert os.getenv("TEST_API_KEY") == None


def test_api_key(api_key):
    assert os.getenv("TEST_API_KEY") == "abc123"
======================================= test session starts ========================================
platform linux -- Python 3.11.11, pytest-8.3.4, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.11.11/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/lunch-time-python/lunch-time-python
configfile: pytest.ini
plugins: anyio-4.6.2.post1, dash-2.18.2
collecting ... 
collected 2 items

t_e84a8210ab5448f490f08e002ec41699.py::test_missing_api_key 
PASSED                           [ 50%]
t_e84a8210ab5448f490f08e002ec41699.py::test_api_key 
PASSED                                   [100%]

======================================== 2 passed in 0.01s =========================================
In [12]:
%%ipytest -vv

# a parameterized fixture: test will be repeated for each parameter
@pytest.fixture(params=["red", "green", "blue", "yellow"])
def colour(request):
    return request.param


def test_colour(colour):
    assert len(colour) >= 3
======================================= test session starts ========================================
platform linux -- Python 3.11.11, pytest-8.3.4, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.11.11/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/lunch-time-python/lunch-time-python
configfile: pytest.ini
plugins: anyio-4.6.2.post1, dash-2.18.2
collecting ... 
collected 4 items

t_e84a8210ab5448f490f08e002ec41699.py::test_colour[red] 
PASSED                               [ 25%]
t_e84a8210ab5448f490f08e002ec41699.py::test_colour[green] 
PASSED                             [ 50%]
t_e84a8210ab5448f490f08e002ec41699.py::test_colour[blue] 
PASSED                              [ 75%]
t_e84a8210ab5448f490f08e002ec41699.py::test_colour[yellow] 
PASSED                            [100%]

======================================== 4 passed in 0.02s =========================================

Test grouping¶

  • put functions into a class whose name begins with Test
  • a class can request a fixture, all member functions then have this fixture
In [13]:
%%ipytest -vv


class TestMath:
    def test_add(self):
        assert 1 + 1 == 2

    def test_mul(self):
        assert 2 * 2 == 4
======================================= test session starts ========================================
platform linux -- Python 3.11.11, pytest-8.3.4, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.11.11/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/lunch-time-python/lunch-time-python
configfile: pytest.ini
plugins: anyio-4.6.2.post1, dash-2.18.2
collecting ... 
collected 2 items

t_e84a8210ab5448f490f08e002ec41699.py::TestMath::test_add 
PASSED                             [ 50%]
t_e84a8210ab5448f490f08e002ec41699.py::TestMath::test_mul 
PASSED                             [100%]

======================================== 2 passed in 0.01s =========================================

Marking tests¶

  • mark tests with attributes using @pytest.mark decorator
  • common use cases
    • skipif to conditionally skip a test
      • e.g. depending on python version or platform
    • xfail to mark a test that is expected to fail
      • e.g. a test that documents a known bug that is not yet fixed
  • can also mark a test class to mark all tests within that class
  • can also create your own custom markers
In [14]:
%%ipytest -vv

import sys


@pytest.mark.xfail(reason="bug from issue #123")
def test_add():
    assert 1 + 1 == 3


@pytest.mark.skipif(not sys.platform.startswith("win"), reason="windows only test")
def test_mul():
    assert 2 * 2 == 4
======================================= test session starts ========================================
platform linux -- Python 3.11.11, pytest-8.3.4, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.11.11/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/lunch-time-python/lunch-time-python
configfile: pytest.ini
plugins: anyio-4.6.2.post1, dash-2.18.2
collecting ... 
collected 2 items

t_e84a8210ab5448f490f08e002ec41699.py::test_add 
XFAIL (bug from issue #123)                  [ 50%]
t_e84a8210ab5448f490f08e002ec41699.py::test_mul 
SKIPPED (windows only test)                  [100%]

================================== 1 skipped, 1 xfailed in 0.08s ===================================

Parameterizing tests¶

  • can parameterize tests using the @pytest.mark.parameterize decorator
  • takes comma-delimeted list of arguments as a string
  • followed by a list of tuples of argument values
In [15]:
%%ipytest -vv


@pytest.mark.parametrize("n", [1, 2, 3])
def test_n(n):
    assert n > 0
======================================= test session starts ========================================
platform linux -- Python 3.11.11, pytest-8.3.4, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.11.11/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/lunch-time-python/lunch-time-python
configfile: pytest.ini
plugins: anyio-4.6.2.post1, dash-2.18.2
collecting ... 
collected 3 items

t_e84a8210ab5448f490f08e002ec41699.py::test_n[1] 
PASSED                                      [ 33%]
t_e84a8210ab5448f490f08e002ec41699.py::test_n[2] 
PASSED                                      [ 66%]
t_e84a8210ab5448f490f08e002ec41699.py::test_n[3] 
PASSED                                      [100%]

======================================== 3 passed in 0.01s =========================================
In [16]:
%%ipytest -vv


@pytest.mark.parametrize("n,n_squared", [(1, 1), (2, 4), (3, 9)])
def test_n(n, n_squared):
    assert n * n == n_squared
======================================= test session starts ========================================
platform linux -- Python 3.11.11, pytest-8.3.4, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.11.11/x64/bin/python
cachedir: .pytest_cache
rootdir: /home/runner/work/lunch-time-python/lunch-time-python
configfile: pytest.ini
plugins: anyio-4.6.2.post1, dash-2.18.2
collecting ... 
collected 3 items

t_e84a8210ab5448f490f08e002ec41699.py::test_n[1-1] 
PASSED                                    [ 33%]
t_e84a8210ab5448f490f08e002ec41699.py::test_n[2-4] 
PASSED                                    [ 66%]
t_e84a8210ab5448f490f08e002ec41699.py::test_n[3-9] 
PASSED                                    [100%]

======================================== 3 passed in 0.01s =========================================

Summary¶

  • pytest is very easy to get started with and use
  • just writing test functions with assertions already provides a lot of value
  • fixtures allow you to provide context to your test functions
  • parameterizing fixtures and/or tests can turn a single test into many test cases
  • (many) more features at docs.pytest.org