Lunch Time Python¶
Lunch 4: pytest¶
pytest is a widely used Python test framework, which makes it easy to write small and readable tests, and also offers more advanced features such as fixtures and mocks. There is also a large ecosystem of plugins providing additional functionality.
Press Spacebar
to go to the next slide (or ?
to see all navigation shortcuts)
Lunch Time Python, Scientific Software Center, Heidelberg University
Why write tests?¶
- ensure correctness
- maintain correctness
- find bugs earlier and more easily
- allow refactoring without fear
- allow others to contribute without unknowingly breaking stuff
- can complement documentation as examples of use
- give others confidence in your code
pytest installation¶
- Conda:
conda install pytest
- Pip:
python -m pip install pytest
Run the tests¶
- run
pytest -v
orpython -m pytest -v
================ test session starts =======================
platform linux -- Python 3.10.2, pytest-7.0.0, pluggy-1.0.0
rootdir: /home/liam/test
plugins: anyio-3.5.0
collected 1 item
test_math.py::test_add PASSED
[100%]
====================== 1 passed in 0.00s ====================
What just happened?¶
- pytest looks for all files that start with
test_
- it collects all functions in these files that start with
test_
- it runs them all, and reports PASS/FAIL for each assertion in each function
Some things we didn't do¶
- import a test library
- inherit from some base test class
- use some special
assertEqual
function - register our test file or test cases
Simple pytest test strategy¶
- for each file
abc.py
, add atest_abc.py
- for each function
foo()
inabc.py
, add atest_foo()
totest_abc.py
- inside
test_foo()
assert things involvingfoo()
that should be true - that's it
ipytest¶
disclaimer¶
- For demonstration purposes, these tests will be written inside this jupyter notebook
- We'll use a helper library
ipytest
to call pytest on them from inside the notebook - But generally I would recommend putting functions and tests into files and running pytest directly
In [1]:
if "google.colab" in str(get_ipython()):
!pip install ipytest -qqq
import ipytest
import pytest
ipytest.autoconfig()
In [2]:
%%ipytest -vv
def test_math():
assert 1 + 1 == 2
======================================= test session starts ======================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.10.14/x64/bin/python3 cachedir: .pytest_cache rootdir: /home/runner/work/lunch-time-python/lunch-time-python configfile: pytest.ini plugins: dash-2.17.1, anyio-3.6.2 collecting ... collected 1 item t_b8e4d078ebe942be89f60b5046bf47db.py::test_math PASSED [100%] ======================================== 1 passed in 0.01s =========================================
Failing tests¶
In [3]:
%%ipytest -q
def f(x):
return 2 * x
def g(x):
return f(x) + 3
def test_math():
a = 2
b = 3
assert f(a) * g(b) == 36
. [100%]
In [4]:
%%ipytest -q
def test_list():
a = [1, 2, 5, 8]
b = [1, 2, 5, 8]
assert a == b
. [100%]
Exceptions¶
In [5]:
%%ipytest -q
def test_exception():
my_list = [1, 2, 3]
with pytest.raises(IndexError):
my_list[5]
. [100%]
In [6]:
%%ipytest -q
def test_exception():
my_list = [1, 2, 3]
with pytest.raises(Exception) as e:
my_list[5]
assert e.type == IndexError
assert "out of range" in str(e.value)
. [100%]
Temporary files¶
- often need to write to a temporary file in a test
- simply add
tmp_path
as an argument to your test function - pytest will provide a unique temporary path object for each test
- this is an example of a fixture
In [7]:
%%ipytest -qs
def test_write(tmp_path):
print(tmp_path)
assert str(tmp_path) != ""
/tmp/pytest-of-runner/pytest-0/test_write0
.
Monkey-patching¶
- a fixture to temporarily modify an object, dict or environment variable
- all modifications are undone after the test is finished
- add
monkeypatch
as an argument to your test function - provides various methods, e.g.
monkeypatch.setattr(obj, name, value)
monkeypatch.setenv(name, value)
monkeypatch.syspath_prepend(path)
monkeypatch.chdir(path)
In [8]:
%%ipytest -qs
import os
def test_env(monkeypatch):
assert os.getenv("TEST_API_KEY") == None
monkeypatch.setenv("TEST_API_KEY", "abc123")
assert os.getenv("TEST_API_KEY") == "abc123"
.
Fixtures¶
- a way to provide context (e.g. data or environment) to a test
- test "requests" a fixture by declaring it as an argument
- various built-in fixtures (
tmp_path
,monkeypatch
, ...) - you can create your own with the
@pytest.fixture
decorator - for each test function argument, pytest looks for a fixture with the same name
- fixtures can themselves request other fixtures
In [9]:
%%ipytest -vv
# a fixture to provide some data to a test
@pytest.fixture
def colours():
return ["red", "green", "blue"]
def test_colours(colours):
assert colours[0] == "red"
======================================= test session starts ======================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.10.14/x64/bin/python3 cachedir: .pytest_cache rootdir: /home/runner/work/lunch-time-python/lunch-time-python configfile: pytest.ini plugins: dash-2.17.1, anyio-3.6.2 collecting ... collected 1 item t_b8e4d078ebe942be89f60b5046bf47db.py::test_colours PASSED [100%] ======================================== 1 passed in 0.01s =========================================
In [10]:
%%ipytest -vv
@pytest.fixture
def colours():
return ["red", "green", "blue"]
# a fixture that itself requests another fixture
@pytest.fixture
def sorted_colours(colours):
return sorted(colours)
def test_colours(sorted_colours):
assert sorted_colours[0] == "blue"
======================================= test session starts ======================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.10.14/x64/bin/python3 cachedir: .pytest_cache rootdir: /home/runner/work/lunch-time-python/lunch-time-python configfile: pytest.ini plugins: dash-2.17.1, anyio-3.6.2 collecting ... collected 1 item t_b8e4d078ebe942be89f60b5046bf47db.py::test_colours PASSED [100%] ======================================== 1 passed in 0.01s =========================================
In [11]:
%%ipytest -vv
# a fixture that uses monkeypatch to set an environment variable
@pytest.fixture
def api_key(monkeypatch):
monkeypatch.setenv("TEST_API_KEY", "abc123")
def test_missing_api_key():
assert os.getenv("TEST_API_KEY") == None
def test_api_key(api_key):
assert os.getenv("TEST_API_KEY") == "abc123"
======================================= test session starts ======================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.10.14/x64/bin/python3 cachedir: .pytest_cache rootdir: /home/runner/work/lunch-time-python/lunch-time-python configfile: pytest.ini plugins: dash-2.17.1, anyio-3.6.2 collecting ... collected 2 items t_b8e4d078ebe942be89f60b5046bf47db.py::test_missing_api_key PASSED [ 50%] t_b8e4d078ebe942be89f60b5046bf47db.py::test_api_key PASSED [100%] ======================================== 2 passed in 0.01s =========================================
In [12]:
%%ipytest -vv
# a parameterized fixture: test will be repeated for each parameter
@pytest.fixture(params=["red", "green", "blue", "yellow"])
def colour(request):
return request.param
def test_colour(colour):
assert len(colour) >= 3
======================================= test session starts ======================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.10.14/x64/bin/python3 cachedir: .pytest_cache rootdir: /home/runner/work/lunch-time-python/lunch-time-python configfile: pytest.ini plugins: dash-2.17.1, anyio-3.6.2 collecting ... collected 4 items t_b8e4d078ebe942be89f60b5046bf47db.py::test_colour[red] PASSED [ 25%] t_b8e4d078ebe942be89f60b5046bf47db.py::test_colour[green] PASSED [ 50%] t_b8e4d078ebe942be89f60b5046bf47db.py::test_colour[blue] PASSED [ 75%] t_b8e4d078ebe942be89f60b5046bf47db.py::test_colour[yellow] PASSED [100%] ======================================== 4 passed in 0.02s =========================================
Test grouping¶
- put functions into a class whose name begins with
Test
- a class can request a fixture, all member functions then have this fixture
In [13]:
%%ipytest -vv
class TestMath:
def test_add(self):
assert 1 + 1 == 2
def test_mul(self):
assert 2 * 2 == 4
======================================= test session starts ======================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.10.14/x64/bin/python3 cachedir: .pytest_cache rootdir: /home/runner/work/lunch-time-python/lunch-time-python configfile: pytest.ini plugins: dash-2.17.1, anyio-3.6.2 collecting ... collected 2 items t_b8e4d078ebe942be89f60b5046bf47db.py::TestMath::test_add PASSED [ 50%] t_b8e4d078ebe942be89f60b5046bf47db.py::TestMath::test_mul PASSED [100%] ======================================== 2 passed in 0.01s =========================================
Marking tests¶
- mark tests with attributes using
@pytest.mark
decorator - common use cases
skipif
to conditionally skip a test- e.g. depending on python version or platform
xfail
to mark a test that is expected to fail- e.g. a test that documents a known bug that is not yet fixed
- can also mark a test class to mark all tests within that class
- can also create your own custom markers
In [14]:
%%ipytest -vv
import sys
@pytest.mark.xfail(reason="bug from issue #123")
def test_add():
assert 1 + 1 == 3
@pytest.mark.skipif(not sys.platform.startswith("win"), reason="windows only test")
def test_mul():
assert 2 * 2 == 4
======================================= test session starts ======================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.10.14/x64/bin/python3 cachedir: .pytest_cache rootdir: /home/runner/work/lunch-time-python/lunch-time-python configfile: pytest.ini plugins: dash-2.17.1, anyio-3.6.2 collecting ... collected 2 items t_b8e4d078ebe942be89f60b5046bf47db.py::test_add XFAIL (bug from issue #123) [ 50%] t_b8e4d078ebe942be89f60b5046bf47db.py::test_mul SKIPPED (windows only test) [100%] ================================== 1 skipped, 1 xfailed in 0.10s ===================================
Parameterizing tests¶
- can parameterize tests using the
@pytest.mark.parameterize
decorator - takes comma-delimeted list of arguments as a string
- followed by a list of tuples of argument values
In [15]:
%%ipytest -vv
@pytest.mark.parametrize("n", [1, 2, 3])
def test_n(n):
assert n > 0
======================================= test session starts ======================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.10.14/x64/bin/python3 cachedir: .pytest_cache rootdir: /home/runner/work/lunch-time-python/lunch-time-python configfile: pytest.ini plugins: dash-2.17.1, anyio-3.6.2 collecting ... collected 3 items t_b8e4d078ebe942be89f60b5046bf47db.py::test_n[1] PASSED [ 33%] t_b8e4d078ebe942be89f60b5046bf47db.py::test_n[2] PASSED [ 66%] t_b8e4d078ebe942be89f60b5046bf47db.py::test_n[3] PASSED [100%] ======================================== 3 passed in 0.02s =========================================
In [16]:
%%ipytest -vv
@pytest.mark.parametrize("n,n_squared", [(1, 1), (2, 4), (3, 9)])
def test_n(n, n_squared):
assert n * n == n_squared
======================================= test session starts ======================================== platform linux -- Python 3.10.14, pytest-8.2.2, pluggy-1.5.0 -- /opt/hostedtoolcache/Python/3.10.14/x64/bin/python3 cachedir: .pytest_cache rootdir: /home/runner/work/lunch-time-python/lunch-time-python configfile: pytest.ini plugins: dash-2.17.1, anyio-3.6.2 collecting ... collected 3 items t_b8e4d078ebe942be89f60b5046bf47db.py::test_n[1-1] PASSED [ 33%] t_b8e4d078ebe942be89f60b5046bf47db.py::test_n[2-4] PASSED [ 66%] t_b8e4d078ebe942be89f60b5046bf47db.py::test_n[3-9] PASSED [100%] ======================================== 3 passed in 0.02s =========================================
Summary¶
- pytest is very easy to get started with and use
- just writing test functions with assertions already provides a lot of value
- fixtures allow you to provide context to your test functions
- parameterizing fixtures and/or tests can turn a single test into many test cases
- (many) more features at docs.pytest.org