Andreas Bergström

Full Stack Web Developer

Running tests in parallel with Django and PyCharm

2019-08-08 - 880 words - 5 minutes to read

These days it is hard to come across any consumer device that only has one CPU-core. And even though most computers released during the last decade comes with multiple cores, there are plenty of software that defaults to non-parallel execution. The unittest package in Python’s standard library is sadly one of those.

But there is still hope. The powerful pytest framework and Django’s own testing framework does support running test in parallel, which can vastly decrease the time required to run through a suite of tests. Let us take a look at the latter and see how we can integrate it into PyCharm’s interface.

The testing framework in Django is based on unittest, but comes with its own testrunner which handles parallel test execution. It is not the default configuration so you have to provide the –parallel flag, this is what the documentation says:

–parallel [N]

Runs tests in separate parallel processes. Since modern processors have multiple cores, this allows running tests significantly faster.

By default –parallel runs one process per core according to multiprocessing.cpu_count(). You can adjust the number of processes either by providing it as the option’s value, e.g. –parallel=4, or by setting the DJANGO_TEST_PROCESSES environment variable.

So by just doing manage.py test –parallel the tests should run in parallel optimized according to the amount of available cores. By using multiprocessing.cpu_count() in a REPL prompt I can determine that Python sees 4 cores on my system:

>> import multiprocessing
>> multiprocessing.cpu_count()
4

This means that if I have 4 test cases which each take around 1 second to finish, they should be able run in parallel and finish in about 1 second instead of 4 seconds. Here are the simple dummy tests:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import time

from django.test import TestCase


class TestDummy(TestCase):
    def test_dummy(self):
        time.sleep(1)
        self.assertEqual(1, 1)


class TestDummy2(TestCase):
    def test_dummy(self):
        time.sleep(1)
        self.assertEqual(1, 1)


class TestDummy3(TestCase):
    def test_dummy(self):
        time.sleep(1)
        self.assertEqual(1, 1)


class TestDummy4(TestCase):
    def test_dummy(self):
        time.sleep(1)
        self.assertEqual(1, 1)

Save it in a app where Django can locate it and run them by:

(venv) ➜  paralleltests git:(master) ✗ python manage.py test
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
....
----------------------------------------------------------------------
Ran 4 tests in 4.014s

OK
Destroying test database for alias 'default'...

We can see that the 4 tests executed one by one taking 4 seconds. Let us now add the –parallel flag:

(venv) ➜  paralleltests git:(master) ✗ python manage.py test --parallel
Creating test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
System check identified no issues (0 silenced).
....
----------------------------------------------------------------------
Ran 4 tests in 1.071s

OK
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...
Destroying test database for alias 'default'...

This time the 4 tests only took around 1 second to execute, that’s a pretty significant performance improvement! For larger projects and 6 or 8-core CPUs (with 12 or 16 cores visible to the OS), this simple flag can save huge amounts of time on larger tests suites.

Configuring PyCharm

If you are using PyCharm you might wish to integrate the –parallel flag into its run configuration interface, so you can run parallel tests straight from the IDE.

There seems to be some gotchas to this however. If you open up a default run configuration it should look something similar to this:

PyCharm scratch files in Project view

Enable the options field and enter “–parallel” into it:

PyCharm scratch files in Project view

That should do it but if you save it and run your tests, you will likely get this error:

django_test_manage.py test: error: argument --parallel: invalid int value: 'core.tests'

Huh, why does it seem like the argument is no longer optional? If you look at the top of the current run session, you will see the command PyCharm used to initiate manage.py test:

... /pycharm/django_test_manage.py" test --parallel core.tests ...

PyCharm injects the option flags before the target which will by accident supply the target as a argument to –parallel. The proper order should be test core.tests –parallel.

There are 2 solutions to this. Either simply remove the target field in the run configuration:

PyCharm scratch files in Project view

Or just supply the argument to –parallel (the amount of parallel processes that should be launched):

PyCharm scratch files in Project view

There is no potential performance overhead to this, as the documentation states the amount of actual launched processed will never be more than the amount of test cases:

Django distributes test cases — unittest.TestCase subclasses — to subprocesses. If there are fewer test cases than configured processes, Django will reduce the number of processes accordingly.

Finally, test cases that are executed parallel should not be independent on each other in any way. As the order which they are launched can be completely random no test case should expect another test case to have finished or even started. Neither should they depend on the same resources such as files. Also note that each test case will have its own database, as opposed to non-parallel execution.

If you are using pytest and want to run the tests in parallel from PyCharm, see the documentation

comments powered by Disqus