Parallel Test Execution using pytest-xdist and Selenium

Running Selenium tests in parallel can significantly reduce test execution time, especially for large test suites. pytest-xdist is a powerful plugin for pytest that allows you to distribute your tests across multiple CPUs or machines.

In this guide, we’ll look at how to set up parallel test execution using pytest-xdist and Selenium in a Python test framework.

✅ Prerequisites

Make sure you have the following installed:

bash

Copy

Edit

pip install pytest selenium pytest-xdist

Optional for better logging/debugging:

bash

Copy

Edit

pip install pytest-html

📁 Example Project Structure

Copy

Edit

tests/

├── test_google_search.py

├── test_login.py

conftest.py

🧪 Sample Selenium Test: test_google_search.py

python

Copy

Edit

from selenium import webdriver

from selenium.webdriver.common.by import By

import time

def test_google_search():

    driver = webdriver.Chrome()

    driver.get("https://www.google.com")

    search_box = driver.find_element(By.NAME, "q")

    search_box.send_keys("pytest-xdist")

    search_box.submit()

    time.sleep(2)

    assert "pytest-xdist" in driver.title

    driver.quit()

⚙️ Setting Up conftest.py (Optional Fixture)

python

Copy

Edit

import pytest

from selenium import webdriver

@pytest.fixture

def driver():

    driver = webdriver.Chrome()

    yield driver

    driver.quit()

You can then pass driver into your test functions.

🚀 Running Tests in Parallel

You can now run your tests in parallel using the -n flag:

bash

Copy

Edit

pytest -n 4

This tells pytest-xdist to run tests across 4 CPU cores.

Additional Options:

-n auto: Automatically detect the number of available CPU cores.

--dist loadscope: Group tests in the same module to the same worker (useful for test setup dependencies).

--maxfail=2: Stop the test suite after 2 failures.

🔍 Important Considerations

1. Thread Safety

Each test must be independent. Avoid shared resources (e.g., writing to the same file or database).

2. WebDriver Conflicts

Ensure a new browser instance per test, or use a Selenium Grid or remote driver for better scalability.

3. CI Integration

Parallel tests work well in CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins). Be mindful of:

Environment limits (e.g., Docker containers, virtual display/Xvfb for GUI browsers)

Setting PYTEST_ADDOPTS=-n auto in your test environment

🔧 Advanced: Using Selenium Grid with pytest-xdist

To distribute tests across multiple machines or containers, combine pytest-xdist with Selenium Grid:

python

Copy

Edit

webdriver.Remote(

    command_executor='http://localhost:4444/wd/hub',

    desired_capabilities={'browserName': 'chrome'}

)

Now, each test can be distributed to a different node in the grid.

✅ Summary

Feature Benefit

pytest-xdist Distribute tests across CPU cores

Parallel with Selenium Reduces total test runtime

Selenium Grid + xdist Scales parallel tests across machines

Learn Selenium Python Training in Hyderabad

Read More

Building a Page Object Model (POM) in Python

Selenium with Pytest: Writing and Running Tests

Creating a Data-Driven Framework with Selenium and Python

Using unittest Framework with Selenium in Python

Visit Our Quality Thought Training in Hyderabad

Get Directions