ci: add QEMU target to uni-test-app to be able to run in ci

- Add pytest_unit_test_app.py with per-group test functions (each gets
  a fresh QEMU boot to handle the single esp_matter::start() constraint)
- Add build_unit_test_app_qemu and pytest_unit_test_app_qemu CI jobs
- Disable WiFi and use QEMU virtual Ethernet in sdkconfig.defaults
- Register host_test and qemu pytest markers
- Document local QEMU test setup in README
This commit is contained in:
Shubham Patil
2026-03-17 14:26:57 +05:30
parent 5f799b4f5a
commit fe57fa6cf1
7 changed files with 172 additions and 3 deletions
+1
View File
@@ -12,3 +12,4 @@ _build/
tools/chip-tool/
.zap/
.DS_Store
pytest_embedded_log
+55
View File
@@ -612,6 +612,61 @@ pytest_esp32h2_esp_matter_dut:
fi
tags: ["esp32h2", "esp_matter_dut"]
build_unit_test_app_qemu:
extends:
- .build_examples_template
artifacts:
paths:
- "examples/unit_test_app/build/*.bin"
- "examples/unit_test_app/build/flasher_args.json"
- "examples/unit_test_app/build/config/sdkconfig.json"
- "examples/unit_test_app/build/bootloader/*.bin"
- "examples/unit_test_app/build/partition_table/*.bin"
- "examples/unit_test_app/build/build_log.txt"
when: always
expire_in: 4 days
script:
- cd ${ESP_MATTER_PATH}/examples/unit_test_app
- idf.py set-target esp32c3 build
pytest_unit_test_app_qemu:
stage: target_test
image: ${DOCKER_IMAGE_NAME}:chip_${CHIP_SHORT_HASH}_idf_${IDF_CHECKOUT_REF}
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event" || $CI_COMMIT_BRANCH == "main" || $CI_PIPELINE_SOURCE == "push"
needs:
- build_unit_test_app_qemu
before_script:
- *add_gitlab_ssh_key
- *get_build_caches
- *setup_idf
- cd ${ESP_MATTER_PATH}
- mkdir -p ${REPOS_PATH}
- *setup_matter
- *update_build_caches
variables:
REPOS_PATH: "$CI_PROJECT_DIR/repos"
script:
- cd ${ESP_MATTER_PATH}
- apt-get update && apt-get install -y -q libslirp0
- python ${IDF_PATH}/tools/idf_tools.py install qemu-riscv32
- eval "$(python ${IDF_PATH}/tools/idf_tools.py export)"
- pip install -r tools/ci/requirements-pytest.txt
- pytest examples/unit_test_app/pytest_unit_test_app.py
--target esp32c3
-m qemu
--embedded-services idf,qemu
--junitxml=XUNIT_RESULT.xml
--qemu-extra-args="-global driver=timer.esp32c3.timg,property=wdt_disable,value=true"
artifacts:
paths:
- "pytest_embedded_log/"
reports:
junit: XUNIT_RESULT.xml
when: always
expire_in: 4 days
tags: ["host_test"]
build_upstream_examples:
resource_group: build_upstream_examples
extends:
+59 -2
View File
@@ -29,6 +29,51 @@ Once flashed, the test menu will appear in the serial monitor. You can:
- Enter a test number to run a specific test
- Enter `*` to run all tests
## Running Tests with QEMU (no hardware needed)
You can run the unit tests locally under QEMU emulation without physical hardware. This is the same method used in CI.
### Prerequisites
- Install QEMU for RISC-V (esp32c3):
```bash
python3 -m pip install pytest-embedded-qemu
```
- Ensure the QEMU binary (`qemu-system-riscv32`) is available. Install it via ESP-IDF tools:
```bash
$IDF_PATH/tools/idf_tools.py install qemu-riscv32
source $IDF_PATH/export.sh # re-source to pick up the new tool in PATH
```
### Build and Run
```bash
cd examples/unit_test_app
idf.py -DSDKCONFIG_DEFAULTS="sdkconfig.defaults;sdkconfig.defaults.qemu" set-target esp32c3 build
# Run all QEMU test groups (each gets a fresh QEMU reboot)
pytest pytest_unit_test_app.py \
--target esp32c3 \
-m qemu \
--embedded-services idf,qemu \
--qemu-extra-args="-global driver=timer.esp32c3.timg,property=wdt_disable,value=true"
# Run a single test group
pytest pytest_unit_test_app.py \
--target esp32c3 \
-m qemu \
--embedded-services idf,qemu \
--qemu-extra-args="-global driver=timer.esp32c3.timg,property=wdt_disable,value=true" \
-k test_get_val
```
### Why multiple test functions?
Each test file has its own `setup_*()` function that calls `esp_matter::start()`, and there is no teardown/stop.
Since only one setup can succeed per boot, tests are grouped so each group runs after a fresh QEMU reboot.
Each pytest function (eg: `test_get_val`, `test_get_val_type`, `test_update_report`) gets its own QEMU instance.
## Extending the Tests
### Adding tests to existing component
@@ -45,8 +90,20 @@ list(APPEND srcs_list "your_new_test_file.cpp")
Please refer to components/esp_matter/test directory for comprehensive structure and example.
- After adding the new component tests, you need to add the component to the TEST_COMPONENTS list in CMakeLists.txt
- Append the component name to the TEST_COMPONENTS list. For example, if you add a new component called "new_component", you need to add it to the TEST_COMPONENTS list in CMakeLists.txt:
- Append the component name to the TEST_COMPONENTS list. For example, if you add a new component called "new_component",
you need to add it to the TEST_COMPONENTS list in CMakeLists.txt:
```cmake
set(TEST_COMPONENTS "esp_matter new_component" CACHE STRING "List of components to test")
```
```
### For running them in the CI,
- Add the test group to the `pytest_unit_test_app.py` file with the appropriate marker and test function name.
```python
@pytest.mark.host_test
@pytest.mark.qemu
@pytest.mark.esp32c3
def test_my_unit_tests(dut: QemuDut) -> None:
run_group(dut, 'my_test_group')
```
@@ -0,0 +1,44 @@
# SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Apache-2.0
import pytest
from pytest_embedded_qemu.dut import QemuDut
def run_group(dut: QemuDut, group: str, timeout: int = 120) -> None:
"""Run all Unity cases matching a group tag, then verify no failures.
pytest-embedded records Unity results without raising on failure,
so we check dut.testsuite afterwards to surface failures to pytest.
"""
cases = [c for c in dut.test_menu if group in c.groups]
assert cases, f'No cases for group "{group}" (parsed {len(dut.test_menu)} total)'
dut.run_all_single_board_cases(group=group, timeout=timeout)
failed = dut.testsuite.failed_cases
if failed:
names = [tc.name for tc in failed]
pytest.fail(f'{len(failed)} failed in [{group}]: {", ".join(names)}')
@pytest.mark.host_test
@pytest.mark.qemu
@pytest.mark.esp32c3
def test_get_val(dut: QemuDut) -> None:
run_group(dut, 'get_val')
@pytest.mark.host_test
@pytest.mark.qemu
@pytest.mark.esp32c3
def test_get_val_type(dut: QemuDut) -> None:
run_group(dut, 'get_val_type')
@pytest.mark.host_test
@pytest.mark.qemu
@pytest.mark.esp32c3
def test_update_report(dut: QemuDut) -> None:
run_group(dut, 'report')
run_group(dut, 'update')
+6 -1
View File
@@ -18,10 +18,15 @@ CONFIG_PARTITION_TABLE_OFFSET=0xC000
# Enable chip shell
CONFIG_ENABLE_CHIP_SHELL=y
# Disable WiFi AP
# Disable WiFi — unit tests don't need networking and WiFi PHY calibration hangs in QEMU
CONFIG_ENABLE_WIFI_STATION=n
CONFIG_ENABLE_WIFI_AP=n
CONFIG_ESP_WIFI_SOFTAP_SUPPORT=n
# Use QEMU virtual Ethernet instead of WiFi
CONFIG_ETH_USE_OPENETH=y
CONFIG_ENABLE_ETHERNET_TELEMETRY=y
#enable lwIP route hooks
CONFIG_LWIP_HOOK_IP6_ROUTE_DEFAULT=y
CONFIG_LWIP_HOOK_ND6_GET_GW_DEFAULT=y
@@ -0,0 +1,5 @@
# Disable WiFi — unit tests don't need networking and WiFi PHY calibration hangs in QEMU
CONFIG_ENABLE_WIFI_STATION=n
# Use QEMU virtual Ethernet instead of WiFi
CONFIG_ETH_USE_OPENETH=y
CONFIG_ENABLE_ETHERNET_TELEMETRY=y
+2
View File
@@ -25,6 +25,8 @@ markers =
esp32s3: support esp32s3 target
# env markers
esp_matter_dut: esp matter runner which have single dut
host_test: test runs on host machine (not on target hardware)
qemu: test runs under QEMU emulation
# log related
log_cli = True