Add a buildv2 test that reconfigures the test app with both GCC and
clang toolchains and asserts that the selected IDF_TOOLCHAIN value
is propagated to sdkconfig. The clang case is skipped when a clang
compiler is not available in PATH.
In the cmakev2 build framework, the selected toolchain was not
consistently propagated to sdkconfig generation. For clang builds,
this produced a configuration that was incompatible with the clang
toolchain and broke compilation.
Ensure the toolchain selection is observed consistently so sdkconfig
and component configuration reflect the intended toolchain for both
GCC and clang builds.
Made-with: Cursor
Remove the temporary esp32s31 disable from usb/host build-test rules.
Add esp32s31 and esp32h4 to the usb_host_msc and usb_host_lib pytest
matrices. Keep the tests temp-skipped in CI until usb_host_flash_disk
runners exist.
PMP entry 3 (SOC_DROM_MASK_HIGH, TOR mode) in the memprot path
was incorrectly granted RW permission on esp32h21 and esp32c61.
The mask ROM data region is inherently read-only; remove the W bit.
Also added necessary tests to check voilations and re-enabled
tests for ESP32P4
Add a new CMake function esp_partition_flash_binary() that provides a
unified API for registering partition data binaries to be flashed. It
replaces the direct esptool_py_flash_target calls scattered across
components (spiffs, fatfs, nvs_flash) with a single function that:
- Resolves partition offset from the partition table automatically
- Determines encryption requirements (auto-detect or ALWAYS_PLAINTEXT)
- Creates per-partition flash targets (e.g. idf.py <partition>-flash)
- Optionally includes the binary in `idf.py flash` via FLASH_IN_PROJECT
On the linux target, the function registers binaries for pre-loading
into the emulated flash. A build-time manifest (linux_flash_data.txt)
is generated via file(GENERATE), and partition_linux.c reads it at
runtime to copy each binary into the memory-mapped flash buffer at
the correct offset.
The partition_ops example is updated to use the new function and
includes a custom_partition with pre-built data to demonstrate the
full workflow, including on the linux target.
Profiling shows that parsing the linker fragment (.lf) files with
pyparsing is the single largest cost in ldgen (~58% of total runtime).
parse_fragment_file() rebuilds its grammar from scratch on every call
because the conditional parser closure has to capture sdkconfig to
evaluate if/elif/else at parse time.
Add a cache that pickles the parsed FragmentFile object list to
<output>.lfcache. The cache is keyed on fragment file paths+mtimes
plus sdkconfig and kconfig mtimes — sdkconfig is included because
fragment if/elif/else blocks are evaluated against it at parse time.
This avoids the pyparsing cost on rebuilds where ldgen has to run
but the fragment files themselves are unchanged — for example when
a function is added or removed in a source file. Cache hits produce
the same sections.ld as a full parse, and any pickle load failure
falls through to a normal parse, so cache corruption or a Python
upgrade cannot produce a wrong build.
Print "Skipping linker fragment parsing, fragment files unchanged"
on the lf cache hit path so build logs show when the optimization
fired — this makes it easy to diagnose user reports of unexpected
ldgen behavior.
The LDGEN_NO_CACHE environment variable disables the lf cache. When
set, ldgen falls through to a normal parse without consulting or
writing the cache file.
Measured on wifi_station (205 libs, 67 .lf files), median of 10 runs:
before after
rebuild, section change, .lf unchanged 5.82s 1.34s
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
After running objdump on all libraries, extract section names from the
stored raw output using regex and compute a fingerprint (MD5 hash of
section names + mtimes of fragment files, sdkconfig, and kconfig).
If the fingerprint matches the cached value from the previous run,
skip the expensive pyparsing + generation step entirely and touch
the output file to update its timestamp. The fingerprint file is
stored next to the output (e.g. sections.ld.fingerprint) in the build
directory.
This avoids the pyparsing bottleneck for the common case of editing
function bodies without adding or removing symbols — section names
in the object files are unchanged, so the generated sections.ld
would be identical. When sections do change, the full generation
runs and the fingerprint is updated.
Print "Skipping linker script generation, section names unchanged"
on the fingerprint hit path so build logs show when the optimization
fired. This makes it easy to diagnose user reports of unexpected
ldgen behavior — the build log directly shows whether the cache was
used.
Add an LDGEN_NO_CACHE environment variable to disable the cache as a
workaround in case the optimization causes problems. When set, ldgen
prints "Linker script generation caches disabled by LDGEN_NO_CACHE"
and runs full generation as before, leaving any existing fingerprint
file untouched.
Measured on wifi_station (205 libs, 67 .lf files), median of 10 runs:
before after
rebuild, section names unchanged 5.82s 0.82s
Closes https://github.com/espressif/esp-idf/issues/18408
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Sahil Yadav
Sergei Silnov
Frantisek Hrbata
Sudeep Mohanty
Igor Masar
Roland Dobai
Jiang Jiang Jian
Linyan Liu
yinqingzhao
Roland Dobai
Fu Hanxi
Marek Fiala
Marius Vikhammer
Chen Jichang
igor.udot
Alexey Gerenkov
Adam Múdry
nilesh.kale
Erhan Kurubas
Alexey Lapshin
Island
C.S.M
Mahavir Jain
Armando (Dou Yiwen)
Guillaume Souchere
Radim Karniš
Euripedes Rocha
Ashish Sharma
Daniel Paul