The idf_component_optional_requires compatibility function in cmakev2
currently attempts to mimic the behavior of cmakev1 by adding a
dependency only if a component is already included in the build. In
cmakev1, this is handled by checking the BUILD_COMPONENTS list, which is
created during an early evaluation phase. Because cmakev2 removed this
early phase to allow for configuration-based component dependencies, the
build system does not inherently know which components will be part of
the build beforehand. To compensate, the current compatibility function
relies on the TARGET_EXISTS generator expression to determine if a
component should be linked.
This approach poses a problem because generator expressions are not
evaluated until the generation phase, but cmakev2 processes linker
scripts during the configuration phase by recursively scanning targets
linked to a library. Because the scanner does not recognize and cannot
evaluate generator expressions, any component linked optionally through
generator expression is skipped. If that component carries a linker
script, the script is never added to the library interface, resulting in
build failures. Since cmakev2 aims to support multiple libraries, a
component might also exist globally, causing TARGET_EXISTS to evaluate
to true, yet still be missed during a specific library's recursive scan,
leading to the same omission of necessary linker scripts.
To resolve this, the implementation of idf_component_optional_requires
is changed to check if a component is known to the build system in
COMPONENTS_DISCOVERED build property. If so, it is explicitly included
and linked directly via its interface target. While this may pull more
components into a build than the previous generator expression method,
potentially increasing build times, it ensures that dependency trees are
fully visible to the library target scanner. This serves as a practical
middle ground that maintains compatibility with existing components and
ensures build stability. This approach allows for the gradual fixing of
idf_component_optional_requires usage in components for cmakev2, with
the eventual goal of removing its usage in cmakev2 entirely.
This change also fixes cases where idf_component_optional_requires is
used to conditionally add requirements based on configuration options.
In cmakev2, the presence of a configuration option does not guarantee[1]
that a component has been included via add_subdirectory, unlike the
behavior in cmakev1. With this change even constructions like
```cmake
if(CONFIG_VFS_SUPPORT_IO)
idf_component_optional_requires(PRIVATE vfs)
endif()
```
will now work in cmakev2, as the updated idf_component_optional_requires
explicitly includes the required component if it is available to the
build system.
Closes https://github.com/espressif/esp-idf/issues/18133
[1] https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/build-system-v2.html#id9
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit reorganizes SPI flash header files to better reflect their
visibility and intended usage:
1. Rename `esp_flash_port/` to `esp_flash_chips/`:
- Better reflects that these headers are for chip driver implementations
- All chip driver headers moved to `esp_flash_chips/` directory
- Added README.md explaining semi-public nature of these headers
2. Move internal headers to `esp_private/`:
- `esp_flash_internal.h` -> `esp_private/esp_flash_internal.h`
- `memspi_host_driver.h` -> `esp_private/memspi_host_driver.h`
3. Move chip driver related headers to `esp_flash_chips/`:
- `esp_private/esp_flash_types.h` -> `esp_flash_chips/esp_flash_types.h`
- `spi_flash/spi_flash_defs.h` -> `esp_flash_chips/spi_flash_defs.h`
- `spi_flash_override.h` -> `esp_flash_chips/spi_flash_override.h`
- All `spi_flash_chip_*.h` headers moved to `esp_flash_chips/`
4. Code improvements:
- Remove unused includes (e.g., `spi_flash_override.h` from `cache_utils.c`)
- Use public API `esp_flash_get_size()` instead of direct member access
- Add `esp_flash_is_quad_mode` to linker.lf for IRAM placement
5. Documentation updates:
- Add README.md in `esp_flash_chips/` explaining semi-public headers
- Update programming guide with warnings about internal headers
- Update both English and Chinese documentation
6. Update all references across the codebase:
- Update includes in `spi_flash` component
- Update `bootloader_support`, `app_update`, `esp_tee`, `espcoredump`
- Update example projects
Breaking changes:
- Headers moved to new locations require include path updates
- `custom_flash_driver` example temporarily disabled until external
components are updated
This commit refactors the SPI flash component to improve encapsulation and
modularity by moving internal types and functions to private headers, and
reorganizing initialization code.
Key changes:
1. Move PSRAM frequency constraint macro from soc_caps.h to mspi_ll.h
- Rename SOC_SPI_MEM_PSRAM_FREQ_AXI_CONSTRAINED to
MSPI_TIMING_LL_PSRAM_FREQ_AXI_CONSTRAINED
- Move macro definition to chip-specific mspi_ll.h files (C5, C61, H4, P4, S31)
- Update usage in clk_utils.c and esp_flash_spi_init.c
- Remove old macro from all soc_caps.h files
2. Move internal types to private headers
- Move esp_flash_t structure to esp_private/esp_flash_types.h
- Move esp_flash_os_functions_t to esp_private/spi_flash_os.h
- Update all internal files to include private headers
- Keep forward declarations in public esp_flash.h
3. Move chip driver header to internal directory
- Move spi_flash_chip_driver.h to esp_flash_port/spi_flash_chip_driver.h
- Update all references to use new path
- Add esp_private/esp_flash_types.h include to the moved header
4. Refactor initialization functions
- Move init_flash from esp_system/startup_funcs.c to
spi_flash/esp_flash_spi_init.c
- Create new init_pm_flash_freq_limit function in startup_funcs.c
to call esp_pm_flash_freq_limit_init() conditionally
- Update system_init_fn.txt with new function locations
5. Improve API encapsulation
- Replace direct access to esp_flash_t->size in
esp_partition_register_external() with esp_flash_get_size() API
- Move esp_flash_is_quad_mode from inline function to regular function
in esp_flash_api.c
6. Update component dependencies
- Add esp_driver_gpio to spi_flash component PRIV_REQUIRES
- Remove unused includes and clean up header dependencies
These changes improve code organization by clearly separating public APIs
from internal implementation details, making the codebase more maintainable
and reducing the risk of breaking changes to internal structures.
Add a simple test to verify that the buildv2_test_app can be built for
the Linux target.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, cmakev2 is being tested only in backward-compatible mode by
using the existing cmakev1 tests with the cmakev2 test application. We
also need to add tests specific to cmakev2, and it is convenient to
reuse the existing build system testing framework. Let's add a `buildv2`
subdirectory to the existing `tools/test_build_system` directory and use
the `pytest_collection_modifyitems` hook to ignore tests in this
directory unless the `--buildv2` option is used.
Without the `--buildv2` option, only the existing cmakev1 tests are
executed and tests in `buildv2` directory are skipped. With the
`--buildv2` option, the existing cmakev1 tests run with the cmakev2
testing application for backward compatibility testing, and all cmakev2
tests within the `buildv2` subdirectory are also executed.
Note: we cannot use the `pytest_ignore_collect` hook, because the
`--buildv2` option is not known to the pytest, so the
`config.getoption('--buildv2', False)` returns always False. We would
likely need to add the `--buildv2` option in the conftest.py in the
esp-idf root directory.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
fix: lsadjf las jflasjfl aslfsald asl fsadlf sladsal jfsadfas
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This adds a clear header to the pytest output, indicating which build
system version is currently being tested.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, idf_build_generate_metadata only accepts binary targets for
which it generates metadata (project_description.json). On Linux
targets, binary images are not generated, but we still need to generate
project_description.json. Extend the current function to accept both
executable and binary targets and ensure project_description.json is
generated when a Linux target is used.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Some components simply return when included in a build for the Linux
target. This is silently ignored in the `idf_component_include`, and the
component interface target is linked as requirement in
`idf_component_register`. This has the side
effect of such components being reported as included, with their
configuration displayed in the menuconfig for included components,
rather than in the submenu for excluded components. For example, the
`bt` component, if added as a dependency in `idf_component_register`,
will be displayed in menuconfig as an included component for Linux
target. The `idf_component_include` function sets the component's
`COMPONENT_REAL_TARGET` property to `NOTFOUND` in such situations. Use
this information when config.env is generated put such components into
excluded submenu. Note that we cannot avoid linking empty component interface
into library, because there might be recursive dependencies and at the
time when the component is included we might not now if it has a real
target or not.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the CONFIG_APP_BUILD_GENERATE_BINARIES option is used to
determine whether binary images will be generated in
idf_project_default. At present, this option is enabled even for the
Linux target, where the esptool_py component is not included, preventing
the generation of binary images. To address this issue, verify if the
target for esptool_py is present.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The IDF_TOOLCHAIN build property is currently incorrectly set to the
default `gcc` value for the linux target, whereas it should be empty.
This misconfiguration causes confusion for components like `soc`, which
adjust toolchain options based on the
IDF_TOOLCHAIN(CONFIG_IDF_TOOLCHAIN_GCC) build property's setting.
When sdkconfig is generated, the IDF_TOOLCHAIN build property is passed
as an environmental variable to kconfgen, and the CONFIG_IDF_TOOLCHAIN
configuration option is set based on this variable. Additionally, the
CONFIG_IDF_TOOLCHAIN_GCC and CONFIG_IDF_TOOLCHAIN_CLANG configuration
options are set accordingly. Subsequently, CONFIG_IDF_TOOLCHAIN_GCC is
used in several places, such as `components/soc/project_include.cmake`,
to configure the toolchain (compiler flags) by invoking functions from
`tools/cmake/toolchain_flags.cmake`, which is included only for
non-linux targets. As a result the configuration fails, because
functions from `tools/cmake/toolchain_flags.cmake` are not available on
linux target.
Since the IDF_TOOLCHAIN cmake cache variable is actually set in the
`tools/cmake/toolchain.cmake` file, the IDF_TOOLCHAIN build property
should be set after the toolchain is initialized in cmakev2's project
initialization. Note that each toolchain file, except for linux,
includes `toolchain.cmake`, which in turn includes
`toolchain_flags.cmake`. This means the IDF_TOOLCHAIN cmake cache
variable is set for every target except linux, because the toolchain
file for linux is empty. As a result CONFIG_IDF_TOOLCHAIN is empty and
CONFIG_IDF_TOOLCHAIN_GCC not set as for cmakev1.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The dependency chain currently tracks component names that are included
recursively with `idf_component_include`. However, these component names
can be ambiguous because a component may be referenced by different
names, such as with a namespace. Additionally, `idf_component_include`
can accept anything that `__get_component_interface` accepts, meaning
even the component interface target can be used to include the component
in the build. To uniquely identify each component, use component
interface targets instead of names in the dependency chain.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
esp-idf uses imported targets as dummy targets that are never linked.
Previous CMake versions would ignore these and not error on unset
IMPORTED_LOCATION if they are never actually linked. CMake 4.2 and newer
errors during codemodel-v2 api queries when imported targets are missing
IMPORTED_LOCATION, so set a dummy location that would error when
actually linked, which fixes the error during api queries.
Closes https://github.com/espressif/esp-idf/pull/18103
In some instances, the COMPONENTS_DISCOVERED build property is used to
walk through the component list and obtain component properties, such as
during configuration preparation or the inclusion of project_include
files. Since we know the component interfaces from the
COMPONENT_INTERFACES build property, we can switch to the faster
`__idf_component_get_property_unchecked` method to obtain component properties
in these cases. This change reduces the reconfiguration time by 0.4
seconds on my computer.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The commit f951ae5b18 ("feat(cmakev2): Added component validation
checks for ...") introduced support for validating component sources and
include directories. It heavily relies on iterating through the
registered components to obtain their properties. For instance, it
examines all components for each component source file to ensure that
the source files do not originate from other components. However, it
turns out that `idf_component_get_property` is a bottleneck in this
process, causing reconfiguration to take 3-4 times longer than before
the introduction of this check. Since we know of all the component
interfaces, we can bypass the checks performed by
`idf_component_get_property` and instead use the raw version,
`__idf_component_get_property_unchecked`, which operates much faster.
Following are hello_world example reconfigure times before and after.
before:
-- Configuring done (11.1s)
-- Generating done (0.2s)
after:
-- Configuring done (3.9s)
-- Generating done (0.2s)
A new library property LIBRARY_COMPONENT_INTERFACES_LINKED is added that
keeps component interfaces linked to the library.
Fixes: f951ae5b18 ("feat(cmakev2): Added component validation checks for ..")
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Retrieve the value of the specified component property quickly. The
existing `idf_component_get_property` function performs various checks
to identify the component interface target, which keeps the component
properties, and accepts a component name, target, target alias, or
interface. While `idf_component_get_property` uses a cache to identify
the component interface relatively quickly, it is still much slower
compared to the raw `get_property`. The
`__idf_component_get_property_unchecked` function provides a faster way how to
obtain component property if the component interface is already known.
It skips all the checks, so it must be used carefully.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The MINIMAL_BUILD build property and the COMPONENTS variable are both
used to determine the initial component list for the build.
Currently, if the COMPONENTS variable is set, the MINIMAL_BUILD logic
is ignored during component selection, but the MINIMAL_BUILD build
property remains set. This leads to an inconsistent state where
menuconfig displays information indicating MINIMAL_BUILD is active,
even though it was ignored in favor of the COMPONENTS variable.
Fix this by setting the MINIMAL_BUILD property to OFF if the
COMPONENTS variable is used.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Chen Jichang
Roland Dobai
Fu Hanxi
Erhan Kurubas
Alexey Lapshin
Frantisek Hrbata
luoxu
Armando (Dou Yiwen)
Mahavir Jain
harshal.patil
Marius Vikhammer
Michael (XIAO Xufeng)
david zuhn
igor.udot
morris
Aditya Patwardhan
Song Ruo Jing
Ferdinand Bachmann
Chen Chen