When the component manager is disabled via IDF_COMPONENT_MANAGER=0, the
cmakev2 build system still entered the full component manager flow
(__fetch_components_from_registry), which called
__download_managed_component(). That function returned early with
result=0 without creating the expected output file, violating the
post-condition in __download_component_level_managed_components() that
checks result==0 => file exists, causing a fatal error.
Instead of patching the low-level function to write an empty stub file,
this commit properly skips the entire component manager flow when the
manager is disabled:
- Move __init_component_manager() to idf.cmake global initialization
sequence alongside other __init_*() calls, so IDF_COMPONENT_MANAGER
build property is available early.
- Set __SDKCONFIG_ORIG to the real sdkconfig path in __init_kconfig()
as its default value. Previously it was only set inside
__create_sdkconfig_orig_copy() and had a defensive fallback in
__create_base_kconfgen_command(). The default ensures __SDKCONFIG_ORIG
is always valid regardless of whether the component manager runs.
- Guard __create_sdkconfig_orig_copy() with an IDF_COMPONENT_MANAGER
check. The sdkconfig backup exists solely to preserve unknown Kconfig
options from managed components during intermediate kconfgen rounds.
When the manager is disabled, no managed components exist, so the
backup is unnecessary.
- Guard __fetch_components_from_registry() call in project.cmake behind
IDF_COMPONENT_MANAGER == 1. When disabled, only the manifest warning
is issued. No download loop runs, no temp files are created, and no
"Component manager round N..." messages are printed.
- Remove the now-redundant IDF_COMPONENT_MANAGER guard from
__download_managed_component(), since it is only reachable when the
manager is enabled.
Closes https://github.com/espressif/esp-idf/issues/18372
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The IDF_TOOLCHAIN build property is currently incorrectly set to the
default `gcc` value for the linux target, whereas it should be empty.
This misconfiguration causes confusion for components like `soc`, which
adjust toolchain options based on the
IDF_TOOLCHAIN(CONFIG_IDF_TOOLCHAIN_GCC) build property's setting.
When sdkconfig is generated, the IDF_TOOLCHAIN build property is passed
as an environmental variable to kconfgen, and the CONFIG_IDF_TOOLCHAIN
configuration option is set based on this variable. Additionally, the
CONFIG_IDF_TOOLCHAIN_GCC and CONFIG_IDF_TOOLCHAIN_CLANG configuration
options are set accordingly. Subsequently, CONFIG_IDF_TOOLCHAIN_GCC is
used in several places, such as `components/soc/project_include.cmake`,
to configure the toolchain (compiler flags) by invoking functions from
`tools/cmake/toolchain_flags.cmake`, which is included only for
non-linux targets. As a result the configuration fails, because
functions from `tools/cmake/toolchain_flags.cmake` are not available on
linux target.
Since the IDF_TOOLCHAIN cmake cache variable is actually set in the
`tools/cmake/toolchain.cmake` file, the IDF_TOOLCHAIN build property
should be set after the toolchain is initialized in cmakev2's project
initialization. Note that each toolchain file, except for linux,
includes `toolchain.cmake`, which in turn includes
`toolchain_flags.cmake`. This means the IDF_TOOLCHAIN cmake cache
variable is set for every target except linux, because the toolchain
file for linux is empty. As a result CONFIG_IDF_TOOLCHAIN is empty and
CONFIG_IDF_TOOLCHAIN_GCC not set as for cmakev1.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_build_generate_depgraph function creates a component dependency
graph in dot (graphviz) format for a specified executable. It uses
existing helper functions from cmakev1, ensuring that the generated dot
files are produced in the same manner as in cmakev1. While adjustments
might be needed in the future if necessary, the current implementation
is intended to offer the same functionality as cmakev1. Similar to
cmakev1, the dot files are only generated only when the
__BUILD_COMPONENT_DEPGRAPH_ENABLED build property is set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_create_size_report function allows for the creation of size
report targets based on the generated link map file. The size report
targets are created using the TARGET option name: "<target>",
"<target>-files", and "<target>-components". These size report targets
are added to the idf_default_project with the TARGET set to "size",
resulting in the creation of "size", "size-files", and "size-components"
targets for the default project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit adds support for uf2 targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/uf2.cmake.
- Adds the idf_create_uf2() function to create the uf2 targets. This
function now takes the executable as an argument thus allowing the uf2
target to be created per-executable.
- idf_project_default() is updated to create the uf2 targets.
This commit adds support for dfu targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/dfu.cmake file on similar lines as the
tools/cmake/dfu.cmake file.
- Adds the idf_create_dfu() function to create the dfu targets. This
function now takes the executable as an argument thus allowing the dfu
target to be created per-executable.
- idf_project_default() is updated to create the dfu target.
The component can be referenced by multiple identifiers such as the
component name, which is derived from the component directory name,
component target, interface, or aliases. All component properties are
attached to the component interface target, which is also used for
declaring component dependencies. The cmakev2 build system has a
function called __get_component_interface, which is responsible for
identifying the component interface based on the given component
identifier. Since this function is called frequently, it needs to be
reasonably fast.
This introduces a new __idf_component_interface_cache INTERFACE target,
which serves as a mapping cache between component identifiers and the
component interface. The cache for each component is initialized in the
__init_component function, which introduces a component to the build
system. Currently, the component interface search is conducted by
examining the COMPONENTS_DISCOVERED and COMPONENT_INTERFACES lists
stored as build properties. Since the build system is aware of most
component identifiers, such as component name, target, and alias, during
component initialization, it can add mappings between component
identifiers and the component interface to a cache, which is built as
the components are initialized. This cache is used in the
__get_component_interface function instead of looking into the
COMPONENTS_DISCOVERED or COMPONENT_INTERFACES lists. This significantly
speeds up the component interface search and also makes the code much
simpler and more readable.
The component interface cache also completely replaces the existing
component name resolution, which was introduced because of the component
manager, and the cache used for resolved component names. This is
possible because all the necessary information is available during
component initialization when the component interface cache is
populated. The ambiguity of components is resolved based on component
source/priority.
Here is an example of the component interface mapping for the
espressif__led_strip component to the idf_espressif__led_strip interface
target. The component name, without the namespace, is referred to as the
short name. In this example, it is led_strip.
- espressif__led_strip -> idf_espressif__led_strip # name(directory name)
- idf_espressif__led_strip -> idf_espressif__led_strip # interface
- idf::espressif__led_strip -> idf_espressif__led_strip # alias
- _idf_espressif__led_strip -> idf_espressif__led_strip # real target
- led_strip -> idf_espressif__led_strip # short name
When another component with the same espressif__led_strip name is
initialized with a different priority, there is no need to change
anything in the cache, because the mapping stays the same.
The cache must be updated when two components share the same short name
but belong to different namespaces. This situation is likely uncommon.
For instance, consider espressif__led_strip and my__led_strip. If
my__led_strip has a higher priority, the cache is updated to reflect the
short name as follows:
- led_strip -> idf_my__led_strip
If both components have the same priority, the short name mapping for
led_strip is entirely removed. Conversely, if my__led_strip has a lower
priority, no short name mapping is added for it.
The short name is also added to the COMPONENT_SHORT_NAME property of the
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the check is based on the existence of sdkconfig, but there
may be situations where sdkconfig exists without containing
CONFIG_IDF_TARGET. Perform a target consistency check only if the target
is identified in the main sdkconfig file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The function arguments in the documentation comments were using field
list, which caused text overflow in the generated documentation and
generally resulted in poor formatting. Let's use paragraphs for
the argument descriptions instead.
The documentation comments are written in reStructuredText, but
currently, they use inconsistent indentation. Standardize all the
documentation comments to use a four-character indentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit updates the order of initialization for build system v2. All
non-project initializations are handled in idf.cmake and all
project-specific initializations are now moved to idf_project_init().
The `idf_build_generate_metadata()` function is used to generate project
metadata, specifically the `project_description.json` file, for a given
executable target. The output file path can be specified as an argument
to the function, allowing metadata generation for multiple executables.
It identifies the library, created by the `idf_build_library` function,
that is linked to the executable based on the `LINK_LIBRARIES`
executable target property. The `LIBRARY_INTERFACES` build property
tracks all library interfaces created with the `idf_build_library`
function, so the targets in the `LINK_LIBRARIES` executable property are
checked against this list to find the library. The library contains
information about components linked to it in the
`LIBRARY_COMPONENTS_LINKED` library property. These components are
processed and added to the `build_component_info` dictionary in the
`project_description.json` along with other information.
Note that, compared to cmakev1, the `all_component_info` dictionary in
`project_description` does not include some information by default, such
as included directories. This information is available in cmakev1 due to
early component evaluation, which is not done in cmakev2. This will
likely require some adjustments on the hints side.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Integrate the ldgen into cmakev2. With this change, it becomes possible
to actually link the project executables.
In cmakev2, the handling of linker scripts is deferred to
idf_build_library, unlike in cmakev1, where linker scripts were added
and generated during the target_linker_script call. In cmakev2, the
target_linker_script only adds the linker scripts and templates, along
with the output filenames for the linker scripts generated from the
templates, to the component property. When idf_build_library is called
and all the requested components are included, it uses the
__get_target_dependencies function to obtain all transitively linked
targets to the library interface target. These targets are mapped to the
components, and the LIBRARY_COMPONENTS_LINKED library property is set.
It contains all components linked to the library interface target. The
components from LIBRARY_COMPONENTS_LINKED are used to collect linker
fragments and linker scripts utilized in the library. Additionally, all
targets transitively linked to the library are used to identify archive
files used in the library. This includes component archives and archives
added with the add_prebuilt_library function. The archives and
ldfragments related to the components linked to the library are used
when ldgen generates the linker scripts from templates.
The linker scripts, both static and generated by ldgen, are added to the
library interface link options and INTERFACE_LINK_DEPENDS property. For
generated linker scripts, a custom target is created and added as a
dependency for the library interface to ensure they are generated before
the link.
The difference compared to cmakev1 is that the generated linker scripts,
currently only sections.ld, are not global in the project but are
generated per library. This means there might be multiple versions of
sections.ld depending on the components included in the library. For
example, a component like esp_system may be linked to multiple library
interface targets, each with a different set of components. This results
in different sets of fragment files and library archives and different
versions of the sections.ld linker script. This should ensure proper
dependencies between targets. In other words, if a component changes its
linker fragment, only executables linked to libraries using this
component should be re-linked. As a consequence of this approach, the
generated linker scripts for different libraries need to have different
names or be stored in different directories to avoid overwriting the
linker script for one library with the linker script for another library
using the same component. This is handled with a suffix, which is based
on the library interface target name and appended to the generated
linker script. So, for example, there is no sections.ld, but instead
sections.ld_fatfs_lib or sections.ld_hello_world_lib. As a next step, we
can add a DEFAULT option to idf_build_library and avoid adding the
suffix for the default library.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Based on the configuration, set the IDF_TARGET build property. This is
also set by cmakev1 and can be used to easily determine the architecture
for the project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add the IDF_BUILD_VER_TAG variables to include the build system version
tag. These variables can be used in files, such as CMake toolchain
files, which are shared between cmakev1 and cmakev2. This allows for the
inclusion of files from the currently running build system version. For
example:
include($ENV{IDF_PATH}/tools/cmake${IDF_BUILD_VER_TAG}/utilities.cmake).
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The compat.cmake file is intended to include functions necessary for
backward compatibility with cmakev1.
Add the check_expected_tool_version function, which is used by some
project_include.cmake files to verify if the tool's version matches the
installed tool. This functionality is used, for example, by esp_common.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For backward compatibility, since externalproject_add is used by
project_include.cmake in the bootloader component. The ExternalProject
should probably be included there instead.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Set the global variables IDF_BUILD_V2 and IDF_BUILD_VER, along with the
build properties and environmental variables. These are intended to be
used by components to determine which build system version they are
evaluated under and to make adjustments based on this information.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current approach involves transparently calling __init_build within
idf_build_library and performing all post-project() initialization
there. The issue with this method is that project_include.cmake files
also need to be included during post-project() initialization, but they
should be included in the global scope. This cannot be achieved within
the __init_build function unless it is converted into a macro. Although
using a macro is a potential solution, it risks causing global scope
pollution. Another complication is the location where project() can be
invoked; it must be explicitly[1] stated in the project's
CMakeLists.txt. This requirement conflicts with our intention to wrap
it within the idf_project_default helper. Given these challenges, it
makes sense to introduce an explicit idf_project_init macro, where all
post-project() initialization occurs, including the inclusion of
project_include.cmake files. While we can still encapsulate this within
idf_project_default, for the plumbing commands (idf_build_library,
idf_build_executable, ...), it will need to be explicitly called after
the project() invocation.
Usage for default project:
cmake_minimum_required(VERSION 3.22)
include($ENV{IDF_PATH}/tools/cmakev2/idf.cmake)
project()
idf_project_default()
Usage for plumbing commands:
cmake_minimum_required(VERSION 3.22)
include($ENV{IDF_PATH}/tools/cmakev2/idf.cmake)
project()
idf_project_init()
idf_build_library()
...
[1] https://cmake.org/cmake/help/latest/command/project.html#usage
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the target consistency check with sdkconfig is performed even
if sdkconfig is not present, by comparing the selected target with the
target defined in sdkconfig.defaults. This results in an inconsistency
report when a different target is set in the IDF_TARGET environment
or cmake variable compared to sdkconfig.defaults. The behavior in
cmakev1 is to prioritize IDF_TARGET and ignore values in
sdkconfig.defaults. Perform the consistency check only if the
sdkconfig file exists.
Additionally, skip this consistency check if a set-target action is
taking place, as the old target in sdkconfig is being replaced with a
new value anyway.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a core function of the build system, responsible for including the
specified component identified by name, into the build
process.
Currently, the non-project components written in cmakev1 are ignored and
not evaluated with the add_subdirectory command because there are no
shims for the cmakev1 API yet. However, this allows for the evaluation
of project components written using the cmakev2 approach, which is
closer to the native CMake usage.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Initialize submodules that are not yet initialized, and issue a warning
for submodules that do not match the recorded hash in the git tree.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Initialize the IDF_VER build property using the version.txt file, or use
git-describe. If neither is available, fall back to the value of the
IDF_VERSION environment variable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The Git tool will be used in various parts of the build system, such as
retrieving the IDF version using git-describe or checking submodules.
Similar to the Python interpreter, add an initialization function to
identify the Git executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The following functions are added to tools/cmakev2/kconfig.cmake:
- __collect_kconfig_files_from_components(): Collect Kconfig files from
components.
- __collect_kconfig_files_from_bootloader_components(): Collect Kconfig
files from bootloader components.
- __collect_kconfig_files_from_directory(): Collect Kconfig files from
a directory.
This commit adds the tools/cmakev2/kconfig.cmake file for Kconfig
processing. The file provides the following functions:
- __init_kconfig(): Placeholder to initialize Kconfig build properties,
collect kconfig files and generate the initial sdkconfig.
- __generate_sdkconfig(): Placeholder to generate sdkconfig output files.
Determine the IDF_TOOLCHAIN from the following sources in this order:
environmental variable and CMake cache variable. Ensure there are no
inconsistencies between the values set in these different locations.
Set the IDF_TOOLCHAIN and IDF_TOOLCHAIN_FILE build properties. Also,
configure the IDF_TOOLCHAIN CMake cache variable and set the
CMAKE_TOOLCHAIN_FILE global variable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Determine the IDF_TARGET from the following sources in this order:
environmental variable, CMake cache variable, and sdkconfig files.
Ensure there are no inconsistencies between the values set in these
different locations.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Based on the environmental variables, CMake cache variables, or default
values, set the DKCONFIG and SDKCONFIG_DEFAULTS build properties.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Determine the Python interpreter and verify package dependencies if the
CMake cache variable PYTHON_DEPS_CHECKED is not set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The version.cmake file should be the sole file used from the cmakev1
build system. There's no need to maintain the IDF_VERSION information in
two separate locations.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Discover component directories and initialize components within them.
This process does not include managed components, which should be added
separately at a later stage. To facilitate this, some minimal
functionalities are introduced, such as build properties, component
properties, and other helper functions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Frantisek Hrbata
Sudeep Mohanty