Add a GENERATE_SDKCONFIG build property that controls whether kconfgen
writes the sdkconfig file (--output config) during the configuration
step. It defaults to 1 (enabled).
When building the bootloader as a subproject, the set of components
(and their Kconfig files) differs from the main project. Running
kconfgen with --output config in this context rewrites the main
project's sdkconfig. Even when the content is identical, the timestamp
update causes ninja to detect sdkconfig as newer than build.ninja
outputs (e.g. cmake_install.cmake), triggering an infinite CMake
re-run loop.
Setting GENERATE_SDKCONFIG to 0 in a subproject prevents this,
matching the behaviour of cmakev1's __OUTPUT_SDKCONFIG property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When the component manager is disabled via IDF_COMPONENT_MANAGER=0, the
cmakev2 build system still entered the full component manager flow
(__fetch_components_from_registry), which called
__download_managed_component(). That function returned early with
result=0 without creating the expected output file, violating the
post-condition in __download_component_level_managed_components() that
checks result==0 => file exists, causing a fatal error.
Instead of patching the low-level function to write an empty stub file,
this commit properly skips the entire component manager flow when the
manager is disabled:
- Move __init_component_manager() to idf.cmake global initialization
sequence alongside other __init_*() calls, so IDF_COMPONENT_MANAGER
build property is available early.
- Set __SDKCONFIG_ORIG to the real sdkconfig path in __init_kconfig()
as its default value. Previously it was only set inside
__create_sdkconfig_orig_copy() and had a defensive fallback in
__create_base_kconfgen_command(). The default ensures __SDKCONFIG_ORIG
is always valid regardless of whether the component manager runs.
- Guard __create_sdkconfig_orig_copy() with an IDF_COMPONENT_MANAGER
check. The sdkconfig backup exists solely to preserve unknown Kconfig
options from managed components during intermediate kconfgen rounds.
When the manager is disabled, no managed components exist, so the
backup is unnecessary.
- Guard __fetch_components_from_registry() call in project.cmake behind
IDF_COMPONENT_MANAGER == 1. When disabled, only the manifest warning
is issued. No download loop runs, no temp files are created, and no
"Component manager round N..." messages are printed.
- Remove the now-redundant IDF_COMPONENT_MANAGER guard from
__download_managed_component(), since it is only reachable when the
manager is enabled.
Closes https://github.com/espressif/esp-idf/issues/18372
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev2 kconfig module sets sdkconfig output paths using internal
property names (__SDKCONFIG_HEADER, __SDKCONFIG_CMAKE, etc.), but
components like ULP read the public names (SDKCONFIG_HEADER,
SDKCONFIG_CMAKE). This results in empty values being passed to the ULP
sub-project, causing its CMake configure step to fail.
Add public aliases matching the cmake v1 property names for backward
compatibility.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The sdkconfig file may contain configuration options defined in Kconfig
files of managed components. Since kconfgen runs before the component
manager fetches these components, the Kconfig definitions for managed
component options are not yet available. The kconfgen --output config
flag regenerates sdkconfig from kconfiglib's internal state, which only
knows about options with loaded Kconfig definitions. This causes unknown
options (i.e., those from managed components) to be silently dropped
from sdkconfig during intermediate regeneration rounds.
Note that kconfgen's --config flag (used for reading sdkconfig) only
performs deprecated option name replacement and does NOT drop unknown
options. The problem is exclusively in --output config, which writes a
fresh sdkconfig from the parsed Kconfig tree state.
Fix this by introducing a __SDKCONFIG_ORIG build property that provides
an indirection layer for the --config input path:
- Before the component manager runs: __SDKCONFIG_ORIG points to a copy
of the original sdkconfig (build/sdkconfig.orig), created by the new
__create_sdkconfig_orig_copy() function. This copy preserves all
original options, including those from managed components.
- During intermediate kconfgen runs: --config reads from the preserved
copy (so unknown options survive as input), while --output config
writes to the real sdkconfig (unknown options may be dropped there,
but this is harmless since kconfgen always reads from the copy).
- After the component manager completes: __SDKCONFIG_ORIG is reset to
point to the real sdkconfig and __BASE_KCONFGEN_CMD is rebuilt, so
that subsequent operations (menuconfig, save-defconfig, confserver)
read and write the actual sdkconfig file directly.
The flow is:
__create_sdkconfig_orig_copy()
-> __SDKCONFIG_ORIG = build/sdkconfig.orig
__generate_sdkconfig()
-> --config build/sdkconfig.orig --output config project/sdkconfig
__fetch_components_from_registry():
loop:
download_components()
__generate_sdkconfig()
-> --config build/sdkconfig.orig --output config project/sdkconfig
if success: break
endloop
-> __SDKCONFIG_ORIG = project/sdkconfig
-> rebuild __BASE_KCONFGEN_CMD
idf_create_menuconfig() / save-defconfig / confserver
-> uses --config project/sdkconfig (the real file)
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Some components simply return when included in a build for the Linux
target. This is silently ignored in the `idf_component_include`, and the
component interface target is linked as requirement in
`idf_component_register`. This has the side
effect of such components being reported as included, with their
configuration displayed in the menuconfig for included components,
rather than in the submenu for excluded components. For example, the
`bt` component, if added as a dependency in `idf_component_register`,
will be displayed in menuconfig as an included component for Linux
target. The `idf_component_include` function sets the component's
`COMPONENT_REAL_TARGET` property to `NOTFOUND` in such situations. Use
this information when config.env is generated put such components into
excluded submenu. Note that we cannot avoid linking empty component interface
into library, because there might be recursive dependencies and at the
time when the component is included we might not now if it has a real
target or not.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In some instances, the COMPONENTS_DISCOVERED build property is used to
walk through the component list and obtain component properties, such as
during configuration preparation or the inclusion of project_include
files. Since we know the component interfaces from the
COMPONENT_INTERFACES build property, we can switch to the faster
`__idf_component_get_property_unchecked` method to obtain component properties
in these cases. This change reduces the reconfiguration time by 0.4
seconds on my computer.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a helper function `__should_generate_sdkconfig()` that checks
whether the sdkconfig file should be generated. The
`__generate_sdkconfig()` function can be called multiple times, such as
when the initial sdkconfig is generated at the start of the build
process and later after additional components are fetched by the
component manager. There might be no components fetched by the
component manager, for example, in the hello_world example, or the
downloaded components may not contain any configuration files. In such
cases, there is no need to regenerate the sdkconfig. This helper
function stores the list of configuration files in the
`__PREV_KCONFIGS`, `__PREV_KCONFIG_PROJBUILDS`, and
`__PREV_SDKCONFIG_RENAMES` build properties at its end, and at the
beginning, it compares them with the current lists of configuration
files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a revised version of the existing __create_confserver_target
function. It creates a specified confserver target for a given
executable. The kconfig_menus.json file, used by the IDEs, is generated
when the confserver starts. This differs from the previous behavior,
where kconfig_menus.json was created globally along with other sdkconfig
formats. The reason for this change is that kconfig_menus.json contains
the Kconfig menu hierarchy and it is not just a flat option-value
format. It needs to accurately reflect which configurations for which
components are included or excluded. The kconfig_menus.json is generated
at `build/kconfig_menus.json`, where IDEs expect it. This means the file
is overwritten every time the server starts by kconfig_menus.json
version for given executable, so only one confserver can run at a time.
This is likely acceptable, as I don't believe it's possible to safely
run multiple instances of confserver due to the potential race
conditions when the sdkconfig file are generated.
In the future, we may include the location of kconfig_menus.json in
project_description.json so it can be easily identified by IDEs for
each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the initial sdkconfig is generated, the
kconfig_menus.json format is also created alongside other formats like
CMake, JSON, or C header. The kconfig_menus.json depends on the Kconfig
hierarchy and cannot be generated globally. It must be generated for
each executable to ensure that included and excluded components are
correctly positioned within the Kconfig menu hierarchy. Remove the
automatic generation of kconfig_menus.json.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current base kconfgen command is stored in the __BASE_KCONFGEN_CMD
build property, and it includes the --env-file option pointing to the
`build/config.env` file. The `build/config.env` file does not separate
component configuration files into included and excluded components,
because it is created before any executable is added and we need it for
the initial sdkconfig generation. The global `build/config.env` can be
used for every sdkconfig format (cmake, json, txt), but not for the
kconfig_menus.json format, because kconfig_menus.json relies on the
Kconfig layout and hierarchy. Let's allow specifying the --env-file for
each kconfgen invocation and remove it from the base kconfgen command.
This allows you to specify an --env-file for each executable, each with
its own config.env file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __run_kconfgen function does not take any arguments. The output
formats and related files are stored in the __KCONFGEN_OUTPUTS_CMD build
property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is basically a renamed version of __create_save_defconfig_target.
The creation of the save-defconfig target is done in
`idf_project_default()`.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For a specified executable, create a menuconfig target using the name
provided in the TARGET option. The function generates a config.env file
specific to the executable, where Kconfig files for components linked to
the executable and Kconfig files for components not linked to it are
separated. This separation allows for a clear visual distinction between
the configuration of components that are linked and those that are not
linked to the executable within the menuconfig.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Generate the config.env file for the specified executable. The
configuration file will be stored in the build directory, within a
directory named after the executable target name. The kconfigs*.in
files, which are generated by prepare_kconfig_files.py, will also be
stored in this directory.
This function primarily prepares the arguments for the
__create_config_env_file function based on the components linked to the
executable, ensuring that component Kconfig files are stored in the
appropriate kconfigs*.in files, depending on whether the component is
linked to the executable.
The directory where the generated files are stored is added to the
executable CONFIG_ENV_DIR property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __create_config_env_file function generates a configuration file for
the prepare_kconfig_files.py script. Currently, it specifies the paths
for the kconfigs*.in files, which are generated by
prepare_kconfig_files.py, to be located in the build directory. The
kconfigs*.in files, along with the configuration file, must also be
generated for each executable, as each executable may have a
different set of linked components. Set the paths for the
kconfigs*.in files to be generated by the prepare_kconfig_files.py
script in the same directory as the configuration file.
Also allow explicit specification of which component configuration files
should be placed in different kconfigs*.in files. For sdkconfig
generation, it is not relevant whether a component's Kconfig file is
placed in the kconfigs.in or kconfigs_excluded.in file. This distinction
is only important for visually separating the configuration of included
and excluded components in the menuconfig. For the main sdkconfig, all
component Kconfig files are placed in the kconfigs.in file, and the
generated kconfigs*.in files are stored in the build directory. For
executables, the generated kconfigs*.in files will be stored in separate
directories for each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The excluded Kconfig files cannot be collected globally, they must be
collected per executable. The distinction between included and excluded
Kconfig files is relevant only for menuconfig, which is specific to each
executable, as each executable may have a different set of linked
components. For the global sdkconfig file and its CMake and JSON
variants, whether the Kconfig files are excluded or not is irrelevant.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the cmakev2 implementation of Kconfig name consistency has a
slightly different warning message compared to cmakev2, causing the test
`test_build.py::test_build_with_misspelled_kconfig` to fail. Make the
warning messages consistent with cmakev1.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The function arguments in the documentation comments were using field
list, which caused text overflow in the generated documentation and
generally resulted in poor formatting. Let's use paragraphs for
the argument descriptions instead.
The documentation comments are written in reStructuredText, but
currently, they use inconsistent indentation. Standardize all the
documentation comments to use a four-character indentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current config.env.in template cannot be easily extended and shared
between cmakev1 and cmakev2. Although the root Kconfig is shared and
sources COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE and
COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE, these environment
variables cannot be set to empty strings. This issue arises unless we
also adjust cmakev1 and esp-docs, which use prepare_kconfig_files.py, to
properly set COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE and
COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE.
If an environment variable is set to an empty string and sourced in
Kconfig, kconfgen will expand the variable with expandvars(), resulting
in an empty value. This expanded variable is then appended to the root
Kconfig or the currently processed Kconfig directory path, depending on
whether osource or orsource is used, and passed to iglob. The iglog will
return the directory as a result, and when Kconfig attempts to open the
file, it will encounter an error because it is a directory.
To resolve this issue and enable sharing of the root Kconfig file, a new
config_buildv2.env.in template has been added, which includes
environment variables for the excluded components.
For cmakev1, kconfgen will encounter the osource command for
COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE. Since this variable is not
defined in config.env.in, expandvars() will return the environment
variable name instead of expanding it, and iglob will not match
anything. Because osource is used for excluded components, the
non-existent file for osource will be skipped.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Pass the environmental variables COMPONENT_KCONFIGS_EXCLUDED_SOURCE_FILE
and COMPONENT_KCONFIGS_PROJBUILD_EXCLUDED_SOURCE_FILE to menuconfig.
This will display the configuration for excluded components under new
menus in menuconfig. Additionally, set the IDF_BUILD_V2 environment
variable to make these new menus visible. The IDF_BUILD_V2 variable is
also passed to all kconfgen commands.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The following functions are added to tools/cmakev2/kconfig.cmake:
- __generate_kconfig_outputs(): Generates Kconfig output files.
- __create_base_kconfgen_command(): Creates command for running
kconfgen.
- __run_kconfgen(): Invokes kconfgen tool.
The following functions are added to tools/cmakev2/kconfig.cmake:
- __setup_kconfig_environment(): Creates Kconfig environment for Kconfig
output generation.
- __get_init_config_version(): Fetch ESP-IDF version and set the
environment variable.
- __create_config_env_file(): Create the config.env file for kconfgen.
The following functions are added to tools/cmakev2/kconfig.cmake:
- __collect_kconfig_files_from_components(): Collect Kconfig files from
components.
- __collect_kconfig_files_from_bootloader_components(): Collect Kconfig
files from bootloader components.
- __collect_kconfig_files_from_directory(): Collect Kconfig files from
a directory.
This commit adds the tools/cmakev2/kconfig.cmake file for Kconfig
processing. The file provides the following functions:
- __init_kconfig(): Placeholder to initialize Kconfig build properties,
collect kconfig files and generate the initial sdkconfig.
- __generate_sdkconfig(): Placeholder to generate sdkconfig output files.
Frantisek Hrbata
Roland Dobai
Jan Beran
Sudeep Mohanty