This commit removes picolibc flags from project.cmake file since they
are added by the esp_libc component when project_include.cmake files are
evaluated.
This commit moves the project initialization by moving the
__init_project_configuration() after the sdkconfig is generated and
included. This is because __init_project_configuration() depends on
config options.
This commit adds component validation checks for cmakev2. The file
component_validation.cmake is ported to cmakev2 and updated to work in
the buildv2 environment.
The function __set_component_cmakev1_properties() sets cmakev1
properties for cmakev2 components to maintain backword copatibility.
However, the function was setting SOURCES property instead of SRCS
property as was intended.
The idf_build_generate_depgraph function creates a component dependency
graph in dot (graphviz) format for a specified executable. It uses
existing helper functions from cmakev1, ensuring that the generated dot
files are produced in the same manner as in cmakev1. While adjustments
might be needed in the future if necessary, the current implementation
is intended to offer the same functionality as cmakev1. Similar to
cmakev1, the dot files are only generated only when the
__BUILD_COMPONENT_DEPGRAPH_ENABLED build property is set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The build system keeps track of each component source. Currently
there are four types of sources:
1. "project_components" - project components
2. "project_extra_components" - components from EXTRA_COMPONENT_DIRS
3. "project_managed_components" - custom project dependencies managed by the IDF Component Manager
4. "idf_components" - ESP-IDF built-in components, typically under /components
This can be used to identify the component libraries that are likely to
change during application development and pass them to ldgen as mutable
libraries. Add all components with "project_components" as their source
as mutable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a helper function `__should_generate_sdkconfig()` that checks
whether the sdkconfig file should be generated. The
`__generate_sdkconfig()` function can be called multiple times, such as
when the initial sdkconfig is generated at the start of the build
process and later after additional components are fetched by the
component manager. There might be no components fetched by the
component manager, for example, in the hello_world example, or the
downloaded components may not contain any configuration files. In such
cases, there is no need to regenerate the sdkconfig. This helper
function stores the list of configuration files in the
`__PREV_KCONFIGS`, `__PREV_KCONFIG_PROJBUILDS`, and
`__PREV_SDKCONFIG_RENAMES` build properties at its end, and at the
beginning, it compares them with the current lists of configuration
files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_create_size_report function allows for the creation of size
report targets based on the generated link map file. The size report
targets are created using the TARGET option name: "<target>",
"<target>-files", and "<target>-components". These size report targets
are added to the idf_default_project with the TARGET set to "size",
resulting in the creation of "size", "size-files", and "size-components"
targets for the default project.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The idf_project_init macro is evaluated within the global variable
scope, and currently, there are a few variables still set. Let's ensure
all variables are properly unset to avoid polluting the global variable
namespace.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The MINIMAL_BUILD property is not relevant in cmakev2, as only
components explicitly linked through targets are included in the build
by design. Display a warning when this outdated build property is set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The link flag for generating the map file is currently added globally to
the link_options. If multiple executables are generated, the link map
file is overwritten by the last created executable. Since cmakev2
supports building multiple executables, the link map file options need
to be set for each executable individually. To address this, add a new
MAPFILE_TARGET option to the idf_build_executable function. When set,
the link map file will be generated by the linker, and a target
specified in the MAPFILE_TARGET option will be created for the map file.
This also splits the idf_project_default macro. Only the
idf_project_init macro needs be called within the global scope, as it
includes the project_include.cmake files and the cmake version of the
configuration. The remaining functionality of the idf_project_default
macro is implemented in a __project_default helper function to avoid
polluting the global variable space.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the `idf_build_generate_metadata` function takes an
executable target as an argument, and the binary name is stored in the
`EXECUTABLE_BINARY` property. This approach is inconvenient because a
single executable might generate multiple binary images, making it
unreliable to store the binary image name in the executable property due
to the N:1 relationship. To address this, pass the binary image target
instead of the executable target to the `idf_build_generate_metadata`
function. This change is facilitated by linking the binary target,
executable target, and library interface targets with the following
properties:
binary:EXECUTABLE_TARGET -> executable:LIBRARY_INTERFACE -> library
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
With commit b719292b75 ("refactor(build-system): Simplify flash target creation"),
partition table and bootloader components are correctly adding their
dependency on the flash target, so there is no need for this to be done
in the build system. This removes the temporary workaround.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit adds support for uf2 targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/uf2.cmake.
- Adds the idf_create_uf2() function to create the uf2 targets. This
function now takes the executable as an argument thus allowing the uf2
target to be created per-executable.
- idf_project_default() is updated to create the uf2 targets.
This commit adds support for dfu targets for cmake2. The following
changes have been made:
- Adds a new tools/cmakev2/dfu.cmake file on similar lines as the
tools/cmake/dfu.cmake file.
- Adds the idf_create_dfu() function to create the dfu targets. This
function now takes the executable as an argument thus allowing the dfu
target to be created per-executable.
- idf_project_default() is updated to create the dfu target.
The COMPONENTS and EXCLUDE_COMPONENTS variables are unused in cmakev2. A
deprecation warning will be printed if they are set. This change also
includes a helper functions for printing deprecation warnings.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 dropped generation of prefix_map_gdbinit, reflect this
change also in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 added esp_stdio to common requires, add it in cmakev2 too
for backward compatibility.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the toolchain CMake files use the remove_duplicated_flags
function from utilities.cmake. The cmakev2 implementation also includes
this function for backward compatibility. Move the
remove_duplicated_flags function to a separate file,
deduplicate_flags.cmake, so it can be shared between cmakev1 and
cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Using idf_component_register is the preferred method for creating new
components for cmakev2. This approach ensures compatibility with both
versions of the build system. The KCONFIG and KCONFIG_PROJBUILD options
have been removed from the API documentation, but are retained in code
in case a cmakev1 component uses these options to warn about
incompatibility. Also remove a note about `project_include.cmake`,
because cmakev2 includes all project_include files.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the archive file name is set only in the cmakev1
idf_component_register shim. The predictable component archive file name
is important, for example, for usage in linker fragments. Ensure that
the cmakev2 component also has the archive file name set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev2 component sources are set, the SRCS target
property is used. This is obviously wrong because the correct CMake
property is SOURCES.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Rename the LINKER_SCRIPTS_STATIC component property to LINKER_SCRIPTS.
This property stores linker scripts that are not processed by ldgen,
which essentially includes all of them. The only linker script processed
by ldgen is sections.ld, which is handled by the esp_system component.
This implies that there is likely no practical use case for other
components to utilize ldgen processed linker scripts. This change is
purely cosmetic to allow components to add linker scripts with:
idf_component_set_property(${COMPONENT_TARGET} LINKER_SCRIPTS linker_script.ld APPEND)
instead of
idf_component_set_property(${COMPONENT_TARGET} LINKER_SCRIPTS_STATIC linker_script.ld APPEND)
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, we include numerous functions in the automatically generated
documentation for the build system API. Let's begin with only the
essential functions and gradually add more to the API based on requests
and actual needs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This commit adds a workaround in idf_build_component shim to avoid
initialization of a duplicate managed component. The component manager
when running on a Windows system returns duplicate managed components
upon successive invocations.
The current approach involves wrapping the library target within the
library interface target with the whole-archive flags, such as for the
GCC linker. However, this does not work as expected because the library
target is expanded, and the whole-archive flags are also applied to
other library targets that the wrapped library target depends on. IOW
the whole-archive flags surround multiple archives, not just the one
requrested. Generally, using linker flags like whole-archive in the
library INTERFACE(INTERFACE_LINK_LIBRARIES) does not seem to work as
CMake may perform deduplication and rearrange the flags [1].
The proof of concept used `CMAKE_LINK_LIBRARY_USING_<FEATURE>` with the
WHOLE_ARCHIVE feature, which was introduced in CMake 3.24 to specify how
the library target should be linked. There are two issues with this.
First, this feature is only available from CMake 3.24 onwards, while our
minimum CMake version is set to 3.22. More importantly, all occurrences
of a library on the link line are wrapped. For example, if a library
like `vfs` appears multiple times on the link line due to dependencies,
each occurrence will be wrapped with whole-archive, causing the linker
to complain about multiple symbol definitions. Therefore, even though
WHOLE_ARCHIVE is recommended for handling whole-archive linkage, it does
not seem suitable for our purposes. This was overlooked in the PoC
because only a simple testing component was used to check the
WHOLE_ARCHIVE behavior.
One way to address this issue is to adopt the same approach used in
cmakev1, which involves specifying the whole-archive flags when linking
component library targets to the final executable. In this case, CMake
retains the flags without any alteration, unlike when the flags are
specified in INTERFACE_LINK_LIBRARIES for the component interface
targets. While this approach is feasible, it would alter the current
logic, where we have a single library interface for the entire idf
library that can be linked to the executable. This change would also
complicate the direct use of the idf library, as it would no longer be
possible to simply link it to the executable. Instead, the executable
would need to correctly link the component libraries with the
whole-archive flags. We could encapsulate this process within the
idf_build_executable function, but projects that only use
idf_build_library would need to implement the same solution.
It appears possible to address this issue by using target_link_options
instead. This allows to specify exactly what should appear on the link
command line. One side effect of this approach is that the library
appears on the link command line multiple times: first when used with
target_link_options and second when used in target_link_libraries.
However, this does not seem to pose a problem. The flags specified with
target_link_options appear on the command line first, ensuring that the
whole-archive is prioritized, and the component archive libraries may be
repeated on the link line anyway due to dependencies. Essentially, the
final link command line is very similar to the one from cmakev1. We may
revisit this approach in the future, but for now, it seems to work as
expected.
[1] https://gitlab.kitware.com/cmake/cmake/-/issues/20078
[2] https://cmake.org/cmake/help/latest/variable/CMAKE_LINK_LIBRARY_USING_FEATURE.html
[3] https://discourse.cmake.org/t/automatically-wrapping-a-static-library-
in-whole-archive-no-whole-archive-when-used-during-linking/5883
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev1 properties for INCLUDE_DIRS and
PRIV_INCLUDE_DIRS are constructed in the
__set_component_cmakev1_properties function, the return values from the
get_target_property function for INCLUDE_DIRECTORIES and
INTERFACE_INCLUDE_DIRECTORIES properties are not checked. If a component
target does not set e.g. INCLUDE_DIRECTORIES property,
get_target_property will return a value such as `include_dirs-NOTFOUND`.
This value is subsequently passed to __get_relative_paths in the PATHS
argument, causing the file(RELATIVE_PATH) call in __get_relative_paths
to fail with an error.
```
CMake Error at /home/fhrbata/work/esp-idf/tools/cmakev2/utilities.cmake:235 (file):
file RELATIVE_PATH must be passed a full path to the file:
include_dirs-NOTFOUND
```
Fix this by explicitly set the PATHS to an empty list before passing it
to the __get_relative_paths function if the property is not set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
idf_build_executable() creates a dummy source file to create the
executable target. Since this file is created unconditionally everytime
CMake configuration happens, a source file change is registered which
causes CMake to execute other targets even though no other source file
changed.
In cmakev1, certain arguments provided to the idf_component_register
function are stored as component properties. These properties are used
to generate the project_description.json file, which other tools rely
on. Since the idf_component_register function is obsolete in cmakev2, we
need to recreate component properties such as INCLUDE_DIRS,
PRIV_INCLUDE_DIRS, REQUIRES, and PRIV_REQUIRES, which were previously
provided by the idf_component_register function in cmakev1. To achieve
this, let's examine the component's real target cmake properties and
reconstruct them to maintain compatibility with cmakev1.
The real target cmake properties may include generator expressions,
which are ignored. This should be acceptable, as they were not addressed
in cmakev1 either, and handling them is likely not feasible.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Return the difference between two lists, meaning the elements that are
present in the first list but not in the second.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This helper function removes entries containing generator expressions
from the given list. The list is modified in place.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a helper function that returns a list of paths relative to a
given base directory for a list of input paths.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
When a linker script file with a .in extension is added using the
target_linker_script function, it is processed with the C preprocessor.
The linker scripts are preprocessed only once, even if they are used in
multiple libraries, because they are the same.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
During the initialization of a component in the __init_component
function, add the COMPONENT_BUILD_DIR property. This can be used to
store component-specific generated files, such as preprocessed linker
scripts.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component has been renamed, update the name in the common
requirements for the cmakev1 components.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component can be referenced by multiple identifiers such as the
component name, which is derived from the component directory name,
component target, interface, or aliases. All component properties are
attached to the component interface target, which is also used for
declaring component dependencies. The cmakev2 build system has a
function called __get_component_interface, which is responsible for
identifying the component interface based on the given component
identifier. Since this function is called frequently, it needs to be
reasonably fast.
This introduces a new __idf_component_interface_cache INTERFACE target,
which serves as a mapping cache between component identifiers and the
component interface. The cache for each component is initialized in the
__init_component function, which introduces a component to the build
system. Currently, the component interface search is conducted by
examining the COMPONENTS_DISCOVERED and COMPONENT_INTERFACES lists
stored as build properties. Since the build system is aware of most
component identifiers, such as component name, target, and alias, during
component initialization, it can add mappings between component
identifiers and the component interface to a cache, which is built as
the components are initialized. This cache is used in the
__get_component_interface function instead of looking into the
COMPONENTS_DISCOVERED or COMPONENT_INTERFACES lists. This significantly
speeds up the component interface search and also makes the code much
simpler and more readable.
The component interface cache also completely replaces the existing
component name resolution, which was introduced because of the component
manager, and the cache used for resolved component names. This is
possible because all the necessary information is available during
component initialization when the component interface cache is
populated. The ambiguity of components is resolved based on component
source/priority.
Here is an example of the component interface mapping for the
espressif__led_strip component to the idf_espressif__led_strip interface
target. The component name, without the namespace, is referred to as the
short name. In this example, it is led_strip.
- espressif__led_strip -> idf_espressif__led_strip # name(directory name)
- idf_espressif__led_strip -> idf_espressif__led_strip # interface
- idf::espressif__led_strip -> idf_espressif__led_strip # alias
- _idf_espressif__led_strip -> idf_espressif__led_strip # real target
- led_strip -> idf_espressif__led_strip # short name
When another component with the same espressif__led_strip name is
initialized with a different priority, there is no need to change
anything in the cache, because the mapping stays the same.
The cache must be updated when two components share the same short name
but belong to different namespaces. This situation is likely uncommon.
For instance, consider espressif__led_strip and my__led_strip. If
my__led_strip has a higher priority, the cache is updated to reflect the
short name as follows:
- led_strip -> idf_my__led_strip
If both components have the same priority, the short name mapping for
led_strip is entirely removed. Conversely, if my__led_strip has a lower
priority, no short name mapping is added for it.
The short name is also added to the COMPONENT_SHORT_NAME property of the
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is a revised version of the existing __create_confserver_target
function. It creates a specified confserver target for a given
executable. The kconfig_menus.json file, used by the IDEs, is generated
when the confserver starts. This differs from the previous behavior,
where kconfig_menus.json was created globally along with other sdkconfig
formats. The reason for this change is that kconfig_menus.json contains
the Kconfig menu hierarchy and it is not just a flat option-value
format. It needs to accurately reflect which configurations for which
components are included or excluded. The kconfig_menus.json is generated
at `build/kconfig_menus.json`, where IDEs expect it. This means the file
is overwritten every time the server starts by kconfig_menus.json
version for given executable, so only one confserver can run at a time.
This is likely acceptable, as I don't believe it's possible to safely
run multiple instances of confserver due to the potential race
conditions when the sdkconfig file are generated.
In the future, we may include the location of kconfig_menus.json in
project_description.json so it can be easily identified by IDEs for
each executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the initial sdkconfig is generated, the
kconfig_menus.json format is also created alongside other formats like
CMake, JSON, or C header. The kconfig_menus.json depends on the Kconfig
hierarchy and cannot be generated globally. It must be generated for
each executable to ensure that included and excluded components are
correctly positioned within the Kconfig menu hierarchy. Remove the
automatic generation of kconfig_menus.json.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current base kconfgen command is stored in the __BASE_KCONFGEN_CMD
build property, and it includes the --env-file option pointing to the
`build/config.env` file. The `build/config.env` file does not separate
component configuration files into included and excluded components,
because it is created before any executable is added and we need it for
the initial sdkconfig generation. The global `build/config.env` can be
used for every sdkconfig format (cmake, json, txt), but not for the
kconfig_menus.json format, because kconfig_menus.json relies on the
Kconfig layout and hierarchy. Let's allow specifying the --env-file for
each kconfgen invocation and remove it from the base kconfgen command.
This allows you to specify an --env-file for each executable, each with
its own config.env file.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __run_kconfgen function does not take any arguments. The output
formats and related files are stored in the __KCONFGEN_OUTPUTS_CMD build
property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This is basically a renamed version of __create_save_defconfig_target.
The creation of the save-defconfig target is done in
`idf_project_default()`.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
For a specified executable, create a menuconfig target using the name
provided in the TARGET option. The function generates a config.env file
specific to the executable, where Kconfig files for components linked to
the executable and Kconfig files for components not linked to it are
separated. This separation allows for a clear visual distinction between
the configuration of components that are linked and those that are not
linked to the executable within the menuconfig.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Generate the config.env file for the specified executable. The
configuration file will be stored in the build directory, within a
directory named after the executable target name. The kconfigs*.in
files, which are generated by prepare_kconfig_files.py, will also be
stored in this directory.
This function primarily prepares the arguments for the
__create_config_env_file function based on the components linked to the
executable, ensuring that component Kconfig files are stored in the
appropriate kconfigs*.in files, depending on whether the component is
linked to the executable.
The directory where the generated files are stored is added to the
executable CONFIG_ENV_DIR property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Sudeep Mohanty
Frantisek Hrbata
Daniel Paul
Roland Dobai
Peter Dragun