The function __set_component_cmakev1_properties() sets cmakev1
properties for cmakev2 components to maintain backword copatibility.
However, the function was setting SOURCES property instead of SRCS
property as was intended.
Currently, the archive file name is set only in the cmakev1
idf_component_register shim. The predictable component archive file name
is important, for example, for usage in linker fragments. Ensure that
the cmakev2 component also has the archive file name set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev2 component sources are set, the SRCS target
property is used. This is obviously wrong because the correct CMake
property is SOURCES.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, we include numerous functions in the automatically generated
documentation for the build system API. Let's begin with only the
essential functions and gradually add more to the API based on requests
and actual needs.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The current approach involves wrapping the library target within the
library interface target with the whole-archive flags, such as for the
GCC linker. However, this does not work as expected because the library
target is expanded, and the whole-archive flags are also applied to
other library targets that the wrapped library target depends on. IOW
the whole-archive flags surround multiple archives, not just the one
requrested. Generally, using linker flags like whole-archive in the
library INTERFACE(INTERFACE_LINK_LIBRARIES) does not seem to work as
CMake may perform deduplication and rearrange the flags [1].
The proof of concept used `CMAKE_LINK_LIBRARY_USING_<FEATURE>` with the
WHOLE_ARCHIVE feature, which was introduced in CMake 3.24 to specify how
the library target should be linked. There are two issues with this.
First, this feature is only available from CMake 3.24 onwards, while our
minimum CMake version is set to 3.22. More importantly, all occurrences
of a library on the link line are wrapped. For example, if a library
like `vfs` appears multiple times on the link line due to dependencies,
each occurrence will be wrapped with whole-archive, causing the linker
to complain about multiple symbol definitions. Therefore, even though
WHOLE_ARCHIVE is recommended for handling whole-archive linkage, it does
not seem suitable for our purposes. This was overlooked in the PoC
because only a simple testing component was used to check the
WHOLE_ARCHIVE behavior.
One way to address this issue is to adopt the same approach used in
cmakev1, which involves specifying the whole-archive flags when linking
component library targets to the final executable. In this case, CMake
retains the flags without any alteration, unlike when the flags are
specified in INTERFACE_LINK_LIBRARIES for the component interface
targets. While this approach is feasible, it would alter the current
logic, where we have a single library interface for the entire idf
library that can be linked to the executable. This change would also
complicate the direct use of the idf library, as it would no longer be
possible to simply link it to the executable. Instead, the executable
would need to correctly link the component libraries with the
whole-archive flags. We could encapsulate this process within the
idf_build_executable function, but projects that only use
idf_build_library would need to implement the same solution.
It appears possible to address this issue by using target_link_options
instead. This allows to specify exactly what should appear on the link
command line. One side effect of this approach is that the library
appears on the link command line multiple times: first when used with
target_link_options and second when used in target_link_libraries.
However, this does not seem to pose a problem. The flags specified with
target_link_options appear on the command line first, ensuring that the
whole-archive is prioritized, and the component archive libraries may be
repeated on the link line anyway due to dependencies. Essentially, the
final link command line is very similar to the one from cmakev1. We may
revisit this approach in the future, but for now, it seems to work as
expected.
[1] https://gitlab.kitware.com/cmake/cmake/-/issues/20078
[2] https://cmake.org/cmake/help/latest/variable/CMAKE_LINK_LIBRARY_USING_FEATURE.html
[3] https://discourse.cmake.org/t/automatically-wrapping-a-static-library-
in-whole-archive-no-whole-archive-when-used-during-linking/5883
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, when the cmakev1 properties for INCLUDE_DIRS and
PRIV_INCLUDE_DIRS are constructed in the
__set_component_cmakev1_properties function, the return values from the
get_target_property function for INCLUDE_DIRECTORIES and
INTERFACE_INCLUDE_DIRECTORIES properties are not checked. If a component
target does not set e.g. INCLUDE_DIRECTORIES property,
get_target_property will return a value such as `include_dirs-NOTFOUND`.
This value is subsequently passed to __get_relative_paths in the PATHS
argument, causing the file(RELATIVE_PATH) call in __get_relative_paths
to fail with an error.
```
CMake Error at /home/fhrbata/work/esp-idf/tools/cmakev2/utilities.cmake:235 (file):
file RELATIVE_PATH must be passed a full path to the file:
include_dirs-NOTFOUND
```
Fix this by explicitly set the PATHS to an empty list before passing it
to the __get_relative_paths function if the property is not set.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
In cmakev1, certain arguments provided to the idf_component_register
function are stored as component properties. These properties are used
to generate the project_description.json file, which other tools rely
on. Since the idf_component_register function is obsolete in cmakev2, we
need to recreate component properties such as INCLUDE_DIRS,
PRIV_INCLUDE_DIRS, REQUIRES, and PRIV_REQUIRES, which were previously
provided by the idf_component_register function in cmakev1. To achieve
this, let's examine the component's real target cmake properties and
reconstruct them to maintain compatibility with cmakev1.
The real target cmake properties may include generator expressions,
which are ignored. This should be acceptable, as they were not addressed
in cmakev1 either, and handling them is likely not feasible.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
During the initialization of a component in the __init_component
function, add the COMPONENT_BUILD_DIR property. This can be used to
store component-specific generated files, such as preprocessed linker
scripts.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The component can be referenced by multiple identifiers such as the
component name, which is derived from the component directory name,
component target, interface, or aliases. All component properties are
attached to the component interface target, which is also used for
declaring component dependencies. The cmakev2 build system has a
function called __get_component_interface, which is responsible for
identifying the component interface based on the given component
identifier. Since this function is called frequently, it needs to be
reasonably fast.
This introduces a new __idf_component_interface_cache INTERFACE target,
which serves as a mapping cache between component identifiers and the
component interface. The cache for each component is initialized in the
__init_component function, which introduces a component to the build
system. Currently, the component interface search is conducted by
examining the COMPONENTS_DISCOVERED and COMPONENT_INTERFACES lists
stored as build properties. Since the build system is aware of most
component identifiers, such as component name, target, and alias, during
component initialization, it can add mappings between component
identifiers and the component interface to a cache, which is built as
the components are initialized. This cache is used in the
__get_component_interface function instead of looking into the
COMPONENTS_DISCOVERED or COMPONENT_INTERFACES lists. This significantly
speeds up the component interface search and also makes the code much
simpler and more readable.
The component interface cache also completely replaces the existing
component name resolution, which was introduced because of the component
manager, and the cache used for resolved component names. This is
possible because all the necessary information is available during
component initialization when the component interface cache is
populated. The ambiguity of components is resolved based on component
source/priority.
Here is an example of the component interface mapping for the
espressif__led_strip component to the idf_espressif__led_strip interface
target. The component name, without the namespace, is referred to as the
short name. In this example, it is led_strip.
- espressif__led_strip -> idf_espressif__led_strip # name(directory name)
- idf_espressif__led_strip -> idf_espressif__led_strip # interface
- idf::espressif__led_strip -> idf_espressif__led_strip # alias
- _idf_espressif__led_strip -> idf_espressif__led_strip # real target
- led_strip -> idf_espressif__led_strip # short name
When another component with the same espressif__led_strip name is
initialized with a different priority, there is no need to change
anything in the cache, because the mapping stays the same.
The cache must be updated when two components share the same short name
but belong to different namespaces. This situation is likely uncommon.
For instance, consider espressif__led_strip and my__led_strip. If
my__led_strip has a higher priority, the cache is updated to reflect the
short name as follows:
- led_strip -> idf_my__led_strip
If both components have the same priority, the short name mapping for
led_strip is entirely removed. Conversely, if my__led_strip has a lower
priority, no short name mapping is added for it.
The short name is also added to the COMPONENT_SHORT_NAME property of the
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, cmakev2 evaluates each component using add_subdirectory with
the EXCLUDE_FROM_ALL option. The intention was to exclude all components
from being built by default unless they are explicitly linked to an
executable based on dependencies. This approach aims to avoid building
components that are included but not actually linked to the executable.
However, this has the side effect of preventing components from adding
their custom targets to the "all" target, which is used, for example, by
esp_phy. Generally, we should not restrict components from adding
targets to "all". Since components are only included if explicitly
requested by default, removing the EXCLUDE_FROM_ALL option should be
acceptable.
The downside is that if IDF_INCLUDE_ALL_COMPONENTS is set and the user
runs `idf.py build`, all components will be built, even if they are not
linked to the executable.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The function arguments in the documentation comments were using field
list, which caused text overflow in the generated documentation and
generally resulted in poor formatting. Let's use paragraphs for
the argument descriptions instead.
The documentation comments are written in reStructuredText, but
currently, they use inconsistent indentation. Standardize all the
documentation comments to use a four-character indentation.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The cmakev1 sets __PREFIX as a component property. To maintain backward
compatibility, set it in cmakev2 as well.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Integrate the ldgen into cmakev2. With this change, it becomes possible
to actually link the project executables.
In cmakev2, the handling of linker scripts is deferred to
idf_build_library, unlike in cmakev1, where linker scripts were added
and generated during the target_linker_script call. In cmakev2, the
target_linker_script only adds the linker scripts and templates, along
with the output filenames for the linker scripts generated from the
templates, to the component property. When idf_build_library is called
and all the requested components are included, it uses the
__get_target_dependencies function to obtain all transitively linked
targets to the library interface target. These targets are mapped to the
components, and the LIBRARY_COMPONENTS_LINKED library property is set.
It contains all components linked to the library interface target. The
components from LIBRARY_COMPONENTS_LINKED are used to collect linker
fragments and linker scripts utilized in the library. Additionally, all
targets transitively linked to the library are used to identify archive
files used in the library. This includes component archives and archives
added with the add_prebuilt_library function. The archives and
ldfragments related to the components linked to the library are used
when ldgen generates the linker scripts from templates.
The linker scripts, both static and generated by ldgen, are added to the
library interface link options and INTERFACE_LINK_DEPENDS property. For
generated linker scripts, a custom target is created and added as a
dependency for the library interface to ensure they are generated before
the link.
The difference compared to cmakev1 is that the generated linker scripts,
currently only sections.ld, are not global in the project but are
generated per library. This means there might be multiple versions of
sections.ld depending on the components included in the library. For
example, a component like esp_system may be linked to multiple library
interface targets, each with a different set of components. This results
in different sets of fragment files and library archives and different
versions of the sections.ld linker script. This should ensure proper
dependencies between targets. In other words, if a component changes its
linker fragment, only executables linked to libraries using this
component should be re-linked. As a consequence of this approach, the
generated linker scripts for different libraries need to have different
names or be stored in different directories to avoid overwriting the
linker script for one library with the linker script for another library
using the same component. This is handled with a suffix, which is based
on the library interface target name and appended to the generated
linker script. So, for example, there is no sections.ld, but instead
sections.ld_fatfs_lib or sections.ld_hello_world_lib. As a next step, we
can add a DEFAULT option to idf_build_library and avoid adding the
suffix for the default library.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Currently, the common components are initialized in the idf_project_init
macro, which means they are included even for cmakev2 components.
However, cmakev2 components are expected to explicitly specify all their
dependencies instead of relying on common components being automatically
linked. Therefore, common components are only relevant within the
context of cmakev1 components. To address this, move the
__init_common_components function call to idf_component_register and
include common components only when they are truly needed for a cmakev1
component.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add embedded source files to the component target sources. This is
intentionally managed in idf_component_include instead of
idf_component_register, allowing even cmakev2 components to set the
component's EMBED_FILES and EMBED_TXTFILES properties. Although it might
be more convenient to call the target_add_binary_data helper function
directly in cmakev2.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a component interface alias only after the actual target is created,
meaning the component is included. The alias has a well-defined name and
can be used in generator expressions like
`$<$<TARGET_EXISTS:idf::esp_netif>:>` without needing to retrieve the
COMPONENT_ALIAS property.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The __DEPENDENCY_CHAIN was not properly created because the component
names were not added to it. Fix this by maintaining it correctly.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
This removes the temporary restriction that only allowed components with
project_components as their source to be evaluated.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
During the initialization of a component in the __init_component
function, store the full path of the project_include.cmake file in the
component's __PROJECT_INCLUDE property, if it exists. Include the
project_include.cmake files for all discovered components at the global
scope within the idf_project_init macro.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Add a core function of the build system, responsible for including the
specified component identified by name, into the build
process.
Currently, the non-project components written in cmakev1 are ignored and
not evaluated with the add_subdirectory command because there are no
shims for the cmakev1 API yet. However, this allows for the evaluation
of project components written using the cmakev2 approach, which is
closer to the native CMake usage.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
The following functions are added to tools/cmakev2/kconfig.cmake:
- __collect_kconfig_files_from_components(): Collect Kconfig files from
components.
- __collect_kconfig_files_from_bootloader_components(): Collect Kconfig
files from bootloader components.
- __collect_kconfig_files_from_directory(): Collect Kconfig files from
a directory.
The `idf_build_(set|get)_property` and
`idf_component_(set|get)_property` functions share a lot of similar
code. Move these common parts into new `__(set|get)_property` helper
functions. With the upcoming `idf_build_library` API function, we might
need to add properties for the interface target created for the library,
which would otherwise lead to yet another code duplication for setting
and getting library interface properties.
Update the current implementations of `idf_build_(set|get)_property` and
`idf_component_(set|get)_property` to utilize these new helper
functions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
With the default signature of `cmake_parse_arguments`, without using
`PARSE_ARGV`, it's not possible to determine if options were not
specified or set as an empty string (or empty list)[1]. If an empty
string is passed to the `PATHS` option, the variable parsed by
`cmake_parse_arguments` is not defined. This issue can be addressed by
using `PARSE_ARGV`, but this approach only works for functions and
requires CMake version 3.31 or newer. Additionally, when `PARSE_ARGV` is
used for multiple value option, the values are not concatenated into a
single list, which is inconvenient, as the lists are instead escaped. If
the `PATHS` option is not defined, set it to an empty string. This
allows passing an empty string, as well as a mix of lists and
individual strings, through PATHS.
The behaviour can be seen with a simple example:
$ cmake -P test.cmake
```test.cmake
cmake_minimum_required(VERSION 3.22)
function(test)
set(options)
set(one_value)
set(multi_value PATHS)
cmake_parse_arguments(ARG "${options}" "${one_value}" "${multi_value}" ${ARGN})
#cmake_parse_arguments(PARSE_ARGV 0 ARG "${options}" "${one_value}" "${multi_value}")
message("PATHS: ${ARG_PATHS}")
endfunction()
test(PATHS "one;two;three" "four" "five;six" "seven")
```
```
PATHS: one;two;three;four;five;six;seven
vs
PATHS: one\;two\;three;four;five\;six;seven
```
Also update the current usage of __get_absolute_paths, as the check for
empty PATHS is no longer necessary.
[1] https://cmake.org/cmake/help/latest/policy/CMP0174.html#policy:CMP0174
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Discover component directories and initialize components within them.
This process does not include managed components, which should be added
separately at a later stage. To facilitate this, some minimal
functionalities are introduced, such as build properties, component
properties, and other helper functions.
Signed-off-by: Frantisek Hrbata <frantisek.hrbata@espressif.com>
Sudeep Mohanty
Frantisek Hrbata