Merge branch 'feat/esp_idf_sbom' into 'master'

feat(tools): esp-idf-sbom integration into idf.py

Closes IDF-15120

See merge request espressif/esp-idf!45696

docs/en/api-guides/tools/idf-sbom.rst

@@ -28,55 +28,49 @@ The component in an SBOM is described with information such as its name and vers
     * - Timestamp
       - Record of the date and time of the SBOM data assembly.
 
-To generate NTIA compliant SBOM file for an ESP-IDF project, you can use the `esp-idf-sbom`_ tool. This tool generates SBOM file in the Software Package Data Exchange (`SPDX`_) format version 2.2 and allows to scan it for known vulnerabilities against the National Vulnerability Database (`NVD`_) based on the Common Platform Enumeration (`CPE`_).
+To generate an NTIA‑compliant SBOM file for an ESP‑IDF project, use the ``idf.py sbom-create`` command. This command generates an SBOM file in Software Package Data Exchange (`SPDX`_) format, version 2.2. The generated SBOM can then be scanned for known vulnerabilities in the National Vulnerability Database (`NVD`_) using Common Platform Enumeration (`CPE`_) identifiers with the ``idf.py sbom-check`` command.
 
-This document outlines the basic usage and features of `esp-idf-sbom`_. For more detailed information, please refer to the documentation within the `esp-idf-sbom`_ project.
+The ``idf.py sbom-create`` and ``idf.py sbom-check`` commands provide basic integration of the `esp-idf-sbom`_ tool into ``idf.py``. For more detailed information, see the documentation in the `esp-idf-sbom`_ project.
 
+Generating SPDX SBOM File for ESP-IDF Project
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Installation
-^^^^^^^^^^^^
-Currently, `esp-idf-sbom`_ is not integrated into the ESP-IDF and needs to be installed separately from PyPI by using
+The generated SPDX SBOM file contains information derived from the project’s build artifacts. To ensure these artifacts are up-to-date, the ``idf.py sbom-create`` command depends on the ``idf.py build`` command. If the project has not yet been built, or if source files have changed, a build is automatically triggered before the SBOM is generated. This guarantees that the resulting SBOM always accurately reflects the current state of the project.
+
+To generate a default minimal SPDX SBOM file for your project, run the following command in your project directory.
 
 .. code-block:: bash
 
-    $ pip install esp-idf-sbom
+    $ idf.py sbom-create
 
-Once installed, the `esp-idf-sbom`_ tool can be invoked as a Python module using the command ``python -m esp_idf_sbom``, or as a console script with the ``esp-idf-sbom`` command.
+By default, the SPDX SBOM file is created in the build directory and named after the project, with the ``.spdx`` extension. For example, for the ``hello_world`` example project, the file will be located at ``build/hello_world.spdx`` by default. The output location of the generated SPDX SBOM file can be changed using the ``--spdx-file`` option. For more information and additional options, see ``idf.py sbom-create --help``.
 
+The ``idf.py sbom-create`` command generates the default SPDX SBOM file for a project. If more control is required over how the SPDX SBOM file is generated and what information it contains, refer to the `esp-idf-sbom`_ tool documentation. That documentation also provides detailed information about the component layout within SPDX SBOM files and the manifest files used to describe components.
 
-Generating SBOM for ESP-IDF project
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-To generate the SBOM file for your project, start by building it using ``idf.py build``. The `esp-idf-sbom`_ tool requires the ``project_description.json`` file as input, which is created during the project's configuration. This file contains essential metadata, including details about components and references to build artifacts necessary for the `esp-idf-sbom`_ tool to produce the SBOM file. Typically, you can find it in the project's ``build`` directory.
+Checking SPDX SBOM File for Vulnerabilities
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-To create a default SPDX SBOM file for your project, use
+The ``idf.py sbom-check`` command scans a project's SPDX SBOM file for known vulnerabilities. While this command is the primary method within ESP-IDF, you can also use any other third-party utility that supports the SPDX format, such as `cve-bin-tool`_.
+
+To check an SPDX SBOM file for vulnerabilities, use the following syntax:
 
 .. code-block:: bash
 
-    $ esp-idf-sbom create -o sbom.spdx build/project_description.json
+    $ idf.py sbom-check --spdx-file <path_to_file>
 
-In the `SPDX` format, the project's application and its components are represented as ``SPDX Packages``, linked by ``SPDX relationships`` to illustrate the supply chain. By default, the generated SBOM contains an SPDX Package for each component involved in the build process. This implies that the SBOM might list components that were built but not ultimately included in the project's final binary image, as none of their symbols were used. Although these components appear in the SBOM, they do not have a relationship with the project's application ``SPDX Package``. For additional information on the SPDX format, please see the `SPDX specification`_.
-
-To generate a minimal SBOM file that includes only the components linked to the project's application, use
+For example, to scan the SPDX SBOM file generated for the ``hello_world`` example, run:
 
 .. code-block:: bash
 
-    $ esp-idf-sbom create --rem-unused --rem-config -o sbom.spdx build/project_description.json
-
-The SPDX SBOM produced might also include other information, such as licenses. The level of information in the generated SBOM can be modified using command-line options. For further details, please refer to `esp-idf-sbom`_.
+    $ idf.py sbom-check --spdx-file build/hello_world.spdx
 
 
-Checking SBOM for vulnerabilities
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+By default, the ``idf.py sbom-check`` command uses a local mirror of the `NVD`_ database. You can choose between two modes of operation:
 
-The `esp-idf-sbom`_ tools comes with a built-in vulnerability checker, but you have the option to use other tools supporting the SPDX SBOM format. For example, `cve-bin-tool`_ is a free and open-source tool that allows to scan SPDX SBOM for vulnerabilities.
+* **Local Mirror (Default):** This requires approximately 900 MB of disk space. While the initial population of the database may take some time, this method is convenient for frequent, fast, and offline scans.
+* **NVD REST API:** Use the ``--nvd-api`` option to query the NVD database directly online. This is better suited for occasional or ad hoc scans where you want to avoid the disk space overhead of a local mirror.
 
-To check the SBOM for vulnerabilities with `esp-idf-sbom`_ tool, use
-
-.. code-block:: bash
-
-    $ esp-idf-sbom check sbom.spdx
-
-By default, this will create a report table that summarizes and details any potential vulnerabilities found during the scan. Below is an example of shortened report for a hypothetical project that intentionally uses an outdated version of the ``expat`` component, demonstrating what a vulnerability detection report might look like.
+The command by default generates a report table summarizing any potential vulnerabilities found. Below is an example of a shortened report for a hypothetical project that intentionally uses an outdated version of the ``expat`` component to demonstrate the detection capabilities. For more information and additional options, see ``idf.py sbom-check --help``.
 
 .. code-block:: bash
 
@@ -159,9 +153,24 @@ By default, this will create a report table that summarizes and details any pote
 
     [... additional lines removed ...]
 
+Checking Components for Vulnerabilities
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-For detailed information about the component layout within the SPDX SBOM, the manifest files used to describe components, and all available options, please visit the `esp-idf-sbom`_ project GitHub page.
+In addition to build artifacts, the ``idf.py sbom-create`` command uses SBOM manifest files located in component directories as a primary data source. You can use ``idf.py sbom-check`` to scan these component manifest files directly for vulnerabilities without needing to generate a complete project SBOM file.
 
+If ``idf.py sbom-check`` is run without any options, it recursively searches the current directory for all SBOM manifest files and scans them.
+
+For example, to check for vulnerabilities in your currently checked-out version of ESP-IDF, run the following from the ESP-IDF root directory:
+
+.. code-block:: bash
+
+    idf.py sbom-check
+
+To scan a specific directory instead of the current one, use the ``--path`` option:
+
+.. code-block:: bash
+
+    idf.py sbom-check --path <path_to_directory>
 
 .. _esp-idf-sbom: https://github.com/espressif/esp-idf-sbom
 .. _SBOM: https://en.wikipedia.org/wiki/Software_supply_chain

tools/idf.py

@@ -544,7 +544,7 @@ def init_cli(verbose_output: list | None = None) -> Any:
         def _print_closing_message(self, args: PropertyDict, actions: KeysView) -> None:
             # print a closing message of some kind,
             # except if any of the following actions were requested
-            if any(t in str(actions) for t in ('flash', 'dfu', 'uf2', 'uf2-app', 'qemu')):
+            if any(t in str(actions) for t in ('flash', 'dfu', 'uf2', 'uf2-app', 'qemu', 'sbom-create')):
                 print('Done')
                 return
 

tools/requirements/requirements.core.txt

@@ -20,6 +20,7 @@ esp-idf-nvs-partition-gen
 esp-idf-size
 esp-idf-diag
 esp-idf-panic-decoder
+esp-idf-sbom
 pyclang
 construct
 rich

tools/test_build_system/test_common.py

@@ -397,3 +397,9 @@ def test_hints_components_loading(
     assert 'HINT FROM PROJECT COMPONENT' in ret.stderr, (
         'Hint from project component should be displayed in build output'
     )
+
+
+def test_sbom_create_cmd(idf_py: IdfPyFunc, test_app_copy: Path) -> None:
+    logging.info('Test if sbom-create command works correctly')
+    idf_py('sbom-create', '--spdx-file', 'test_app.spdx')
+    assert (test_app_copy / 'test_app.spdx').is_file()