espressif/esp-idf
1
0
Fork 0
mirror of https://github.com/espressif/esp-idf.git synced 2026-04-27 19:13:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(ble_log_console): add CLI entry point, build scripts, and docs

Browse Source 
- Click-based CLI entry point (console.py) with --port, --baudrate,
  --log-dir options and interactive Launch Screen when --port omitted
- Deprecation warning for --output CLI flag
- 'ls' subcommand to list saved capture files
- PyInstaller build scripts (build.sh, build.bat, build_exe.py)
- Convenience run scripts (run.sh, run.bat)
- README rewrite with installation, usage, architecture docs, and
  troubleshooting guide
- User guides in English and Chinese
- Remove old single-file ble_log_console.py
- Register new scripts in executable-list.txt
This commit is contained in:
Zhou Xiao
2026-03-23 02:10:56 +08:00
parent e40fd6753d
commit adcf6cb75f
11 changed files with 1176 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/bt/ble_log_console/README.md
+425
View File
@@ -0,0 +1,425 @@
# BLE Log Console
A Textual-based TUI tool for real-time capture, parsing, and display of BLE Log frames from UART DMA output. Designed for both Espressif internal developers and ESP-IDF customers.
**User Guide**: [English](docs/User-Guide-EN.md) | [中文](docs/User-Guide-CN.md)
## Table of Contents
- [Features](#features)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [Usage](#usage)
- [Firmware Configuration](#firmware-configuration)
- [How It Works](#how-it-works)
- [Offline Analysis](#offline-analysis)
- [Keyboard Shortcuts](#keyboard-shortcuts)
- [Building Executable](#building-executable)
- [Architecture](#architecture)
- [Development](#development)
- [Troubleshooting](#troubleshooting)
## Features
- **Real-time frame parsing** with automatic checksum mode detection (XOR/Sum × Full/Header-only)
- **Frame sync state machine** with loss detection and recovery (SEARCHING → CONFIRMING → SYNCED → CONFIRMING_LOSS)
- **Internal frame decoding**: INIT_DONE (firmware version), INFO, ENH_STAT (per-source write/loss counters), FLUSH
- **UART redirect display**: When firmware uses UART PORT 0, redirected `ESP_LOG` output is decoded from `REDIR` frames and displayed as ASCII log lines
- **Dimmed internal logs**: Console-generated messages (sync, warnings, errors) are dimmed to visually separate from user application logs
- **Live status panel**: Sync state, RX bytes, transport speed (current + max), frame rate
- **Per-source frame loss warnings**: Real-time `[WARN]` notifications when firmware reports new frame loss, with source name (e.g., `LL_TASK`)
- **Per-source statistics view**: Press `d` to open a modal overlay showing written/lost frame and byte counts per source
- **Raw binary capture**: All received bytes are saved to a `.bin` file for [offline analysis](#offline-analysis)
- **Scrollable log view** with auto-scroll toggle
## Prerequisites
### 1. ESP-IDF Environment
BLE Log Console runs within the ESP-IDF Python environment. You must source `export.sh` before use:
```bash
cd <esp-idf-root>
. ./export.sh
```
This sets up the Python virtual environment at `~/.espressif/python_env/` which includes all required dependencies (`textual`, `pyserial`, `click`, etc.).
### 2. Firmware Configuration
The target ESP32 device must have the BLE Log module enabled and configured for UART DMA output. Configure via `idf.py menuconfig`:
```
Component config → Bluetooth → BT Logs → Enable BLE Log Module (Experimental) [y]
```
Then select the transport peripheral and UART settings:
```
Component config → Bluetooth → BT Logs → BLE Log Module
→ Peripheral Selection → UART DMA
→ UART DMA Configuration
→ UART Port Number (default: 0)
→ Baud Rate (default: 3000000)
→ TX GPIO Number (set to match your hardware)
```
#### Quick Setup: Critical-Log-Only Mode
The simplest way to enable BLE Log with UART DMA output:
```
Component config → Bluetooth → BT Logs → Enable critical-log-only mode [y]
```
This automatically enables the BLE Log Module, selects UART DMA as the default peripheral, and restricts each stack (Controller/Host/Mesh) to critical logs only.
#### Recommended Kconfig Options
| Kconfig Option | Recommended | Why |
|----------------|-------------|-----|
| `CONFIG_BT_LOG_CRITICAL_ONLY` | `y` | One-click setup — enables BLE Log + UART DMA + compression |
| `CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE` | `3000000` | 3 Mbps — balances throughput and reliability |
| `CONFIG_BLE_LOG_LL_ENABLED` | `y` (auto) | Auto-enabled by ESP BLE Controller detection |
> **Note**: Payload checksum (XOR, full scope) and enhanced statistics are always enabled — no Kconfig options needed.
> **Note on UART PORT 0**: When `CONFIG_BLE_LOG_PRPH_UART_DMA_PORT=0`, the firmware automatically wraps `ESP_LOG` output in BLE Log frames (`BLE_LOG_SRC_REDIR`). The console decodes and displays these as regular ASCII log lines. See the [BLE Log module README](../../../components/bt/common/ble_log/README.md#uart-redirect-port-0) for details.
### 3. Hardware Connection
Connect the ESP32 UART TX pin to a USB-to-serial adapter:
```
ESP32 TX GPIO ──────── USB-Serial RX
ESP32 GND ──────── USB-Serial GND
```
Ensure your USB-serial adapter supports the configured baud rate (3 Mbps by default). Adapters based on CP2102N, CH343, or FT232H are recommended.
## Installation
No separate installation is needed. The `textual` and `textual-fspicker` packages are included in ESP-IDF's core requirements (`tools/requirements/requirements.core.txt`) and installed automatically by `./install.sh`.
Verify the dependency is available:
```bash
. ./export.sh
python -c "import textual; print(textual.__version__)"
```
## Usage
### Interactive Mode (Launch Screen)
Run `python console.py` with no arguments to open the **Launch Screen** — an interactive TUI where you can select the serial port, baud rate, and log directory before starting capture:
```bash
cd <esp-idf-root>
. ./export.sh
cd tools/bt/ble_log_console
python console.py
```
The Launch Screen lets you browse available ports and configure options without memorising CLI flags.
### Capture Mode (CLI)
Pass `--port` directly to skip the Launch Screen and start capture immediately:
```bash
cd <esp-idf-root>
. ./export.sh
cd tools/bt/ble_log_console
# Basic usage (--port is now optional; omit to use Launch Screen)
python console.py --port /dev/ttyUSB0
# With custom baud rate
python console.py --port /dev/ttyUSB0 --baudrate 2000000
# With custom log directory
python console.py --port /dev/ttyUSB0 --log-dir /tmp/my_captures
# With custom output file (deprecated — prefer --log-dir)
python console.py --port /dev/ttyUSB0 --output /tmp/my_capture.bin
# Short form
python console.py -p /dev/ttyUSB0 -b 3000000 -d /tmp/my_captures
```
#### Options
| Option | Short | Default | Description |
|--------|-------|---------|-------------|
| `--port` | `-p` | (optional) | UART port device path (e.g., `/dev/ttyUSB0`, `COM3`). Omit to use Launch Screen. |
| `--baudrate` | `-b` | `3000000` | Baud rate — must match `CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE` |
| `--log-dir` | `-d` | current working directory | Directory where capture `.bin` files are saved |
| `--output` | `-o` | auto-generated | *(Deprecated)* Explicit output file path — use `--log-dir` instead |
When `--log-dir` is not specified, capture files are saved to the **current working directory** with a timestamp-based filename:
```
<cwd>/ble_log_YYYYMMDD_HHMMSS.bin
```
### List Saved Captures (`ls`)
List all previously saved `.bin` capture files, sorted by most recent first:
```bash
python console.py ls
```
Example output:
```
Captures in /tmp/ble_log_console:
2026-03-17 14:30:25 2.3 MB ble_log_20260317_143025.bin
2026-03-17 10:15:03 512.0 KB ble_log_20260317_101503.bin
2026-03-16 18:42:11 1.1 MB ble_log_20260316_184211.bin
```
These `.bin` files contain raw binary data exactly as received from UART, suitable for [offline analysis](#offline-analysis).
## Firmware Configuration
### Checksum Mode Detection
The console automatically detects the firmware's checksum mode by probing all 4 combinations during the SEARCHING phase:
| Firmware Config | Console Detects |
|-----------------|-----------------|
| XOR checksum + Full scope | `XOR / Header+Payload` |
| XOR checksum + Header-only scope | `XOR / Header` |
| Sum checksum + Full scope | `Sum / Header+Payload` |
| Sum checksum + Header-only scope | `Sum / Header` |
The detected mode is logged in the log view after sync is achieved (3 consecutive valid frames).
### Enhanced Statistics (ENH_STAT)
The firmware periodically emits `INTERNAL` frames containing per-source write/loss counters (enhanced statistics is always enabled). The console decodes these and uses them as the authoritative source of frame and byte loss. Loss counters are baselined on the first ENH_STAT received per source, so the console only shows loss since it started.
When new frame loss is detected in an ENH_STAT report, a `[WARN]` notification is displayed in the log view with the source name and incremental loss count. Press `d` at any time to view a per-source breakdown of written and lost frames/bytes.
## How It Works
### Sync State Machine
```
frame valid
┌──────────┐ ──────────────▶ ┌────────────────┐
│ SEARCHING │ │ CONFIRMING_SYNC │ ──(N frames)──▶ SYNCED
└──────────┘ ◀────────────── └────────────────┘
frame invalid
frame invalid
┌────────┐ ──────────────▶ ┌─────────────────┐
│ SYNCED │ │ CONFIRMING_LOSS │ ──(M+1 fails)──▶ SEARCHING
└────────┘ ◀────────────── └─────────────────┘
frame valid
```
- **N** = 3 (sync confirmation threshold)
- **M** = 3 (loss tolerance — consecutive failures before resync)
### Frame Format
The console parses the standard BLE Log frame format:
```
[payload_len: 2B LE][frame_meta: 4B LE][payload: variable][checksum: 4B LE]
└── Header (6B) ──┘ └── Tail (4B) ──┘
```
- `frame_meta` = `source_code[7:0] | frame_sn[31:8]`
- For most sources, payload starts with 4-byte `os_ts` (OS timestamp in ms)
- For `REDIR` source (code 8), payload is raw ASCII (no `os_ts` prefix)
### REDIR Frame Decoding
When the firmware uses UART PORT 0, `ESP_LOG` output is wrapped in frames with source code `REDIR` (8). The console:
1. Extracts the raw ASCII payload from each REDIR frame
2. Buffers partial lines across frames (a single log line may span multiple frames due to batch sealing)
3. Emits complete lines to the log view on each `\n` boundary
## Offline Analysis
### Raw Binary Capture
Every byte received from UART is saved to the output `.bin` file **before** parsing. This ensures the capture is complete and unmodified, regardless of parser state or sync loss.
Use `python console.py ls` to find saved captures.
### Parsing with BLE Log Analyzer
The saved `.bin` files can be parsed offline using the **BLE Log Analyzer**'s `ble_log_parser_v2` module for detailed analysis:
- Frame-by-frame decoding with source filtering
- HCI log extraction and conversion to btsnoop format (for Wireshark)
- Timestamp reconstruction and event correlation
- Link Layer log decoding
The binary format is identical whether captured by BLE Log Console, a logic analyzer, or any other tool — the parser reads the same frame structure documented above.
> **Tip**: The `bt_hci_to_btsnoop` tool at `tools/bt/bt_hci_to_btsnoop/` can convert extracted HCI logs to btsnoop format for analysis in Wireshark.
## Keyboard Shortcuts
All shortcuts are **case-insensitive** (e.g., `Q` and `q` both quit).
| Key | Action |
|-----|--------|
| `q` | Quit the application |
| `Ctrl+C` | Quit the application |
| `c` | Clear the log view |
| `s` | Toggle auto-scroll (on by default) |
| `d` | Show per-source frame statistics (press `Escape` to return) |
| `h` | Show keyboard shortcuts (press `Escape` to return) |
| `r` | Reset chip via DTR/RTS toggle |
## Building Executable
To distribute BLE Log Console as a standalone single-file executable (no Python installation required on the target machine), use the provided `build_exe.py` script with [PyInstaller](https://pyinstaller.org/):
```bash
pip install pyinstaller
cd tools/bt/ble_log_console
python build_exe.py
```
The executable is written to `dist/ble_log_console` (Linux/macOS) or `dist\ble_log_console.exe` (Windows). Copy it to any machine and run it directly — no ESP-IDF environment needed.
> **Note**: Build the executable on the same OS/architecture as the target machine. PyInstaller does not cross-compile.
## Architecture
```
console.py (Click CLI)
│
▼
BLELogApp (Textual App)
│
├── Backend Worker (thread)
│ │
│ ├── UART Transport ── open_serial() ── raw .bin file
│ │
│ ├── FrameParser ── sync state machine + checksum auto-detection
│ │ │
│ │ └── ParsedFrame { source_code, frame_sn, payload, os_ts_ms }
│ │
│ ├── InternalDecoder ── decode INIT_DONE, INFO, ENH_STAT, FLUSH
│ │
│ └── StatsAccumulator ── RX bytes, BPS, FPS, firmware-reported loss
│
└── Frontend (Textual widgets)
├── LogView ── scrollable RichLog with styled output
├── StatusPanel ── fixed bottom bar with live stats
├── StatsScreen ── modal overlay for per-source statistics (d key)
└── ShortcutScreen ── modal overlay for keyboard shortcuts (h key)
```
### Source Layout
```
src/
__init__.py # Python 3.10 guard + textual dependency check
app.py # Textual App — wires backend worker to frontend
backend/
models.py # Enums, dataclasses, Textual Message types
checksum.py # XOR and Sum checksum (matches firmware impl)
frame_parser.py # Sync state machine with checksum auto-detection
internal_decoder.py # INTERNAL frame payload decoder
stats/ # Statistics sub-package
__init__.py # Re-exports StatsAccumulator
accumulator.py # Thin composition of sub-modules
transport.py # RX bytes, BPS, FPS tracking
firmware_loss.py # ENH_STAT loss delta tracking
firmware_written.py # ENH_STAT write tracking
sn_gap.py # SN gap detection
peak_burst.py # 1ms window peak write burst
traffic_spike.py # Wire saturation detection
uart_transport.py # Serial port helpers, file I/O
frontend/
log_view.py # RichLog wrapper with color-coded write methods
shortcut_screen.py # Modal screen for keyboard shortcuts
stats_screen.py # Modal screen for per-source statistics
status_panel.py # Reactive status bar (sync, speed, help hint)
tests/
helpers.py # Synthetic frame builder for tests
test_checksum.py # Checksum algorithm tests
test_frame_parser.py # State machine + auto-detection tests
test_internal_decoder.py # Internal frame decoding tests
test_stats.py # Stats accumulator and firmware loss tests
```
## Development
### Running Tests
```bash
cd <esp-idf-root>
. ./export.sh
cd tools/bt/ble_log_console
python -m pytest tests/ -v
```
### Linting & Formatting
```bash
python -m ruff format src/ tests/
python -m ruff check --fix src/ tests/
```
### Type Checking
```bash
python -m mypy src/backend/
```
## Troubleshooting
### "UART port not found"
- Check the device is connected: `ls /dev/ttyUSB*` (Linux) or `ls /dev/tty.usb*` (macOS)
- Ensure you have permission: `sudo usermod -aG dialout $USER` (Linux, then re-login)
- On WSL, USB devices need [usbipd-win](https://github.com/dorssel/usbipd-win) to pass through
### Sync stays in SEARCHING
- **Baud rate mismatch**: Ensure `--baudrate` matches `CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE`
- **Wrong port**: Verify you're connected to the correct UART TX pin
- **Firmware not running**: Check the device has booted and BLE Log is initialized
- **Signal integrity**: At 3 Mbps, use short cables and ensure solid GND connection
### No ESP_LOG output
When using UART PORT 0, `ESP_LOG` is redirected through BLE Log frames. If you don't see log lines:
- Ensure the firmware has `CONFIG_BLE_LOG_PRPH_UART_DMA_PORT=0`
- The console automatically decodes REDIR frames — no extra configuration needed
- Logs are flushed by a 1-second periodic timer, so there may be a short delay
### High frame loss
- Press `d` to view per-source loss counters (enhanced statistics is always enabled)
- Increase buffer sizes: `CONFIG_BLE_LOG_LBM_TRANS_SIZE`, `CONFIG_BLE_LOG_LBM_LL_TRANS_SIZE`
- Add more LBMs: `CONFIG_BLE_LOG_LBM_ATOMIC_LOCK_TASK_CNT`
- Increase baud rate if your adapter supports higher speeds
### Import errors
```
ModuleNotFoundError: No module named 'textual'
```
Re-run the ESP-IDF installer:
```bash
cd <esp-idf-root>
./install.sh
. ./export.sh
```
tools/bt/ble_log_console/build.bat
+57
View File
@@ -0,0 +1,57 @@
@echo off
rem SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD
rem SPDX-License-Identifier: Apache-2.0
rem Build a single-file BLE Log Console executable via PyInstaller.
rem The executable is placed in the caller's working directory.
rem All intermediate build artifacts are cleaned up automatically.
setlocal
set "SCRIPT_DIR=%~dp0"
set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%"
set "CALLER_DIR=%cd%"
rem Derive IDF_PATH (three levels up from script directory)
for %%I in ("%SCRIPT_DIR%\..\..\..") do set "IDF_PATH=%%~fI"
echo Activating ESP-IDF environment ...
call "%IDF_PATH%\export.bat" > nul 2>&1
if %errorlevel% neq 0 (
echo ERROR: Failed to activate ESP-IDF environment.
exit /b 1
)
echo Installing build dependencies ...
python -m pip install --quiet textual textual-fspicker pyinstaller
if %errorlevel% neq 0 (
echo ERROR: Failed to install dependencies.
exit /b 1
)
echo Building executable ...
cd /d "%SCRIPT_DIR%"
python build_exe.py
if %errorlevel% neq 0 (
echo ERROR: Build failed.
cd /d "%CALLER_DIR%"
exit /b 1
)
set "EXE_NAME=ble_log_console.exe"
if exist "dist\%EXE_NAME%" (
move /y "dist\%EXE_NAME%" "%CALLER_DIR%\%EXE_NAME%" > nul
echo.
echo Executable ready: %CALLER_DIR%\%EXE_NAME%
) else (
echo ERROR: Build produced no executable.
cd /d "%CALLER_DIR%"
exit /b 1
)
rem Remove intermediate artifacts
if exist "%SCRIPT_DIR%\build" rd /s /q "%SCRIPT_DIR%\build"
if exist "%SCRIPT_DIR%\dist" rd /s /q "%SCRIPT_DIR%\dist"
del /q "%SCRIPT_DIR%\*.spec" 2>nul
cd /d "%CALLER_DIR%"
tools/bt/ble_log_console/build.sh
Executable
+41
View File
@@ -0,0 +1,41 @@
#!/usr/bin/env bash
# SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Apache-2.0
# Build a single-file BLE Log Console executable via PyInstaller.
# The executable is placed in the caller's working directory.
# All intermediate build artifacts are cleaned up automatically.
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
IDF_PATH="$(cd "$SCRIPT_DIR/../../.." && pwd)"
CALLER_DIR="$(pwd)"
export IDF_PATH
echo "Activating ESP-IDF environment ..."
# shellcheck source=/dev/null
. "$IDF_PATH/export.sh" > /dev/null 2>&1
echo "Installing build dependencies ..."
python -m pip install --quiet textual textual-fspicker pyinstaller
echo "Building executable ..."
cd "$SCRIPT_DIR"
python build_exe.py
# Move executable to caller's directory and clean up
EXE_NAME="ble_log_console"
if [ -f "dist/$EXE_NAME" ]; then
mv "dist/$EXE_NAME" "$CALLER_DIR/$EXE_NAME"
echo ""
echo "Executable ready: $CALLER_DIR/$EXE_NAME"
else
echo "ERROR: Build produced no executable." >&2
exit 1
fi
# Remove intermediate artifacts
rm -rf "$SCRIPT_DIR/build" "$SCRIPT_DIR/dist" "$SCRIPT_DIR"/*.spec
cd "$CALLER_DIR"
tools/bt/ble_log_console/build_exe.py
+62
View File
@@ -0,0 +1,62 @@
# SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Apache-2.0
"""Build a single-file executable for BLE Log Console using PyInstaller.
Usage:
pip install pyinstaller
python build_exe.py
"""
import subprocess
import sys
def main() -> None:
cmd = [
sys.executable,
'-m',
'PyInstaller',
'console.py',
'--onefile',
'--name',
'ble_log_console',
'--hidden-import',
'textual',
'--hidden-import',
'textual.drivers',
'--hidden-import',
'textual.css',
'--hidden-import',
'textual_fspicker',
'--hidden-import',
'serial',
'--hidden-import',
'serial.tools',
'--hidden-import',
'serial.tools.list_ports',
'--hidden-import',
'serial.tools.list_ports_common',
'--hidden-import',
'serial.tools.list_ports_linux',
'--hidden-import',
'serial.tools.list_ports_windows',
'--hidden-import',
'serial.tools.list_ports_osx',
'--collect-data',
'textual',
'--collect-data',
'textual_fspicker',
'--noconfirm',
]
print(f'Running: {" ".join(cmd)}')
result = subprocess.run(cmd, check=False)
if result.returncode != 0:
print(f'\nBuild failed (exit code {result.returncode}).', file=sys.stderr)
print('If you see hidden import errors, add the missing module to the cmd list above.', file=sys.stderr)
sys.exit(result.returncode)
print('\nBuild complete. Executable: dist/ble_log_console')
if __name__ == '__main__':
main()
tools/bt/ble_log_console/console.py
+93
View File
@@ -0,0 +1,93 @@
# SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Apache-2.0
"""BLE Log Console entry point.
Usage:
python console.py # interactive setup
python console.py --port /dev/ttyUSB0 # direct connect
python console.py ls # list saved files
"""
from datetime import datetime
from pathlib import Path
from typing import Optional
import click
from src.app import BLELogApp
from src.backend.models import format_bytes
from src.backend.uart_transport import validate_uart_port
@click.group(invoke_without_command=True)
@click.option('--port', '-p', default=None, help='UART port. If omitted, shows interactive setup.')
@click.option('--baudrate', '-b', type=int, default=3_000_000, show_default=True, help='Baud rate')
@click.option(
'--log-dir', '-d', type=click.Path(), default=None, help='Log save directory. Default: current working directory.'
)
@click.option(
'--output',
'-o',
type=click.Path(),
default=None,
hidden=True,
help='[Deprecated] Output binary file path. Use --log-dir instead.',
)
@click.pass_context
def cli(ctx: click.Context, port: Optional[str], baudrate: int, log_dir: Optional[str], output: Optional[str]) -> None:
"""BLE Log Console — real-time BLE log monitor."""
if ctx.invoked_subcommand is not None:
return
# Resolve log directory
resolved_log_dir: Optional[Path] = None
if output is not None:
# Legacy --output: treat as full file path, use its parent as log_dir
click.echo(
'Warning: --output is deprecated and the filename is ignored. '
'Use --log-dir instead. Saving to directory: ' + str(Path(output).parent),
err=True,
)
resolved_log_dir = Path(output).parent
elif log_dir is not None:
resolved_log_dir = Path(log_dir)
if port is not None:
error = validate_uart_port(port)
if error:
raise click.BadParameter(error, param_hint="'--port'")
app = BLELogApp(
port=port,
baudrate=baudrate,
log_dir=resolved_log_dir,
)
app.run()
@cli.command(name='ls')
@click.option(
'--dir',
'-d',
'log_dir',
type=click.Path(exists=True),
default=None,
help='Directory to list. Default: current directory.',
)
def list_files(log_dir: Optional[str]) -> None:
"""List saved binary capture files."""
search_dir = Path(log_dir) if log_dir else Path.cwd()
files = sorted(search_dir.glob('ble_log_*.bin'), key=lambda f: f.stat().st_mtime, reverse=True)
if not files:
click.echo(f'No captures found in {search_dir}')
return
click.echo(f'Captures in {search_dir}:\n')
for f in files:
size = f.stat().st_size
mtime = datetime.fromtimestamp(f.stat().st_mtime).strftime('%Y-%m-%d %H:%M:%S')
size_str = format_bytes(size)
click.echo(f' {mtime} {size_str:>10} {f.name}')
if __name__ == '__main__':
cli()
tools/bt/ble_log_console/docs/User-Guide-CN.md
+221
View File
@@ -0,0 +1,221 @@
# BLE Log Console 用户指南
## 简介
BLE Log Console 是一个基于终端的实时 BLE 日志捕获与解析工具。它通过 UART DMA 接收 ESP32 固件发出的 BLE Log 帧，实时解析并展示在终端界面中，同时将原始二进制数据保存到文件供离线分析。
## 准备工作
### 1. 固件配置
在 `idf.py menuconfig` 中启用 BLE Log 模块：
**最简配置（推荐）：**
```
Component config → Bluetooth → BT Logs → Enable critical-log-only mode [y]
```
勾选即可一键启用 BLE Log 模块 + UART DMA 输出 + 仅关键日志模式。
**手动配置：**
```
Component config → Bluetooth → BT Logs → Enable BLE Log Module (Experimental) [y]
Component config → Bluetooth → BT Logs → BLE Log Module
→ Peripheral Selection → UART DMA
→ UART DMA Configuration
→ UART Port Number (默认: 0)
→ Baud Rate (默认: 3000000)
→ TX GPIO Number (根据硬件设置)
```
| 配置项 | 推荐值 | 说明 |
|--------|--------|------|
| `CONFIG_BT_LOG_CRITICAL_ONLY` | `y` | 一键启用，包含 BLE Log + UART DMA |
| `CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE` | `3000000` | 3 Mbps，兼顾吞吐量和稳定性 |
> **关于 UART PORT 0**：当配置为 PORT 0 时，固件会自动将 `ESP_LOG` 输出包装为 BLE Log 帧（`REDIR` source），Console 会自动解码并显示为普通日志行。
### 2. 硬件连接
**PORT 0（推荐）：** 直接通过 USB 线连接开发板的 UART 口即可，无需额外接线。
**其他 PORT：** 需要将指定的 TX GPIO 连接到外部 USB 串口适配器：
```
ESP32 TX GPIO ──────── USB 串口适配器 RX
ESP32 GND ──────── USB 串口适配器 GND
```
确保 USB 串口适配器支持所配置的波特率（默认 3 Mbps）。推荐使用 CP2102N、CH343 或 FT232H 芯片的适配器。
### 3. ESP-IDF 环境
使用前必须先 source ESP-IDF 环境：
```bash
cd <esp-idf 根目录>
. ./export.sh
```
无需额外安装依赖，`textual` 已包含在 ESP-IDF 核心依赖中。
## 启动
```bash
cd <esp-idf 根目录>
. ./export.sh
cd tools/bt/ble_log_console
```
### 交互模式（启动界面）
不带参数运行即可打开启动界面（Launch Screen），在 TUI 中选择串口、波特率和日志保存目录：
```bash
python console.py
```
### 直连模式（CLI）
传入 `--port` 跳过启动界面，直接开始捕获：
```bash
# 基本用法
python console.py -p /dev/ttyUSB0
# 指定波特率（须与固件配置一致）
python console.py -p /dev/ttyUSB0 -b 3000000
# 指定日志保存目录
python console.py -p /dev/ttyUSB0 -d /tmp/my_captures
```
| 参数 | 缩写 | 默认值 | 说明 |
|------|------|--------|------|
| `--port` | `-p` | （可选） | 串口设备路径，如 `/dev/ttyUSB0`、`COM3`。省略时打开启动界面。 |
| `--baudrate` | `-b` | `3000000` | 波特率，须与固件 `CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE` 一致 |
| `--log-dir` | `-d` | 当前工作目录 | 捕获文件保存目录 |
捕获文件保存到当前工作目录（或 `--log-dir` 指定的目录），文件名按时间戳自动生成：
```
ble_log_YYYYMMDD_HHMMSS.bin
```
## 界面说明
启动后，界面分为两个区域：
### 日志区域（上方）
滚动显示实时日志，包括：
- **`[INFO]`**（绿色）：系统信息，如固件版本
- **`[WARN]`**（黄色）：丢帧警告，格式为 `Frame loss [LL_TASK]: 5 frames, 200 bytes`，表示该 source 新增了丢帧
- **`[SYNC]`**（青色）：同步状态变化
- 普通文本：UART redirect 输出的 `ESP_LOG` 日志（仅 PORT 0 时出现）
### 状态栏（下方）
固定显示在底部，实时更新：
```
Sync: SYNCED Checksum: XOR | Header+Payload
RX: 1.2 MB Speed: 2.8 Mbps Max: 2.9 Mbps Rate: 3421 fps
Frame Loss: 12 frames, 480 bytes
```
- **Sync**: 同步状态（SEARCHING → CONFIRMING → SYNCED → CONFIRMING_LOSS）
- **Checksum**: 自动检测到的校验模式
- **RX**: 累计接收字节数
- **Speed / Max**: 当前/峰值传输速度
- **Rate**: 当前帧率
- **Frame Loss**: 自 Console 启动以来的累计丢帧数和丢失字节数
## 快捷键
| 按键 | 功能 |
|------|------|
| `q` | 退出 |
| `Ctrl+C` | 退出 |
| `c` | 清屏（清除日志区域） |
| `s` | 切换自动滚动（默认开启） |
| `d` | 打开每个 Source 的帧统计详情（按 `Escape` 或 `d` 关闭） |
| `h` | 显示快捷键帮助（按 `Escape` 关闭） |
| `r` | 通过 DTR/RTS 复位芯片 |
### 帧统计详情（`d` 键）
按 `d` 键会弹出一个覆盖层，以表格形式展示每个 BLE Log Source 自 Console 启动以来的写入和丢失统计：
```
┌─────────── Per-Source Frame Statistics (since console start) ───────────┐
│ Source │ Written Frames │ Written Bytes │ Lost Frames │ Lost Bytes │
│ LL_TASK │ 12345 │ 56.7 KB │ 5 │ 200 B │
│ LL_HCI │ 890 │ 34.2 KB │ 0 │ 0 B │
│ HOST │ 456 │ 12.1 KB │ 0 │ 0 B │
│ HCI │ 234 │ 8.5 KB │ 2 │ 80 B │
└─────────────────────────────────────────────────────────────────────────┘
```
有丢帧的行会以红色高亮显示。
## 查看历史捕获文件
```bash
python console.py ls
```
输出示例：
```
Captures in /tmp/ble_log_console:
2026-03-17 14:30:25 2.3 MB ble_log_20260317_143025.bin
2026-03-17 10:15:03 512.0 KB ble_log_20260317_101503.bin
```
这些 `.bin` 文件包含 UART 接收到的原始二进制数据，可使用 BLE Log Analyzer 的 `ble_log_parser_v2` 模块进行离线解析（HCI 日志提取、btsnoop 转换等）。
## 常见问题
### 状态一直停在 SEARCHING
- **波特率不匹配**：确认 `--baudrate` 与固件 `CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE` 一致
- **串口错误**：确认连接的是正确的 UART TX 引脚
- **固件未运行**：确认设备已启动且 BLE Log 已初始化
- **信号质量**：3 Mbps 下请使用短线缆，确保 GND 连接可靠
### 出现 Buffer overflow warning
表示解析器内部缓冲区累积超过 8 KB 未能解析出有效帧。通常发生在：
- 设备启动初期，UART 上还没有有效的 BLE Log 帧数据
- 波特率不匹配导致接收到的全是乱码
如果只在启动时出现一次，属于正常现象；如果持续出现，请检查波特率和硬件连接。
### 丢帧严重
- 按 `d` 查看各 Source 的丢帧详情
- 增大固件 buffer：`CONFIG_BLE_LOG_LBM_TRANS_SIZE`、`CONFIG_BLE_LOG_LBM_LL_TRANS_SIZE`
- 增加 LBM 数量：`CONFIG_BLE_LOG_LBM_ATOMIC_LOCK_TASK_CNT`
- 提高波特率（需适配器支持）
### 看不到 ESP_LOG 输出
- 确认固件配置了 `CONFIG_BLE_LOG_PRPH_UART_DMA_PORT=0`
- Console 会自动解码 REDIR 帧，无需额外配置
- 日志由 1 秒周期定时器刷新，可能有短暂延迟
### 提示 `ModuleNotFoundError: No module named 'textual'`
重新运行 ESP-IDF 安装脚本：
```bash
cd <esp-idf 根目录>
./install.sh
. ./export.sh
```
tools/bt/ble_log_console/docs/User-Guide-EN.md
+221
View File
@@ -0,0 +1,221 @@
# BLE Log Console User Guide
## Introduction
BLE Log Console is a terminal-based tool for real-time capture and parsing of BLE Log frames from ESP32 firmware via UART DMA. It displays parsed frames in an interactive TUI and saves the raw binary data to a file for offline analysis.
## Prerequisites
### 1. Firmware Configuration
Enable the BLE Log module in `idf.py menuconfig`:
**Quick setup (recommended):**
```
Component config → Bluetooth → BT Logs → Enable critical-log-only mode [y]
```
This enables the BLE Log module, selects UART DMA as the transport, and restricts each stack to critical logs only — all in one toggle.
**Manual configuration:**
```
Component config → Bluetooth → BT Logs → Enable BLE Log Module (Experimental) [y]
Component config → Bluetooth → BT Logs → BLE Log Module
→ Peripheral Selection → UART DMA
→ UART DMA Configuration
→ UART Port Number (default: 0)
→ Baud Rate (default: 3000000)
→ TX GPIO Number (set to match your hardware)
```
| Config Option | Recommended | Description |
|---------------|-------------|-------------|
| `CONFIG_BT_LOG_CRITICAL_ONLY` | `y` | One-click setup — enables BLE Log + UART DMA |
| `CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE` | `3000000` | 3 Mbps — balances throughput and reliability |
> **About UART PORT 0**: When configured for PORT 0, the firmware automatically wraps `ESP_LOG` output in BLE Log frames (`REDIR` source). The console decodes and displays these as regular log lines.
### 2. Hardware Connection
**PORT 0 (recommended):** Simply connect the development board's UART port via USB — no additional wiring needed.
**Other PORTs:** Connect the designated TX GPIO to an external USB-to-serial adapter:
```
ESP32 TX GPIO ──────── USB-Serial RX
ESP32 GND ──────── USB-Serial GND
```
Ensure your USB-serial adapter supports the configured baud rate (3 Mbps by default). Adapters based on CP2102N, CH343, or FT232H are recommended.
### 3. ESP-IDF Environment
Source the ESP-IDF environment before use:
```bash
cd <esp-idf-root>
. ./export.sh
```
No additional installation is needed — `textual` is included in ESP-IDF's core dependencies.
## Getting Started
```bash
cd <esp-idf-root>
. ./export.sh
cd tools/bt/ble_log_console
```
### Interactive Mode (Launch Screen)
Run with no arguments to open the Launch Screen — a TUI where you can select the serial port, baud rate, and log directory before starting capture:
```bash
python console.py
```
### Capture Mode (CLI)
Pass `--port` to skip the Launch Screen and start capture immediately:
```bash
# Basic usage
python console.py -p /dev/ttyUSB0
# With custom baud rate (must match firmware config)
python console.py -p /dev/ttyUSB0 -b 3000000
# With custom log directory
python console.py -p /dev/ttyUSB0 -d /tmp/my_captures
```
| Option | Short | Default | Description |
|--------|-------|---------|-------------|
| `--port` | `-p` | (optional) | Serial port path, e.g., `/dev/ttyUSB0`, `COM3`. Omit to use Launch Screen. |
| `--baudrate` | `-b` | `3000000` | Baud rate — must match `CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE` |
| `--log-dir` | `-d` | current working directory | Directory where capture `.bin` files are saved |
Capture files are saved to the current working directory (or `--log-dir` if specified) with a timestamp-based filename:
```
ble_log_YYYYMMDD_HHMMSS.bin
```
## UI Overview
The interface has two areas:
### Log View (upper area)
A scrollable area displaying real-time logs:
- **`[INFO]`** (green): System information, e.g., firmware version
- **`[WARN]`** (yellow): Frame loss warnings, e.g., `Frame loss [LL_TASK]: 5 frames, 200 bytes`, indicating new frame loss on that source
- **`[SYNC]`** (cyan): Sync state transitions
- Plain text: `ESP_LOG` output via UART redirect (PORT 0 only)
### Status Panel (bottom bar)
Fixed at the bottom, updated in real time:
```
Sync: SYNCED Checksum: XOR | Header+Payload
RX: 1.2 MB Speed: 2.8 Mbps Max: 2.9 Mbps Rate: 3421 fps
Frame Loss: 12 frames, 480 bytes
```
- **Sync**: Sync state (SEARCHING → CONFIRMING → SYNCED → CONFIRMING_LOSS)
- **Checksum**: Auto-detected checksum mode
- **RX**: Total received bytes
- **Speed / Max**: Current / peak transport speed
- **Rate**: Current frame rate
- **Frame Loss**: Cumulative lost frames and bytes since console start
## Keyboard Shortcuts
| Key | Action |
|-----|--------|
| `q` | Quit |
| `Ctrl+C` | Quit |
| `c` | Clear the log view |
| `s` | Toggle auto-scroll (on by default) |
| `d` | Open per-source frame statistics (press `Escape` or `d` to close) |
| `h` | Show keyboard shortcuts (press `Escape` to close) |
| `r` | Reset chip via DTR/RTS toggle |
### Frame Statistics Detail (`d` key)
Pressing `d` opens a modal overlay showing per-source write and loss statistics since the console started:
```
┌─────────── Per-Source Frame Statistics (since console start) ───────────┐
│ Source │ Written Frames │ Written Bytes │ Lost Frames │ Lost Bytes │
│ LL_TASK │ 12345 │ 56.7 KB │ 5 │ 200 B │
│ LL_HCI │ 890 │ 34.2 KB │ 0 │ 0 B │
│ HOST │ 456 │ 12.1 KB │ 0 │ 0 B │
│ HCI │ 234 │ 8.5 KB │ 2 │ 80 B │
└─────────────────────────────────────────────────────────────────────────┘
```
Rows with frame loss are highlighted in red.
## Viewing Saved Captures
```bash
python console.py ls
```
Example output:
```
Captures in /tmp/ble_log_console:
2026-03-17 14:30:25 2.3 MB ble_log_20260317_143025.bin
2026-03-17 10:15:03 512.0 KB ble_log_20260317_101503.bin
```
These `.bin` files contain raw binary data as received from UART. They can be parsed offline using the BLE Log Analyzer's `ble_log_parser_v2` module for HCI log extraction, btsnoop conversion, and more.
## Troubleshooting
### Sync stays in SEARCHING
- **Baud rate mismatch**: Ensure `--baudrate` matches `CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE`
- **Wrong port**: Verify you are connected to the correct UART TX pin
- **Firmware not running**: Confirm the device has booted and BLE Log is initialized
- **Signal quality**: At 3 Mbps, use short cables and ensure a solid GND connection
### Buffer overflow warning
This means the parser's internal buffer exceeded 8 KB without successfully parsing any frames. Common causes:
- Device is still booting and no valid BLE Log frames have been sent yet
- Baud rate mismatch causing all received data to be garbage
If it only appears once at startup, this is normal. If persistent, check the baud rate and hardware connection.
### High frame loss
- Press `d` to view per-source loss details
- Increase firmware buffers: `CONFIG_BLE_LOG_LBM_TRANS_SIZE`, `CONFIG_BLE_LOG_LBM_LL_TRANS_SIZE`
- Add more LBMs: `CONFIG_BLE_LOG_LBM_ATOMIC_LOCK_TASK_CNT`
- Increase baud rate (if your adapter supports it)
### No ESP_LOG output
- Confirm the firmware has `CONFIG_BLE_LOG_PRPH_UART_DMA_PORT=0`
- The console decodes REDIR frames automatically — no extra configuration needed
- Logs are flushed by a 1-second periodic timer, so there may be a short delay
### `ModuleNotFoundError: No module named 'textual'`
Re-run the ESP-IDF installer:
```bash
cd <esp-idf-root>
./install.sh
. ./export.sh
```
tools/bt/ble_log_console/run.bat
+31
View File
@@ -0,0 +1,31 @@
@echo off
rem SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD
rem SPDX-License-Identifier: Apache-2.0
rem BLE Log Console launcher for Windows CMD.
rem Works from any directory. All arguments are forwarded to console.py.
setlocal
set "SCRIPT_DIR=%~dp0"
set "SCRIPT_DIR=%SCRIPT_DIR:~0,-1%"
rem Derive IDF_PATH (three levels up from script directory)
for %%I in ("%SCRIPT_DIR%\..\..\..") do set "IDF_PATH=%%~fI"
echo Activating ESP-IDF environment ...
call "%IDF_PATH%\export.bat" > nul 2>&1
if %errorlevel% neq 0 (
echo ERROR: Failed to activate ESP-IDF environment.
exit /b 1
)
echo Installing extra dependencies ...
python -m pip install --quiet textual textual-fspicker
if %errorlevel% neq 0 (
echo ERROR: Failed to install dependencies.
exit /b 1
)
python "%SCRIPT_DIR%\console.py" %*
exit /b %errorlevel%
tools/bt/ble_log_console/run.sh
Executable
+22
View File
@@ -0,0 +1,22 @@
#!/usr/bin/env bash
# SPDX-FileCopyrightText: 2026 Espressif Systems (Shanghai) CO LTD
# SPDX-License-Identifier: Apache-2.0
# BLE Log Console launcher.
# Works from any directory: ./run.sh, or /full/path/to/run.sh
# All arguments are forwarded to console.py.
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
IDF_PATH="$(cd "$SCRIPT_DIR/../../.." && pwd)"
export IDF_PATH
echo "Activating ESP-IDF environment ..."
# shellcheck source=/dev/null
. "$IDF_PATH/export.sh" > /dev/null 2>&1
echo "Installing extra dependencies ..."
python -m pip install --quiet textual textual-fspicker
exec python "$SCRIPT_DIR/console.py" "$@"
tools/ci/exclude_check_tools_files.txt
+1 -2
View File
@@ -40,6 +40,5 @@ tools/ci/cleanup_ignore_lists.py
tools/ci/artifacts_handler.py
tools/ci/get_known_failure_cases_file.py
tools/unit-test-app/**/*
tools/bt/bt_hci_to_btsnoop.py
tools/bt/README.md
tools/ci/dynamic_pipelines/templates/known_generate_test_child_pipeline_warnings.yml
tools/bt/**/*
tools/ci/executable-list.txt
+2
View File
@@ -46,6 +46,8 @@ examples/system/ota/otatool/otatool_example.py
examples/system/ota/otatool/otatool_example.sh
install.fish
install.sh
tools/bt/ble_log_console/build.sh
tools/bt/ble_log_console/run.sh
tools/check_python_dependencies.py
tools/ci/build_template_app.sh
tools/ci/check_api_violation.sh