esp-idf/tools/bt/ble_log_console

BLE Log Console

A Textual-based TUI tool for real-time capture, parsing, and display of BLE Log frames from UART DMA output. Designed for both Espressif internal developers and ESP-IDF customers.

User Guide: English | 中文

Table of Contents

• Features
• Prerequisites
• Installation
• Usage
• Firmware Configuration
• How It Works
• Offline Analysis
• Keyboard Shortcuts
• Building Executable
• Architecture
• Development
• Troubleshooting

Features

• Real-time frame parsing with automatic checksum mode detection (XOR/Sum × Full/Header-only)
• Frame sync state machine with loss detection and recovery (SEARCHING → CONFIRMING → SYNCED → CONFIRMING_LOSS)
• Internal frame decoding: INIT_DONE (firmware version), INFO, ENH_STAT (per-source write/loss counters), FLUSH
• UART redirect display: When firmware uses UART PORT 0, redirected ESP_LOG output is decoded from REDIR frames and displayed as ASCII log lines
• Dimmed internal logs: Console-generated messages (sync, warnings, errors) are dimmed to visually separate from user application logs
• Live status panel: Sync state, RX bytes, transport speed (current + max), frame rate
• Per-source frame loss warnings: Real-time [WARN] notifications when firmware reports new frame loss, with source name (e.g., LL_TASK)
• Per-source statistics view: Press d to open a modal overlay showing written/lost frame and byte counts per source
• Raw binary capture: All received bytes are saved to a .bin file for offline analysis
• Scrollable log view with auto-scroll toggle

Prerequisites

1. ESP-IDF Environment

BLE Log Console runs within the ESP-IDF Python environment. You must source export.sh before use:

cd <esp-idf-root>
. ./export.sh

This sets up the Python virtual environment at ~/.espressif/python_env/ which includes all required dependencies (textual, pyserial, click, etc.).

2. Firmware Configuration

The target ESP32 device must have the BLE Log module enabled and configured for UART DMA output. Configure via idf.py menuconfig:

Component config → Bluetooth → BT Logs → Enable BLE Log Module (Experimental)  [y]

Then select the transport peripheral and UART settings:

Component config → Bluetooth → BT Logs → BLE Log Module
  → Peripheral Selection → UART DMA
  → UART DMA Configuration
      → UART Port Number        (default: 0)
      → Baud Rate               (default: 3000000)
      → TX GPIO Number          (set to match your hardware)

Quick Setup: Critical-Log-Only Mode

The simplest way to enable BLE Log with UART DMA output:

Component config → Bluetooth → BT Logs → Enable critical-log-only mode  [y]

This automatically enables the BLE Log Module, selects UART DMA as the default peripheral, and restricts each stack (Controller/Host/Mesh) to critical logs only.

Recommended Kconfig Options

Kconfig Option	Recommended	Why
CONFIG_BT_LOG_CRITICAL_ONLY	y	One-click setup — enables BLE Log + UART DMA + compression
CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE	3000000	3 Mbps — balances throughput and reliability
CONFIG_BLE_LOG_LL_ENABLED	y (auto)	Auto-enabled by ESP BLE Controller detection

Note

: Payload checksum (XOR, full scope) and enhanced statistics are always enabled — no Kconfig options needed.

Note on UART PORT 0: When CONFIG_BLE_LOG_PRPH_UART_DMA_PORT=0, the firmware automatically wraps ESP_LOG output in BLE Log frames (BLE_LOG_SRC_REDIR). The console decodes and displays these as regular ASCII log lines. See the BLE Log module README for details.

3. Hardware Connection

Connect the ESP32 UART TX pin to a USB-to-serial adapter:

ESP32 TX GPIO ──────── USB-Serial RX
ESP32 GND     ──────── USB-Serial GND

Ensure your USB-serial adapter supports the configured baud rate (3 Mbps by default). Adapters based on CP2102N, CH343, or FT232H are recommended.

Installation

No separate installation is needed. The textual and textual-fspicker packages are included in ESP-IDF's core requirements (tools/requirements/requirements.core.txt) and installed automatically by ./install.sh.

Verify the dependency is available:

. ./export.sh
python -c "import textual; print(textual.__version__)"

Usage

Interactive Mode (Launch Screen)

Run python console.py with no arguments to open the Launch Screen — an interactive TUI where you can select the serial port, baud rate, and log directory before starting capture:

cd <esp-idf-root>
. ./export.sh
cd tools/bt/ble_log_console

python console.py

The Launch Screen lets you browse available ports and configure options without memorising CLI flags.

Capture Mode (CLI)

Pass --port directly to skip the Launch Screen and start capture immediately:

cd <esp-idf-root>
. ./export.sh
cd tools/bt/ble_log_console

# Basic usage (--port is now optional; omit to use Launch Screen)
python console.py --port /dev/ttyUSB0

# With custom baud rate
python console.py --port /dev/ttyUSB0 --baudrate 2000000

# With custom log directory
python console.py --port /dev/ttyUSB0 --log-dir /tmp/my_captures

# With custom output file (deprecated — prefer --log-dir)
python console.py --port /dev/ttyUSB0 --output /tmp/my_capture.bin

# Short form
python console.py -p /dev/ttyUSB0 -b 3000000 -d /tmp/my_captures

Options

Option	Short	Default	Description
--port	-p	(optional)	UART port device path (e.g., /dev/ttyUSB0, COM3). Omit to use Launch Screen.
--baudrate	-b	3000000	Baud rate — must match CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE
--log-dir	-d	current working directory	Directory where capture .bin files are saved
--output	-o	auto-generated	(Deprecated) Explicit output file path — use --log-dir instead

When --log-dir is not specified, capture files are saved to the current working directory with a timestamp-based filename:

<cwd>/ble_log_YYYYMMDD_HHMMSS.bin

List Saved Captures (ls)

List all previously saved .bin capture files, sorted by most recent first:

python console.py ls

Example output:

Captures in /tmp/ble_log_console:

  2026-03-17 14:30:25     2.3 MB  ble_log_20260317_143025.bin
  2026-03-17 10:15:03   512.0 KB  ble_log_20260317_101503.bin
  2026-03-16 18:42:11     1.1 MB  ble_log_20260316_184211.bin

These .bin files contain raw binary data exactly as received from UART, suitable for offline analysis.

Firmware Configuration

Checksum Mode Detection

The console automatically detects the firmware's checksum mode by probing all 4 combinations during the SEARCHING phase:

Firmware Config	Console Detects
XOR checksum + Full scope	XOR / Header+Payload
XOR checksum + Header-only scope	XOR / Header
Sum checksum + Full scope	Sum / Header+Payload
Sum checksum + Header-only scope	Sum / Header

The detected mode is logged in the log view after sync is achieved (3 consecutive valid frames).

Enhanced Statistics (ENH_STAT)

The firmware periodically emits INTERNAL frames containing per-source write/loss counters (enhanced statistics is always enabled). The console decodes these and uses them as the authoritative source of frame and byte loss. Loss counters are baselined on the first ENH_STAT received per source, so the console only shows loss since it started.

When new frame loss is detected in an ENH_STAT report, a [WARN] notification is displayed in the log view with the source name and incremental loss count. Press d at any time to view a per-source breakdown of written and lost frames/bytes.

How It Works

Sync State Machine

                  frame valid
  ┌──────────┐  ──────────────▶  ┌────────────────┐
  │ SEARCHING │                  │ CONFIRMING_SYNC │ ──(N frames)──▶ SYNCED
  └──────────┘  ◀──────────────  └────────────────┘
                  frame invalid

                  frame invalid
  ┌────────┐  ──────────────▶  ┌─────────────────┐
  │ SYNCED │                   │ CONFIRMING_LOSS  │ ──(M+1 fails)──▶ SEARCHING
  └────────┘  ◀──────────────  └─────────────────┘
                  frame valid

• N = 3 (sync confirmation threshold)
• M = 3 (loss tolerance — consecutive failures before resync)

Frame Format

The console parses the standard BLE Log frame format:

[payload_len: 2B LE][frame_meta: 4B LE][payload: variable][checksum: 4B LE]
 └── Header (6B) ──┘                                       └── Tail (4B) ──┘

• frame_meta = source_code[7:0] | frame_sn[31:8]
• For most sources, payload starts with 4-byte os_ts (OS timestamp in ms)
• For REDIR source (code 8), payload is raw ASCII (no os_ts prefix)

REDIR Frame Decoding

When the firmware uses UART PORT 0, ESP_LOG output is wrapped in frames with source code REDIR (8). The console:

1. Extracts the raw ASCII payload from each REDIR frame
2. Buffers partial lines across frames (a single log line may span multiple frames due to batch sealing)
3. Emits complete lines to the log view on each \n boundary

Offline Analysis

Raw Binary Capture

Every byte received from UART is saved to the output .bin file before parsing. This ensures the capture is complete and unmodified, regardless of parser state or sync loss.

Use python console.py ls to find saved captures.

Parsing with BLE Log Analyzer

The saved .bin files can be parsed offline using the BLE Log Analyzer's ble_log_parser_v2 module for detailed analysis:

• Frame-by-frame decoding with source filtering
• HCI log extraction and conversion to btsnoop format (for Wireshark)
• Timestamp reconstruction and event correlation
• Link Layer log decoding

The binary format is identical whether captured by BLE Log Console, a logic analyzer, or any other tool — the parser reads the same frame structure documented above.

Tip

: The bt_hci_to_btsnoop tool at tools/bt/bt_hci_to_btsnoop/ can convert extracted HCI logs to btsnoop format for analysis in Wireshark.

Keyboard Shortcuts

All shortcuts are case-insensitive (e.g., Q and q both quit).

Key	Action
q	Quit the application
Ctrl+C	Quit the application
c	Clear the log view
s	Toggle auto-scroll (on by default)
d	Show per-source frame statistics (press Escape to return)
h	Show keyboard shortcuts (press Escape to return)
r	Reset chip via DTR/RTS toggle

Building Executable

To distribute BLE Log Console as a standalone single-file executable (no Python installation required on the target machine), use the provided build_exe.py script with PyInstaller:

pip install pyinstaller
cd tools/bt/ble_log_console
python build_exe.py

The executable is written to dist/ble_log_console (Linux/macOS) or dist\ble_log_console.exe (Windows). Copy it to any machine and run it directly — no ESP-IDF environment needed.

Note

: Build the executable on the same OS/architecture as the target machine. PyInstaller does not cross-compile.

Architecture

console.py (Click CLI)
    │
    ▼
BLELogApp (Textual App)
    │
    ├── Backend Worker (thread)
    │     │
    │     ├── UART Transport ── open_serial() ── raw .bin file
    │     │
    │     ├── FrameParser ── sync state machine + checksum auto-detection
    │     │     │
    │     │     └── ParsedFrame { source_code, frame_sn, payload, os_ts_ms }
    │     │
    │     ├── InternalDecoder ── decode INIT_DONE, INFO, ENH_STAT, FLUSH
    │     │
    │     └── StatsAccumulator ── RX bytes, BPS, FPS, firmware-reported loss
    │
    └── Frontend (Textual widgets)
          ├── LogView ── scrollable RichLog with styled output
          ├── StatusPanel ── fixed bottom bar with live stats
          ├── StatsScreen ── modal overlay for per-source statistics (d key)
          └── ShortcutScreen ── modal overlay for keyboard shortcuts (h key)

Source Layout

src/
  __init__.py                  # Python 3.10 guard + textual dependency check
  app.py                       # Textual App — wires backend worker to frontend
  backend/
    models.py                  # Enums, dataclasses, Textual Message types
    checksum.py                # XOR and Sum checksum (matches firmware impl)
    frame_parser.py            # Sync state machine with checksum auto-detection
    internal_decoder.py        # INTERNAL frame payload decoder
    stats/                     # Statistics sub-package
      __init__.py              # Re-exports StatsAccumulator
      accumulator.py           # Thin composition of sub-modules
      transport.py             # RX bytes, BPS, FPS tracking
      firmware_loss.py         # ENH_STAT loss delta tracking
      firmware_written.py      # ENH_STAT write tracking
      sn_gap.py                # SN gap detection
      peak_burst.py            # 1ms window peak write burst
      traffic_spike.py         # Wire saturation detection
    uart_transport.py          # Serial port helpers, file I/O
  frontend/
    log_view.py                # RichLog wrapper with color-coded write methods
    shortcut_screen.py         # Modal screen for keyboard shortcuts
    stats_screen.py            # Modal screen for per-source statistics
    status_panel.py            # Reactive status bar (sync, speed, help hint)
tests/
  helpers.py                   # Synthetic frame builder for tests
  test_checksum.py             # Checksum algorithm tests
  test_frame_parser.py         # State machine + auto-detection tests
  test_internal_decoder.py     # Internal frame decoding tests
  test_stats.py                # Stats accumulator and firmware loss tests

Development

Running Tests

cd <esp-idf-root>
. ./export.sh
cd tools/bt/ble_log_console
python -m pytest tests/ -v

Linting & Formatting

python -m ruff format src/ tests/
python -m ruff check --fix src/ tests/

Type Checking

python -m mypy src/backend/

Troubleshooting

"UART port not found"

• Check the device is connected: ls /dev/ttyUSB* (Linux) or ls /dev/tty.usb* (macOS)
• Ensure you have permission: sudo usermod -aG dialout $USER (Linux, then re-login)
• On WSL, USB devices need usbipd-win to pass through

Sync stays in SEARCHING

• Baud rate mismatch: Ensure --baudrate matches CONFIG_BLE_LOG_PRPH_UART_DMA_BAUD_RATE
• Wrong port: Verify you're connected to the correct UART TX pin
• Firmware not running: Check the device has booted and BLE Log is initialized
• Signal integrity: At 3 Mbps, use short cables and ensure solid GND connection

No ESP_LOG output

When using UART PORT 0, ESP_LOG is redirected through BLE Log frames. If you don't see log lines:

• Ensure the firmware has CONFIG_BLE_LOG_PRPH_UART_DMA_PORT=0
• The console automatically decodes REDIR frames — no extra configuration needed
• Logs are flushed by a 1-second periodic timer, so there may be a short delay

High frame loss

• Press d to view per-source loss counters (enhanced statistics is always enabled)
• Increase buffer sizes: CONFIG_BLE_LOG_LBM_TRANS_SIZE, CONFIG_BLE_LOG_LBM_LL_TRANS_SIZE
• Add more LBMs: CONFIG_BLE_LOG_LBM_ATOMIC_LOCK_TASK_CNT
• Increase baud rate if your adapter supports higher speeds

Import errors

ModuleNotFoundError: No module named 'textual'

Re-run the ESP-IDF installer:

cd <esp-idf-root>
./install.sh
. ./export.sh