> For the complete documentation index, see [llms.txt](https://docs.theseus.us/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.theseus.us/cyclops/developers/vns_bench-dev-toolkit.md). # vns\_bench dev toolkit ## Prerequisites * Make sure you have python 3.10+ available on your system * Install [Foxglove Studio](https://foxglove.dev/download) * Optional: set `FOXGLOVE_STUDIO_BIN=/path/to/foxglove-studio` if the binary is not on your `$PATH` Install `sshpass` (to connect to the VPS): ```bash # Install sshpass sudo apt install sshpass ``` ## Quick Start ### Installation ```bash # For production use pip install "https://theseus-public-releases.s3.us-west-1.amazonaws.com/vns_bench-0.1.0-py3-none-any.whl" # Verify installation vns_bench --help ``` ### Setting up Foxglove Open Foxglove Studio. Download the `default_layout.json` below and [upload it on Foxglove Studio](https://docs.foxglove.dev/docs/visualization/layouts): * Toggle the `default_layout` drop down on the menu bar * Choose "Import from file" * Select the default\_layout.json file and import it to Foxglove {% file src="/files/oe1r0uOIn7vXaIs2AtPH" %} Here's what you should see once you configure Foxglove Studio and run the `vns_bench analyze` command on your data:
The tool gives some statistics on how the VPS perfomed during a flight and lets you see the tracks for both the VPS and GPS. ### Analyzing Recordings The `analyze` command detects and processes flights from your recordings. *Make sure you connect and power the VPS before running this.* {% hint style="success" %} **Joe, the command you want is:** ```bash # Pull specific recording and visualize vns_bench analyze --remote --folder 76_0af56fa4-b899-4380-b462-f03f213177a5 ``` {% endhint %} > Specify or select a recording when running `vns_bench analyze`. `vns_bench` automatically calculates metrics for your flight and opens Foxglove for visualization. #### Available commands Remote logs are downloaded to your current directory and kept for future reference. ```bash # Analyze and visualize (default - auto-opens in Foxglove) vns_bench analyze /path/to/recording/ # Pull from device and auto-visualize (downloads to current directory) vns_bench analyze --remote # Analyze without launching Foxglove (just export MCAP) vns_bench analyze --remote --no-launch # Skip visualization entirely vns_bench analyze --remote --no-export # Analyze all recordings in a directory vns_bench analyze /path/to/recordings/ --batch # Output results as JSON vns_bench analyze /path/to/recording/ --json ``` #### Remote Mode The `--remote` flag allows pulling logs directly from a NanoPi edge device before analysis: * Connects via SSH to the device (default: `pi@192.168.218.100`) * Lists available recordings with sizes * Downloads to your current directory: * Structured logs (`structured_logs/` directory with `.jsonl` files) * Telemetry data (`telemetry/` directory with `.jsonl` and `.idx` files) * Legacy log files (`.log` files) * Metadata files (`.json` files) * Excludes measurement data (camera images - too large) * Automatically uses structured logs if available, or converts legacy logs if needed * Runs flight detection analysis * Exports to MCAP and launches Foxglove if flight detected * Keeps all downloaded files for future reference Remote options: * `--host`: SSH host (default: `pi@192.168.218.100`) * `--password`: SSH password (default: `pi`) * `--folder`: Specific recording folder to download (if not provided, interactive prompt) * `--base-dir`: Base directory on device (default: `/mnt/nvme`) ### Metrics within Foxglove {% hint style="info" %} Version 1.7 and earlier of Micro VPS may only show limited support for metrics. Please reach out if you are interested in a metric that is not accessible from your data. {% endhint %} #### Position * `/vns/gps_location`, `/vns/map_match_location` - LocationFix (for Map) * `/vns/gps_pose`, `/vns/map_match_pose` - PoseInFrame (for 3D) * `/tf` - Frame transforms #### Events * `/vns/events` - Diagnostic log messages (WARN, ERROR, INFO) with structured fields #### Performance * `/vns/horizontal_error_m` - GPS vs Map Match error (meters) * `/vns/convergence_m` - Convergence indicator * `/vns/horizontal_error_rollup`, `/vns/convergence_rollup` - Rolling stats #### Execution Metrics * **Timing**: `/vns/execution/map_match_total_ms`, `patch_extract_ms`, `localization_ms`, `map_match_publish_ms`, `frame_prep_ms` * **Map Reload**: `/vns/execution/map_reload_total_ms`, `map_reload_wait_ms`, `map_reload_grid_ms` * **System Health**: `/vns/execution/work_queue_depth`, `vio_velocity_mps`, `distance_since_realignment_m` * **Factor graph**: `/vns/execution/ins_calibration_scale`, `ins_calibration_heading_deg` * **Counters**: `/vns/execution/odom_skip`, `realignment_blocked`, `map_match_publish`, `waypoint_detected` #### Grouped Metrics (multi-series plotting) * `/vns/groups/latencies` - All timing metrics with `metric_name` tag * `/vns/groups/system_health` - Queue depth, VIO velocity, realignment distance * `/vns/groups/calibration` - INS scale and heading * `/vns/groups/map_reload` - Map reload timing breakdown #### Counter Breakdowns (aggregated tables) * `/vns/counters/odom_skip_breakdown` - Skip counts by reason * `/vns/counters/realignment_blocked_breakdown` - Block counts by reason * `/vns/counters/map_match_publish_breakdown` - Publish counts by status #### Summary & Debug * `/vns/summary_table` - Flight metrics (for Table panel) * `/vns/summary` - Raw JSON summary * `/vns/debug/nadir`, `/vns/debug/belief_grid` - Debug images ### Troubleshooting The `mav_manager` module uses the [EXTENDED\_SYS\_STATE](https://mavlink.io/en/messages/common.html#EXTENDED_SYS_STATE) mav message to determine if the drone is in the air or on the ground. When analyzing logs, vns\_bench looks at the `landed_state` recorded in the mav manager logs to check if the recording is an actual flight. Older versions of ardupilot might not support this message leading to the following output. ```bash holden@kyobancha:~/Downloads/81_9d$ vns_bench analyze 81_9d9b5ba2-8b78-410b-9673-ffccc24913ed/ ℹ Detected VIO legacy logs, converting... Parsing global.log... Parsed 33 lines Parsing map_match_gps_fusion.log... Parsed 164517 lines Parsing map_matcher.log... Parsed 88015 lines Parsing vio_transformer.log... Parsed 746 lines Parsing vio_health.log... Parsed 3720 lines Parsing vio_supervisor.log... Parsed 84 lines Parsing mav_manager.log... Parsed 138 lines Parsing scheduler.log... Parsed 5 lines Parsing timesync.log... Parsed 8729 lines Conversion complete: Variant: VIO Metrics: 65879 Poses: 37649 Output: /home/holden/.vns_bench/converted/81_9d9b5ba2-8b78-410b-9673-ffccc24913ed ✔ Converted 65879 metrics, 37649 poses ⚠ No flight detected ``` If you encounter this issue make sure you have a local copy of the logs downloaded, then try running the following command instead: ```bash vns_bench export /local/path/to/recording/ ``` The export command doesn't do the flight detection check and should open up the recording in foxglove.