vns_bench dev toolkit

Prerequisites

• Make sure you have python 3.10+ available on your system

• Install Foxglove Studio

• Optional: set FOXGLOVE_STUDIO_BIN=/path/to/foxglove-studio if the binary is not on your $PATH

Install sshpass (to connect to the VPS):

# Install sshpass
sudo apt install sshpass

Quick Start

Installation

# For production use
pip install "https://theseus-public-releases.s3.us-west-1.amazonaws.com/vns_bench-0.1.0-py3-none-any.whl"

# Verify installation
vns_bench --help

Setting up Foxglove

Open Foxglove Studio. Download the default_layout.json below and upload it on Foxglove Studio:

• Toggle the default_layout drop down on the menu bar

• Choose "Import from file"

• Select the default_layout.json file and import it to Foxglove

6KB

default_layout.json

Open

Here's what you should see once you configure Foxglove Studio and run the vns_bench analyze command on your data:

The tool gives some statistics on how the VPS perfomed during a flight and lets you see the tracks for both the VPS and GPS.

Analyzing Recordings

The analyze command detects and processes flights from your recordings. Make sure you connect and power the VPS before running this.

Joe, the command you want is:

# Pull specific recording and visualize
vns_bench analyze --remote --folder 76_0af56fa4-b899-4380-b462-f03f213177a5

Specify or select a recording when running vns_bench analyze. vns_bench automatically calculates metrics for your flight and opens Foxglove for visualization.

Available commands

Remote logs are downloaded to your current directory and kept for future reference.

# Analyze and visualize (default - auto-opens in Foxglove)
vns_bench analyze /path/to/recording/

# Pull from device and auto-visualize (downloads to current directory)
vns_bench analyze --remote

# Analyze without launching Foxglove (just export MCAP)
vns_bench analyze --remote --no-launch

# Skip visualization entirely
vns_bench analyze --remote --no-export

# Analyze all recordings in a directory
vns_bench analyze /path/to/recordings/ --batch

# Output results as JSON
vns_bench analyze /path/to/recording/ --json

Remote Mode

The --remote flag allows pulling logs directly from a NanoPi edge device before analysis:

• Connects via SSH to the device (default: [email protected])

• Lists available recordings with sizes

• Downloads to your current directory:

  • Structured logs (structured_logs/ directory with .jsonl files)

  • Telemetry data (telemetry/ directory with .jsonl and .idx files)

  • Legacy log files (.log files)

  • Metadata files (.json files)

  • Excludes measurement data (camera images - too large)

• Automatically uses structured logs if available, or converts legacy logs if needed

• Runs flight detection analysis

• Exports to MCAP and launches Foxglove if flight detected

• Keeps all downloaded files for future reference

Remote options:

• --host: SSH host (default: [email protected])

• --password: SSH password (default: pi)

• --folder: Specific recording folder to download (if not provided, interactive prompt)

• --base-dir: Base directory on device (default: /mnt/nvme)

Metrics within Foxglove

Version 1.7 and earlier of Micro VPS may only show limited support for metrics. Please reach out if you are interested in a metric that is not accessible from your data.

Position

• /vns/gps_location, /vns/map_match_location - LocationFix (for Map)

• /vns/gps_pose, /vns/map_match_pose - PoseInFrame (for 3D)

• /tf - Frame transforms

Events

• /vns/events - Diagnostic log messages (WARN, ERROR, INFO) with structured fields

Performance

• /vns/horizontal_error_m - GPS vs Map Match error (meters)

• /vns/convergence_m - Convergence indicator

• /vns/horizontal_error_rollup, /vns/convergence_rollup - Rolling stats

Execution Metrics

• Timing: /vns/execution/map_match_total_ms, patch_extract_ms, localization_ms, map_match_publish_ms, frame_prep_ms

• Map Reload: /vns/execution/map_reload_total_ms, map_reload_wait_ms, map_reload_grid_ms

• System Health: /vns/execution/work_queue_depth, vio_velocity_mps, distance_since_realignment_m

• Factor graph: /vns/execution/ins_calibration_scale, ins_calibration_heading_deg

• Counters: /vns/execution/odom_skip, realignment_blocked, map_match_publish, waypoint_detected

Grouped Metrics (multi-series plotting)

• /vns/groups/latencies - All timing metrics with metric_name tag

• /vns/groups/system_health - Queue depth, VIO velocity, realignment distance

• /vns/groups/calibration - INS scale and heading

• /vns/groups/map_reload - Map reload timing breakdown

Counter Breakdowns (aggregated tables)

• /vns/counters/odom_skip_breakdown - Skip counts by reason

• /vns/counters/realignment_blocked_breakdown - Block counts by reason

• /vns/counters/map_match_publish_breakdown - Publish counts by status

Summary & Debug

• /vns/summary_table - Flight metrics (for Table panel)

• /vns/summary - Raw JSON summary

• /vns/debug/nadir, /vns/debug/belief_grid - Debug images

Troubleshooting

The mav_manager module uses the EXTENDED_SYS_STATE mav message to determine if the drone is in the air or on the ground. When analyzing logs, vns_bench looks at the landed_state recorded in the mav manager logs to check if the recording is an actual flight. Older versions of ardupilot might not support this message leading to the following output.

holden@kyobancha:~/Downloads/81_9d$ vns_bench analyze 81_9d9b5ba2-8b78-410b-9673-ffccc24913ed/
ℹ Detected VIO legacy logs, converting...
Parsing global.log...
  Parsed 33 lines
Parsing map_match_gps_fusion.log...
  Parsed 164517 lines
Parsing map_matcher.log...
  Parsed 88015 lines
Parsing vio_transformer.log...
  Parsed 746 lines
Parsing vio_health.log...
  Parsed 3720 lines
Parsing vio_supervisor.log...
  Parsed 84 lines
Parsing mav_manager.log...
  Parsed 138 lines
Parsing scheduler.log...
  Parsed 5 lines
Parsing timesync.log...
  Parsed 8729 lines

Conversion complete:
  Variant: VIO
  Metrics: 65879
  Poses: 37649
  Output: /home/holden/.vns_bench/converted/81_9d9b5ba2-8b78-410b-9673-ffccc24913ed
✔ Converted 65879 metrics, 37649 poses
⚠ No flight detected

If you encounter this issue make sure you have a local copy of the logs downloaded, then try running the following command instead:

vns_bench export /local/path/to/recording/

The export command doesn't do the flight detection check and should open up the recording in foxglove.