# vns\_bench dev toolkit

## Prerequisites

* Make sure you have python 3.10+ available on your system
* Install [Foxglove Studio](https://foxglove.dev/download)
* Optional: set `FOXGLOVE_STUDIO_BIN=/path/to/foxglove-studio` if the binary is not on your `$PATH`

Install `sshpass` (to connect to the VPS):

```bash
# Install sshpass
sudo apt install sshpass
```

## Quick Start

### Installation

```bash
# For production use
pip install "https://theseus-public-releases.s3.us-west-1.amazonaws.com/vns_bench-0.1.0-py3-none-any.whl"

# Verify installation
vns_bench --help
```

### Setting up Foxglove

Open Foxglove Studio. Download the `default_layout.json` below and [upload it on Foxglove Studio](https://docs.foxglove.dev/docs/visualization/layouts):

* Toggle the `default_layout` drop down on the menu bar
* Choose "Import from file"
* Select the default\_layout.json file and import it to Foxglove

{% file src="/files/oe1r0uOIn7vXaIs2AtPH" %}

Here's what you should see once you configure Foxglove Studio and run the `vns_bench analyze` command on your data:

<figure><img src="/files/xpxTP9dciT5orxZn3hsO" alt=""><figcaption></figcaption></figure>

The tool gives some statistics on how the VPS perfomed during a flight and lets you see the tracks for both the VPS and GPS.

### Analyzing Recordings

The `analyze` command detects and processes flights from your recordings. *Make sure you connect and power the VPS before running this.*

{% hint style="success" %}
**Joe, the command you want is:**

```bash
# Pull specific recording and visualize
vns_bench analyze --remote --folder 76_0af56fa4-b899-4380-b462-f03f213177a5
```

{% endhint %}

> Specify or select a recording when running  `vns_bench analyze`. `vns_bench` automatically calculates metrics for your flight and opens Foxglove for visualization.

#### Available commands

Remote logs are downloaded to your current directory and kept for future reference.

```bash
# Analyze and visualize (default - auto-opens in Foxglove)
vns_bench analyze /path/to/recording/

# Pull from device and auto-visualize (downloads to current directory)
vns_bench analyze --remote

# Analyze without launching Foxglove (just export MCAP)
vns_bench analyze --remote --no-launch

# Skip visualization entirely
vns_bench analyze --remote --no-export

# Analyze all recordings in a directory
vns_bench analyze /path/to/recordings/ --batch

# Output results as JSON
vns_bench analyze /path/to/recording/ --json
```

#### Remote Mode

The `--remote` flag allows pulling logs directly from a NanoPi edge device before analysis:

* Connects via SSH to the device (default: `pi@192.168.218.100`)
* Lists available recordings with sizes
* Downloads to your current directory:
  * Structured logs (`structured_logs/` directory with `.jsonl` files)
  * Telemetry data (`telemetry/` directory with `.jsonl` and `.idx` files)
  * Legacy log files (`.log` files)
  * Metadata files (`.json` files)
  * Excludes measurement data (camera images - too large)
* Automatically uses structured logs if available, or converts legacy logs if needed
* Runs flight detection analysis
* Exports to MCAP and launches Foxglove if flight detected
* Keeps all downloaded files for future reference

Remote options:

* `--host`: SSH host (default: `pi@192.168.218.100`)
* `--password`: SSH password (default: `pi`)
* `--folder`: Specific recording folder to download (if not provided, interactive prompt)
* `--base-dir`: Base directory on device (default: `/mnt/nvme`)

### Metrics within Foxglove

{% hint style="info" %}
Version 1.7 and earlier of Micro VPS may only show limited support for metrics. Please reach out if you are interested in a metric that is not accessible from your data.
{% endhint %}

#### Position

* `/vns/gps_location`, `/vns/map_match_location` - LocationFix (for Map)
* `/vns/gps_pose`, `/vns/map_match_pose` - PoseInFrame (for 3D)
* `/tf` - Frame transforms

#### Events

* `/vns/events` - Diagnostic log messages (WARN, ERROR, INFO) with structured fields

#### Performance

* `/vns/horizontal_error_m` - GPS vs Map Match error (meters)
* `/vns/convergence_m` - Convergence indicator
* `/vns/horizontal_error_rollup`, `/vns/convergence_rollup` - Rolling stats

#### Execution Metrics

* **Timing**: `/vns/execution/map_match_total_ms`, `patch_extract_ms`, `localization_ms`, `map_match_publish_ms`, `frame_prep_ms`
* **Map Reload**: `/vns/execution/map_reload_total_ms`, `map_reload_wait_ms`, `map_reload_grid_ms`
* **System Health**: `/vns/execution/work_queue_depth`, `vio_velocity_mps`, `distance_since_realignment_m`
* **Factor graph**: `/vns/execution/ins_calibration_scale`, `ins_calibration_heading_deg`
* **Counters**: `/vns/execution/odom_skip`, `realignment_blocked`, `map_match_publish`, `waypoint_detected`

#### Grouped Metrics (multi-series plotting)

* `/vns/groups/latencies` - All timing metrics with `metric_name` tag
* `/vns/groups/system_health` - Queue depth, VIO velocity, realignment distance
* `/vns/groups/calibration` - INS scale and heading
* `/vns/groups/map_reload` - Map reload timing breakdown

#### Counter Breakdowns (aggregated tables)

* `/vns/counters/odom_skip_breakdown` - Skip counts by reason
* `/vns/counters/realignment_blocked_breakdown` - Block counts by reason
* `/vns/counters/map_match_publish_breakdown` - Publish counts by status

#### Summary & Debug

* `/vns/summary_table` - Flight metrics (for Table panel)
* `/vns/summary` - Raw JSON summary
* `/vns/debug/nadir`, `/vns/debug/belief_grid` - Debug images

### Troubleshooting

The `mav_manager`  module uses the [EXTENDED\_SYS\_STATE](https://mavlink.io/en/messages/common.html#EXTENDED_SYS_STATE) mav message to determine if the drone is in the air or on the ground. When analyzing logs, vns\_bench looks at the `landed_state` recorded in the mav manager logs to check if the recording is an actual flight. Older versions of ardupilot might not support this message leading to the following output.

```bash
holden@kyobancha:~/Downloads/81_9d$ vns_bench analyze 81_9d9b5ba2-8b78-410b-9673-ffccc24913ed/
ℹ Detected VIO legacy logs, converting...
Parsing global.log...
  Parsed 33 lines
Parsing map_match_gps_fusion.log...
  Parsed 164517 lines
Parsing map_matcher.log...
  Parsed 88015 lines
Parsing vio_transformer.log...
  Parsed 746 lines
Parsing vio_health.log...
  Parsed 3720 lines
Parsing vio_supervisor.log...
  Parsed 84 lines
Parsing mav_manager.log...
  Parsed 138 lines
Parsing scheduler.log...
  Parsed 5 lines
Parsing timesync.log...
  Parsed 8729 lines

Conversion complete:
  Variant: VIO
  Metrics: 65879
  Poses: 37649
  Output: /home/holden/.vns_bench/converted/81_9d9b5ba2-8b78-410b-9673-ffccc24913ed
✔ Converted 65879 metrics, 37649 poses
⚠ No flight detected
```

If you encounter this issue make sure you have a local copy of the logs downloaded, then try running the following command instead:

```bash
vns_bench export /local/path/to/recording/
```

The export command doesn't do the flight detection check and should open up the recording in foxglove.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.theseus.us/micro-vps/validation/vns_bench-dev-toolkit.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.