# DustFly V4 / V5 — known-map goal navigation through a fly connectome

## Stručně česky

V4 se učí nové **mapové rozhraní před mušší sítí**. Zná současnou a cílovou NAV oblast a vybírá další průchod. Geometrická část z něj vytvoří směrový podnět pro FC2; EPG dostává natočení těla. Celý graf 166 700 neuronů potom skutečně počítá aktivitu. Pohyb se odvozuje pouze z 24 PFL3 buněk. Vnitřní váhy z V3 ani motorový převodník V2 se v této fázi nemění.

V5 přidává **PPO učení motorového převodníku**. Ten dostává jen PFL3 aktivity a upravuje zatáčení a rychlost. Učí se pokusy na 256 situacích. Odměna zvýhodňuje postup a úspěch a penalizuje čas, zablokování a nedokončený pokus. Tělo nemá vyšší fyzickou rychlost než V4. Neučíme znovu vnitřní mušší spoje.

„Libovolný cíl“ znamená nový dosažitelný bod **na stejné známé mapě**. Všechny NAV oblasti se mohly objevit při učení. Toto není generalizace na neznámou mapu, navigace jen z obrazu ani celá hra CS2. V4/V5 mají více vstupních informací než V1–V3. Výsledky V4/V5 se proto porovnávají mezi sebou na odděleném testu.

Ve statickém webu jsou připravené záznamy; nový zadaný cíl počítá lokální Python laboratoř. Diagram ukazuje skutečně spočítanou spojitou aktivitu, ne náhodné blikání. Nožičky 3D modelu jsou ilustrativní animace, nikoli biomechanický výstup jednotlivých neuronů.

## Provenance and retained dynamics

MaleCNS v1.0 is an anatomical dataset, not a pretrained game policy. Our “Original” is the hand-configured DustFly rate simulation built on this anatomy. The graph retains 166,700 neurons, 25,582,938 directed aggregated edges and 124,177,617 anatomical contacts. An edge can aggregate many contacts; we do not equate these counts.

The prepared signed CSR matrix uses postsynaptic rows and incoming absolute-contact normalization. ACh is approximated as positive; GABA, glutamate and histamine as negative; unspecified and some modulatory types as positive. These are simplifying physiological assumptions. Dynamics use 10 ms neural steps, 50 ms time constant:

```
drive = clip(-0.05 + 2.5 * W @ activity + stimulus, 0, 1)
activity += (0.01 / 0.05) * (drive - activity)
```

Every locomotion step integrates 100 ms of this full graph. V3 trained gains on 8,145 existing PFL3 incoming edges; 2,907 changed. V4/V5 inherit that checkpoint unchanged. Edge signs and topology remain fixed. V2/V3 training is documented separately in METHODS-V2.md and METHODS-V3.md.

## Exact neural interfaces

The downloadable `neuron-interfaces.csv` lists each selected MaleCNS body ID, prepared-array index, instance and role.

| Population | Selection | Use and rationale |
|---|---:|---|
| R1–R6 | 3,335 of 3,377 cells | Visual input; assign the strongest contact-weighted annotated L1/L2/L3 hex column, map its location to the artificial depth image. Cells without a usable column are not image inputs. |
| EPG | all 46 annotated cells | Current heading, inspired by head-direction coding. Instance `_L#` / `_R#` determines one of eight preferred columns. |
| FC2A/B/C | 18 + 27 + 47 = 92 | Desired heading, inspired by goal-direction coding. Instance `_C#` assigns one of nine phases. Combining all three subtypes is an engineering simplification. |
| PFL3 | all 24 cells | Motor readout; 12 left and 12 right projections inferred by direct and short indirect connectivity toward left/right DNa02, not soma side alone. |
| PFL2 | 12 cells | Monitored; the manual V1 brake coefficient is zero. Still part of the recurrent graph. |
| DNa02 | 2 cells | Connectivity reference for motor laterality; not the direct actuator readout. |
| L1/L2/L3/L5 | 7,114 cells | Constant lamina input 0.02. |

Right-heading mirroring and phase offsets (EPG 2.74889 rad, FC2 5.497787 rad) are calibrated interface parameters, not measured single-cell preferred directions. V1 feeds direct goal bearing into FC2. V2/V3 feed the learned visual adapter's proposed heading. V4/V5 feed the map interface's waypoint bearing. All versions retain retinal input. V4/V5 replace the V2 visual MLP, rather than adding a hidden direct actuator path.

## V4 training and information boundary

`prepare_navigation_v4.py` parses 2,243 NAV regions with 5,148 directed adjacency edges and at most six outgoing choices. The graph has 83 strongly connected components; the largest contains 1,891 regions. Directed Dijkstra distances between region centres provide 3,576,864 reachable ordered state/goal pairs excluding equal regions. Targets classify the next adjacent region on a shortest path. These are teacher labels, not recorded human actions.

The map network has **2,429,958 parameters**:

```
current region -> embedding[128] ┐
goal region    -> embedding[128] ┴-> concatenate[256] -> 256 SiLU -> 256 SiLU -> 6
goal embedding · candidate-edge embedding ----------------------------------┘
```

Invalid neighbour logits are masked. Cross-entropy and AdamW use batch 8,192, weight decay 0.0001, gradient norm cap 2. Seed 240926. Initial phase: 12,000 steps, learning rate 0.002. Refinement from that checkpoint: 10,000 additional steps, learning rate 0.0003. Uniform pair batches alternate with batches sampled from the latest classification errors. All pairs are rechecked every 1,000 updates. The refined model has zero teacher-label errors. This is exact fitting on known graph regions, **not a heldout navigation metric**.

GPU fitting took approximately 24.3 + 20.3 seconds on RTX 5090, excluding graph preprocessing and development navigation. The runtime cache `actions.npy` contains this trained network's argmax decisions, not a copied teacher lookup table. Verification recomputes all 3,576,864 neural outputs and checks exact equality. Every selected next edge strictly decreases the teacher graph distance, ruling out learned discrete route cycles for reachable pairs. It does not guarantee continuous walking success.

Runtime inputs explicitly include current NAV region, goal region, world position, goal coordinates, local portal geometry, heading and depth. Repeated cached choices assemble a learned discrete route. A geometry helper selects nearby portal/region waypoints using up to 16 route steps and approximately 2.2 m lookahead, testing local straight-line visibility with the same mover. The waypoint's bearing stimulates FC2. Runtime imports neither Dijkstra nor the teacher distance table. It nevertheless contains a learned planning interface and geometry assistance; we do not attribute map memory to internal fly synapses.

The frozen V2 motor decoder reads only 24 PFL3 activities, standardized with its learned means/scales. Its MLP is 24 → 64 SiLU → 64 SiLU → 2. It estimates a directional angle:

```
angle = atan2(output[1], output[0])
turn  = clip(3 * angle, -2.8, 2.8)             # rad/s
speed = 4.5 * exp(-2.5 * abs(angle))          # m/s
```

## V5 PPO training

The V4 route interface, V3 connectome weights and V2 angle decoder are frozen. A new actor consumes only the same 24 standardized PFL3 activities: 24 → 64 tanh → 64 tanh → 2. A diagonal Gaussian distribution has trainable log standard deviations initialized to −2, clamped to [−3.5, −0.7]. Final-layer zero initialization makes deterministic initial commands equal to V4.

```
turn  = clip(3 * angle + tanh(latent[0]), -2.8, 2.8)
speed = clip(4.5 * exp(-2.5 * abs(angle)) + 2 * tanh(latent[1]), 0, 4.5)
```

Two V5 action parameterizations were compared using development data only. The additive form above allows an offset of up to ±1 rad/s. A second `gain` run restricts steering to:

```
turn = clip(3 * angle * (1 + 0.25 * tanh(latent[0])), -2.8, 2.8)
```

The gain variant preserves the fly decoder's steering sign and changes its magnitude by at most 25% before clipping. It uses the same speed equation, reward, training situations and PPO settings. It was introduced after a late additive checkpoint produced one much slower development route. This is an explicit development design change, not final-test tuning. The checkpoint metadata field `turn_mode` records the selected variant; old checkpoints without that field use `additive`.

Training samples latent actions; final evaluation uses actor means. PPO log probabilities refer to the sampled unsquashed Gaussian latents. The deterministic tanh mapping is the same during training and evaluation. Zero PFL3 activity explicitly stops movement in inference.

The deployed actor has 5,890 parameters; two log standard deviations and a 12,001-parameter critic are training-only parameters. The training-only critic takes 24 PFL3 activities plus distance potential/200 and elapsed time/120: 26 → 96 tanh → 96 tanh → 1. It cannot affect inference commands. The potential approximates remaining route length using teacher graph distance and local distance to the current area's centre; inside the goal area it uses Euclidean goal distance. Nonfinite distances use 250. Reward per 100 ms step:

```
previous_potential - next_potential - 0.03
- 0.3 * blocked + 20 * success - 10 * timeout
```

This engineered shaping is not a claim of exact potential-based policy invariance or globally optimal time. Teacher distances enter reward and training critic only, never the actor or inference controller.

PPO configuration: clipped ratio 0.2, discount 0.995, GAE lambda 0.95, Adam learning rate 0.0003 and epsilon 1e-5, four epochs, minibatch 1,024, rollout 32 environments × 128 steps, entropy coefficient 0.002. Value loss is half squared error and contributes with coefficient 0.5. Gradient norm cap 0.5; stop further epochs if approximate KL exceeds 0.03. Seed 280926. Each environment uses its own full connectome activity; the shared anatomical weights remain frozen. Episodes sample only the 256 fixed RL-training situations. Completed environments reset body and neural state. Timeout is terminal with the explicit failure penalty.

Route and PPO training enable PyTorch's `high` float32 matrix-multiplication precision setting. Evaluation runs in fresh processes with the default precision setting. We do not promise bitwise equality across GPU batches or hardware; the route-cache equivalence check explicitly uses the training precision setting.

Training traces, candidate evaluations, the selected checkpoint and exact step counts are included in `outputs/v5/selection.json` and `training-*.json`. A resumed run starts fresh environment slots and optimizer state, loads the saved actor/critic weights, and records that fact. No final-test outcomes are used to choose a checkpoint.

## Development, final test and environment

Split generation uses seeds 250926 (32 development), 260926 (100 final) and 270926 (256 RL training). Half the starts are sampled near the historical T-spawn regions; half are sampled across the map. Goal regions have teacher graph distance between 12 and 220 m. Positions mix 65% region centre and 35% a Dirichlet combination of polygon vertices; initial headings are uniform. Region sampling uses a clipped first-triangle area proxy, **not uniform sampling over all walkable surface area**.

All NAV regions can occur in teacher training. Final continuous positions and heading combinations are distinct from the teacher's region-centre labels and not used for checkpoint selection. The test therefore measures new continuous situations on a known graph, not unseen-region or unseen-map generalization.

V4 development selected a learned current-area portal controller and a deterministic surface-height repair. The initial waypoint-progress implementation reached 1/8 pilot goals; the revised local portal controller reached 7/8. A false NAV height step blocked the remaining case. Repairing heights produced 32/32 development successes. These are development changes and are disclosed, not hidden as final-test outcomes.

V4/V5 use identical NAV movement with 100 ms actions subdivided into steps no longer than 8 cm, permitted upward step below 0.5 m and downward step below 4 m, speed cap 4.5 m/s and turning cap 2.8 rad/s. Candidate surfaces are bounded to three local adjacency hops. The vectorized implementation is verified against 500 randomized cases from the original mover.

Historical V1–V3 used a single plane from each polygon's first three vertices, with mean-height fallback for degenerate planes. V4/V5 use a fan of nondegenerate triangles and barycentric heights. At the diagnosed area-214 entrance, this removes a false 0.711 m step. It remains an approximation: zero-area triangles are skipped and some collinear intermediate edge vertices need not be reproduced exactly. All new split Z coordinates were deterministically corrected before final evaluation; XY, region IDs and seeds stayed unchanged. Both new models share this repair. Historical recordings retain their original geometry calculation.

Final success means horizontal distance ≤2 m and vertical difference <0.6 m within **120 simulated seconds**. All failures count. We report success count, mean successful duration, mean duration with every failure scored as 120 s, and paired time differences for cases completed by both models. Paired bootstrap confidence intervals resample matched cases; they quantify uncertainty for this sampled test, not reliability everywhere on Dust2. V1–V3 historical B tests had a 60 s limit and different information and surface heights, so their headline results are not pooled with the new benchmark.

## Replays and visualization

Eight showcase situations are fixed: the original five goals plus final-list indices 1, 33 and 65 selected before final outcomes. Every model's attempt is published, including failures. Historical and new models share XY start/goal coordinates; Z follows their disclosed surface model. The eight replays per model are illustrations, separate from the 100-case paired benchmark. Recomputing in different GPU batch sizes can create slight floating-point trajectory differences; no visually best run is selected.

Each new frame records actual position, yaw, commanded speed and turn, local cue, six population summaries, depth, and 176 selected single-cell values. The neural diagram projects actual soma coordinates using PCA and shows the 240 strongest direct anatomical contact counts among those sampled cells. The lines do not show trained weight changes. Recorded speed is the motor command; the NAV mover can reduce actual displacement. The website labels requested speed and indicates movement-limited commands. Values are little-endian float16 encoded in base64. This is continuous model activity, not spikes.

Legacy recordings originally lacked single-cell traces. They were reconstructed by running the frozen full graph along recorded headings and neural cues, rendering depth at the recorded rounded poses. Population rates and motor angles were compared to the original outputs before attachment. Maximum population-rate discrepancy across the 15 recordings was 0.00001428; maximum motor-angle discrepancy was below 0.000001 rad. Original trajectories and outcomes are unchanged. This is a verified reconstruction, not a claim of bitwise recovery of the old GPU state.

The fly body's mesh comes from the Apache-2.0 flybody asset in Google DeepMind's MuJoCo Menagerie, upstream TuragaLab/flybody. We remove bristles, weld mesh seam vertices, decimate and convert the neutral articulated pose to GLB (15,445 triangles, approximately 383 KB). Heading and speed follow the recorded body command; leg motion is a simple illustrative gait. The browser does not simulate MuJoCo physics, individual motor joints, ground-contact biomechanics or the upstream learned flybody policies.

The camera and body orientation interpolate between stored 100 ms poses for smooth playback; the neural diagram shows recorded sample values without invented intermediate activity. The static website uses local HTML, CSS, JavaScript, fonts, Draco decoder, geometry and JSON recordings. It performs no full-connectome inference and contacts no GPU API. For a genuinely new goal, use the separate loopback-only local lab. Site and ZIP contain the same prepared results.

## Reproduction

Use the prepared project with `de_dust2.nav`, `de_dust2_simply.glb`, prepared MaleCNS data, V2/V3 checkpoints and CUDA PyTorch on a suitable GPU. Dependency versions are in `requirements.txt`. The downloadable experiment archives exclude the large anatomical source data, map assets and gameplay videos; they include code, smaller trained weights, protocols and results. Keep the published files intact and use separate tags for repeat runs.

```
python scripts/prepare_navigation_v4.py
python scripts/repair_v4_surface_heights.py
python scripts/train_route_v4.py --tag repeat-initial
python scripts/train_route_v4.py --tag repeat-refined --resume repeat-initial --steps 10000 --lr 0.0003
python scripts/train_rl_v5.py --route outputs/v4/checkpoint --updates 60 --tag repeat-ppo
python scripts/train_rl_v5.py --route outputs/v4/checkpoint --updates 90 --tag repeat-gain --turn-mode gain
python scripts/run_v4.py --model general --split development --tag repeat
python scripts/run_v4.py --model fast --split development --motor outputs/v5/checkpoint/motor.pt --tag repeat
python scripts/verify_v5.py
```

The prepared project's selected `route.pt`, `actions.npy` and `route.json` live in `outputs/v4/checkpoint`; the selected RL weights live in `outputs/v5/checkpoint/motor.pt`. Repeat training saves separate subfolders and does not automatically overwrite the selection. Freeze both selected models before evaluating a new final split. Running `prepare_navigation_v4.py` in an existing project preserves split files; its metadata should be regenerated only in a reproduction copy.

Start the local goal UI with `start-lab.ps1`, then open `http://127.0.0.1:8765/lab/`. The local worker validates finite coordinates, walkable surfaces and learned route reachability, permits one GPU job at a time, and rejects foreign browser origins. V1–V3 remain available as historical controls.

Asset rebuilding only (not needed for inference) additionally requires `mujoco==3.13.0` and `fast-simplification==0.2.0`. The avatar exporter reads the downloaded, licensed flybody source under `data/flybody`.

## References and limits

- [MaleCNS v1.0](https://male-cns.janelia.org/download/), HHMI Janelia and collaborators including Google Research, CC BY 4.0 — anatomy and annotations.
- [Google's reconstruction overview](https://blog.google/innovation-and-ai/technology/research/male-fruit-fly-brain-map/) — original user-provided context.
- [Mussells Pires et al., Nature 2024](https://www.nature.com/articles/s41586-023-07006-3), *Converting an allocentric goal into an egocentric steering signal* — EPG/FC2/PFL3 functional inspiration.
- [Westeinde et al., Nature 2024](https://www.nature.com/articles/s41586-024-07039-2), *Transforming a head direction signal into a goal-oriented steering command* — PFL3 and descending steering pathways.
- [Schulman et al., 2017](https://arxiv.org/abs/1707.06347), *Proximal Policy Optimization Algorithms* — PPO method.
- [MuJoCo Menagerie flybody](https://github.com/google-deepmind/mujoco_menagerie/tree/main/flybody) and [upstream flybody](https://github.com/TuragaLab/flybody) — body geometry only, Apache 2.0.
- [DoomFly](https://github.com/nftechie/doomfly), [StonkFly](https://github.com/nftechie/stonkfly), [FlyBrain](https://github.com/Chieler/flybrain) — community inspiration for interactive connectome experiments; their policies and code are not imported by our runtime.
- [Awpy](https://github.com/pnxenopoulos/awpy) — Source NAV format reference; attribution in THIRD_PARTY_NOTICES.md.

There is no comparison against a conventional route follower or shuffled neural graph in this release. We cannot isolate a performance advantage of biological topology. The neural model, artificial senses, metre-scale body and simplified CS2 movement are engineering choices. There is no shooting, enemy interaction, jumping, smoke, bomb planting or complete CS2 physics. DustFly is an independent experiment, not an official Google, Janelia or Valve product.