See and receive a goal
A depth image shows nearby obstacles. Position tells the system where the body is and where it should go.
MESOCORTAI / DUSTFLY EXPERIMENT
Can a fruit fly rush B?
A fly neural network. Our 3D replica of Dust2. Watch an image and a goal become movement.
RECORDED COMPUTATION
All views show the same moment.
Depth image
Light = near
Compass
FacingGoal bearingGoal is straight ahead
0.0 / — sRun outcomeLoading…
What am I watching? V7 · Image + own experience
The same inputs and network as V6. During training, it also receives correct headings for situations reached through its own decisions. This DAgger procedure teaches it to correct its mistakes.
The route used to train the original visual adapter. These attempts share a starting location on the map. The website plays a saved computation; a new goal can be computed in the local lab.
Output command: 0.0 m/s · 0.0°/s.
PFL L 0.0000 · PFL R 0.0000
01 / THE IDEA
A wiring map cannot walk on its own.
Researchers reconstructed a fly’s neurons and their connections. We built a simulated network from that wiring and added vision, position, a goal and a virtual body. The experiment asks what needs to learn for this system to find its way.
Activate the fly network
Our input adapter proposes a heading. Fly neurons process this cue together with body heading and the image.
Take the next step
The motor adapter reads output neurons and determines turning and speed. A new view starts the next cycle.
This describes the new V6–V8 versions. Their coordinates are supplied, not estimated from the image. “Rush B” is navigation in our 3D replica of Dust2.
02 / EIGHT VERSIONS, ONE CONNECTOME
What changed in each version?
The number marks a stage of the experiment. A later version is not necessarily better. Select any version.
V6
Image + position + goal
Every 100 ms, it reads a fresh depth image, current pose and target coordinates. A trained adapter proposes a heading for the fly network. Walking uses no saved route or decision table.
- Input
- Current 64 × 36 depth image, current XYZ and heading, target XYZ. No saved route.
- What learned
- A map-based teacher creates examples: fresh image, pose and goal → suitable heading. Training updates the visual adapter. During walking the teacher is absent; the network predicts a fresh heading from current inputs.
- Output
- Turning and speed derived from the fly network’s output activity. The network stays in the control loop in every version.
Inference parameters and what the count means
27,382,366 neural-network weights and biases used in this version’s forward computation. Of these, 25,582,938 belong to the fly graph. This count does not say how many neurons are active at a particular moment.
1,793,538 visual-adapter parameters + 5,890 inherited V2 motor parameters. The adapter actually runs at every step.
We exclude activity state, normalization buffers, sensors, physics and training critics. This is not a count of trainable parameters.
Open the interactive architecture: original vs. our versions →03 / A FRESH DECISION AT EVERY STEP
What happens without a saved route?
We compare V6, V7 and V8 on the same new situations. They know the map from training; final position–goal pairs are kept separate.
V6–V8 training and development tests are in progress. Final results will appear after completion and model freezing.
How much training did V6–V8 receive?
These are the executed training runs for the selected checkpoints. Development situations guided selection; the final test ran only after the models were frozen.
Development success is not a final result. Exact iterations, discarded pilots, random seeds, hashes and learning curves are available in the evidence JSON and methods.
Training record and curves · JSONV6–V8 code and selected weights · ZIPFull V6–V8 methods →04 / CONTEXT
Building on an established research direction.
Anatomical wiring can form the basis of a computational model whose interfaces or eligible internal weights are trained. Our contribution is a concrete, inspectable navigation experiment with synchronized inputs, activity and movement.
Comparison with other projects and limits of the conclusions
- Lappalainen et al. / flyvis ↗ — published research trains a connectome-constrained network on a visual task and validates its neural predictions. Our navigation project does not yet have that biological validation.
- FlyDoom ↗ — describes visual encoders, sparse fly recurrence, different training modes and PPO. It is a methodological example; we did not reproduce its results.
- fly-craftax ↗ — uses PPO to train a readout from descending-neuron activity and includes control runs. It provides a relevant example of training an interface around a connectome.
- DoomFly ↗ — simulates the same size of retained MaleCNS graph. At review, it reported failed validation gates for its candidate V6; an appealing demo is not automatically evidence of learning.
Awesome Fly is a project directory, not a certification of results. This experiment and its measured behaviour can be shared. We do not yet establish that fly wiring outperforms another equally trained network, or that the model faithfully reproduces a living fly.
Awesome Fly ↗Full comparison and sources →Explore the full experimentInteractive architecture · V1–V5 · exact neurons · training · references
02
Original vs. our versions
Signal flow: surroundings → adapter → fly network → adapter → movement. Highlighted components show where learning happened.
Click any component. Both versions will show the same step side by side.
Original model
↺ New view after moving → next cycle
Image + position + goal
↺ New view after moving → next cycle
03 / 05
Fly network
Numbers representing neural activity change with inputs and connections. This activity is actually computed through the retained anatomical fly graph.
Original
166,700 neurons and 25,582,938 directed connections. The weights and simple dynamics are our approximation built on MaleCNS.
Our version V6
The full graph with weights previously trained in V3. V6 and V7 do not change them. Activity for all 166,700 neurons is still computed while walking.
A data-flow schematic, not a map of neuron positions. All eight versions compute fly network activity; the browser shows their recorded results.
V4/V5
From one route to goals across the map
Version four gets map assistance. Version five learns from experience how to use its movement outputs better.
“I choose the next passage to the goal.”
Our new input adapter learned which neighbouring map area to choose for a given goal. It turns the passage between areas into a heading cue for the fly network. Activity in its PFL3 neurons then determines movement.
Inside the fly network, V3 weights stay fixed. The new navigation component before it is trained.
“A smoother journey earns a reward.”
V5 builds on V4. It repeatedly walks training routes and tries small changes to speed and turning. It earns rewards for progress and success, and loses reward for time, blockage and an unfinished attempt.
The motor adapter learns from PFL3 activity alone. Internal fly connections and the map adapter remain fixed.
What “arbitrary goal” means here
A point on a reachable walkable surface of this known map. V4 and V5 know their position and target NAV area. Training covered all reachable area pairs; we test new coordinates inside them. This is not an unseen map or navigation from images alone. No finite test guarantees success everywhere.
How the new versions learned
- Prepare correct choices
An offline teacher finds shortest paths on the NAV graph. For every reachable “I am here → I want to go there” pair, it identifies the next neighbouring area. This produced 3,576,864 examples.
- Train the map adapter
A small neural network predicts one of at most six neighbouring areas. Incorrect choices are repeated more often during further training. The final network matches the teacher on all training pairs. This is training accuracy, not walking success.
- Convert a choice into a neural cue
We cache the trained network’s choices. During walking, we assemble a route from them and use local passage geometry to select a nearby visible point. Its bearing feeds FC2; body heading feeds EPG. Every movement still comes from fly activity.
Final test: 100 new situations
V4 and V5 received identical starts, goals, headings, map assistance and time limits. Half the starts are near T spawn; half are across the map. All regions were known during training.
| Model | Goals reached | Successful time | Time including failures |
|---|---|---|---|
| V4 · map navigation | … | — | — |
| V5 · RL | … | — | — |
Success: within 2 m horizontally and less than 0.6 m vertically of the goal within 120 seconds. Times are simulated. The final column counts every failure as 120 s.
Why we keep these results separate from the original B test
V1–V3 had only depth and compass, a 60 s limit and the original NAV height approximation. V4–V5 add map localization, a 120 s limit and repaired triangulated surface heights. The repair removed false steps in polygons whose first vertices are collinear. V4 and V5 are compared under identical conditions.
03
What the experiment is built from
Anatomy from researchers
MaleCNS v1.0 specifies which neurons are connected. Our prepared graph contains 166,700 neurons and 25,582,938 directed connections between them. Activity, signs and the conversion of anatomical contacts into weights are simulator approximations.
Our 3D replica of Dust2
The simplified 3D map generates depth views and powers the lightweight web viewer. The navigation mesh serves the offline teacher and constrains body movement. V4–V5 also use it in the learned map interface.
V1–V3 receive while walking
Depth ahead · its heading · direct goal bearing
V4–V5 additionally receive
Map localization · target NAV area · passage geometry
04
Original stages: V1–V3 training
Training means repeating examples, measuring error and updating eligible parameters. Each version allowed a different part of the system to change.
V3
We directly trained internal connection strengths
We started with the working V2. We locked its visual and motor adapters. We supplied random pairs of “where I am facing” and “where I want to go”, then measured error in the network’s directional response.
- 01
Stimulus
102,400 angle pairs. Their correct relative direction is known exactly.
- 02
Network response
We compute neural activity and read it with the frozen motor adapter.
- 03
Error and update
Deviation from the target response determines small changes to eligible internal weights.
- 04
Validate and freeze
Validation on different angles and 20 development starts. Then freezing and final evaluation.
8 145connections eligible to change
2 907weights actually changed
~236 straining on an RTX 5090
This training targeted a more accurate directional response, not map navigation. Proposing a route heading from an image still depends on the visual adapter trained in V2.
05
Original V1–V3 results
We separate reaching B, directional response accuracy and a few examples at other goals. Each test answers a different question.
When does an attempt count as successful?
Within 60 simulated seconds, the body gets within 2 metres of the goal horizontally with a height difference below 0.6 metres. All failures count. Reported times are simulated time, not computation time.
A / The same 100 starts, goal B
| Version | Reached B | Mean successful time |
|---|---|---|
| Original model | … | — |
| Learned interfaces | … | — |
| Trained internal connections | … | — |
New positions and headings in seven T-spawn areas, but the same map and route. The final 100 starts were not used for tuning. V3 reuses the previously published V2 test; it is not a new independent test of V2.
Internal learning did not increase navigation success.
V2 already achieved 100/100. V3 also achieved 100/100 and was slightly slower on average. This does not establish that changing internal weights improves the B route.
B / Did the internal network’s response change?
The network received 2,048 additional angle pairs. We measure how many degrees its response differs from the target direction on average. Lower is better. Select the time since the stimulus:
At this time V3 is more accurate. The earliest response at 100 ms has approximately 16% less error. Bar scale: 0–3°.
C / Five goals, all 15 attempts
Each cell opens its recording above. One attempt per goal and version illustrates behaviour, not statistical reliability.
| Goal | 1 · Original | 2 · Interfaces | 3 · Internal |
|---|
Loading results…
Goals were selected before V3 results: three points from the recorded V2 route, one fixed point outside it, plus the original B. The displayed start is the first predefined test start. We did not select the best trajectory.
↗
Which neurons we connected — and why
Anatomy comes from MaleCNS. Functional inspiration comes from fly orientation research. The specific sensors, calibrations and CS2 movement mappings are our engineering.
EPG
Where the body faces
Research links EPG to heading representation. We therefore send the current body angle here, distributed across neurons according to their columns.
Selecting inputs and outputs does not mean we simulate only these cells. Activity evolves across the entire prepared graph between input and movement. The network does not establish the biological correctness of our interface; that would require further experiments.
Download the exact neuron list: MaleCNS ID, instance and roleReferences and inspiration
MaleCNS v1.0
HHMI Janelia and collaborators including Google Research. Source of connectivity, cell types, instances and anatomical coordinates. CC BY 4.0.
Data and licence Original Google articleMussells Pires et al. · Nature, 2024
Converting an allocentric goal into an egocentric steering signal. Inspiration for the relationship between heading, desired direction and motor output. The paper does not validate our CS2 interface.
Research paperWesteinde et al. · Nature, 2024
Transforming a head direction signal into a goal-oriented steering command. Support for the motor role of PFL3 and its relationship to descending DNa02 neurons. Our output-side assignment additionally uses the specific MaleCNS connectivity.
Research paperSchulman et al. · PPO, 2017
Proximal Policy Optimization Algorithms. We use clipped PPO, actor–critic and GAE. Reward, environment, inputs and the combination with the fly network were designed for this experiment.
Original methodflybody · MuJoCo Menagerie
Simplified geometry from the public flybody model, Apache 2.0. GLB conversion reduced it to 15,445 triangles. We use the body shape; this website does not run the original biomechanical simulator or its trained leg controller.
Geometry used Upstream project Apache 2.0DoomFly · StonkFly · FlyBrain
Projects supplied as inspiration for connecting a connectome to an interactive environment and showing neural activity. We do not reuse their checkpoints or claim identical implementations or that their results establish ours.
06
Clearing things up
A small glossary
- Connectome
- A map of connections between neurons. It is not ready-made behaviour.
- Activity
- A number describing a neuron’s current response in our simulation.
- Connection weight
- How strongly one neuron affects another. V3 training changes selected weights.
- Adapter
- Our component translating sensors into network input, or activity into movement.
- Freezing
- Parameters stay fixed. During an attempt, activity changes but weights no longer learn.
- EPG / FC2 / PFL3
- Names of neural populations. In our implementation: heading, desired direction and motor output.
07
Evidence and running it yourself
The website presents a frozen experiment. Switching language or explanation level does not change the models or results.
I only want to host the website
Unzip the showcase archive and upload all its contents to static hosting. The entry point is index.html. Keep every subfolder. It also works in a website subdirectory and needs no Python, Node.js or GPU at runtime.
Use an HTTP server for local testing. Opening a file through file:// may block map and data loading.
I want to try a completely new goal
Use the separate local lab included in the project. It validates a point on the navigation mesh and computes a new attempt on the GPU. The model does not keep learning during the attempt, and reaching the goal is not guaranteed.
The eight goals on this website are prepared attempts, so they work on ordinary static hosting.
Sources and credits
Anatomy data: MaleCNS v1.0, HHMI Janelia and collaborators including Google Research, CC BY 4.0. Dust2 / Counter-Strike 2: Valve, assets supplied by the user. DustFly is an independent experiment, not a Google or Valve product.
Take the experiment to your website
All prepared recordings, 3D views and explanations in English and Czech. Unzip and upload to static hosting.