# Project handoff — TRI_PROJ2 herding (2026-05-16)

Context for a fresh model picking this project up. Project deadline: **2026-06-04**.
Branch: `test/johnny8`. Last commits: `876e14e` (LSTM), `dd5ac66` (core fixes).

---

## What this project is

Group G25 course project: an autonomous shepherd dog that herds 1–10 sheep through a gate into a pen. Two worlds (rectangular `field`, circular `field_round`), two drives (`differential`, `mecanum`), and five control strategies:

- `strombom` — analytical Strömbom collect/drive heuristic
- `sequential` — analytical single-target pin-and-push baseline
- `universal` — analytical teacher used to collect BC demos
- `bc` — MLP policy trained via behaviour cloning of `universal`
- `rl` — KL-regularised PPO fine-tune of `bc`

The dog perceives sheep only through a front-mounted LiDAR (`protos/ShepherdDog.proto`).
A 2D Gym env (`training/herding_env.py`) is used for training and headless evaluation;
Webots is used for sim-to-deployment validation.

See `docs/project.md` for the formal course objectives. See
`~/.claude/projects/-home-jalf-code-TRI-PROJ2/memory/` for the running notes
(`project_state.md`, `dagger_results.md`, `lstm_results.md`, `webots_perception_gap.md`).

---

## What's working today

Everything below is **verified**, with command lines you can copy-paste.

### Analytical strategies (Strömbom, Sequential, Universal)

Work in Webots with **GT bypass** (`HERDING_USE_GT=1`) — 12/12 trials across
both worlds × {5, 10 sheep}. User has signed off on GT bypass for these
analytical baselines (they take a position list as input; GT vs LiDAR is a
perception-layer concern, not a strategy concern).

Validated by `webots_sweep_gt.log` (full matrix, all OK).

### Gym performance (clean 360° LiDAR sim, default tracker)

```
BC diff/field:  96% avg (90-100% across n=1..10)
RL diff/field:  99% avg (90-100%)
BC diff/round:  58%  ← weak combo
RL diff/round:  58%  ← weak combo
BC mec/field:   86%
RL mec/field:   90%
BC mec/round:   73%
RL mec/round:   79%
```

Plus a Stage-2 `rl_fast` time-penalty pass on diff/field and mec/field
(`rl_fast_*` directories) that slightly accelerates time-to-pen with similar
success.

### Webots LiDAR — 360° proto variant (`protos/ShepherdDog360.proto`)

Created today as a robustness ablation. v1 policies (trained on default 360°
gym LiDAR) transfer cleanly:

```
strombom/sequential/universal:  12/12 OK
bc diff (5 and 10 sheep, both worlds):  3/4 OK (only diff/field n=10 timed out)
bc mecanum:                              0/4 — separate dynamics gap
rl any:                                  0/4 — RL more brittle than BC, unexpectedly
```

Validated by `webots_sweep_360.log`.

---

## What does NOT work (despite multiple attempts)

**Any learned policy (BC, RL, DAgger, LSTM) in Webots LiDAR with the
canonical 140° FOV proto.** All hit the same wall: tracker phantom-track
patterns from real Webots LiDAR don't match what the gym FP-injection model
produces, so policies trained on the gym proxy can't handle the obs they see
in Webots.

Approaches tried today (all detailed in `~/.claude/projects/.../memory/`):

| Approach | Gym proxy | Webots LiDAR 140° |
|---|---|---|
| v1 MLP + frame stack, clean training | 99% | 0/5 |
| DAgger (3 rounds, privileged teacher labels) | 12% → 38% on proxy | 0/5 |
| LSTM RecurrentPPO from scratch, 3M steps | 69% clean / 2% proxy | 0/5 |

Diagnosis: gym `HERDING_WEBOTS` preset (`herding/config.py`) is an
approximation but not faithful to actual Webots LiDAR. Real Webots produces
~4 phantom tracks per step for 5 real sheep due to wall/post/leg returns;
gym injection uses a Poisson process at static anchor points which is
distributionally different.

---

## Critical bug fixes shipped today

If you're picking this up, these are real bugs that took hours to find:

1. **Webots controllers were silently crashing on numpy import.** Webots
   launched them under system `python3` (no numpy). Fixed by adding
   `runtime.ini` files at `controllers/{shepherd_dog,sheep}/runtime.ini`
   that point Webots to the conda env's python.

2. **FP_RATE mismatch BC=0 vs RL=2 poisoned PPO.** Default in Makefile was
   `FP_RATE=2.0` for RL but `--fp-rate 0.0` hard-coded for BC demos. PPO
   stalled at 0% success for 1.46M steps. Now `FP_RATE=0.0` consistent.

3. **Tracker phantom-penned tracks.** `pen_latch_depth=0.5` was too shallow
   (FPs at y≈-15 latched and lived forever). Now 2.0, and penned tracks
   decay at `forget_steps × 8` instead of being eternal.

4. **HERDING_WEBOTS preset tuning** in `herding/config.py` —
   `max_new_tracks_per_step=1`, `static_reject=1.2`. Reduces phantom-track
   spawning rate but doesn't eliminate it.

---

## Recommended path to a strong June 4 deliverable

You don't need to fix the 140° LiDAR gap — there's a defensible story
already. The article framing writes itself:

> "Wide-FOV (360°) LiDAR enables clean sim-to-real transfer of learned
> shepherding policies. Narrow-FOV (140°) introduces phantom-track noise
> that current policies cannot fully reject — closing this gap is future
> work, likely requiring either a faithful gym-side LiDAR model or
> Webots-in-the-loop training."

Concrete deliverable plan:

1. **Demo video and screenshots**: use the 360° proto for BC/RL demonstrations
   and GT bypass for analyticals on 140°. All combos covered.
2. **Quantitative results**: gym eval already gives success%, mean steps.
   Add a flock-dispersion metric (`max(distances from CoM)` at end of
   episode) — about 30 lines in `eval.py`.
3. **Collision tracking**: add a counter in `HerdingEnv.step()` for
   `dog-sheep distance < 0.30 m`. Currently the env knows about
   `COLLISION_DIST` but doesn't expose it in info. ~20 lines.
4. **Mecanum**: the mecanum Webots dynamics gap is **separate** from the
   perception issue. `tools/calibrate_mecanum.sh` exists for this. Run
   it and see if it gives matching dynamics. This is the most valuable
   remaining technical task — closing the mecanum gap would let you
   complete the "diff vs mecanum" extra-merit comparison in
   `docs/project.md`.
5. **Round world**: gym performance is ~58-79% across approaches. The
   curved walls break Strömbom's "stand behind the centroid" geometry
   (the position behind sometimes lies outside the field). Two cheap
   tweaks worth trying: (a) a per-episode `W_RADIUS` reward bonus for
   compact flocks (gather-first behavior), (b) curriculum on the env's
   `difficulty` knob (already wired in `HerdingEnv`).

Bonuses still on the table (from `docs/project.md` extra merit):
- **Multi-shepherd axis-split** — user's idea, ~1 day work. Each dog
  computes one component of the analytical Strömbom action. No multi-agent
  RL needed.
- **Robustness / DR ablation** — FP/wheel-slip knobs exist; run an ablation
  table.

---

## Repository layout (essentials)

```
herding/
  config.py            # HerdingConfig dataclasses, HERDING_DEFAULT / HERDING_WEBOTS presets
  control/             # strombom.py, sequential.py, universal.py (analytical teachers)
  perception/          # lidar_sim.py, lidar_perception.py, sheep_tracker.py
  world/               # diffdrive.py kinematics, flocking_sim.py, geometry.py (PEN_*/GATE_*/FIELD_*)

training/
  herding_env.py       # Gym env: HerdingEnv. ~560 lines. Step/reset/reward/obs.
  bc/
    collect.py         # Demo collector — supports --privileged and --dagger-policy
    pretrain.py        # MLP BC trainer (MSE + 1-cos loss)
  rl/
    train.py           # KL-regularised PPO fine-tune of BC
    train_lstm.py      # NEW today: RecurrentPPO (sb3-contrib) from scratch
  eval.py              # Env-side evaluator; supports MLP + LSTM policies
  runs/                # Trained artifacts (bc_*, rl_*, rl_fast_*, lstm_*)
    v1_clean/          # Backup of pre-DAgger artifacts

controllers/
  shepherd_dog/
    shepherd_dog.py    # Webots controller. Mode selection via HERDING_MODE env.
    policy_loader.py   # Auto-detects MLP vs LSTM zip. Handles obs / state.
    runtime.ini        # ← critical, points Webots to conda python
  sheep/
    runtime.ini        # ← same fix

protos/
  ShepherdDog.proto       # canonical 140° FOV (matches the physical robot)
  ShepherdDog360.proto    # 360° variant for the FOV ablation / fallback delivery
  ShepherdDogMecanum.proto
  Sheep.proto

worlds/
  field.wbt              # rectangular world
  field_round.wbt        # circular world

tools/
  run_webots.sh          # launcher: tools/run_webots.sh N MODE DRIVE WORLD
  webots_sweep.sh        # full LiDAR sweep across all modes × drives × worlds
  webots_sweep_gt.sh     # same but with HERDING_USE_GT=1
  dagger_round.sh        # NEW today: one-shot DAgger collect + train
  calibrate_mecanum.sh   # mecanum dynamics calibration (not run today)

Makefile                 # Top-level: make train_all, make eval_all, etc.
```

---

## Quick commands

```bash
# Run pytest (111 tests, all passing)
make test

# Train one combo end-to-end (BC → RL → eval, ~1h on 2 cores)
make DRIVE=differential WORLD=field

# Train all 4 combos (~5h)
make train_all

# Eval an existing policy directory in gym
python -m training.eval --policy training/runs/rl_differential_field \
    --max-flock 10 --max-steps 15000 --n-seeds 10 \
    --drive-mode differential --world field

# Webots — analytical, GT bypass (this works for all combos)
HERDING_USE_GT=1 tools/run_webots.sh 5 strombom differential field

# Webots — BC with the 360° proto (currently the 140° proto is active;
# swap by editing protos/ShepherdDog.proto or use the 360° variant directly)
tools/run_webots.sh 5 bc differential field

# Headless full sweep (~80 min)
tools/webots_sweep.sh webots_sweep.log

# Train LSTM (sb3-contrib must be installed)
python -m training.rl.train_lstm \
    --out training/runs/lstm_differential_field \
    --total-timesteps 3000000 --use-webots-preset \
    --drive-mode differential --world field
```

---

## Hardware/environment

- 3.8 GB RAM, 8 GB swap, 2 cores. Memory pressure is real — saw the
  OS OOM-kill RL training during chained `train_all` once. If you re-run
  full pipelines, monitor memory and consider splitting.
- Conda env: `tir` at `/home/jalf/miniconda3/envs/tir/`. Has SB3,
  sb3-contrib, PyTorch, gymnasium. Webots controllers point to this
  python via the new `runtime.ini` files.
- Webots installed at `/usr/local/webots/`. Headless mode requires
  `xvfb-run -a` (no X display on this machine).

---

## What I'd suggest for a fresh attempt at the 140° LiDAR gap

If the user wants you to keep pushing on it, the highest-EV experiment
not yet tried is:

**Consensus tracker** — modify `herding/perception/sheep_tracker.py` to
require K consecutive detections within a small radius before promoting
a track to "real." Phantom tracks from sporadic wall returns wouldn't
survive the K-step consensus; real sheep continuously visible in FOV
would. The current `max_new_tracks_per_step=1` rate-limits new tracks
but every detection still spawns one immediately.

Implementation sketch: add a "candidate" track type that doesn't appear
in `get_positions()`. After K (e.g. 3-5) consecutive matched detections,
promote candidate → real track. Roughly 30-50 lines of code.

This is a tracker-level fix at deploy time only, so it wouldn't require
retraining the policies — v1 BC/RL should transfer cleanly if the tracker
output looks more like what they were trained on (one position per real
sheep, no phantoms).

I would NOT recommend more architectural training experiments (DAgger
round 4, larger LSTM, etc.) — three independent approaches today already
showed the bottleneck is upstream of the policy.