jalf/TIR_PROJ
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Webots sim-to-real fixes, DAgger pipeline, 360° proto variant

Browse Source 
Today's session worked across the full Webots delivery stack — found and
fixed a cluster of bugs blocking the BC/RL transfer, then explored
training-side mitigations for the residual perception gap.

Bug fixes:
- Makefile FP_RATE default 2.0 → 0.0: BC demos used fp_rate=0 but RL
  fine-tune defaulted to fp_rate=2, poisoning the BC obs distribution
  and stalling PPO at 0% success across 1.46M+ steps.
- controllers/{shepherd_dog,sheep}/runtime.ini: Webots was launching
  controllers under system python3 (no numpy) and they were crashing
  silently. Pinned to the conda tir env.
- herding/config.py HERDING_WEBOTS preset: pen_latch_depth 0.5 → 2.0,
  max_new_tracks_per_step 3 → 1, static_reject 0.8 → 1.2. Stops phantom
  FPs near the gate from latching as permanently-penned tracks.
- herding/perception/sheep_tracker.py: penned tracks now decay at
  forget_steps × 8 instead of living forever. Adds get_positions
  min_freshness filter for deploy-time use.

Training/eval matches deployment:
- training/bc/collect.py: --dagger-policy flag for DAgger rollouts
  (policy drives, teacher labels) + --use-webots-preset for matched
  140° tracker + DR config.
- controllers/shepherd_dog/shepherd_dog.py: scan-fallback (0, 0.6) when
  BC/RL sees empty sheep_positions — recovers from FOV gaps.

Tooling:
- tools/dagger_round.sh: one-shot DAgger round (collect + concat + bc).
- tools/webots_sweep_gt.sh: full sweep with HERDING_USE_GT=1 for the
  perception-gap diagnosis matrix.
- protos/ShepherdDog360.proto: 360° FOV variant for the FOV-ablation
  comparison. Canonical proto stays at 140° per project spec.

Artifacts: v1 BC/RL policies for all 4 (drive × world) combos trained
in clean gym (success: diff/field 90-100%, diff/round 58%, mec/field
60-100%, mec/round 50-100%). DAgger r1/r2 BCs for diff/field show
12%→38% progression on gym HERDING_WEBOTS proxy but did not close
to actual Webots LiDAR (0/5 throughout). Next: LSTM policy or
learned tracker per the project-state memory.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Johnny Fernandes
2026-05-16 17:21:02 +00:00
parent c61df91950
commit dd5ac669e5
34 changed files with 2336 additions and 188 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tools/calibrate_mecanum.sh
Executable
+57
View File
@@ -0,0 +1,57 @@
#!/usr/bin/env bash
# Measure the actual velocity response of the Webots mecanum robot and
# compare against the gym's first-order kinematics prediction.
#
# Uses HERDING_MODE=calibrate in the shepherd_dog controller, which applies
# a known fixed action for N steps, records GPS displacement, and computes
# the relative error vs gym prediction.
#
# Usage:
# bash tools/calibrate_mecanum.sh [N_STEPS]
# N_STEPS : steps to hold each action (default 150, ≈ 2.4 s real-time)
#
# Output:
# calibrate_mecanum.log — per-axis results printed and written here
#
# Target: < 10% relative error on each axis.
# If errors are high, tune coulombFriction / forceDependentSlip in
# tools/run_webots.sh (mecanum contactProperties block).
set -euo pipefail
N_STEPS="${1:-150}"
ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )"
LOG="$ROOT/calibrate_mecanum.log"
export PATH="/home/jalf/miniconda3/envs/tir/bin:$PATH"
echo "Running mecanum calibration (N_STEPS=$N_STEPS)..."
echo "Results will be written to: $LOG"
truncate -s 0 "$LOG"
run_calib() {
local vx="$1" vy="$2" om="$3"
echo " Testing vx=$vx vy=$vy om=$om ..."
rm -f "$ROOT/training/.run_done"
timeout --kill-after=15s 60 \
xvfb-run -a \
env WEBOTS_HEADLESS=1 WEBOTS_EXTRA_ARGS="--stdout --stderr" \
HERDING_MODE=calibrate HERDING_DRIVE=mecanum HERDING_WORLD=field \
CALIB_VX="$vx" CALIB_VY="$vy" CALIB_OM="$om" \
CALIB_N_STEPS="$N_STEPS" \
bash "$ROOT/tools/run_webots.sh" 0 calibrate mecanum field \
2>&1 | grep -E "cmd=|gym|webots|error" || true
pkill -9 -f "webots-bin|Xvfb" 2>/dev/null || true
sleep 1
}
# Three test vectors: pure-x, pure-y, diagonal
run_calib 0.5 0.0 0.0
run_calib 0.0 0.5 0.0
run_calib 0.35 0.35 0.0
echo ""
echo "=== Calibration results ==="
cat "$LOG" 2>/dev/null || echo "(no results written — check controller output above)"
echo ""
echo "Target: <10% error on each axis."
echo "If errors are high, tune coulombFriction / forceDependentSlip in"
echo "tools/run_webots.sh (mecanum contactProperties block)."
tools/dagger_round.sh
Executable
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env bash
# Run one DAgger round on a (drive, world) combo.
#
# Usage: tools/dagger_round.sh <drive> <world> [seeds_per_n] [round_idx]
#
# Collects DAgger demos using the current BC policy as the actor and the
# universal teacher as the labeller, in the HERDING_WEBOTS preset env
# (140° FOV, tight tracker — matches deployment). Concatenates with the
# original BC demos, re-trains BC, and saves to runs/bc_dagger_<combo>/.
set -euo pipefail
ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )"
cd "$ROOT"
DRIVE="${1:-differential}"
WORLD="${2:-field}"
SEEDS="${3:-15}"
ROUND="${4:-1}"
TAG="${DRIVE}_${WORLD}"
ORIG_DEMOS="training/bc/demos_${TAG}.npz"
DAGGER_DEMOS="training/bc/dagger${ROUND}_${TAG}.npz"
COMBINED_DEMOS="training/bc/combined${ROUND}_${TAG}.npz"
BC_DIR="training/runs/bc_${TAG}"
OUT_DIR="training/runs/bc_dagger${ROUND}_${TAG}"
case "$WORLD" in
field_round)
EPOCHS=150
;;
*)
EPOCHS=60
;;
esac
echo "=== DAgger round ${ROUND}: ${DRIVE}/${WORLD} ==="
echo " Actor policy: ${BC_DIR}/policy.zip"
echo " Output: ${OUT_DIR}/policy.zip"
# 1. Collect DAgger demos: BC drives, teacher labels (privileged + HERDING_WEBOTS).
python -m training.bc.collect \
--teacher universal --out "$DAGGER_DEMOS" \
--seeds-per-n "$SEEDS" --subsample 3 \
--frame-stack 4 --drive-mode "$DRIVE" --world "$WORLD" \
--max-steps 30000 \
--privileged --use-webots-preset \
--fp-rate 0.0 --action-smooth 0.55 --wheel-slip-std 0.05 \
--dagger-policy "$BC_DIR"
# 2. Concatenate original demos + dagger demos.
python - <<PY
import numpy as np
orig = np.load("${ORIG_DEMOS}")
dag = np.load("${DAGGER_DEMOS}")
obs = np.concatenate([orig["obs"], dag["obs"]], axis=0)
act = np.concatenate([orig["actions"], dag["actions"]], axis=0)
np.savez("${COMBINED_DEMOS}", obs=obs, actions=act,
meta=np.concatenate([orig["meta"], dag["meta"]], axis=0))
print(f"[combine] orig={orig['obs'].shape[0]} + dagger={dag['obs'].shape[0]} = {obs.shape[0]}")
PY
# 3. Re-train BC on combined demos.
python -m training.bc.pretrain \
--demos "$COMBINED_DEMOS" --out "$OUT_DIR" \
--epochs "$EPOCHS" --net-arch 512,512
echo "=== DAgger round ${ROUND} done: ${OUT_DIR}/policy.zip ==="
tools/run_webots.sh
+20 -17
View File
@@ -38,12 +38,12 @@ MODE=${2:-bc}
DRIVE=${3:-differential}
WORLD=${4:-field}
if (( N < 1 || N > 10 )); then
echo "N must be 1..10, got $N" >&2; exit 1
if (( N < 0 || N > 10 )); then
echo "N must be 0..10, got $N" >&2; exit 1
fi
case "$MODE" in
bc|rl|strombom|sequential|universal) ;;
*) echo "MODE must be bc|rl|strombom|sequential|universal, got '$MODE'" >&2; exit 1 ;;
bc|rl|strombom|sequential|universal|calibrate) ;;
*) echo "MODE must be bc|rl|strombom|sequential|universal|calibrate, got '$MODE'" >&2; exit 1 ;;
esac
case "$DRIVE" in
differential|mecanum) ;;
@@ -83,29 +83,31 @@ cp "$SRC" "$DST"
if [[ "$DRIVE" == "mecanum" ]]; then
sed -i 's|"../protos/ShepherdDog.proto"|"../protos/ShepherdDogMecanum.proto"|' "$DST"
sed -i 's|^ShepherdDog {|ShepherdDogMecanum {|' "$DST"
# Inject mecanum contact properties after the existing contactProperties block.
# Inject mecanum contact properties into the contactProperties array.
# Strategy: find the closing ' ]' that ends the contactProperties block
# (it sits at 2-space indent, immediately before the WorldInfo closing brace)
# and insert just before it.
python3 -c "
import re, sys
with open(sys.argv[1], 'r') as f:
with open('$DST', 'r') as f:
txt = f.read()
# Find the closing ']' of contactProperties and insert before it.
mec = '''
ContactProperties {
mec = ''' ContactProperties {
material1 \"MecanumWheel\"
coulombFriction [
2
1.0
]
bounce 0
forceDependentSlip [
10
0.01
]
softCFM 0.0001
}'''
# Insert before the first ']' that closes contactProperties [...]
txt = re.sub(r'(contactProperties\s*\[[^\]]*)(\])', r'\1' + mec + r'\2', txt, count=1)
with open(sys.argv[1], 'w') as f:
}
'''
# The contactProperties array closes with ' ]\n}' (2-space indent ] then WorldInfo }).
# Insert the new block just before that closing ].
txt = txt.replace('\n ]\n}', '\n' + mec + ' ]\n}', 1)
with open('$DST', 'w') as f:
f.write(txt)
" "$DST"
"
fi
# Comment out sheep N+1..10 by prefixing the matching Sheep { ... } line.
@@ -129,6 +131,7 @@ HERDING_MODE=$MODE
HERDING_POLICY_DIR=$RESOLVED_POLICY_DIR
HERDING_DRIVE=$DRIVE
HERDING_WORLD=$WORLD
HERDING_USE_GT=${HERDING_USE_GT:-0}
EOF
export HERDING_MODE="$MODE"
tools/webots_sweep.sh
Executable
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env bash
# Headless Webots sweep across modes, drives, worlds, and flock sizes.
# Runs sequentially; each trial gets a hard 150s wall-clock timeout.
# Results are written to webots_sweep.log (tab-separated) and printed.
#
# Usage: bash tools/webots_sweep.sh [output_log]
set -euo pipefail
ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )"
OUT="${1:-$ROOT/webots_sweep.log}"
TIMEOUT_S=120 # ~80k steps in fast headless mode — covers slow-converging physics
# Webots uses its own python3; put the conda env first so all deps resolve.
export PATH="/home/jalf/miniconda3/envs/tir/bin:$PATH"
# Columns: mode drive world n_sheep success steps
printf "%-12s %-14s %-12s %7s %7s %s\n" \
"mode" "drive" "world" "n_sheep" "success" "steps" | tee "$OUT"
printf '%s\n' "$(printf '%-12s %-14s %-12s %7s %7s %s' \
'----' '-----' '-----' '-------' '-------' '-----')" | tee -a "$OUT"
run_trial() {
local mode="$1" drive="$2" world="$3" n="$4" policy_dir="${5:-}"
local done_file="$ROOT/training/.run_done"
rm -f "$done_file"
local extra_env=()
extra_env+=(WEBOTS_HEADLESS=1)
extra_env+=(WEBOTS_EXTRA_ARGS="--stdout --stderr")
extra_env+=(HERDING_USE_GT=0)
if [[ -n "$policy_dir" ]]; then
extra_env+=(HERDING_POLICY_DIR="$ROOT/$policy_dir")
fi
local raw
raw=$(
timeout --kill-after=15s "$TIMEOUT_S" \
xvfb-run -a \
env "${extra_env[@]}" \
bash "$ROOT/tools/run_webots.sh" "$n" "$mode" "$drive" "$world" \
2>&1
) || true
# Webots-bin and Xvfb can survive the timeout; kill any orphans now.
pkill -9 -f "webots-bin|Xvfb" 2>/dev/null || true
sleep 1
local success="FAIL"
local steps="timeout"
if echo "$raw" | grep -q "\[dog\] all .* sheep penned at step"; then
success="OK"
steps=$(echo "$raw" | grep "\[dog\] all .* sheep penned at step" \
| grep -oP 'step \K[0-9]+')
fi
printf "%-12s %-14s %-12s %7s %7s %s\n" \
"$mode" "$drive" "$world" "$n" "$success" "$steps" | tee -a "$OUT"
}
# ---------------------------------------------------------------------------
# Analytic baselines (differential only — that's the story context)
# strombom / sequential: canonical baselines
# universal: the actual teacher used to collect BC demos
# ---------------------------------------------------------------------------
for mode in strombom sequential universal; do
for world in field field_round; do
for n in 5 10; do
run_trial "$mode" differential "$world" "$n"
done
done
done
# ---------------------------------------------------------------------------
# BC — world-specific policies
# ---------------------------------------------------------------------------
for drive in differential mecanum; do
for world in field field_round; do
for n in 5 10; do
run_trial bc "$drive" "$world" "$n" \
"training/runs/bc_${drive}_${world}"
done
done
done
# ---------------------------------------------------------------------------
# RL_FAST — MODE=rl with explicit HERDING_POLICY_DIR pointing to rl_fast dirs
# (run_webots.sh rejects "rl_fast" as a mode; "rl" + policy override is correct)
# ---------------------------------------------------------------------------
for drive in differential mecanum; do
for world in field field_round; do
for n in 5 10; do
run_trial rl "$drive" "$world" "$n" \
"training/runs/rl_fast_${drive}_${world}"
done
done
done
echo ""
echo "Sweep complete. Results saved to: $OUT"
tools/webots_sweep_gt.sh
Executable
+100
View File
@@ -0,0 +1,100 @@
#!/usr/bin/env bash
# Headless Webots sweep across modes, drives, worlds, and flock sizes.
# Runs sequentially; each trial gets a hard 150s wall-clock timeout.
# Results are written to webots_sweep.log (tab-separated) and printed.
#
# Usage: bash tools/webots_sweep.sh [output_log]
set -euo pipefail
ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )"
OUT="${1:-$ROOT/webots_sweep.log}"
TIMEOUT_S=120 # ~80k steps in fast headless mode — covers slow-converging physics
# Webots uses its own python3; put the conda env first so all deps resolve.
export PATH="/home/jalf/miniconda3/envs/tir/bin:$PATH"
# Columns: mode drive world n_sheep success steps
printf "%-12s %-14s %-12s %7s %7s %s\n" \
"mode" "drive" "world" "n_sheep" "success" "steps" | tee "$OUT"
printf '%s\n' "$(printf '%-12s %-14s %-12s %7s %7s %s' \
'----' '-----' '-----' '-------' '-------' '-----')" | tee -a "$OUT"
run_trial() {
local mode="$1" drive="$2" world="$3" n="$4" policy_dir="${5:-}"
local done_file="$ROOT/training/.run_done"
rm -f "$done_file"
local extra_env=()
extra_env+=(WEBOTS_HEADLESS=1)
extra_env+=(WEBOTS_EXTRA_ARGS="--stdout --stderr")
extra_env+=(HERDING_USE_GT=1)
if [[ -n "$policy_dir" ]]; then
extra_env+=(HERDING_POLICY_DIR="$ROOT/$policy_dir")
fi
local raw
raw=$(
timeout --kill-after=15s "$TIMEOUT_S" \
xvfb-run -a \
env "${extra_env[@]}" \
bash "$ROOT/tools/run_webots.sh" "$n" "$mode" "$drive" "$world" \
2>&1
) || true
# Webots-bin and Xvfb can survive the timeout; kill any orphans now.
pkill -9 -f "webots-bin|Xvfb" 2>/dev/null || true
sleep 1
local success="FAIL"
local steps="timeout"
if echo "$raw" | grep -q "\[dog\] all .* sheep penned at step"; then
success="OK"
steps=$(echo "$raw" | grep "\[dog\] all .* sheep penned at step" \
| grep -oP 'step \K[0-9]+')
fi
printf "%-12s %-14s %-12s %7s %7s %s\n" \
"$mode" "$drive" "$world" "$n" "$success" "$steps" | tee -a "$OUT"
}
# ---------------------------------------------------------------------------
# Analytic baselines (differential only — that's the story context)
# strombom / sequential: canonical baselines
# universal: the actual teacher used to collect BC demos
# ---------------------------------------------------------------------------
for mode in strombom sequential universal; do
for world in field field_round; do
for n in 5 10; do
run_trial "$mode" differential "$world" "$n"
done
done
done
# ---------------------------------------------------------------------------
# BC — world-specific policies
# ---------------------------------------------------------------------------
for drive in differential mecanum; do
for world in field field_round; do
for n in 5 10; do
run_trial bc "$drive" "$world" "$n" \
"training/runs/bc_${drive}_${world}"
done
done
done
# ---------------------------------------------------------------------------
# RL_FAST — MODE=rl with explicit HERDING_POLICY_DIR pointing to rl_fast dirs
# (run_webots.sh rejects "rl_fast" as a mode; "rl" + policy override is correct)
# ---------------------------------------------------------------------------
for drive in differential mecanum; do
for world in field field_round; do
for n in 5 10; do
run_trial rl "$drive" "$world" "$n" \
"training/runs/rl_fast_${drive}_${world}"
done
done
done
echo ""
echo "Sweep complete. Results saved to: $OUT"