# Prop-01 E1 §5 success-criteria scorecard

| Field | Value |
| --- | --- |
| **Protocol** | `LAX-PROP-2026-09-04-01a` (active); §5 thresholds frozen from `LAX-PROP-2026-09-04-01` |
| **Artifact** | `sims/sr-ladder/results/prop01_live_e1_full.json` + `.md` |
| **Grid** | E1-reduced full: 8 laws × 20 seeds × N∈{50,100,400} × σ∈{0.0,0.05} × {A0_pysr, A4_physo} = **1920** cells |
| **partial** | `false` (1920/1920) |
| **Soft-pad** | OFF (`mean_soft_pad_s=0.0`); N=200 dropped vs STLS {50,100,200,400} |
| **prop01_support_claim (artifact)** | `false` — **not flipped by this scorecard** |
| **Scored** | 2026-09-05 (America/New_York) from on-disk aggregates only |

**Sources cited:** proposal §5 in `proposals/2026-09-05-prop-01a-constraint-aware-symbolic-regression.md` (mirrors frozen §5 in `proposals/2026-09-04-prop-01-constraint-aware-symbolic-regression.md`); metrics lock `sims/sr-ladder/methods_metrics.md` (τ=0.2, R\*=0.25, C\*=0.5).

---

## Preconditions for any support claim (protocol §5 / 01a banner)

Declare support only on **Track Lit-backend** with A0=PySR, A4=PhySO, **matched compute**, **≥20 seeds**.

| Check | Measured | Status |
| --- | --- | --- |
| Lit backends | `arms`: A0_pysr (PySR 2.2.1), A4_physo (PhySO 1.2.0); both `available: true` | Met |
| ≥20 seeds | `seeds` = 0..19 → **20** | Met |
| Matched wall | `matched_wall.arm_means_within_band` = **true**; A0 mean wall **13.474…** s, A4 **12.808…** s, `arm_means_rel_gap` **0.0494…** (band ±0.2); paired within-band **780/960** (`frac_paired_within_band` **0.8125**) | Met (arm-means band) |
| Soft-pad / N grid | Soft-pad OFF; N grid **{50,100,400}** only (no 200) | Noted — limits sample-efficiency resolution |
| Cell accept/fail | **1678** accept / **242** fail; A0 **934/960** accept, A4 **744/960** accept; A4 zero-eq cells **0** | Context only (not a §5 gate) |

Preconditions alone do **not** authorize flipping `support_claim`; all four §5 gates must clear with measured evidence.

---

## Gate 1 — Sample efficiency (§5 criterion 1)

**Criterion (proposal §5):** Full-constraint arm (A4 / PhySO) achieves recovery-rate ≥ **0.8** at an \(N\) that is **≤ 0.5×** the unconstrained arm’s (A0 / PySR) \(N\) for ≥ **5 of 7** recoverable systems (median across seeds).

**How computed:** From `aggregate[]`, per recoverable law at σ=0.0 (clean): smallest `n_samples` with `recovery_rate ≥ 0.8` over `n_seeds=20`; then test \(N_{A4} ≤ 0.5\,N_{A0}\). Recoverable laws = all except `dim_violating`.

| Law | A0 min \(N\) @ ≥0.8 | A4 min \(N\) @ ≥0.8 | Ratio / note | Counts toward ≥5/7? |
| --- | ---: | ---: | --- | --- |
| `linear_decay` | 50 (rate 1.000) | 50 (rate 1.000) | 50/50 = 1.0 > 0.5 | **No** |
| `harmonic_oscillator` | 50 (1.000) | 50 (0.950) | 1.0 > 0.5 | **No** |
| `damped_oscillator` | 50 (0.950) | **none** (A4 rates all 0.000) | A4 never ≥0.8 | **No** |
| `lotka_volterra` | **none** (0.000 all N) | **none** (0.000) | neither ≥0.8 | **No** |
| `diffusion_1d` | **none** (A0 0.000 all N) | 50 (1.000) | A0 \(N\) undefined on grid; A4 hits 0.8 | **Not counted as clear pass** (ratio Incomplete) |
| `burgers_1d` | **none** (peak A0 0.450) | **none** (peak A4 0.550) | neither ≥0.8 | **No** |
| `gravity_toy` | 50 (1.000) | 50 (0.950) | 1.0 > 0.5 | **No** |

**Systems clearly meeting ≤0.5×:** **0 / 7**. At most one Incomplete efficiency-gain case (`diffusion_1d`). Required: **≥5 / 7**.

**Verdict: Fail**

---

## Gate 2 — Noise robustness (§5 criterion 2)

**Criterion (proposal §5):** Under the mid noise schedule, A4 recovery rate ≥ A0 recovery rate − **0.05** (no systematic rejection of correct laws).

**How computed:** Mid noise = `noise_sigma_rel` **0.05**. Pooled over recoverable laws × N × 20 seeds from `records` (also equals mean of `aggregate` recovery_rate rows): A0 **0.4167** (420 cells), A4 **0.5452** (420 cells). Per `(law, N)` from `aggregate`.

| Pool | A0 recovery | A4 recovery | A4 ≥ A0 − 0.05? |
| --- | ---: | ---: | --- |
| All recoverable × {50,100,400} @ σ=0.05 | **0.4167** | **0.5452** | **Yes** (0.5452 ≥ 0.3667) |

**Per-(law,N) failures (A4 < A0 − 0.05):** all three `damped_oscillator` mid cells — A0 recovery **0.650 / 0.700 / 0.850** vs A4 **0.000 / 0.000 / 0.000** (deltas **−0.650 / −0.700 / −0.850**). Other 18/21 `(law,N)` pairs pass the numeric margin.

**Parenthetical (“no systematic rejection”):** `damped_oscillator` shows A4 recovery identically 0 under mid noise while A0 recovers majority–high rates → systematic rejection on a recoverable law.

**Verdict: Fail** (pooled mean clears −0.05, but criterion’s no-systematic-rejection clause fails on measured `damped_oscillator` mid-noise rows)

---

## Gate 3 — Clean fail (§5 criterion 3)

**Criterion (proposal §5):** On constraint-violating generators, A4 clean-fail rate ≥ **0.9** while residual gate \(R^*\) is held fixed from recoverable systems.

**How computed:** Violator = `dim_violating`. Clean-fail in this artifact = fraction of seeds with **not** `accepted` (R ≥ R\* = 0.25; see `methods_metrics.md` / runner aggregate). From `records`: A4 × `dim_violating` = **120** cells; clean-fail count = not accepted → rate **0.7833**. From `aggregate` A4 violator rows:

| σ | N | A4 `clean_fail_rate` | A4 `accept_rate` |
| ---: | ---: | ---: | ---: |
| 0.0 | 50 | 0.750 | 0.250 |
| 0.0 | 100 | 0.800 | 0.200 |
| 0.0 | 400 | 0.750 | 0.250 |
| 0.05 | 50 | 0.800 | 0.200 |
| 0.05 | 100 | 0.750 | 0.250 |
| 0.05 | 400 | 0.850 | 0.150 |

Mean of those six rates = **0.7833**; **min = 0.750**; **max = 0.850**. **None ≥ 0.9.** Metrics lock confirms R\* = 0.25 (`metrics_locked.R_star`).

**Verdict: Fail**

---

## Gate 4 — Non-triviality (§5 criterion 4)

**Criterion (proposal §5):** Conservation-only ablation does **not** achieve success criteria (1)–(2) via constant/trivial identities alone (residual on dynamics still required).

**How computed:** Requires a conservation-only arm (protocol A3). This E1 artifact `arms` = **[`A0_pysr`, `A4_physo`] only** — no A3 / conservation-only ablation on lit backends.

**Verdict: Incomplete** (cannot evaluate from E1-reduced full 1920 results)

---

## Overall: can `support_claim` flip?

| Gate | Verdict |
| --- | --- |
| §5.1 Sample efficiency | **Fail** |
| §5.2 Noise robustness | **Fail** |
| §5.3 Clean fail | **Fail** |
| §5.4 Non-triviality | **Incomplete** |
| **Flip `prop01_support_claim` to true?** | **No** |

**Conclusion:** `support_claim` **must remain false**. Protocol 01a / §5 requires **all** gates to hold on lit backends with matched compute and ≥20 seeds. This run meets the lit-backend / seed / arm-mean wall-band preconditions and finishes the 1920-cell grid (`partial=false`), but measured recovery curves do not deliver ≥5/7 systems with A4 reaching 0.8 at ≤0.5× A0’s \(N\); mid-noise shows systematic A4 rejection on `damped_oscillator`; A4 clean-fail on `dim_violating` is **0.7833 &lt; 0.9**; and conservation-only non-triviality was not run. Completing the grid is not §5 success. **Do not flip `prop01_support_claim`.**

---

## Recommended next experiment (concrete)

**E1b — sample-efficiency + clean-fail + A3 ablation patch (lit backends only):**

1. **Expand N grid** to include **N∈{25, 50, 100, 200, 400}** (restore 200; add 25) under the same matched wall-band (±20%) with soft-pad policy documented, still ≥20 seeds, A0=PySR / A4=PhySO — so ≤0.5× ratios are identifiable when both arms already saturate at N=50.
2. **PhySO damped_oscillator repair lane:** increase A4 budget (epochs / `max_n_evaluations`) under the **same** wall-cap matching rule; re-measure mid-noise recovery specifically on `damped_oscillator` until either A4 ≥ A0 − 0.05 per cell or document irreducible fail vs kill criterion (drop &gt; 0.2).
3. **Clean-fail push on `dim_violating`:** keep R\*=0.25 fixed; harden A4 acceptance / units path so A4 clean-fail ≥ 0.9 across the violator grid (target measured rate ≥0.9, not accept-rate cosmetics).
4. **Add A3 conservation-only lit or labeled STLS-surrogate ablation** solely to close §5.4 (surrogate never flips support; lit A3 preferred if PhySO partial mode exists).

Re-score this same scorecard template only after E1b artifacts land; flip `support_claim` **only** if gates 1–4 all **Pass** with measured numbers.

---

## Top failing / incomplete gates (summary)

1. **§5.1 Sample efficiency — Fail** (0/7 clear ≤0.5× passes; need ≥5/7)
2. **§5.3 Clean fail — Fail** (A4 violator clean-fail **0.7833** &lt; **0.9**)
3. **§5.2 Noise robustness — Fail** (`damped_oscillator` mid-noise A4 recovery **0** vs A0 **0.65–0.85**) — with **§5.4 Incomplete** as runner-up (no A3 arm in this grid)