# DESIGN — topic-agnostic skill birth

The decisions below are settled. `interfaces.py` is the frozen contract; this file explains the
*why* and the *how* so three builders implement in parallel without talking. Read
`README.md` for the claim, hypothesis, family plan, and G1–G11 bars.

## 0. The one rule that makes this hard

The decision path may inspect a fixture's DATA and its I/O SHAPE, and may call the fixture's
CHECKER. It may not read the fixture's topic identity. In code: the engine never touches
`TaskFamily.name` or `TaskFamily.expect`, and never compares any value against a topic word.
Everything topic-specific is *data supplied by the family* (examples, checker, negatives,
seeds, probe pool) or a *born skill after adoption*. The engine is the S036 universal core,
generalized from "primitives a domain declares" to "generic operators instantiated over
observed shape."

## 1. Naming the missing skill + writing the standard, deterministically, no model

**Shape inference (`shape(value) -> str`, structural, topic-blind).** `None→"null"`;
`bool/int/float→"scalar"`; `str→"text"`; `list/tuple→"seq[" + "|".join(sorted distinct element
shapes) + "]"`; `dict→"record<n=K>"` (arity K, NOT keys — keys are content and would leak). The
in/out shape of a family is `shape` of the first example's input/answer, asserted stable across
all examples (else `MalformedFamilyError`).

**Name (step 3).** `skill_{in_tag}_to_{out_tag}_{h8}` where `in_tag/out_tag` are the shape tags
with non-alnum stripped, and `h8 = sha256(canonical(examples)).hexdigest()[:8]`, `canonical` =
`json.dumps(sorted example pairs, sort_keys=True, default=repr)`. Deterministic, carries no
topic word, unique per example set (S047 precedent: hash/shape-derived naming is legitimate).

**Standard (steps 4-5, `write_standard`).** Deterministic templating over observed COUNTS +
shape tags (S047: derived templating over I/O shape is "plain English"). `plain_english` reads,
e.g.: *"Skill {name}: {n_train} worked examples each map a {in_shape} to a {out_shape}. A
candidate is correct iff it (a) reproduces every training answer, (b) reproduces the family
oracle on {n_holdout} held-out inputs, (c) never returns a wrong confident answer on {n_adv}
adversarial inputs, (d) matches none of the {n_neg} negative rules, and (e) is the unique
surviving behavior on a discriminating hold-out. Adopt iff exactly one candidate meets (a)-(e);
otherwise refuse and say what is missing."* `success_rule`, `refusal_conditions`,
`contradiction_check` are the machine-checkable decomposition. RECORDED before evaluate() (G3).

## 2. The generic-operator algebra + how the checks compose into adopt/refuse

**Operators (the 10, `GENERIC_OPERATORS`).** Each is a factory `params -> (spec, program)`.
Params are ENUMERATED from observed shape, never from a topic label:
- `normalize(scheme)` — scheme ∈ fixed enum `{casefold, strip, casefold+strip, sort, dedup,
  sort+dedup}`. Applies to text or seq.
- `map(scheme)` — a normalize scheme applied elementwise over a seq/text.
- `extract(selector)` — selector ∈ {observed dict keys} ∪ {observed seq indices} ∪ {split on an
  observed delimiter char present in the inputs, take index i}. Keys/indices/delimiters are
  READ FROM THE DATA (data-driven ≠ topic branch).
- `lookup(table[, field])` — table induced from train `(input-atom → output-atom)`; optionally
  applied after an `extract`/`normalize` (this is where reuse enters, §4).
- `classify(field, threshold|membership)` — label set = distinct answers; rule = compare an
  observed numeric field to a threshold drawn from observed values, or set-membership.
- `reduce(agg)` — agg ∈ `{sum,count,min,max,join,first,last}` over a seq.
- `filter(pred)` / `compare(op)` — generic predicate / relational over observed fields.
- `compose(g,h)` and `sequence([...])` — draw atoms from ADOPTED skills + one generic op
  (the compounding search space; empty until skills exist, exactly like S036
  `_composition_candidates`).

**enumerate_candidates** = primitive instantiations (params from shape) ++ compositions over the
registry, in a FIXED sorted order. `len(...)` is the honest `proposals` count (F001; count ALL,
not post-ranking — F001 is the trap we avoid).

**The decision (`evaluate`, one path for every family):**
1. `train_consistent = [c for c in candidates if all(checker(x, c(x)) for x,_ in examples)]`
   — propose from TRAIN ONLY (G5). (`checker` is the family oracle, F003; equivalently match
   the recorded answer.) Count a `train_check` per candidate.
2. Drop any `c` whose behavior on the probe pool equals a `NegativeRef` behavior
   (`negative_checks`) — kills coincidental train-fit that is actually a known-wrong rule.
3. `survivors = [c for c in train_consistent if passes_holdout(c) and passes_adversarial(c)]`.
   `passes_holdout` = `checker(x, c(x))` on every held-out pair (`holdout_checks`).
   `passes_adversarial` = on each adversarial input, `c` either returns a checker-approved
   answer OR refuses (raises/returns the sentinel) — it must NOT return a confident WRONG
   answer (`adversarial_checks`). A candidate that crashes or wrong-answers on garbage is out.
4. `contradiction_check` (soundness, `contradiction_checks`): re-run step 1 on
   `examples + [poisoned]` where `poisoned` = one example with its answer swapped to a different
   observed answer. Assert the outcome is a REFUSE (0 or the loop cannot uniquely determine) —
   the loop must never silently pick under contradiction. Recorded, not adopted from.
5. **Unique determination (S035/S036 sealed rule):** group survivors by BEHAVIOR = their
   output tuple on the discriminating pool (`holdout` inputs ∪ `probe_pool`).
   - `0` survivors → `RefusalRequest(REACH_LIMIT, …)` (need an operation I don't have).
   - `>1` distinct behaviors → underdetermined: find the single probe input where behaviors
     disagree (S047 `pick_discriminating_unit`) → `RefusalRequest(UNDERDETERMINED, missing_input,
     disagreeing_values)`; if no probe separates them → `INDISTINGUISHABLE`.
   - exactly `1` behavior → require the holdout be DISCRIMINATING (the FULL candidate set yields
     ≥2 distinct behaviors on it — informative regime, S036 `learn`); if not → refuse
     (uninformative holdout). Else ADOPT the survivor.

This is `node_core.engine.learn` / `node_core.loop.learn_unique` generalized: same
0/1/>1 unique-determination skeleton, richer candidate space + adversarial/negative/contradiction
gates layered on. Builder A MAY import node_core for reference primitives but MUST NOT modify it
(G11) and MUST NOT rely on it for the new operator algebra.

## 3. Adversarial + contradiction generation: engine-generic vs family-supplied

**Engine-generated (topic-agnostic, from shape only — `generate_adversarial(shape, seeds)`):**
- shape mutations: reorder / duplicate / drop an element; swap two record fields.
- boundary: empty (`[] / "" / {}`), singleton, longest-seen.
- garbage: type-mismatched injection (a scalar where a seq is expected — derived from the shape
  mismatch, not a topic), a `None` field.
- contradiction: the poisoned-example construction in §2.4 — mechanical, from observed answers.

**Family-supplied (topic-specific — lives ONLY in `families.py`):** the checker/oracle (F003);
`adversarial_seeds` (a few genuinely hard inputs the mutator can't invent — data only, still
decided generically); `negatives` (plausible-wrong rules as programs); `holdout` with oracle
answers; `probe_pool`. The engine's DECISION never branches on any of these being present.

## 4. The compounding path (registry-as-operator-source) — family #4

After an adoption, `enumerate_candidates` includes `lookup(table)` composed over adopted skills:
`sequence([born_skill, generic_op])` and `compose(born_i, born_j)`. Family #4 (label mapping,
`record→text`) is constructed so its ONLY surviving candidate is `lookup ∘ norm_casetrim` — the
lookup key must be case-folded/trimmed (via the skill born in family #1) before the table hits,
so no atomic operator fits and no composition fits UNLESS `norm_casetrim` is already in the
registry. Run with an empty registry the family REFUSES (reach limit); run in the fixed order it
ADOPTS with `deps=("skill_text_to_text_…",)` and a `reuse_edge` is recorded (G7, mirrors S036-G2
`n_consonants=sub(count_letters,count_vowels)` unreachable-before-prereqs, and S051 compounding).
Builder C verifies reach-before by calling `birth` on family #4 against a FRESH engine.

## 5. The refusal case — family #5 (log/file extraction), and the discriminating request

Family #5's examples are consistent with ≥2 distinct generic rules that AGREE on every example
but DISAGREE on some unseen input — e.g. `extract(split(" "), index=1)` vs `extract(split(":"),
index=1)`. The engine reaches "exactly the underdetermined branch," then searches `probe_pool`
for the single input where the surviving behaviors differ and emits: *"I can't tell yet — your
examples fit these {k} rules: …. To decide, give me the answer for {missing_input}; those rules
disagree there (they would give {disagreeing_values})."* Builder B MUST seed `probe_pool` with
at least one separating input so the request is concrete (recoverable gap, not a dead end). The
refusal is verifiable: supplying that one label collapses to the unique rule (S047 resolve
round-trip — optional to demonstrate; the bar G6 only needs the exact request stated).

## 6. Determinism (G10) — the rules every builder follows

Zero RNG, zero wall-clock, zero `hash()` (use `hashlib`). Every enumeration sorted before use;
every set converted to a sorted list before it enters a returned value or `results.json`.
`json.dump(..., sort_keys=True)`. Program closures are rebuilt from specs, never pickled. Any
cross-run difference is a `NonDeterminismError`, raised loud.

## 7. File ownership — three builders, no collisions

Each builder owns their file exclusively and imports the frozen `interfaces.py`. No builder edits
another's file or `interfaces.py`. The two seams (engine↔harness, engine↔families) are both fully
specified in `interfaces.py`, so the builders never need to coordinate.

| file | owner | contents | may import | must NOT |
|---|---|---|---|---|
| `interfaces.py` | (frozen; nobody edits) | dataclasses, enums, engine API, ledger schema, typed errors, pinned shas | stdlib | — |
| `skill_birth.py` | **Builder A** | the engine: shape inference, naming, standard templating, the 10 operators + enumeration, `evaluate` decision, adversarial/contradiction generation, adoption, the F001 ledger | `interfaces`, stdlib, (read-only) `node_core` | read `family.name`/`.expect`; any topic-word branch; RNG/clock/`hash()`; write outside this dir |
| `families.py` | **Builder B** | all 6 `TaskFamily` fixtures per the README plan: examples, holdout, adversarial_seeds, negatives, probe_pool, checker, expect, name. `all_families() -> List[TaskFamily]`. Topic-specific data ONLY | `interfaces`, stdlib | any engine/decision logic; import `skill_birth`; branch on shape |
| `run_skill_birth.py` | **Builder C** | the G1–G11 harness: run `birth` over `all_families()` in order; mechanical G1 AST/token scan of `skill_birth.py`; G7 reach-before check on a fresh engine; G8 confident-error audit via each family checker; G9 ledger reconciliation; G10 double-run byte-identity; G11 sha check + no-external-write check; write `results.json` deterministically | `interfaces`, `skill_birth`, `families`, stdlib, `ast`, `hashlib` | edit A's or B's files; move a bar |

`results.json` is written by Builder C only. `README.md`/`DESIGN.md`/`interfaces.py` already
exist; builders do not recreate them.

## 8. What a builder could misread (read this twice)

- **The reporting label is passed SEPARATELY.** `birth(family, label)` — the engine body uses
  `family.*` data but takes the human label as `label`, so `family.name` is never read inside the
  decision. Builder A: put `label` into `BirthOutcome.family` at return time; never read
  `family.name`. Builder C: pass `family.name` as `label`.
- **Propose from TRAIN ONLY** (G5). Holdout/adversarial/negatives are validation + discrimination,
  never candidate generation. Mixing them in silently breaks G5 and inflates confidence.
- **`proposals` = full enumerated candidate count**, not survivors, not post-ranking (F001/F003).
  Count every candidate you build and every check you run. A sweep counted only after ranking is
  exactly the F001 defect.
- **Data-driven params are allowed; topic branches are not.** Enumerating over observed dict keys,
  seq indices, delimiters, label sets, thresholds is fine (it reads data). `if "amount" in row`
  or `if family.name == …` is a topic branch and fails G1.
- **Adversarial "pass" = approved-answer OR refuse, never a confident wrong answer.** A candidate
  that returns a wrong value on garbage must be eliminated; one that refuses on garbage is fine.
- **Negatives kill coincidental fit.** A candidate matching a `NegativeRef` on the probe pool is a
  wrong rule that happens to fit train — drop it even if it "passed" train.
- **Reuse must be genuine** (G7): family #4's survivor `deps` must NON-trivially name a born skill,
  and the SAME family on a fresh empty registry must REFUSE (reach-before). A composition that
  would also be reachable without the prereq is not compounding.
- **Determinism beats convenience:** sort every set, no `hash()`, rebuild programs from specs.
