APEX Daily Pulse — 2026-11-13
All Passed Is Not All Tested
[✅ Verified] Pillar: Validation and assurance
[❓ Unverified] Estimated duration: 3–5 minutes
Reactivation target: Reject a green summary when required probe results are absent or duplicated.
Prerequisites
- Python 3.12+ and a Bash-compatible terminal.
- Owned equipment; synthetic probe results only.
- No privileges, network access, or external dependencies.
Actions
- 1. Declare the required probe identities before processing results. Run this complete fixture:
python3 - <<'PY'
from collections import Counter
from typing import TypedDict
class Result(TypedDict):
probe_id: str
passed: bool
required = Counter({"P1": 1, "P2": 1})
cases: dict[str, list[Result]] = {
"baseline": [
{"probe_id": "P1", "passed": True},
{"probe_id": "P2", "passed": True},
],
"empty": [],
"missing": [
{"probe_id": "P1", "passed": True},
],
"duplicate": [
{"probe_id": "P1", "passed": True},
{"probe_id": "P1", "passed": True},
],
"failed": [
{"probe_id": "P1", "passed": True},
{"probe_id": "P2", "passed": False},
],
}
def guarded_gate(results: list[Result]) -> bool:
"""Require a nonempty plan and exactly one passing result per probe."""
observed = Counter(row["probe_id"] for row in results)
return (
bool(required)
and observed == required
and all(row["passed"] is True for row in results)
)
observed_decisions: dict[str, bool] = {}
for name, results in cases.items():
naive = all(row["passed"] for row in results)
guarded = guarded_gate(results)
observed_decisions[name] = guarded
print(
f"{name}: records={len(results)} "
f"naive={naive} guarded={guarded}"
)
expected_decisions = {
"baseline": True,
"empty": False,
"missing": False,
"duplicate": False,
"failed": False,
}
drill_passed = observed_decisions == expected_decisions
print(f"drill_passed={drill_passed}")
raise SystemExit(0 if drill_passed else 1)
PY
- 2. Compare the missing and duplicate cases. Both contain only passing results. The duplicate case even has the correct total record count. Neither proves coverage of P2.
- 3. Confirm that only the baseline passes the guarded gate. The other four cases are negative controls.
Observable signal
[❓ Unverified] Expected output; this fixture was not executed in this run:
baseline: records=2 naive=True guarded=True
empty: records=0 naive=True guarded=False
missing: records=1 naive=True guarded=False
duplicate: records=2 naive=True guarded=False
failed: records=2 naive=False guarded=False
drill_passed=True
[✅ Verified] Mechanism: Python's all() returns true for an empty iterable. For a nonempty iterable, it evaluates only the supplied results. It does not establish that required results exist. Comparing identity counts against an independent plan supplies that check.
Success condition
Only the complete, unique, all-passing baseline receives approval. Empty, missing, duplicate, and failed result sets are rejected. The fixture exits zero only when all five guarded decisions match their expected values.
Failure signature
[✅ Verified] empty: naive=True, missing: naive=True, or duplicate: naive=True followed by operational approval demonstrates that the gate checks reported outcomes without proving required coverage.
Python unavailable: POSIX awk alternative
Use a fixed two-probe plan and validate identity counts:
gate() {
awk '
{
total++
if (NF != 2 || ($1 != "P1" && $1 != "P2") || $2 != "PASS")
invalid = 1
seen[$1]++
}
END {
ok = !invalid && total == 2 && seen["P1"] == 1 && seen["P2"] == 1
print (ok ? "PASS" : "BLOCK")
exit(ok ? 0 : 1)
}'
}
printf '%s\n' 'P1 PASS' 'P2 PASS' | gate
printf '%s' '' | gate
printf '%s\n' 'P1 PASS' | gate
printf '%s\n' 'P1 PASS' 'P1 PASS' | gate
printf '%s\n' 'P1 PASS' 'P2 FAIL' | gate
Run the alternative in a shell without automatic exit-on-error; four nonzero statuses are intentional.
[❓ Unverified] Expected decisions, in order: PASS, BLOCK, BLOCK, BLOCK, BLOCK.
Boundary check
[✅ Verified] Hidden assumption: the required-probe plan is correct and independent of returned results. If the producer builds the plan from the same incomplete output, both can agree while coverage is missing. Bind the plan to the intended run and target. Keep the identity-count check in the approval wrapper, not in a manual review step.
Adaptation question
Which APEX gates report success from returned results without comparing them against an independent list of required probes?