What the evidence does not reach

Reliability is not replication

Cross-model agreement establishes that the coding is stable. It says nothing about whether the coded thing is real. These are separate properties, and totalling them as though they were the same currency is the single cleanest attack available to a reviewer.

What was done

The analysis was assigned as a disconfirmation task across separate frontier-model families — twenty-two runs across nine families. Operators were instructed to identify coding instability, alternative orderings, null constructions, cultural dependencies, and statistical weaknesses capable of eliminating the result. Repeated attempts to remove the structure under the stated rules did not succeed.

On the content instrument, cross-model agreement is approximately r = .92.

Why that is not replication

These runs establish cross-model reliability — that the coding is stable and not idiosyncratic to a single architecture. They are not independent replication, and the paper does not treat them as such. Frontier models share training corpora and architectural priors.

Every model coding these narratives has been trained on the near-death literature, including stage models published in the 1970s that describe substantially this ordering. Twenty-two runs across nine families can test whether the coding is stable across architectures. They cannot test whether a shared prior is generating it, because the prior is shared. A no-rubric result is consistent with the elements being natural kinds and equally consistent with the models applying an internalized template; it does not discriminate between them.

The test that does discriminate, and what it returned

Blind human coders are the only design that separates structure-in-the-text from template-in-the-model. Trained coders, insulated from the desired result and governed by thresholds fixed in advance, either recover the same elements and the same order or they do not.

That panel has now reported. Five reviewers, blind to the predicted order and to all machine labels, recorded 168 forward transitions against 9, a forward share of 94.9% against a null of 0.50, and agreed on the content classification at 0.81. The minimum results had been specified in advance, and were met. The thresholds, and what they were set to →

This is what moves the result past cross-model reliability. Twenty-two runs across nine model families could only ever establish that the coding was stable. Reproduction by operators who are not models, and who could not see what was expected, is a different kind of evidence — and it is what Level 5 asks for.

The same correction at table scale

The distinction is applied to the author's own program as strictly as to anyone else's. Two instruments built by the same author, run by the same operator, on corpora drawn from the same literature, produce convergent internal evidence — not independent corroboration. The standard is applied strictly against its author's own work, and no level on the published ladder is claimed on the strength of that convergence.

Until independent replication exists, the order result rests on the primary coded corpus, the held-out validation with the order frozen, the null testing, and the adversarial cross-model analysis — and on the blind human panel that has now reported.