Research Reliability
Publication outcomes
Development cohort ·
Frozen 75-paper RI1 development cohort. Publication yield measures selectivity, not scientific accuracy.
From selected paper to public brief
- 75Selected
- 69Normalized
- 44Classification admitted
- 25Reader generated
- 11Published
Stages are not additive. Of 25 generated readers, 11 were published and 14 remain private. Other selected papers retain their separate outcomes.
Other recorded outcomes and what they mean
These historical dispositions may overlap stage populations. They are not additional publications or a combined failure rate.
- Abstained
- 5
- Capacity held
- 25
- Failed
- 35
- UNKNOWN
- 9
- Reader generated
- A reader draft was retained. Generation alone does not establish eligibility for publication.
- Private Held
- A material evidence, source-quality, execution or content requirement remains unresolved. This does not establish that the source paper is scientifically wrong.
- Abstained
- The pipeline did not proceed with an analysis it could not support under the applicable route or evidence requirements.
- Capacity hold
- The requested stage could not proceed within its configured capacity limits.
- UNKNOWN
- The outcome or accounting is uncertain. No successful result or zero cost is inferred.
- Quarantined
- An output was retained privately because its applicable checks did not admit it.
- Material omission
- A missing finding or qualification would make the reader misleading.
- Wrong lens
- The selected research lens is not supported for the paper.
Standout research
Strongest evidence-grounded and reader-useful briefs, selected for prominent placement.
Useful, qualified analysis
Evidence-bound research with material conditions, limitations and source-quality caveats in view.
Withheld when requirements are unmet
Not released when material evidence, source-quality, execution or content requirements are not satisfied.
Cost and accounting boundaries
- Known model spend
- $6.193
- Committed reservations
- $19.779
- Cumulative ceiling
- $40.00
Actual cost is incomplete. 2 additional selective-successor outcomes remain UNKNOWN. Unknown costs are not counted as zero; token estimates are not an invoice.
Average known cost per generated reader: at least $0.248. Average known cost per published brief: at least $0.563. These allocate cumulative program spend, including failures, across unique readers or new publications.
Development measurements, denominators and caveats
| Measure | Versions |
|---|---|
| Discovered source versions | 1193 |
| Frozen selected versions | 75 |
| Source eligible (selected cohort) | 75 |
| Normalized | 69 |
| Classification admitted | 44 |
| Reader generated | 25 |
| Featured eligible | 4 |
| Ordinary Research Brief eligible | 7 |
| Published in this release | 11 |
| Retained readers kept private | 14 |
| Historical abstentions | 5 |
| Historical failures | 35 |
| Historical UNKNOWN holds | 9 |
| Historical capacity holds | 25 |
| Budget deferred | 0 |
Historical material omissions: 17 / 20 evaluated readers (85.0%). Historical materially unsupported clauses: 48 / 312 evaluated clauses (15.4%).
Sampled human QA: 3 / 8 retained or conditionally retained (37.5%); 5 required corrections. These historical findings precede approved corrections and do not estimate the accuracy of the released briefs.
- Source eligibility covers the selected 75, not all 1,193 discoveries.
- Stage counts are not additive. Historical terminal categories are preserved; a new brief publication decision does not rewrite them.
- Eleven candidate-bound briefs enter this release after offline evidence checks and sampled human review. Four are Featured; seven are ordinary Research Briefs.
- Historical omission and unsupported-clause rates describe predecessor readers before approved corrections, not an accuracy estimate for the released briefs.
- The eight-paper human sample retained three readers, including one conditional caveat, and required five corrections. Changed text has not been assigned a fabricated second human pass.
- Known token-metered spend excludes unknown costs and is not an invoice. Cost per reader or publication is a lower bound using total known cumulative program spend.
- Continuous discovery is separate from model-stage execution and protected publication. Scheduling alone is not evidence of a completed automatic publication.
What these measurements establish
A publication count is not an accuracy score. Evidence reference checks and semantic support evaluation answer different questions. Important limitations remain visible in each brief; evidence review does not establish independent replication.
Scheduled forward operation
New-paper processing is measured separately from the frozen 75-paper experiment and the total public Research catalog. A reader draft is not a published brief.
Oct 5, 2026, 3:00 PM UTC · 0 readers generated · 0 published
Recorded through . 3 unique selected versions; continuation stage attempts do not add another paper to this denominator.
| Measure | Source versions |
|---|---|
| Metadata versions discovered | 500 |
| Selected | 3 |
| Source eligible among selected | 2 |
| PDF acquired | 1 |
| Normalized | 1 |
| Classification admitted | 1 |
| Extraction admitted | 1 |
| Reader generated | 0 |
| Semantic evaluation completed | 0 |
| Published Featured | 0 |
| Published Research Brief | 0 |
| Private Held | 1 |
| Abstained | 0 |
| Failed | 1 |
| UNKNOWN | 1 |
| License rejected | 1 |
| PDF size rejected | 1 |
Discovery stopped at its existing scan bound; whole-source eligibility is not established. Source eligibility is assessed for selected versions, not all discovered records.
Publication yield: undefined — no generated-reader denominator; 0 / 1 classification-admitted papers (0.0%).
Original window and linked continuation
Original scheduled window
to : 2 model calls, 0 synthesis calls, 0 semantic-evaluation calls; 0 readers generated, 0 evaluations completed, 0 publications.
Known model spend $0.009019; committed reservations $0.034925. Accounting is reconciled for this attempt group.
Linked downstream continuation
to : 1 model calls, 1 synthesis calls, 0 semantic-evaluation calls; 0 readers generated, 0 evaluations completed, 0 publications.
Known model spend $0.134144; committed reservations $0.33. Accounting is reconciled for this attempt group.
Linked downstream continuation
to : 0 model calls, 0 synthesis calls, 0 semantic-evaluation calls; 0 readers generated, 0 evaluations completed, 0 publications.
Known model spend $0.00; committed reservations $0.363. Actual accounting is incomplete: 1 reservations remain unresolved; unknown cost is not zero.
Combined window: 3 stage calls across 3 unique selected versions. Known model spend $0.143163; committed reservations $0.727925. Reservations are conservative commitments, not additional final charges.
Average known window cost per generated reader: undefined. Per published brief: undefined. Token-based cost estimates are not an invoice.
Material omissions: unavailable — no valid assessed denominator. Unsupported claims: unavailable — no valid assessed denominator. No human QA judgment is inferred from automatic evaluation.
Historical accounting remains separate: 12 earlier unresolved entries retain $1.396387 in reservations. They are not new calls or zero-cost outcomes in this window.
Private Held means a material content, evidence, source-quality or execution requirement remains unresolved; it does not establish that the source paper is scientifically wrong. The original capacity hold is preserved even when a separately authorized continuation changes the current outcome.
Oct 6, 2026, 3:00 PM UTC · 0 readers generated · 0 published
Recorded through . 3 unique selected versions; continuation stage attempts do not add another paper to this denominator.
| Measure | Source versions |
|---|---|
| Metadata versions discovered | 500 |
| Selected | 3 |
| Source eligible among selected | 3 |
| PDF acquired | 3 |
| Normalized | 3 |
| Classification admitted | 1 |
| Extraction admitted | 1 |
| Reader generated | 0 |
| Semantic evaluation completed | 0 |
| Published Featured | 0 |
| Published Research Brief | 0 |
| Private Held | 1 |
| Abstained | 0 |
| Failed | 0 |
| UNKNOWN | 2 |
| License rejected | 0 |
| PDF size rejected | 0 |
Discovery stopped at its existing scan bound; whole-source eligibility is not established. Source eligibility is assessed for selected versions, not all discovered records.
Publication yield: undefined — no generated-reader denominator; 0 / 1 classification-admitted papers (0.0%).
Original window and linked continuation
Original scheduled window
to : 3 model calls, 0 synthesis calls, 0 semantic-evaluation calls; 0 readers generated, 0 evaluations completed, 0 publications.
Known model spend $0.008004; committed reservations $0.0495. Actual accounting is incomplete: 1 reservations remain unresolved; unknown cost is not zero.
Linked downstream continuation
to : 1 model calls, 0 synthesis calls, 0 semantic-evaluation calls; 0 readers generated, 0 evaluations completed, 0 publications.
Known model spend $0.00; committed reservations $0.014575. Actual accounting is incomplete: 1 reservations remain unresolved; unknown cost is not zero.
Linked downstream continuation
to : 1 model calls, 1 synthesis calls, 0 semantic-evaluation calls; 0 readers generated, 0 evaluations completed, 0 publications.
Known model spend $0.139303; committed reservations $0.363. Accounting is reconciled for this attempt group.
Combined window: 5 stage calls across 3 unique selected versions. Known model spend $0.147307; committed reservations $0.427075. Reservations are conservative commitments, not additional final charges.
Average known window cost per generated reader: undefined. Per published brief: undefined. Token-based cost estimates are not an invoice.
Material omissions: unavailable — no valid assessed denominator. Unsupported claims: unavailable — no valid assessed denominator. No human QA judgment is inferred from automatic evaluation.
Historical accounting remains separate: 12 earlier unresolved entries retain $1.396387 in reservations. They are not new calls or zero-cost outcomes in this window.
Private Held means a material content, evidence, source-quality or execution requirement remains unresolved; it does not establish that the source paper is scientifically wrong. The original capacity hold is preserved even when a separately authorized continuation changes the current outcome.
Processing history and methodology
Snapshot updated . Measurements below retain their original populations and methodology.
Continuous Eval V1 · measurements and model profiles
Continuous Eval V1
Methodology: continuous-eval:v1. Sep 23, 2026, 12:00 AM through Sep 27, 2026, 12:00 AM UTC (end exclusive).
Bounded sample: 75 selected from 1,193 discovered source versions; selection limit 75.
Source-use assessments: 161; known eligible for private processing: 75; source-use holds: 86.
1,032 versions remain unassessed. Eligibility across the full discovery population is not established. Source selection is complete. Selection does not establish processing success or scientific approval.
- research_fast: openai.gpt-6-luna · medium reasoning · AWS_BEDROCK_MANTLE · research-continuous-fast:v2
- research_deep: openai.gpt-6-sol · high reasoning · AWS_BEDROCK_MANTLE · research-continuous-deep:v5
- research_fast: openai.gpt-6-luna · medium reasoning · AWS_BEDROCK_MANTLE · research-continuous-fast:v3
- research_fast: openai.gpt-6-luna · medium reasoning · AWS_BEDROCK_MANTLE · research-continuous-fast:v5
- research_fast: openai.gpt-6-luna · medium reasoning · AWS_BEDROCK_MANTLE · research-continuous-fast:v1
- research_deep: openai.gpt-6-sol · high reasoning · AWS_BEDROCK_MANTLE · research-continuous-deep:v3
- research_fast: openai.gpt-6-luna · medium reasoning · AWS_BEDROCK_MANTLE · research-continuous-fast:v4
| Outcome / stage | Source versions |
|---|---|
| Source versions discovered | 1,193 |
| Source eligible | Unavailable |
| Acquired | 75 |
| Normalized | 69 |
| Classification admitted | 5 |
| Synthesis attempted | 3 |
| Semantically evaluated | 2 |
| Publication eligible | Unavailable |
| Published | 0 |
| Quarantined | 3 |
| Abstained | 2 |
| Capacity deferred | Unavailable |
| Budget deferred | Unavailable |
| Infrastructure failed | 9 |
| Dimension | Observed / assessed |
|---|---|
| Evidence sufficiency | 40 / 40 claims passed |
| Claim coverage | Unavailable |
| Semantic support | 40 / 40 claims passed |
| Scope correctness | 1 / 2 source versions passed |
| Wrong lens | 1 / 2 source versions failed |
| Quantitative validation | Unavailable |
| Limitations preserved | 1 / 2 source versions passed |
| Citation and provenance integrity | Unavailable |
| Normalization dependency | 0 / 40 claims failed |
| Material omission | 1 / 2 source versions failed |
| Section fit | 2 / 2 source versions passed |
| Publication usefulness | 1 / 2 source versions passed |
Observed failure categories
- Material qualification or finding omitted: 1 source versions
- Research lens not supported: 1 source versions
Usage and latency
$1.633543 reserved; $0.52237 estimated from provider token usage; 4 calls with unknown cost. Token estimates are not an invoice.
21 measured stage calls; median 553.7 seconds; 95th percentile 584.9 seconds.
Historical results · Legacy RI1
This earlier publication inventory is retained for context. It is excluded from Continuous Eval measurements.
Legacy RI1
Methodology: legacy-ri1. Current published inventory; no processing-window denominator available.
Comparable model-profile telemetry is unavailable for this inventory.
| Outcome / stage | Source versions |
|---|---|
| Source versions discovered | Unavailable |
| Source eligible | Unavailable |
| Acquired | Unavailable |
| Normalized | Unavailable |
| Classification admitted | Unavailable |
| Synthesis attempted | Unavailable |
| Semantically evaluated | Unavailable |
| Publication eligible | Unavailable |
| Published | 10 |
| Quarantined | Unavailable |
| Abstained | Unavailable |
| Capacity deferred | Unavailable |
| Budget deferred | Unavailable |
| Infrastructure failed | Unavailable |
| Dimension | Observed / assessed |
|---|---|
| Evidence sufficiency | Unavailable |
| Claim coverage | Unavailable |
| Semantic support | Unavailable |
| Scope correctness | Unavailable |
| Wrong lens | Unavailable |
| Quantitative validation | Unavailable |
| Limitations preserved | Unavailable |
| Citation and provenance integrity | Unavailable |
| Normalization dependency | Unavailable |
| Material omission | Unavailable |
| Section fit | Unavailable |
| Publication usefulness | Unavailable |
Observed failure categories
No comparable failure-category data is available in this snapshot.
Usage and latency
Comparable model usage and cost are unavailable.
Unavailable means the retained data cannot establish that measurement. Cohorts are not combined into a headline pass rate. Counts describe automated processing outcomes and do not establish human scientific approval.