Evidence
under adversarial review.

Peptigrade should not run on a single “smart model” reading a few papers and improvising a verdict. The protocol is designed around:

• duplicate claim decomposition
• duplicate and transparent search planning and screening
• duplicate independent extraction
• explicit risk-of-bias appraisal
• adversarial challenge from multiple postures
• arbitration with abstention when evidence is thin
• full provenance from verdict back to source text

The default failure mode is not“confident answer from incomplete evidence.” The default failure mode is UNVALIDATED, SPECULATIVE, or LOW confidence until the workflow earns a stronger conclusion.

Use this protocol for:

• Claim audits — a public claim or popular framing is adjudicated against the literature.
• Peptide × outcome grades — a molecule is graded for a specific use, not as a blanket product.
• Source breakdowns — a podcast, post, paper, or newsletter is decomposed and traced upstream.

Do not use this protocol to:

• provide medical advice
• recommend dosing to an individual
• infer legality from incomplete regulatory information
• validate claims from memory without source retrieval

This protocol is stitched from established frameworks. We do not adopt any one of them wholesale; we borrow the parts that are useful and explicit.

Two principles matter most for AI execution:

1. Systematic-review discipline beats model eloquence.
2. Adversarial review beats single-pass confidence.

These are non-negotiable if the protocol is executed by LLMs.

1. § I

   Evidence first, never memory first

   No model may assert that a claim is supported, contradicted, or untested unless the underlying sources were actually retrieved and logged. A model may use prior knowledge to suggest search terms, but not to decide the verdict.

2. § II

   Separate retrieval, extraction, and judgment

   The same pass that retrieves sources should not be trusted to synthesize them without review. Retrieval, extraction, bias appraisal, and final adjudication should be split into distinct steps or agents.

3. § III

   Require duplicate independent passes for subjective steps

   Claim decomposition, search planning, screening, extraction, and final synthesis all contain judgment. They must be run independently by at least two reviewers or two isolated model contexts before arbitration.

4. § IV

   Force adversarial postures

   At least one reviewer must make the strongest defensible case for the claim; at least one must make the strongest case against. A third reviewer looks for mismatches, omissions, and overgeneralization.

5. § V

   Do not let mechanism substitute for efficacy

   Mechanistic plausibility can raise interest. It cannot validate a human efficacy claim on its own.

6. § VI

   Human claims require human evidence

   Animal and in-vitro evidence can support mechanistic or preclinical sub-claims. They cannot validate a claim about efficacy, safety, or speed of effect in humans.

7. § VII

   Reviews summarize; primary studies decide

   Systematic reviews and meta-analyses are high-value synthesis sources, but they do not replace reading the primary studies that drive the conclusion when a claim is contested or high stakes.

8. § VIII

   Abstention is a valid result

   If the search is incomplete, sources conflict, route/dose/population mismatch is large, or evidence is sparse, the workflow should stop at UNVALIDATED, SPECULATIVE, CONTESTED, or Low confidence rather than force a stronger answer.

9. § IX

   Every public sentence must be traceable

   Every sentence in a verdict summary must map to cited sources, specific evidence snippets or extracted fields, and a documented reasoning step in the audit record.

An AI workflow should emit these artifacts for every reviewed claim.

Duplicate claim decomposition

Before any retrieval runs, the workflow produces two independent packets — Claim Decomposition A and Claim Decomposition B — splitting the source material into atomic sub-claims, assigning qualifiers, and stating what evidence would count as direct support or contradiction.

The reconciliation step is deterministic:

• compare claim boundaries
• compare population / route / outcome / time qualifiers
• compare claim-type assignments
• merge exact duplicates
• escalate materially different framings instead of averaging them away

If the two decomposers disagree on what the claim is actually asserting, the workflow has not earned the right to search yet.

Atomic claim extraction

A claim is the smallest assertion that can be tested against evidence. Long sentences should be split until each sub-claim has a single testable burden.

Every atomic claim is normalized into this frame:

type ClaimManifest = {
  originalText: string;
  normalizedText: string;
  claimType:
    | "empirical"
    | "mechanistic"
    | "comparative"
    | "quantitative"
    | "predictive"
    | "definitional";
  population?: string;
  intervention?: string;
  comparator?: string;
  outcome?: string;
  route?: string;
  timeHorizon?: string;
  scopeKey?: string;
  scopeNotes?: string[];
};

For internal evidence review, the meaningful unit is usually closer to peptide × outcome × population × route × time horizon. Public pages may collapse presentation for readability, but the protocol retains these qualifiers internally. If a qualifier is implicit or unknown, it is marked as such and certainty is capped.

Duplicate and adversarial search planning

Search execution does not begin from a single planner’s instincts. Each finalized ClaimManifest produces:

• Search Plan A — direct-support planner optimized to find the best on-point evidence.
• Search Plan B / contradiction-seeking plan — planner optimized to find null, contradictory, safety, integrity, and scope-mismatch evidence.

These plans are reconciled deterministically. The reconciliation verifies that direct-support and contradiction queries exist for every central sub-claim, that safety / integrity / registry / regulatory checks are present where relevant, that synonyms and scope qualifiers are not missing, and that query coverage is logged before execution.

type SearchPlan = {
  subClaimId: string;
  queryFamilies: Array<{
    family:
      | "direct"
      | "contradiction"
      | "safety"
      | "integrity"
      | "registry"
      | "regulatory";
    query: string;
    rationale: string;
  }>;
  requiredSourceClasses: string[];
  coverageChecklist: string[];
};

Retrieval completeness checklist

For each sub-claim, the planner explicitly considers whether the query set covers:

• canonical peptide name
• synonyms, aliases, development codes
• common misspellings and transliterations
• route terms when route matters
• outcome and endpoint synonyms
• population / indication / disease terms
• comparator or control terminology
• dose or exposure terminology
• human clinical terms
• animal model terms for preclinical sub-claims
• target / pathway terms for mechanistic sub-claims
• negative, null, failed, no-effect terms
• safety and adverse-event terms
• retraction, correction, expression-of-concern terms
• registry identifiers and trial terminology
• regulatory body names and status terminology

“Not searched because irrelevant” is acceptable. Silent omission is not.

Databases and source classes

At minimum, the workflow searches or checks:

• PubMed / MEDLINE
• trial registries (ClinicalTrials.gov and regional equivalents)
• Crossref or DOI resolution
• PubPeer
• Retraction Watch or equivalent retraction source
• relevant regulatory bodies (FDA, EMA, WADA, etc.)

Preprints may be included only when methodology is inspectable, no peer-reviewed equivalent exists, and the verdict is explicitly capped for certainty.

Citation chasing and record linkage

For central claims and decisive studies, retrieval is not complete after keyword search alone. The workflow performs and logs:

• backward citation review of decisive reviews and pivotal primary studies
• forward citation review of decisive primary studies
• registry-to-publication matching for registered trials
• duplicate-report linking across abstracts, articles, supplements, and registry records
• dead-end logging when cited primary sources cannot be retrieved in full text

PRISMA-style search ledger

type SearchDatabaseRecord = {
  database: string;
  queryFamily:
    | "direct"
    | "contradiction"
    | "safety"
    | "integrity"
    | "registry"
    | "regulatory";
  query: string;
  searchedAt: string;
  filters?: string[];
  retrieved: number;
};

type SearchLedger = {
  subClaimId: string;
  databases: SearchDatabaseRecord[];
  totalRetrieved: number;
  afterDeduplication: number;
  screened: number;
  excluded: number;
  included: number;
  fullTextUnavailable: number;
  decisiveSourcesMissingFullText: string[];
  citationChasing: {
    backwardFrom: string[];
    forwardFrom: string[];
    registryMatched: string[];
    duplicateReportsLinked: string[];
  };
};

Screening rules

Screening runs independently in two isolated contexts.

Every exclusion must have a reason code. Screening is not complete until A/B decisions reconcile, exclusion reason codes are finalized, PRISMA counts reconcile to the deduplicated inventory, and decisive full-text gaps are logged.

Duplicate extraction

Study characteristics and outcome data are extracted independently by two reviewers. This is especially important for primary outcomes, adverse events, effect sizes, route and dose, population definitions, and follow-up duration.

For AI workflows, “independent” means the second extractor must not see the first extractor’s filled form before submitting its own.

Evidence-card fields

type EvidenceCard = {
  studyId: string;
  design:
    | "systematic_review"
    | "meta_analysis"
    | "rct"
    | "nonrandomized_human"
    | "case_series"
    | "animal"
    | "in_vitro"
    | "preprint"
    | "registry";
  population: string;
  n?: number;
  route?: string;
  dose?: string;
  comparator?: string;
  endpoint: string;
  endpointType: "clinical" | "surrogate" | "mechanistic" | "safety";
  timepoint?: string;
  resultSummary: string;
  effectSize?: string;
  adverseEvents?: string;
  exactQuotesOrSnippets: string[];
};

The extractor must label whether the endpoint is clinical, surrogate, mechanistic, or safety. This prevents mechanistic wins from being mistaken for clinical wins.

Study-design ladder

When evidence conflicts, we prefer:

1. systematic reviews / meta-analyses of high-quality human trials
2. registered, peer-reviewed human RCTs
3. non-randomized human comparative studies
4. case series / uncontrolled pilots
5. animal in-vivo studies
6. in-vitro / cell / mechanistic studies
7. expert opinion without new data
8. anecdote / testimony

Domain-based bias appraisal

For randomized trials we use RoB 2 style domains:

• randomization process
• deviations from intended interventions
• missing outcome data
• outcome measurement
• selective reporting

For non-randomized human studies we use ROBINS-I style reasoning:

• confounding
• participant selection
• intervention classification
• deviations from intended interventions
• missing data
• outcome measurement
• selective reporting

For every included study, we also flag:

• preregistration or registry record present / absent
• funding source and sponsor
• author conflicts
• replication status
• retraction status
• PubPeer or major integrity concerns

Integrity override

This is the part that makes the protocol AI-ready instead of AI-flavored.

Required roles

If a single model is used for all roles, contexts are isolated and prior outputs are not revealed until arbitration. Better still: heterogeneous models or at least heterogeneous prompting and retrieval contexts.

Required adversarial questions

Before a verdict is finalized, the workflow must answer:

1. What is the strongest evidence for the claim?
2. What is the strongest evidence against the claim?
3. Are the supportive studies actually testing the same population, route, dose, comparator, and endpoint as the claim?
4. Are supportive results clinical outcomes or only mechanistic / surrogate outcomes?
5. Are contradictory or null studies being omitted because they are harder to explain?
6. Are reviews being used to smuggle in unsupported primary-study conclusions?
7. Does the claim generalize beyond the studied tissue, route, timeframe, or population?
8. Is the verdict being driven by one lab, one paper, or one uncontrolled pilot?
9. Is there any integrity signal that should cap certainty regardless of apparent effect?

Overstatement check

Many public claims are not cleanly true or false. They are partly grounded, then overstated. For composite claims, the workflow computes the formal status of each sub-claim and whether the public framing overgeneralizes the evidence.

OVERSTATED exists precisely so a composite claim is not collapsed into VALIDATED or FALSIFIED when the fairer answer is “the underlying phenomenon exists in narrow scope, but the public framing outruns the evidence.”

Every claim audit runs through these stages in order.

1. Stage 01

   Run duplicate claim decomposition

   Input

   source artifact · verbatim claim text · target population (if known)

   Output

   Decomposition A · Decomposition B · reconciled claim manifest · atomic sub-claims

   Gate

   every sub-claim is testable; route / outcome / population explicit; material framing differences reconciled or escalated

2. Stage 02

   Duplicate and adversarial search planning

   Input

   reconciled claim manifest · atomic sub-claims

   Output

   Search Plan A · Search Plan B / contradiction-seeking plan · reconciled query set

   Gate

   direct-support and contradiction queries for every central sub-claim; coverage checklist logged; required source classes explicit

3. Stage 03

   Transparent retrieval and screening

   Input

   normalized sub-claims · reconciled search plan

   Output

   search ledger · candidate sources · screening log

   Gate

   search terms saved; contradictory-evidence search explicit; A/B screening reconciled; PRISMA counts reconcile; full-text gaps logged

4. Stage 04

   Trace provenance recursively

   Input

   screened source set · each claim

   Output

   classification per claim: Direct · Inherited · Speculative

   Gate

   primary source reached or dead end recorded; inherited claims not credited as direct evidence

5. Stage 05

   Extract evidence in duplicate

   Input

   included studies

   Output

   reconciled evidence cards

   Gate

   discrepancies reconciled or escalated; multiple reports merged; exact snippets back every important extracted claim

6. Stage 06

   Appraise bias and integrity

   Input

   evidence cards

   Output

   bias cards with design classification, domain-based assessment, integrity check, replication note

   Gate

   the workflow can explain why higher- and lower-quality studies were weighted differently

7. Stage 07

   Build support and contradiction maps

   Input

   evidence cards · bias cards

   Output

   per sub-claim: supports · contradicts · mentions — plus directness and evidence-type annotations

   Gate

   every sub-claim has a contradiction map, even if empty

8. Stage 08

   Run adversarial review

   Input

   all prior artifacts

   Output

   steelman, skeptic, mismatch, and omitted-evidence reviews

   Gate

   at least one adversarial pass challenged the favored interpretation; unresolved disagreements logged, not buried

9. Stage 09

   Adjudicate status and confidence

   Input

   evidence cards · bias cards · maps · adversarial findings

   Output

   sub-claim statuses · overall status · confidence

   Gate

   final summary does not outrun the evidence cards; confidence capped by unresolved gaps, directness problems, or integrity concerns

10. Stage 10

    Publish with provenance and linting

    Input

    decision log

    Output

    verdict summary · what you can / cannot say · citations · evidence chain · last verified date · structured data · sentence-level provenance · linter report

    Gate

    every public sentence maps to evidence cards and snippets; linter checks pass or are explicitly waived by policy; wording does not exceed allowed strength

The minimum machine-checkable completion spec for a run.

Sentence-level provenance is a first-class object, not a formatting afterthought.

type PublicClaimSentence = {
  sentenceId: string;
  text: string;
  sourceEvidenceCardIds: string[];
  sourceSnippetIds: string[];
  allowedStrength:
    | "directly_supported"
    | "qualified_support"
    | "context_only"
    | "not_allowed";
};

The publication packet refuses to ship sentences marked not_allowed or sentences whose wording exceeds the allowed strength of their linked evidence.

Publication linter

Before publication, deterministic wording checks run:

• no dosing advice
• no individualized medical advice
• no unsupported safety reassurance
• no “proven,” “clinically established,” or “safe” unless policy and evidence status allow it
• no human efficacy wording from animal-only or in-vitro evidence
• no review article cited as if it were primary evidence
• no factual biomedical sentence without citation coverage
• no sentence stronger than the weakest central sub-claim permits

Any failure blocks publication or requires an explicit, logged override.

Eight formal statuses:

Decision rules for composite claims

Do not use a naive “worst sub-claim wins” rule. Instead:

1. identify the central necessary sub-claims
2. identify the supporting but non-central sub-claims
3. classify each sub-claim separately
4. assign the overall status based on the central necessary sub-claims plus any material overstatement

Confidence is separate from status.

Confidence is judged on five axes: search completeness, directness, quality / bias profile, consistency, integrity / provenance clarity. The final confidence does not exceed the weakest material axis.

Confidence caps and status ceilings

Hard rules

• No human efficacy claim may be marked VALIDATED on animal or in-vitro evidence alone.
• No claim may be marked FALSIFIED merely because support is absent; absence of evidence is usually UNVALIDATED.
• No review article may be the sole basis for a decisive status if the primary studies are available.
• No final verdict may ignore contradictory evidence found in search.
• No study may be counted twice across multiple reports.
• No final summary may introduce claims absent from the evidence cards.

Abstention triggers

Cap the verdict at UNVALIDATED or Low confidence when:

• full text is not available for decisive sources
• the claim depends on route / dose / population details not reported in the literature
• only uncontrolled pilots exist
• only one lab drives the literature
• the protocol / registry record materially conflicts with the paper and cannot be reconciled
• serious integrity concerns remain unresolved

Treat the protocol itself as something that must be measured. We track at least:

• decisive-source recall
• false inclusion rate
• false exclusion rate
• critical-field extraction accuracy
• verdict agreement with expert benchmark sets
• rate of overconfident wrong verdicts
• disagreement rates across decomposition, search planning, screening, and extraction

Both modes use the same machinery above.

Independence

If you want fair adversarial review, do not let later reviewers inherit earlier reviewers’ conclusions. Shared context creates correlated errors.

Deterministic checks

Use non-LLM validation wherever possible for:

• DOI / PMID resolution
• deduplication
• publication dates
• registry status
• retraction status
• schema validation of outputs

Calibration

Prompt the arbitrator to justify why a stronger status was earned rather than defaulting to the weaker one. If that justification is thin, downgrade.

Escalation

Escalate to human review when:

• the claim is legally or medically high stakes
• the studies are highly technical and difficult to normalize
• the adversarial reviewers disagree on central sub-claims
• integrity concerns are non-trivial

This protocol is grounded in:

• GRADE certainty-of-evidence logic
• Cochrane RoB 2 and ROBINS-I style bias assessment
• PRISMA 2020 reporting discipline
• Cochrane duplicate independent extraction practice
• SciFact scientific claim verification
• FEVER claim verification and FEVER adversarial evaluation
• post-publication integrity checks via PubPeer, retraction tracking, and trial registries

These anchors matter because they push the workflow toward transparency over vibes, duplication over single-pass confidence, adversarial challenge over confirmation bias, and abstention over overclaiming.

The canonical cadence SLAs — per-trigger response times (same-day, 7-day, 30-day, quarterly, annual) — live in the internal grading-and-reassessment protocol. This is the short version; if the two disagree, the internal protocol wins.

• Re-run a claim audit within 30 days when new peer-reviewed evidence materially affects a central sub-claim.
• Re-run immediately (same day) when a decisive source is retracted, corrected, or flagged for serious integrity issues.
• Re-run same day when a major regulatory change alters the real-world status of the molecule or use case.
• Re-run stale claims at least annually even if no trigger fires; quarterly housekeeping audit across the catalog.

Last updated: 2026-04-20