Methodology

The Treebeard Score is the output of a trust intelligence system — not a static label, but a continuous assessment that updates as new signals arrive.

Trust in autonomous systems is compositional. No single source is sufficient. Registries are incomplete. Certifications are partial. On-chain observations are noisy. The most resilient trust systems combine multiple imperfect signals into a structured view — making disagreement between sources visible and usable, not collapsed into false precision.

Here is how Treebeard produces decision-grade trust intelligence, and how to read what it produces.

Six Signal Categories

Every agent is evaluated across six top-level signal categories. Each category captures a distinct dimension of agent quality. Default weights are shown — actual weights vary by agent type (see Weight Profiles below).

Lending Confidence

For agents rated B- or higher, Treebeard computes a parallel Lending Confidence score (0–100) assessing creditworthiness for DeFi lending and integration exposure decisions. This score uses the same underlying signals but re-weights them for financial risk:

Score interpretation and exposure thresholds are set by the integrator. Treebeard provides the signal; risk decisions remain with the operator.

Agent Type-Dependent Weight Profiles

A one-size-fits-all weighting produces ratings that yield inaccurate results for at least half the agents rated. Each agent type receives a tailored weight profile that reflects which qualities matter most for that agent type. All columns sum to 100%.

Scoring Mechanics

Signals are gathered, normalized, and composed into category scores, then into an overall rating. Three signal types feed the system:

Each category score is the weighted average of its component signals (equal weights within categories for v1). The overall score is the weighted average of category scores using the agent type-dependent weight profile. A safety floor is then applied: an agent's maximum overall score is capped by its Safety & Reliability category score.

Publishable Coverage Gate

Treebeard publishes a score only when an agent has enough signal coverage to produce a confident result. The rating engine evaluates each of the six categories for real data (vs. defaults), counts the categories that pass, and divides by six. If the resulting fraction is below the publishable threshold, the engine refuses to publish a score.

The current threshold is 40% coverage — at least three of the six categories must have real data. Agents that fall below this gate appear in the directory and have full on-chain identity surfaced, but their profile shows a checklist of missing categories instead of a numeric score.

This is methodology rigor, not a coverage limitation. Other services emit a number with thinner data. Treebeard does not. When the gap closes — new feedback events, a github_url surfaces, ERC-8128 detection lands — the score publishes automatically on the next rating pass. No opt-in refresh, no application process.

See an unrated profile in the directory to view the checklist in practice.

Bayesian Weight Calibration

Default weights are Bayesian priors based on expert judgment. Over time, Treebeard replaces intuition with empirical evidence through a structured calibration process.

Anti-Gaming Principles

For each signal, the methodology evaluates the cost to credibly fake that signal. Signals are weighted proportionally to their cost-to-fake. Cheap-to-manipulate metrics (social followers, GitHub stars) carry minimal weight.

Transparency Model

Treebeard publishes extensively — but not everything. The boundary between published and proprietary is drawn deliberately to prevent gaming while maintaining institutional trust.

Version History

Algorithm changes require 30-day advance notice and published rationale. All historical versions remain available for reference.

Weight recalibrations are published quarterly. Methodology version increments are reserved for structural changes to categories, signal types, or scoring mechanics.

Signal Pipeline Roadmap

Treebeard's signal pipeline is expanding. The following additions are under evaluation or planned for 2026:

These additions will be reflected in this methodology document and in a new algorithm version when they ship.

Signal Coverage by Standard

Treebeard's signals are organized by the question they answer, not by protocol name. This is intentional — new standards emerge regularly, and the coverage table below remains accurate as the underlying sources evolve.

Filter chips in the Agent Directory correspond directly to the live rows above. New protocols are added to this table when Treebeard begins indexing them — no empty tabs, no placeholder data.

🌳 The Treebeard Review Panel

AI agents rated by data. Challenged by experts.

What it is

The Treebeard Review Panel is a 25-persona AI simulation that debates whether an agent's quantitative score is fair. Treebeard's formula is blind to context — an agent with 122 endpoints and active x402 revenue can score F/38 because the formula doesn't capture operational complexity. The panel exists to surface exactly those disconnects.

When an agent receives a Treebeard Review, three sequential debates run via Claude Haiku: a bull case (panelists who believe the score is too low), a bear case (panelists who believe the score is fair or too high), and a full-panel verdict synthesizing both. The result is a structured verdict with a panel score, confidence level, vote split, and concrete improvement suggestions.

Panel composition

25 anonymous personas. Individual names and bios are not disclosed. The panel is composed by archetype to ensure genuine debate tension:

• Protocol Engineers ×8
• VC Partners ×4
• Security Specialists ×3
• Business Users ×3
• Product / UX Designers ×2
• Economists ×2
• Regulatory Experts ×2
• End User Representative ×1

Personas are cross-geographic, span ages 24–58, and carry explicit bias profiles designed to surface disagreement rather than produce consensus rubber-stamps.

The dialectical process

Each review runs three sequential AI calls. This structure produces genuine tension — not a single-perspective summary.

How qualitative maps to scoring

The panel verdict does not override the quantitative score — both are displayed independently. The verdict includes a panel-estimated score range (e.g., “Panel believes fair value is 72–78”) which may inform a future methodology update, but does not change the published rating automatically.

Disagreements between panel score and quantitative score are flagged for founder review. If the panel consistently rates a signal category as under-weighted for a given agent archetype, that pattern feeds into the next weight calibration cycle.

What it is not

• Not a replacement for the quantitative score — both are shown side by side
• Not a human review — it is a structured AI simulation
• Not financial or investment advice
• Not a guarantee of agent quality or safety

How improvement suggestions work

The verdict call produces 3–5 specific improvement actions, each with an estimated score impact and difficulty level (Easy / Medium / Hard). These map directly to Treebeard's six signal categories — so builders know exactly which category to target and roughly how many points each action is worth. Suggestions are generated from the agent's actual signal data, not generic advice.

Eligibility

Agents with a quantitative score of 70 (C+) or above are eligible for a Treebeard Review. The threshold is intentionally set below B- (75) to catch undervalued agents — the agents the formula most likely underestimates. Reviews are triggered after significant score changes (±10 points) or grade boundary crossings.