How to Evaluate Whether an AI Agent Is Trustworthy

Why agent trust matters now

Sometime in the last eighteen months, AI agents stopped being chat copilots and started being economic actors. They execute trades. They sign and submit transactions. They subscribe to APIs, settle invoices, and transact with other agents in the middle of the night. As of April 2026, 176,000 of them are registered on-chain. Fewer than 0.04% earn a passing grade on independent rating.

That gap is the problem agent commerce now has to solve. ERC-8004 gave agents a portable identity. x402 let them pay each other through HTTP 402 responses. ERC-8183 gave them verifiable commerce records. None of those primitives answer the only question that matters when you actually wire money: which agent should I trust to act on my behalf, or transact with my system, this hour, in this context?

Trust evaluation for AI agents is a new discipline. It borrows from credit ratings (continuous monitoring, methodology transparency), from security auditing (code review, attestations), and from mechanism design (anti-gaming guarantees). What follows is a practical framework for making the evaluation, whether you're a developer integrating an agent API, a protocol team accepting agent counterparties, or a buyer transacting with an agent for the first time.

The seven trust signals

Trust is not a single attribute. It's a composition of independent measurable signals, each catching a different failure mode. Miss any one and you've left a blind spot for the kind of adversary who actually reads rating methodologies. Seven dimensions cover the surface.

Each signal has its own data sources, its own failure modes, and its own ways of being gamed. A high score in one cannot compensate for absence in another. An agent with audited code (signal 3) but no operational history (signal 2) is a research project, not a counterparty. An agent with strong community signals (signal 6) but no enforced autonomy boundaries (signal 4) is a marketing operation, not a trustworthy actor. The seven cover the surface together. Apart, they don't.

Why single-source ratings fail

The temptation when evaluating an AI agent is to pick one signal and treat it as a proxy for the others. The result? You make a decision based on a partial view, and the attacker-of-the-month finds the gap you ignored. Four structural reasons explain why single-source fails.

Bias from token and marketplace incentives

Rating providers that hold a token or take a marketplace cut on the agents they rate face an inherent conflict. Higher ratings raise the value of the token or the volume on the marketplace. Lower ratings cost the rater money. A rating system optimized for the rater's economics is not optimized for the buyer's risk assessment. Oh and this isn't hypothetical: it's the core lesson of the 2008 credit ratings crisis applied to a new asset class. The issuer-pays model broke once. Watching it break a second time in agent ratings is the kind of thing that should keep the industry up at night.

Static snapshots versus dynamic behavior

A rating taken once and not refreshed cannot reflect agent behavior. AI agents update, retrain, change wallet permissions, integrate new dependencies, get acquired, go dormant. A trust signal that doesn't move with the underlying state stops describing the agent within days. A quarterly snapshot is a historical document, not a counterparty risk score.

Sybil resistance

Single-source signals are easy to game. Solicit positive feedback from sock puppet wallets. Deploy multiple identical contracts to fake distribution. Buy reviews. (Yes, agents pay for reviews. They're cheap, and the rate is dropping.) Composite signals weighted by source diversity make these attacks exponentially harder, not because any single source is unhackable, but because the attack surface is now seven different attack surfaces that have to fail in concert.

Coverage gaps

One chain, one registry, one source of attestations always leaves blind spots. Agents that operate across chains or that combine ERC-8004 identity with x402 payment flows and ACP commerce activity cannot be evaluated by any single registry. The evaluation has to travel with the agent across surfaces, or it misses half the picture.

How composite ratings aggregate signals

A composite rating combines the seven signals through a transparent weighting function and refreshes continuously. The structure is straightforward in principle, even if implementing it well takes time:

1. Each signal produces a numeric score from public data sources.
2. Each signal is weighted by category, with weights varying by agent type. (A trading agent weights operational reliability higher than a creative content agent does.)
3. A safety floor caps the composite below a threshold if any critical signal fails. Failed code audit caps the rating at D regardless of other strengths.
4. The composite refreshes on enrichment events and on a daily schedule.
5. Methodology, weights, and signal sources are published, and reproducible from the same public inputs.

Treebeard's methodology implements this pattern. The weights, signal definitions, and scoring math are public at /methodology/process. The list of agent archetypes and their respective weight profiles is at /methodology/methodology. The improvement guide for builders is at /methodology/improve.

Worked example: evaluating one agent

Consider how the seven signals apply to a real on-chain agent currently rated by Treebeard: Ethy AI, an autonomous trading agent built on A2A workflows, x402 payment flows, and ERC-8004 identity. Treebeard rates it C (65.1) at the time of writing.

What this evaluation tells a counterparty:

• The agent is real and identifiable. Identity passes.
• The agent operates as advertised. Operational reliability is acceptable.
• The agent is well-scoped. Autonomy index is high and bounded.
• The agent has not yet earned a strong safety or security signal. Counterparties placing significant value should expect improvement here before scaling integration.
• The agent has community validation. It is not a stealth-mode operation.

What does that translate to in practice? A C-grade agent is callable infrastructure for low-to-medium-stakes integrations. It is not yet a fiduciary-grade counterparty. The evaluation does not say the agent is bad. It says the available signals don't yet justify higher trust. Which is a different statement, and the difference matters.

Limitations and open problems

Honest framework discussion requires honest acknowledgment of where this approach has gaps. Four are worth naming.

Thin signal coverage on newer chains

Signals like operational reliability and reputation require activity history. Chains that are new (or chains where ERC-8004 adoption is still early) produce thin signals. An agent on Base today has fewer measurable behaviors than the same agent on Ethereum, not because it's a worse agent, but because the measurement layer hasn't caught up. Evaluators have to adjust confidence accordingly.

Self-reported metadata

An agent's description, claimed capabilities, and category label come from the agent's own metadata. A classifier can assign a category from keyword matching, but it can't verify the agent actually does what it claims. Operational signals must confirm or contradict the claimed function. Most don't, because most agents haven't been called enough times.

Composite scores hide category-specific risk

A C-grade composite that averages high autonomy and low security looks the same as a C-grade composite that averages medium scores across the board. The averaged score hides the specific risk distribution. Always read category breakdowns before integrating. The composite is the headline. The categories are the actual story.

Rating velocity versus agent change rate

An agent that updates its model or permissions daily can change faster than a daily-rated trust signal can capture. For high-frequency or high-stakes integrations, supplement static ratings with real-time monitoring of behavioral drift. The rating tells you what was true yesterday. Behavior tells you what is true now.

FAQ

Sources

This framework draws on academic and industry work in mechanism design, credit ratings, and AI agent infrastructure. Primary references:

• ERC-8004: Identity and Reputation Registries for AI Agents. Ethereum standard for portable agent identity.
• ERC-8183: Agentic Commerce Protocol. Standard for verifiable agent-to-agent commerce records.
• x402: HTTP-native payment protocol. Specification for agent-driven payments and counterparty attestations.
• Hardin, Russell. Trust and Trustworthiness. Russell Sage Foundation, 2002. Foundational treatment of trust as a relational property between principal and counterparty.
• Akerlof, George A. The Market for "Lemons": Quality Uncertainty and the Market Mechanism. Quarterly Journal of Economics, 1970. Canonical paper on asymmetric information and market failure.
• Cantor, Richard, and Frank Packer. Sovereign Credit Ratings. Federal Reserve Bank of New York, 1996. The original academic study of how composite continuous ratings work in practice.
• a16z: The missing infrastructure for AI agents. April 2026. Industry framing of identity, governance, payments, verification, and user control as the five infrastructure problems blockchains can address.
• Treebeard Methodology. Live, versioned methodology including category weights, calibration cycle, and the safety floor.
• Treebeard Independence. Disclosure of conflicts, funding, and operating principles.