Dimension 01 · weight 10%Agent boundaries
- What goals are in scope?
- What is explicitly out of scope?
- How does the agent stop?
Weak signal: Open-ended “do whatever is needed” mandates.
Recommended next step: Write a goal contract with stop conditions before tool access expands.
Dimension 02 · weight 10%Tool permissions
- Which tools can it call?
- Are credentials least-privilege?
- Can tools write money, email, or access?
Weak signal: Admin tokens shared with the agent “for convenience.”
Recommended next step: Separate read vs write tools; require approval on write tools with impact.
Dimension 03 · weight 10%Deterministic controls
- Which steps must never be model-chosen?
- Are schemas and validators in the path?
- Is there a state machine underneath?
Weak signal: All control flow is left to free-form model reasoning.
Recommended next step: Keep critical transitions in code; let the model fill bounded slots.
Dimension 04 · weight 10%Memory
- What is stored across turns?
- Who can read agent memory?
- How is memory revoked?
Weak signal: Unbounded memory of sensitive conversations.
Recommended next step: Scope memory by tenant and purpose; expire aggressively.
Dimension 05 · weight 10%Evaluation
- What task suite proves tool choice quality?
- How do you detect loops and thrash?
- Do you measure harmful actions prevented?
Weak signal: Success = “it completed a demo script once.”
Recommended next step: Evaluate tool selection and refusal behavior on fixed scenarios.
Dimension 06 · weight 10%Human approval
- Which actions require a human?
- Is approval binding before execution?
- Who is accountable?
Weak signal: Post-hoc audit instead of pre-action gates.
Recommended next step: Gate irreversible tools behind named human approval.
Dimension 07 · weight 10%Failure handling
- What happens on tool errors?
- How are infinite loops stopped?
- Is partial work rolled back?
Weak signal: Agent retries forever against failing APIs.
Recommended next step: Hard caps on steps, time, and spend; fail closed.
Dimension 08 · weight 10%Observability
- Can you replay a run?
- Are tool calls logged with args redacted?
- Are cost anomalies alerted?
Weak signal: Only final text is stored.
Recommended next step: Trace every tool call and decision with retention policy.
Dimension 09 · weight 10%Security
- How is prompt injection contained?
- Can tools be used for exfiltration?
- Are egress destinations allowlisted?
Weak signal: Agent can call arbitrary URLs with production credentials.
Recommended next step: Allowlist tools and destinations; sanitize untrusted content.
Dimension 10 · weight 10%Rollback
- Can you disable the agent instantly?
- Can tool permissions be revoked?
- Can prior versions be restored?
Weak signal: No kill switch separate from redeploying the whole app.
Recommended next step: Feature-flag the agent and test disablement in staging.