Dimension 01 · weight 10%Business problem clarity
- What decision or workflow changes if this works?
- What does “good enough” look like in measurable terms?
- Who owns success after launch?
Weak signal: AI is justified by curiosity or vendor pressure without a named outcome.
Recommended next step: Write a one-page problem statement with owner, metric, and non-goals before model work expands.
Dimension 02 · weight 10%Data readiness
- Is the required data accessible under clear permissions?
- Is freshness, completeness, and lineage understood?
- Can sensitive fields be excluded or redacted by design?
Weak signal: Teams train or retrieve against unknown, stale, or over-permissioned corpora.
Recommended next step: Inventory sources, owners, access rules, and retention before expanding retrieval or training.
Dimension 03 · weight 10%Architecture readiness
- Is the system RAG, workflow, agent, or hybrid — and why?
- Where does the model stop and deterministic logic begin?
- Can the design fail closed without cascading damage?
Weak signal: A chatbot wrapper is treated as an enterprise system.
Recommended next step: Document components, trust boundaries, and fallbacks as an operable design, not a slide.
Dimension 04 · weight 10%Integration readiness
- Which existing systems must be called, and with what auth?
- Are side effects idempotent and auditable?
- Is there a safe path when APIs are slow or unavailable?
Weak signal: Prototype APIs hit production systems without rate limits, auth scopes, or rollback.
Recommended next step: Map every write path and require explicit approval for consequential mutations.
Dimension 05 · weight 10%Security
- What data can leave the trust boundary?
- How are secrets, tokens, and tool credentials scoped?
- How are prompt injection and data exfiltration risks handled?
Weak signal: Model and tool credentials are shared and over-scoped “to make demos work.”
Recommended next step: Least-privilege credentials, content filters on untrusted input, and egress controls.
Dimension 06 · weight 10%Governance
- Who may approve model or policy changes?
- Which actions require human sign-off?
- How are incidents and misuse reported?
Weak signal: No named owner for model behavior after the pilot team moves on.
Recommended next step: Assign an operating owner and an approval matrix for high-impact actions.
Dimension 07 · weight 10%Evaluation
- What gold-set or task suite proves quality?
- How often is evaluation re-run after changes?
- Do you measure false confidence as well as accuracy?
Weak signal: Quality is judged by a few happy-path demos.
Recommended next step: Build a fixed evaluation set and block promotion when scores regress.
Dimension 08 · weight 10%Human-in-the-loop
- Where must a named human approve before execution?
- Is the review UI fast enough that people will actually use it?
- What happens when reviewers disagree or are offline?
Weak signal: “Human in the loop” means a log file nobody reads.
Recommended next step: Put approval before irreversible actions, with clear ownership and SLA.
Dimension 09 · weight 10%Observability
- Can you inspect prompts, retrievals, tool calls, and decisions?
- Are latency, cost, and error rates monitored?
- Can an incident be reconstructed after the fact?
Weak signal: Production AI is a black box until customers complain.
Recommended next step: Instrument traces for model and tool paths; alert on error and cost anomalies.
Dimension 10 · weight 10%Production operations
- Who on-calls when the system fails?
- How do you roll back model, prompt, or index changes?
- What is the kill switch for automation?
Weak signal: Launch without runbooks, rollback, or a named operator.
Recommended next step: Write runbooks, define rollback, and test the kill switch before wider rollout.