promptstyles.org
Rubric

Evaluation rubric

A rubric for interviews where candidates build with AI. It scores the process, not just the code: how the problem is framed, how the model is directed, how the output is checked and who really owns the result. Share it with candidates before the interview.

Levels

  1. 1

    Operator

    Uses a single style and accepts the output without real review.

  2. 2

    Practitioner

    Switches between describing and dictating, reviews the code and fixes errors.

  3. 3

    Engineer

    Chooses the style by risk and complexity, uses tests and specs as contracts, manages context.

  4. 4

    Architect

    Sets standards for the team, knows when not to use AI, weighs cost, security and maintenance.

Dimensions

Dimension1 · Operator2 · Practitioner3 · Engineer4 · Architect
Problem framingDoes the candidate understand and specify the problem before generating code?Pastes the task into the model as it is.Restates the task and adds the obvious requirements.States requirements, constraints and edge cases, and asks clarifying questions.Questions the problem itself, finds hidden requirements (security, scale, operations) and cuts scope on purpose.
Style selectionDoes the candidate choose how to direct the model, on purpose?Uses one habitual style, whatever the task.Alternates between describing the outcome and dictating the implementation.Picks the style according to risk and complexity, and can explain the choice.Combines styles fluently, switches mid-task when the situation changes and can teach the trade-offs.
VerificationHow does the candidate check that the output is correct?Runs it once. If it looks right, it ships.Reads the diff and fixes visible errors.Uses tests, specs or types as a contract and checks edge cases and failure modes.Designs verification upfront (tests, evals, monitoring) and catches subtle problems in security, performance and concurrency.
Context managementDoes the candidate give the model what it needs, and only that?Gives no context, or dumps everything.Adds relevant files after the model fails.Curates context upfront with relevant files, conventions, constraints and examples.Makes context reusable for the team (rules files, docs, tools) and keeps it current.
Ownership and judgmentDoes the candidate own the result as if they had written every line?Cannot explain parts of the generated code.Explains the code and changes it when asked.Rejects or rewrites weak output and defends design decisions with reasons.Knows when not to use AI and weighs cost, security and maintainability for the team.

How to run the interview

65 minutes in total

  1. Brief 5 min

    Share your AI policy and this rubric. Say which tools are allowed and that you evaluate the process, not only the output.

  2. Framing 10 min

    Present a realistic problem from your domain, not a puzzle. Ask the candidate to restate it, ask questions and outline a plan before prompting.

  3. Build 30 min

    The candidate builds with their usual AI tools and thinks out loud. Watch style choices, the context they provide and how they review output. Do not help with prompts.

  4. Change 10 min

    Introduce a new requirement or a subtle bug. Watch how the candidate verifies and adapts.

  5. Debrief 10 min

    Ask the candidate to explain the code, their choices and what they would do before production. The interview questions on each style page help here.

Scoring

  • Score each dimension from 1 to 4 with written evidence. Write down what you saw, not what you felt.
  • Do not average blindly. Decide upfront which dimensions matter most for the role.
  • A 1 in ownership is a strong no for production roles, whatever the other scores.
  • Calibrate. Interviewers should score the same recorded session together at least once.

Green flags

  • States assumptions before prompting
  • Rejects output that is plausible but wrong
  • Switches style when the risk changes, and says why
  • Writes or asks for tests before trusting the code

Red flags

  • Accepts large diffs without reading them
  • Cannot explain the generated code
  • Loops on "fix it" without a diagnosis
  • Hides AI use, or uses tools that were not allowed

How to prepare as a candidate

  1. Pick one small feature and build it in every style of the taxonomy. Compare the results and how long each one took.
  2. Practice thinking out loud. Say which style you are choosing and why.
  3. Read every diff before you accept it. Be ready to explain any line.
  4. Write tests or acceptance criteria before you prompt, at least once a day.
  5. Keep a rules file (AGENTS.md, CLAUDE.md or similar) in a side project and maintain it.
  6. Ask the company for its AI policy and rubric before the interview. Under this manifesto, that is a fair question.

Edit the rubric on GitHub · taxonomy.json