If engineers in the role will use AI tools, an interview can allow those tools while still assessing the candidate's own judgment. Define what you want to observe, give candidates equivalent access, and evaluate how they inspect and improve the work.
The exercise should produce evidence about engineering ability. A fast, polished output alone is not enough.
State the rules before the interview
Tell candidates which tools are allowed, whether accounts or subscriptions are provided, what material they may enter, and how they should explain tool use. Provide an alternative if a candidate cannot access the chosen tool.
Use a fictional codebase or a purpose-built exercise. Do not ask candidates to upload employer code, customer data, or private work samples to an external service.
Clarify whether you are assessing independent fundamentals, assisted development, or both. If both matter, separate the sections and explain the rules for each.
Design a task with something to verify
Illustrative exercise: A fictional application imports a list of records. The candidate must handle malformed input, avoid duplicate entries, and explain what happens if processing stops halfway through.
Provide a small starting project and acceptance conditions. Leave enough uncertainty for the candidate to ask questions, but do not hide requirements merely to create surprises.
An AI assistant may propose code. The candidate should still decide whether the proposal fits the task.
Use an observation sheet
| Dimension | Evidence to record |
|---|---|
| Problem framing | Clarifies inputs, constraints, and expected behavior |
| Inspection | Reads generated changes and identifies assumptions |
| Verification | Tests ordinary cases and relevant failure cases |
| Debugging | Uses evidence to locate and correct a problem |
| Ownership | Can explain the final code and remaining limitations |
Ask the candidate to walk through a specific path in the code. Change one requirement and discuss what would need to change. These follow-ups make understanding visible without turning the session into a contest over prompting style.
Keep comparisons fair
Use the same task, resources, evaluation criteria, and approximate working time for candidates in the same process. Record tool failures or access problems that affected the session.
Do not compare one person's assisted output with another person's unassisted output as if the conditions were identical. Avoid treating subscription familiarity as a proxy for engineering skill.
Make the result useful to the debrief
Write what the candidate demonstrated and what remains untested. For example: “Found a duplicate-handling bug and added a relevant test; deployment judgment was outside the exercise.”
For broader assessment design, use the bounded work-sample guide. For roles responsible for AI systems themselves, add a separate AI evaluation exercise.