Answer

How do interviewers grade AI use in a coding interview?

Last checked ยท 11 sources

Mostly on the same things as before, with verification watched more closely. Meta reportedly keeps its four competencies: problem solving, code quality, verification and communication. Canva looks for judgment in steering the AI and fixing its mistakes. Rootly reads transcripts for decomposition, course correction, domain understanding and iteration. No company we found publishes a scored rubric.

What do companies say they look for?

Canva's list is the most explicit. Its June 2025 post asks whether candidates know when and how to use AI, how well they break down ambiguous requirements, whether they make sound technical decisions, whether they can find and fix problems in AI generated code, and whether the result meets production standards.

Rootly's reviewers read the transcript for four things: problem decomposition, course correction, domain understanding and iteration quality. Google's pilot, as reported, evaluates prompting, output validation and debugging. Zapier says it watches how candidates prompt, push back on an output and adapt, and values a rough result with visible reasoning over a polished one with none.

What separates strong candidates from weak ones?

Canva describes its strongest candidates as people who asked clarifying questions, handed the AI well defined pieces of work, reviewed and improved what it wrote, and debugged well when it failed. Weak candidates, it says, could code but lacked judgment about steering the tool.

Rootly gives a sharper example. Two take homes had nearly identical code. In one transcript the candidate worked in phases, asked targeted questions and caught the AI's mistake in webhook retry handling. In the other the candidate pasted the whole spec into one prompt, accepted most of the output and kept asking the AI to fix things until they worked. The transcripts decided it.

Shopify's head of engineering named a small but telling sign: asking the AI to fix a trivial bug instead of changing one character by hand.

Does using less AI cost you points?

It can. Shopify's Farhan Thawar said candidates without a copilot tend to lose to those with one, and that he wants most of the work done with AI, though not all of it. At Rippling, interviewing.io reports, AI is optional but a different rubric applies depending on whether you use it.

Using too much costs points too. Hello Interview reports Meta candidates marked down for relying on the assistant as a crutch. The target is deliberate use: the AI does the typing it is good at, and you do the deciding.

Is there research on what works?

Some, and it is small. In a study of 19 Microsoft developers working 33 real issues with Cursor's agent and Claude 3.5 Sonnet, issues solved incrementally succeeded 83% of the time against 38% for handing over the whole issue at once. Supplying expert knowledge was linked to 64% success against 29% without, writing some code by hand to 73% against 36%, and asking for refinements to 68% against 30%.

The same study found no significant link between success and the participants' age, role or years of experience. It is descriptive, from one company, with an early 2025 model, so read it as a pattern worth practising rather than a law.

Can anyone score a transcript reliably?

Not yet, as far as the published evidence goes. In a study of 15 professional developers, a prompt score from an AI judge correlated only 0.11 with years of experience. In a study of 100 university students in Zurich, human rated prompt quality tracked the outcome of the same session (a correlation of 0.479) but also tracked the students' essay grades (0.353), so part of what a prompt score measures is writing.

Anthropic's study of about 400,000 Claude Code sessions found that sessions rated novice reached verified success 15% of the time against 28% to 33% for intermediate and above, with only a modest gap between intermediate and expert. That is why our rubric is for practice, with quoted evidence, and not a score.

Sources

  1. Canva Engineering Blog: Simon Newton on AI in Canva's interviewsCompany's own page. 11 June 2025. Checked 18 October 2026.
  2. Rootly blog: Quentin Rousseau on take homes reviewed through AI transcriptsCompany's own page. 24 February 2026, updated 23 June 2026. Checked 18 October 2026.
  3. Moneywise on Yahoo Finance, reporting Business Insider: Google's AI assisted interview pilotPress report. 14 May 2026. Checked 18 October 2026.
  4. Zapier blog: raising the AI fluency bar in hiringCompany's own page. Undated. Checked 18 October 2026.
  5. The Pragmatic Engineer podcast: Shopify's Farhan Thawar on AI and hiringPress report. 2 July 2025. Checked 18 October 2026.
  6. interviewing.io: guide to Rippling's interview processInterview prep site. Undated. Checked 18 October 2026.
  7. Hello Interview: guide to Meta's AI enabled coding roundInterview prep site. Undated, describes the October 2025 rollout. Checked 18 October 2026.
  8. Kumar and others: 19 Microsoft developers working real issues with an agent (arXiv 2506.12347)Research paper. Version 3, October 2025. Checked 18 October 2026.
  9. Mehra and others: prompt scores of 15 professional developers (arXiv 2607.06074)Research paper. 7 July 2026. Checked 18 October 2026.
  10. Thorgeirsson, Weidmann and Su: what predicts vibe coding proficiency, 100 students (CHI 2026)Research paper. 14 March 2026. Checked 18 October 2026.
  11. Anthropic: about 400,000 Claude Code sessions and the returns to expertiseResearch paper. 16 June 2026. Checked 18 October 2026.

Last checked: .