Hiring for AI Taste

Every AI hire I have regretted looked great on paper. Every AI hire I would hire twice made me uncomfortable at some point in the interview.

Taste, in this context, is the ability to tell, quickly, which problems are worth solving with AI and which are not. It is the thing that separates an engineer who ships a useful agent from one who ships a beautiful demo nobody uses.

I have a four-question screen. I will share it because the people I want to hire will pass it either way, and the people I don't want to hire will game it badly enough that I will catch them.

One: tell me about an AI project you killed. Not paused. Killed. If they have never killed a project, they are not making decisions, they are watching demos. I want to hear the eval they ran, the number that came back wrong, and the calendar invite they sent.

Two: show me a system where you chose not to use an LLM. This is the taste question. The right answer is usually a small ML model, a heuristic, or a regex. The wrong answer is a defensive speech about agent orchestration.

Three: walk me through your eval before your model. If they start with architecture, they are a hobbyist. If they start with how they measure whether the system is working, they are a professional.

Four: what is the stupidest-looking thing you have built that worked? This one is my favourite. The answer is always a rule, a cache, or a hand-written prompt. People who have actually shipped systems have a long list of these. People who haven't, don't.

What I'm really screening for is a bias toward the cheap, the legible, and the falsifiable. Those three adjectives will get you further in production AI than any benchmark on the leaderboard.