The Second User
By Benjamin Evans
She refreshed her email eleven times that morning.
Not because she expected good news. Because silence, after an application, has a particular weight to it. You learn to read the nothing. Three days of nothing usually means the résumé was seen and passed over. A week of nothing means it probably wasn't seen at all. Two weeks and you stop checking — not because you've given up, but because your body has quietly done the math for you and decided the hope isn't worth the cortisol.
She didn't know that a product had already made the decision. An AI-powered screening tool — clean interface, fast results, designed with care — had ranked her 85th out of 300 applicants within four seconds of her submission. The recruiter who opened the tool that afternoon saw a tidy list. The top twenty candidates had green indicators. Hers did not appear on any screen the recruiter ever looked at.
The product worked exactly as designed.
It just wasn't designed for her.
The experience nobody mapped
This is not a story about a flawed algorithm. The ranking may have been perfectly defensible. She may not have been the strongest candidate. That's not the point.
The point is that a product team spent months designing the experience of the person holding the interface — the recruiter. They mapped her workflow. They reduced her clicks. They tested her satisfaction. They measured her time-to-hire. They made the tool feel fast and smart and trustworthy.
Nobody designed the experience of being ranked.
Nobody mapped what it feels like to submit your qualifications into a system that will process them in ways you cannot see, by criteria you cannot know, with consequences you cannot appeal. Nobody tested how a human on the output side of a decision experiences the gap between applying and hearing nothing. Nobody measured the cost of that silence.
This isn't a design oversight. It's a structural blind spot in how we build.
Held by the interface
Almost every AI-powered decision tool has two users. The first user holds the interface. The second user is held by it.
The recruiter and the candidate. The loan officer and the applicant. The content moderator and the creator. The radiologist and the patient. The claims adjuster and the policyholder. In every case, a product team has designed — often beautifully — for the person making the decision. And in almost every case, the person affected by the decision doesn't get a flow. They don't get an onboarding screen. They don't get a loading state or a helpful tooltip or an error message that explains what went wrong.
They get the outcome. Or more often, they get nothing at all.
We have a name for the first user. We call them "the user." We have no name for the second one. We don't have a persona for them. We don't have a journey map. We don't have a single line item in the product requirements document that says: and here is how the person on the other end of this decision will understand what happened to them.
The absence of a name is the tell. You can't design for someone you haven't named.
What scale actually changes
This pattern isn't new. It predates AI by decades.
Every product that makes a decision about a person — approves or denies, ranks or filters, flags or clears — has always had a second user. Credit scores have second users. Background checks have second users. Insurance underwriting has second users. The person affected by the system's output has always been there, on the other side of the wall, experiencing consequences they didn't choose and can't fully see.
What's different now is scale.
A human underwriter reviewing loan applications might process forty in a day. An AI system processes forty thousand. The same design choice — to optimize for the decision-maker and ignore the decision-recipient — now affects orders of magnitude more people per hour. A single interface decision about how a ranking is displayed, which information is surfaced, what threshold triggers a green or red indicator, ripples through thousands of lives before lunch.
And the speed itself creates a new problem. When a human makes a decision about you, there's at least the theoretical possibility of being seen. The human might notice something the criteria missed. Might pause. Might feel the weight of the choice.
AI doesn't pause. It doesn't feel weight. It generates an output at the same speed whether the input is a routine application or someone's last chance at a job before their savings run out. The computational cost of the decision is disconnected from its human cost. Just as a model generates tokens at the same rate for a cookie recipe and a cancer question, a screening tool processes a CEO's nephew and a single mother re-entering the workforce with identical indifference.
The product can't tell the difference. That's not a flaw. That's the design challenge.
The scammer and the grandmother
A verification flow doesn't know whether the person on the other end is a scammer or a grandmother trying to send rent money. The design has to hold both possibilities with care.
I've spent a lot of time thinking about this exact problem — in a context that had nothing to do with AI.
At a fintech company, we built verification flows. Identity checks. The kind of screens where a product asks you to prove who you are before it lets you do what you came to do. These flows have two users too: the company verifying identity, and the person being verified. For years, the product had been designed entirely for the first user — the system. Fast confirmation. Low fraud rates. Clean data.
The second user — the person being asked to hold up an ID to a phone camera, the person waiting for a decision about whether they're allowed to access their own money — had been an afterthought. The flow treated them like suspects. Sterile prompts. No explanation. No timeline. No indication of whether they'd done something wrong or the system was just being cautious.
The design had to hold an impossible tension: the person on the other end of a verification might be a scammer, or might be a grandmother trying to send rent money. The system can't tell. And the old design resolved that ambiguity by defaulting to suspicion — because suspicion was cheaper than care.
We redesigned the flow around a different default: respect. Transparency about what the system needed. Clarity about what would happen next. A tone that acknowledged the person might be exactly who they said they were. The same security. A completely different experience of being secured.
There's a version of safety that looks like a locked door. And there's a version that looks like someone walking you home.
What changed wasn't the decision. It was whether the second user felt like a person or a case number.
The consequences aren't distributed evenly
The AI industry is building thousands of products right now that make decisions about people. Hiring tools, lending platforms, content moderation systems, diagnostic aids, insurance models, risk assessments. Every one of them has a second user. Almost none of them have been designed for one.
And the consequences aren't distributed evenly.
The people most affected by opaque AI decisions are, almost always, the people with the fewest alternatives. The job applicant who can't afford to wait another month. The loan applicant who doesn't have a backup bank. The content creator whose income depends on an algorithm they can't see. The patient whose diagnosis arrives through a system their doctor barely understands.
These aren't edge cases. When a system makes decisions about the most vulnerable people in its orbit without designing for their experience of those decisions — that's not a gap at the margins. That's a failure at the center.
The evidence is already in. In the Netherlands, an automated system used ethnic profiling to flag 26,000 families as childcare benefit fraudsters. Parents were forced to repay tens of thousands of euros. A $27 error led to a $27,000 recovery demand. Over a thousand children were taken into care. The scandal brought down a sitting government. The system worked exactly as designed. The people it was designed to process had no interface, no explanation, and no appeal.1
Virginia Eubanks documented the same pattern in the United States — Indiana denying one million applications for healthcare and food assistance because a new computer system interpreted any administrative error as "failure to cooperate."2 As Eubanks observed: these tools are not disrupters so much as they're amplifiers. They don't create new injustices. They scale the ones we already tolerate.
The hardest thing about this failure is that it's quiet. The second user doesn't file a bug report. They don't leave a one-star review. They don't show up in your NPS survey, because they were never your user in the first place. They find another way. Or they don't.
And the product team never sees the data, because they never built the instrument to capture it.
What designing for the second user would require
So what would it look like to actually design for the second user?
Not perfectly. Not comprehensively. But at all.
It would start with naming them. Literally. A persona, a journey, a set of needs that appear in the requirements document alongside the primary user's needs. Not as a compliance exercise. As a craft commitment. If you're building a product that ranks people, the people being ranked are your users. If you're building a product that approves or denies, the people being denied are your users. Design for them or accept that you haven't.
It would mean designing the output experience with the same rigor as the input experience. What does the candidate see? What does the applicant receive? Not a form letter generated after the fact. An experience — considered, clear, honest — that respects the person's time and stakes. A rejection that explains what it can. A timeline that sets real expectations. An interface, even a simple one, that says: we know you're on the other end of this.
Show Image
It would mean building measurement for the second user's experience. Not just the decision-maker's satisfaction score. The applicant's comprehension. The patient's trust. The creator's sense of fairness. These metrics don't exist in most products because nobody thought to build them. But you can't change what you can't see. The first job is always building the instrument.4
And it would mean asking a question that most product teams never ask, because the answer is uncomfortable: if the second user could see exactly how this system works — the criteria, the weights, the thresholds, the training data — would they feel the process was fair?
If yes, show them.
If no, don't fix the interface. Fix the system.
A new unit of design
Show Image
There's a version of this essay that ends with a framework. A tidy set of principles. A checklist for responsible AI product design.
That's not this essay.
Because the real issue isn't that product teams lack a framework. It's that the fundamental grammar of product design — the user, the flow, the journey, the persona — was built for a world where one person uses the product and the same person experiences the consequences. That grammar breaks when the person affected by the product's decision is someone who never opens it.
We don't need a new checklist. We need a new unit of design. A way of thinking about products that begins with the question: who is affected by this, and what is their experience of being affected?
The recruiter had a good morning. The tool was fast. The shortlist was clear. She moved twelve candidates to the next round before lunch and felt productive.
Eighty-five names below the fold, a woman refreshed her email for the twelfth time.
The product worked exactly as designed.
Every product that makes a decision about a person has two users. We've been designing for one of them. The other has been waiting — in silence, without a flow, without a name — for someone to notice they exist.
Further Reading
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor — Virginia Eubanks (2018). The foundational text on how algorithmic systems scale existing inequality. virginia-eubanks.com/automating-inequality
"Algorithmic Accountability and Public Reason" — Philosophy & Technology (2019). On the obligation of algorithmic decision-makers to justify their systems to "decision-subjects." PMC/NIH
"How Algorithmic Systems Automate Inequality" — TechPolicy.Press (2026). On how algorithmic discrimination shifts the burden of proof onto those with the least information. techpolicy.press
"Examining Perceptions Towards Hiring Algorithms" — ScienceDirect (2022). Nationally representative study finding the public views algorithmic hiring as neither fair nor effective. sciencedirect.com
"Exploration-Based Algorithms Can Improve Hiring Quality and Diversity" — MIT Sloan (2020). Research showing that algorithm design choices directly determine who gets access to opportunity. mitsloan.mit.edu
Dutch Childcare Benefits Scandal (Toeslagenaffaire) — How a fraud-detection algorithm using ethnic profiling wrongly accused 26,000 families and brought down a government. Wikipedia · Amnesty International report
"Amazon Scraps Secret AI Recruiting Tool That Showed Bias Against Women" — Reuters (2018). The case study that demonstrated algorithmic hiring bias at scale. reuters.com
