Bringing AI agents into client servicing
How I approach putting agents in front of client-servicing work: agents read, route and draft, and a person sends every message.
Context
Client servicing in a bank starts in an inbox. Specialists read each request, work out what kind it is, look up the client, and then write a reply and start the paperwork. It is skilled work, and much of it is reading and retyping.
Language models are good at exactly that reading and drafting. The question was not whether agents could do it in a demo, but whether they could do it in a way a bank could put in front of clients and a risk committee could approve.
Constraints
- Nothing reaches a client without a person approving it.
- Client email is untrusted input and may try to instruct the system.
- Agents must not decide who is authorized to request what; that answer lives in systems of record.
- Quality has to be measured before any talk of scaling, in a form risk partners can review.
- Many request types, with very different costs of getting each one wrong.
Options on the table
The fastest on paper, and the hardest to approve. One confidently wrong email to a client undoes the efficiency case, and there was no measured baseline yet to argue it was safe.
Low risk, but it leaves specialists doing the reading, the lookups and the form-filling. The gain is small and hard to measure.
Agents do the reading, classification, extraction and drafting. A specialist reviews one finished artifact and sends it. That puts a single, well-defined control in front of the client and turns every approval, edit and rejection into labeled data.
The decision
A classifier routes each inbound request to either an agentic flow or manual handling. In the agentic flow, agents extract what the request needs and draft the reply and the paperwork for a specialist to review.
"Human always sends" is the operating principle. Quality is measured with golden datasets and LLM-as-judge scoring, against labeled cases, before anyone discusses expanding scope.
My role
Product lead: scope, the human-in-the-loop design, success measures, and alignment with servicing, technology and risk partners.
Defined the evaluation approach, and work on the classification, MCP integration and multi-agent design underneath it.
Outcome
Results are internal, so this write-up covers the method rather than the numbers. The work is designed to produce two things: a measured view of agent quality by request type, and a control model that risk partners could evaluate.
The patterns behind it are written up in Field Notes, with scripted runs you can step through.
What I would change
Start building the golden dataset in the first week, not after the first prototype. The labeled set is the long pole, and every design debate goes faster once it exists.
Agree the release rule (what quality, on which request types, earns a wider rollout) with risk partners before the first eval run.