Where a person belongs in an AI workflow
Keep a person in the loop wherever a wrong result is expensive or hard to undo. Three patterns cover it: review before the action, review after it with a reversal path, and sampled audit of an unattended run. Choose one per decision, set the thresholds that route cases to a person, and design the queue so review takes seconds.
Submit a project AI workflow automationReviewed by David (CEO) · Updated 24 Sep 2026 · 10 min read
OutsourcingVN is operated by Netbase JSC, which scopes and runs accountable project teams, so we have a commercial interest in your decision; the advice below holds whoever builds the workflow. This page is a spoke of the AI workflow automation guide and assumes you have already decided that some of the work should be automated.
The three review patterns
Most arguments about "human in the loop" are arguments about which of three patterns applies. They are not ranked, and one workflow usually uses more than one, because a workflow contains more than one decision.
| Pattern | What the reviewer does | Right when | The trade-off |
|---|---|---|---|
| Review before the action | Approves, edits or rejects the output before anything leaves the system | The action reaches a customer, moves money, changes a record with legal weight, or cannot be undone | Every case waits for a person, so throughput is capped by reviewer capacity |
| Review after the action, with reversal | Checks a flagged subset after the fact and can reverse inside a defined window | The action is cheap to undo and a short-lived error costs less than the delay would | A real undo path must be built, tested and monitored, which people forget to scope |
| Sampled audit of an unattended run | Inspects a scheduled sample, plus every case the workflow flagged itself | The workflow has a measured record on an evaluation set and the worst outcome is tolerable | Errors surface late; the sample must be large enough and drawn without bias |
The delivered record on this site closest to the first pattern is an anonymised classifieds platform where AI-powered content filtering detected and flagged offensive content into an admin moderation dashboard, the administrator deciding. That is a queue, not an automatic removal.
Pick the pattern per decision, not per workflow
The common error is applying one pattern to a whole workflow because it is easier to explain. A support workflow may draft a reply (review before sending), tag the case (unattended, audited weekly) and update a CRM field (act now, reverse if the audit disagrees). Three decisions, three patterns.
Write the decisions out as a list first. For each, answer two questions: what is the worst thing that happens if this is wrong, and how long does it take to put right? Those answers pick the pattern almost mechanically, and if you cannot answer them the honest next step is a mapping exercise, not a build. The same list settles the agent question: an agent choosing its own steps needs review at the boundary of what it may touch, not at the end — the difference described in AI agents versus workflow automation.
Thresholds: confidence, impact, and the pair together
Routing a case to a person is a rule, and the rule needs inputs you can observe.
Confidence is the model's own signal, and alone it is weak. A score is not a probability of being right until you have checked it against labelled cases, and a model can be confidently wrong in exactly the situations you care about. Use confidence to rank a queue, not to decide.
Impact is the business signal, and you control it entirely. Order value, account tier, whether the customer is already in a complaint, whether the record feeds a regulated report: these are facts in your own systems, and they do not drift when a model version changes.
The rule that survives production combines them. Route to a person when impact is high, whatever the confidence, and when confidence is low and impact is anything but trivial. Let the workflow act alone only where both signals are comfortable. Set the first thresholds conservative so the early weeks over-review, then move them on evidence from the evaluation set rather than on how busy the queue feels; where those numbers come from is the subject of AI workflow evaluation and testing.
Designing the queue so review takes seconds
A review step becomes the bottleneck when the reviewer has to redo the work to check it. The design problem is putting enough evidence beside the decision to make the answer obvious at a glance.
A queue item earns its place when one screen shows the proposed output; the input that produced it, quoted rather than summarised; the rule that routed this case here; the sources the output depends on; and the actions — approve, edit and approve, reject with a reason, escalate. The reject reason is not paperwork. It is the evidence for the next threshold change, and a queue that does not capture it throws away its most valuable output.
Three habits keep it fast. Group similar cases so the reviewer stays in one context. Show their own recent decisions so handling stays consistent. Put a visible age on every item, because the failure mode of a review queue is not a wrong decision, it is a queue nobody has looked at since Tuesday. And put the people who will work it in the room when it is designed, as the AI workflow automation service does: a reviewer will tell you in ten minutes which field is missing.
Designing the escalation path
A workflow with no defined exception path is not ready to go live. Not "needs improvement" — not ready. The exception path turns an unpredictable system into an accountable one, and it is designed rather than discovered during the first incident.
-
List the exception classes
At minimum: the model abstains or returns low confidence; the input is malformed; the downstream system refuses the write; a service times out; the output breaks a rule you set; the case falls outside the tested scope.
-
Make abstention a valid output
The workflow must be able to say "I do not know" and have that be a normal, logged path, not an error. A model never allowed to abstain will guess, and a guess enters the system looking like an answer.
-
Decide what a refusal does
When a downstream system rejects the write, the case must not silently disappear. Choose between retry, queue and escalate, and make the partial state visible.
-
Name the owner and the clock
Each class gets a role that receives it and a time within which it is picked up. A queue with no owner is a backlog.
-
Decide the default when nobody acts
Past its window, does an item fail closed with the customer told, or fail open with a safe default? Choose per class, in writing, before launch.
-
Write the reversal
For anything acting before review, specify how a completed action is undone, who may undo it and what the customer sees. No reversal means review belongs before the action.
-
Test the exception path
Put abstentions, refusals, timeouts and out-of-scope inputs into the evaluation set and the pilot. A pilot that runs only clean cases has proved nothing.
Who is accountable for a wrong decision
Accountability does not transfer to software. In every design we will agree to, a named role owns the outcome of each decision class, and the workflow records who or what decided, on what evidence and when.
Three things make that real: a decision log holding the input, the output, the rule that fired, the reviewer and the timestamp, kept long enough to answer a complaint; a separation between whoever tunes the thresholds and whoever owns the business outcome, so loosening one is signed rather than quietly configured; and a written statement of what the workflow must never do, enforced in code rather than in a prompt.
Netbase delivers remote-first from Hanoi in Agile increments with weekly reviews, using AI-assisted engineering under human review, and project teams draw on business analysis, project management, solution architecture, development, QA and UI/UX roles. The weekly review is where threshold and queue changes are discussed with you rather than applied quietly; project delivery explains how they are recorded.
How the review load should fall, and what it means when it does not
A healthy design shows a falling share of cases needing review while the quality of reviewed decisions holds. If you cannot plot that, you are not measuring the workflow.
- Share falls, reject rate stays low. The thresholds are tightening correctly. Keep going, and keep the audit sample.
- Share falls, reject rate rises. Someone loosened the thresholds faster than the evidence justified. Move them back.
- Share is flat for months. Either reject reasons are not feeding back, or the workflow is being applied to cases it never suited. Neither is fixed by a different model.
- Share falls to zero by decision, not by evidence. Removing the person because the queue was inconvenient is not maturity, it is an unmeasured risk transfer.
Workflow families settle at very different floors: AI automation use cases shows where an unattended run is realistic, and an inbound queue like customer support automation keeps a person on the hard cases permanently.
Somebody has to run this queue after handover. Our claim record supports saying that Netbase offers dedicated development teams, on-demand support and fully managed delivery as secondary options alongside fixed-scope project work, within a lifecycle running from discovery through team assembly, agile execution with outcome-based milestones, training and rollout, to ongoing support. It does not support a published coverage window, a response commitment, or any delivery record of Netbase staffing a client's review queue. Managed operations is therefore an assessment-led starting point where scope, coverage and reporting are agreed per engagement, not a shelf product with a service level printed here.
Plan the next step for your project
Common questions
No, if the person decides rather than re-does. The saving comes from the routine cases the workflow finishes alone, not from removing judgement where judgement is needed.
Usually your people at first, because they hold the judgement the workflow is missing. Whether some of it moves to a managed arrangement later is a separate, assessed decision.
That is information, not a defect. It usually means the inputs are worse than assumed or the task was drawn too wide. Narrow the task before you change the model.
Very little. The thresholds, queue and exception path are the architecture; the model is a component inside it, as choosing an AI model for business workflows argues.
Scope the review step with the build
The delivered record here that touches conversational AI is an anonymised WhatsApp Business AI chatbot with CRM integration, built with intent and conversation-flow handling, an LLM API and CRM synchronisation, delivered in milestones from design and prototype through documentation, knowledge transfer and 30 days of support over four to eight weeks. It evidences the build and milestone pattern, and it claims no human hand-over, so it does not evidence the review design on this page; the rules behind that distinction are on the methodology page. When you are ready, submit a project with your decision list and the worst-case answers, and we will scope the review step with the build. OutsourcingVN is operated by Netbase JSC and is Netbase's own outsourcing-services platform.
Work behind this guide
The client, sector, region and CRM product are not named, and no conversation volume or business result is claimed.
Keep ReadingRelated services and solutions
AI workflow automation with evaluation and human control
AI Workflow Automation starts with one operating workflow, one accountable owner and one agreed way to judge the result. The goal is a workflow that handles the routine cases correctly on representative test cases, routes uncertain or high-impact cases to a person, and can be monitored and changed after handover. It is not a promise that every process can or should be automated.
Learn More