AI should improve a workflow, not become an uncontrolled workflow
AI adoption is moving rapidly from isolated chat tools into operational processes.
Businesses are beginning to use AI to:
- classify information;
- summarise documents;
- triage requests;
- extract data;
- recommend next actions;
- draft responses;
- route cases;
- support decisions.
That creates real opportunity.
It also creates a new design problem:
Which decisions should the AI make, which should remain deterministic, and which require a person?
Microsoft’s current workflow guidance makes this distinction explicitly: deterministic business logic should remain in normal code, while human-in-the-loop gates are appropriate when a person needs to make the call.
That is a useful starting principle.
Not every automation problem requires AI
Before adding an AI model, ask whether the task is actually uncertain.
If the business rule is:
“Invoices above R100,000 require finance-director approval.”
you do not need AI.
That is deterministic logic.
If the task is:
“Read this customer complaint and determine whether it appears urgent, routine or potentially high-risk.”
AI may be useful because interpretation is required.
This produces three broad categories.
Deterministic
Rules are known and should produce consistent outcomes.
Examples:
- thresholds;
- permissions;
- calculations;
- routing rules;
- required fields;
- access controls.
AI-assisted
Interpretation is useful, but the output should inform rather than automatically determine a consequential outcome.
Examples:
- summarisation;
- classification;
- extracting themes;
- drafting;
- identifying possible anomalies;
- prioritisation.
Human decision
The consequence, ambiguity or risk warrants explicit human judgment.
Examples:
- disciplinary action;
- significant financial approvals;
- legal determinations;
- high-risk customer actions;
- safety-sensitive decisions;
- irreversible changes.
The architecture should reflect those differences.
Human-in-the-loop should be designed, not added afterwards
Many organisations describe their system as having “human oversight.”
That can mean almost anything.
Effective oversight needs a defined point in the workflow.
For example:
AI assesses → system pauses → reviewer receives evidence → human approves or rejects → workflow continues
Microsoft recommends explicit human approval for consequential or difficult-to-reverse actions and escalation of sensitive or ambiguous cases.
The reviewer also needs enough context to make the decision.
A human approval button without useful evidence is not meaningful oversight.
Start with risk, not novelty
The appropriate level of control depends on what can go wrong.
NIST’s AI Risk Management Framework is designed around identifying and managing AI risks throughout the lifecycle of a system rather than treating AI governance as a once-off compliance exercise.
Ask:
- What happens if the model is wrong?
- Can the outcome be reversed?
- Does it affect a person materially?
- Does it involve money?
- Does it expose sensitive information?
- Does it influence a legal or compliance process?
- Will someone understand why the decision occurred?
- Is the output being checked?
A low-risk content summary does not require the same controls as an automated decision affecting a customer account.
AI outputs should not automatically be treated as facts
Generative AI can produce plausible but incorrect output.
That creates a fundamental workflow-design requirement:
The system must know when verification is required.
NIST’s Generative AI Profile notes that organisations may need additional human review, tracking, documentation and management oversight when generative AI is used.
A practical workflow can therefore differentiate:
Low-risk output
Draft or summarise automatically.
Moderate-risk output
Produce a recommendation with source evidence.
High-risk output
Require explicit human approval.
The objective is not to distrust AI completely.
It is to calibrate trust to consequence.
Traceability matters
If an AI-assisted workflow affects business operations, the organisation should be able to reconstruct what happened.
Depending on the use case, that might include:
- input received;
- model or system used;
- output generated;
- rule triggered;
- person who reviewed it;
- action taken;
- timestamp;
- later correction.
The OECD AI Principles specifically emphasise accountability and traceability across AI-system lifecycles.
This becomes particularly important when someone later asks:
“Why did the system do that?”
Without traceability, the organisation may have no defensible answer.
Transparency should match the context
If an AI system is interacting directly with customers or employees, people should understand what they are interacting with.
The OECD Principles call for appropriate transparency about AI capabilities, limitations and decisions, including enough information for affected people to understand or challenge outputs where appropriate.
Microsoft similarly recommends clear AI disclosure for external-facing agents.
This does not mean every internal automation needs a lengthy AI disclaimer.
It means transparency should be proportionate to the use case.
Data access should be narrower than model capability
An AI model may be capable of processing enormous amounts of information.
That does not mean it should have access to all of it.
AI-enabled workflows should apply familiar security principles:
- least privilege;
- role-based access;
- data minimisation;
- defined retention;
- controlled integrations;
- audit logs.
If an AI assistant only needs customer-order information, it should not automatically receive payroll, HR and executive data as well.
Good AI governance begins with good system architecture.
The model should not control everything
One of the strongest architectural patterns is to separate intelligence from control.
For example:
AI
“This message appears to contain a potential payment-fraud indicator.”
Deterministic rule
Risk score above threshold → stop automated processing.
Human
Review the evidence and decide whether to escalate.
In this pattern, AI contributes judgment without becoming the ultimate control mechanism.
That is materially different from:
AI decides → system acts → nobody reviews.
Microsoft’s workflow guidance explicitly supports separating model-driven reasoning, deterministic workflow logic and human gates depending on who should control each step.
Use AI where ambiguity creates expensive manual work
Strong candidates often share several characteristics:
- high volume;
- repetitive interpretation;
- significant manual reading;
- structured downstream actions;
- measurable outcomes;
- manageable consequences if the AI is wrong.
Examples could include:
Inbound enquiries
AI categorises and routes them.
Documents
AI extracts key information while validation rules check required data.
Operations
AI identifies anomalies for staff review.
Customer service
AI drafts responses while sensitive cases escalate.
Sales
AI summarises activity and highlights follow-up opportunities.
Compliance
AI assists review while authorised people retain decision responsibility.
The purpose is not replacing every human step.
It is reducing the amount of human attention spent on low-value interpretation so people can focus on decisions that actually require judgment.
Pilot the workflow before automating the organisation
AI implementations should begin with a controlled process.
Define:
Input
What does the AI receive?
Task
What exactly should it do?
Allowed output
Recommendation, draft, classification or decision?
Risk threshold
When must it stop?
Human gate
Who reviews sensitive cases?
Evidence
What gets recorded?
Measure
How do we know whether it improves performance?
Then test.
NIST’s AI Resource Center emphasises testing, evaluation, verification and validation as part of operationalising responsible AI.
A workflow that cannot be measured should not be scaled confidently.
Measure more than speed
Automation projects often report only:
“We reduced processing time.”
Useful, but incomplete.
Also measure:
- error rate;
- false positives;
- escalation rate;
- human override rate;
- customer outcome;
- rework;
- cost per case;
- compliance failures;
- user trust.
A faster workflow producing worse decisions is not an improvement.
The Juchepi perspective
At Juchepi Group, we believe AI works best when it sits inside a deliberately designed operating system.
That means separating:
Rules that should always behave consistently
from:
Judgment that AI can assist
from:
Decisions that should remain with people.
The objective is not maximum automation.
It is controlled automation.
A well-designed AI workflow should make the business:
faster where speed matters, more consistent where rules matter, and more deliberate where judgment matters.
Exploring AI-enabled workflow automation?
Juchepi Group helps organisations map workflows, identify appropriate AI use cases and design custom systems combining automation, deterministic controls and human oversight.
Start a technology conversation with Juchepi Group.