A system can be active all day and still fail to move a business forward. This is especially easy to miss with AI-assisted workflows: the dashboard shows attempts, messages, retries, and status changes, but the result the business needed may still be missing.
Activity has a role in diagnosing what happened. It is not the same thing as progress. If a company wants to use AI responsibly in its daily operations, it needs a better way to answer one question: What changed that a person can verify?
Three different things often get called progress
Many operational dashboards blur three distinct categories:
- Activity: a tool ran, an API request was attempted, a document was read, or another retry began.
- Evidence of work: a reviewed file changed, a test passed, a purchase decision was documented, or a customer-facing response was actually sent.
- A completed outcome: the business objective was met, quality was checked, and the responsible person accepted the result.
These are not interchangeable. A worker can be active without creating evidence. It can create evidence without achieving the requested outcome. And it can produce an apparent result that still fails the quality or authorization checks required to put it into use.
That distinction applies just as much to procurement, customer service, and financial workflows as it does to writing software.
A practical example: the buying queue
Consider a hypothetical company that uses an AI assistant to refresh its buying queue. The assistant repeatedly gathers supplier information, checks prices, and reports that it is processing requests. Twenty minutes later, the buying queue still has not changed.
How should the manager evaluate that work? Not by counting the checks. The useful evidence would be an updated queue with traceable supplier data, a clear reason for each recommendation, and any exceptions flagged for human review. Completion would mean that the person accountable for purchasing can actually use the refreshed queue.
Now imagine the assistant successfully updates the queue but marks an unavailable item as immediately purchasable. The artifact changed, yet the workflow still failed an important business test. Progress requires both output and validity.
Five signals worth putting on an AI operations dashboard
1. Last meaningful milestone
Record when the last durable, reviewable result happened. For technical work, that might be a committed change, test result, or approved deployment. For operations, it might be a corrected record, a verified purchase order, or a completed service action. A heartbeat or retry is still diagnostic information, but it should not reset the progress clock.
2. Evidence attached to the milestone
A status update such as “completed the review” is stronger when it links to the work: the changed record, test report, decision note, or actual outbound communication. Managers should not have to guess what “done” means.
3. An accountable owner
Every automated workflow needs a person or team with authority to accept the outcome and handle exceptions. Automation can perform actions, but responsibility for the business process does not disappear.
4. A separate waiting or blocked state
If a workflow is waiting for credentials, a supplier response, a failed integration, or human approval, label it that way. Reporting all of those situations as “working” creates a misleading sense of motion. A useful system distinguishes active execution from waiting, blocked, review, and complete.
5. A recovery procedure
Stalled work needs a defined response: determine the failure, limit retries, preserve the evidence, assign a person, and decide whether to resume, stop, or roll back. Repeating the same failed step indefinitely is not a recovery strategy.
What should the watchdog actually watch?
A simple monitoring rule can look for the time since the last accepted milestone, not only the time since the last tool call. It can also check whether the same failure keeps recurring, whether the task has exceeded its reasonable time budget, and whether it has any evidence of progress at all.
Those checks should be tailored to the work. A supplier quote may legitimately take days. A straightforward data validation should not need hours. The goal is not to force identical thresholds on every process; it is to make expected timing and escalation explicit.
A 30-minute audit any operations team can try
Choose one AI-assisted process and answer five questions:
- What specific business outcome was this automation created to produce?
- What artifact or event would independently demonstrate useful progress?
- Who has the authority to approve the result and handle exceptions?
- What will the system display when it is waiting, blocked, or repeatedly failing?
- When should a human intervene, and how will the process recover safely?
If the answers are unclear, that is a workflow-design problem worth solving before adding more automation. Start with one process, record a few meaningful milestones, and adjust based on what the evidence reveals.
The measure of a useful system
Businesses do not gain much from software that looks busy but leaves decisions, customers, and employees in the same position. A good automation program makes real progress visible, makes failures easier to diagnose, and strengthens the people who remain accountable for the result.
The most useful question is not “How many times did the AI run?” It is “What did it deliver, how was that checked, and who can use it?”
This article discusses general operational design principles. The buying-queue example is hypothetical and does not describe a particular customer’s records.
For interviews or collaboration on business operations, responsible automation, and practical systems design, see the Media & Speaking page.