AI Coworker Product Experiments: Fast Changes Still Need a Decision Rule
Amplitude's Wave case study shows how quickly an AI coworker can turn observed friction into a shipp...
A convincing answer from an AI coworker may be based on a handful of retrieved rows rather than the complete workbook. Learn how to verify scope, calculations, policy, and the source version before acting on it.
On November 16, 2026, AWS announced tabular search for Amazon Bedrock Managed Knowledge Base. Its example is deceptively ordinary: a finance team asks which departments need variance commentary, a question that requires the full profit-and-loss workbook, a threshold in policy, and a revision in a separate memo. The important change is not that an AI coworker can read a spreadsheet. It is that retrieving a few relevant-looking passages is not the same as calculating over every relevant row.
That difference matters to businesses far outside finance. An operations lead asks which orders missed a cutoff; a service manager asks which customers had a second contact; a shop owner asks which products fell below reorder level. If the system searches a workbook as if it were a set of paragraphs, it may find three persuasive examples and silently miss the other thirty. A fluent answer is not evidence that the whole table was examined.
AWS describes a retrieval flow that copies the original CSV or spreadsheet into a network-isolated sandbox, runs code over the whole file when a question requires it, and returns the calculated result with citations and a trace naming the workbook. That is a useful design pattern, not a guarantee that every business question is now safe to automate. Someone still has to establish which files count, whether the policy is current, whether the calculation matches the business definition, and who may act on the answer.
The operational question is therefore not, “Can our AI coworker summarize this workbook?” It is, “Can we reconstruct exactly which workbook, rows, rule, and calculation produced this conclusion?” If the answer is no, keep the output in the research lane. Do not promote it to a decision, a customer promise, or a write to a live system.
Search systems are excellent at finding passages that appear responsive to a question. A table question often has a different shape: it asks for a set, a total, an exception count, or a comparison across all eligible records. The missing record is often the most consequential one, and by definition it does not appear in the answer you were shown.
Imagine a service business with a workbook of open jobs. The manager asks, “Which jobs are past the customer-promised date and still have no owner?” A retrieval system might surface a row with a late job, a policy excerpt defining overdue work, and a note from a supervisor. Those sources can support a plausible response. They cannot establish that every sheet, hidden row, revised date, and unassigned job was considered.
The right query needs a defined population: active jobs as of a stated time, from the approved workbook version, across named sheets and status codes. It needs a rule for blank owners, a timezone for promised dates, and treatment of canceled or duplicate work. Those details are not pedantry. They determine who gets called today and who gets left waiting.
A calculation can be mathematically correct on the wrong subset. Conversely, a complete scan can apply the wrong definition of “late.” Ask for two kinds of evidence. First, population evidence: source identity, file version, sheets, filters, row count before and after filtering, and any files excluded. Second, computation evidence: formula or code, units, date handling, treatment of missing values, and the resulting exception IDs.
AWS's release explicitly distinguishes its tabular-search path from ordinary passage retrieval: the new path runs code over the original file in a sandbox when the question needs a whole-table lookup or calculation. It also says the ordinary Retrieve API still returns passages without running code. The practical lesson is to identify which path your own system actually used. A vendor feature announcement is not a substitute for a trace of this particular answer.
The same distinction applies when a person does the work manually. A well-designed AI coworker can make coverage and calculation easier to inspect than an analyst's undocumented spreadsheet filter. It should not become a new source of unexamined confidence. If the workbook changes during the task, the coworker should either pin a snapshot or tell the reviewer which version the answer reflects.
The AWS finance example depends on a threshold policy, a quarterly memo, and a P&L workbook. A department whose variance crosses a threshold may still be exempt under the memo's revised budget treatment. Another may require commentary under the policy even if its numeric variance looks small, because a nonfinancial condition applies. The answer cannot be derived by searching only the workbook or only the prose.
Build a source-of-truth map for each question. Name the approved policy owner and effective date, the workbook's origin and refresh cadence, and any exception memo that can modify the rule. If two documents conflict, the coworker should surface the conflict and route it to a person authorized to decide which rule applies. It should not choose whichever excerpt is more convenient or more recent-looking.
This is a different problem from simply getting an AI coworker to explain a number. The answer may involve a perfectly computed total while still being wrong about the population or the policy that gives the number meaning. The seam between finding evidence, computing a result, and authorizing an action must remain visible.
Consider a fictional distributor with three inventory sheets: current stock, backorders, and discontinued SKUs. The operations lead asks which products need a purchase request before Friday. The policy says reorder when available stock minus committed orders falls below the SKU-specific minimum; discontinued SKUs must never be reordered; a supplier delay memo changes the lead time for one product family.
Bad: The AI coworker finds a row showing low stock for a familiar product and drafts a purchase request for ten units. It cites the stock sheet but never checks committed orders, the discontinued list, or the supplier memo. The arithmetic on that one row looks fine. The recommendation is not a valid application of the business rule, and a purchase request would turn a research mistake into a live commitment.
Good: It states the workbook version and timestamp, the three sheets examined, the number of active SKUs before and after exclusions, and the exact rule applied. It lists candidate SKUs with source row identifiers, current stock, committed orders, minimum, and lead-time exception. It flags a missing minimum for one SKU as unresolved. It prepares a proposed purchase request for review but cannot send it to a supplier without a separately authorized approval.
The fictional example is deliberately small. It reveals four ways a seemingly useful answer can fail: incomplete input coverage, an outdated rule, an unexamined exception, and an action taken before a human verifies the result. A larger model does not remove those failure modes. A stronger operating contract does.
For an actionable result, require an answer packet rather than a free-form paragraph. The packet should include the question asked; file identifiers and versions; policy identifiers and effective dates; population and exclusion counts; method; exceptions; reviewer; and proposed next step. A short executive summary can sit on top, but the evidence cannot disappear behind it.
This is also why “the answer has citations” is an insufficient acceptance test. A cited cell proves that one cell was seen. It does not prove completeness across sheets or that the cited policy was in force. AWS documentation for agentic retrieval explains that trace events show planning, retrieval, and full-document expansion. A trace naming the workbook helps, but a reviewer must still test the boundary conditions and reconcile the outcome with the approved business definition.
You do not need a banking-scale data platform to apply this discipline. Start with one recurring question that costs your team time and has a clear owner. A weekly exception report is better than a vague request to “analyze all our data.” Run the same question against a known workbook and a deliberately changed copy before granting broader access.
Write the decision in plain language. Specify the business question, the population, the cutoff time, and what will happen if the answer is wrong. “Identify invoices needing review by 4 p.m. Thursday” is testable; “find finance insights” is not.
List authoritative inputs. Record the approved workbook, named sheets, upstream system, policy document, and any current exception memo. Decide who owns each source and how freshness will be checked. A copied file in someone's downloads folder is not automatically the source of truth.
Define the rule outside the prompt. Write the threshold, eligibility criteria, exclusions, date arithmetic, and treatment of blanks as a business specification a person can inspect. Where possible, compute stable metrics with deterministic code or a governed reporting system, then let the coworker explain and investigate the results.
Create adversarial test rows. Include a canceled item that appears late, a duplicate ID, a blank owner, a formula cell that appears empty, an exception on a second sheet, and a revised policy date. Know the expected result before running the test. If the answer only works on clean rows, it is not ready for ordinary business data.
Require coverage evidence. Ask for sheet names, row counts, filter counts, source timestamps, exception identifiers, and the calculation method. Compare a small sample by hand and one full result against a trusted report. If the system cannot expose what it examined, treat its output as a lead for investigation rather than a decision-ready result.
Separate proposed from permitted actions. A coworker can prepare a case with evidence and a draft correction while only a named person can approve a ledger adjustment, supplier order, customer refund, or public statement. Scope the connected account to the smallest read and write rights the task actually needs. A persuasive explanation never confers authority.
Recheck after changes. Rerun the test when the workbook schema, policy, source connector, model, or computation route changes. Keep the old expected result and the new trace. If the same question suddenly yields a different population, pause downstream writes until an owner explains the difference.
This is intent engineering in operational form: the business defines what a good outcome is and which errors must stop the workflow. The model may plan the investigation, but the source and action boundaries are enforced elsewhere. For a broader view of why an AI coworker differs from a one-off tool, the distinction is responsibility across repeated work, not merely the ability to answer a question once.
AWS is clear that tabular search fits calculations and lookups over table files stored with documents, while recurring governed metrics, large warehouse joins, dashboards, and reporting controls belong in business-intelligence tools. That limitation should guide small teams too. If the question is asked every morning and the answer triggers payments or customer commitments, move its core metric into a tested reporting pipeline instead of repeatedly improvising over a changing workbook.
Use the AI coworker for the variable part: investigate exceptions, compare the result with a policy memo, assemble a review packet, and ask for the missing judgment. Keep the arithmetic and eligibility definition deterministic where possible. That division lets the coworker handle ambiguity without making it the sole authority on what the numbers mean.
A useful acceptance exercise is to change one row in a copy of the workbook and predict the exact difference in the output. Then change only the policy memo and predict what should change without touching the numbers. Finally remove access to a source file and confirm the coworker reports incomplete evidence rather than presenting the previous answer as current. Those three tests expose different failures: computation, policy interpretation, and missing-source handling.
Q: Can an AI coworker reliably analyze a spreadsheet? A: It can, for a defined question, when the system examines the complete relevant table, applies a documented rule, and provides evidence of the source version and calculation. Reliability must be tested on your own messy rows and exception cases, not inferred from a polished summary.
Q: Why isn't a citation to a spreadsheet cell enough? A: A citation shows where one claim came from. It does not prove the answer considered every eligible row, other sheets, policy exceptions, or the current version of the workbook. Ask for the population and exclusion counts as well as specific citations.
Q: Should we use an AI coworker instead of our reporting dashboard? A: No. Keep stable recurring metrics in governed reporting or deterministic code. Use the coworker to investigate unusual cases, connect a calculation to current policy, and prepare a reviewable recommendation when the question changes from week to week.
Q: What if the workbook and policy document disagree? A: Do not let the coworker silently select one. Record both sources and their effective dates, stop any consequential write, and route the conflict to the policy owner. The reviewer should decide whether the workbook needs correction or the rule has changed.
Q: Can we let the coworker place orders based on its spreadsheet analysis? A: Begin with read-only investigation and a proposed order. Grant a narrow write capability only after coverage tests, explicit authorization limits, a named approver, and an outcome check have been demonstrated. A successful calculation is not itself permission to spend.
Q: How often should we retest the workflow? A: Retest whenever a source layout, policy, connector, model, or calculation path changes. Also include a known exception row in regular checks, because a workflow can remain fluent after it has stopped seeing the right evidence.
If your team needs AI coworkers that can investigate real business records without turning a plausible answer into an unchecked action, explore Associates AI Teammates and view pricing. The platform gives your team persistent, model-agnostic Teammates with explicit roles and human decision boundaries; your people stay accountable for the calls that matter.
Written by
Founder, Associates AI
Mike is a self-taught technologist who has spent his career proving that unconventional thinking produces the most powerful solutions. He built Associates AI on the belief that every business — regardless of size — deserves AI that actually works for them: custom-built, fully managed, and getting smarter over time. When he's not building agent systems, he's finding the outside-of-the-box answer to problems that have existed for generations.
More from the blog
Amplitude's Wave case study shows how quickly an AI coworker can turn observed friction into a shipp...
A shared AI coworker can coordinate a team without making every private request, retrieved record, o...
The next AI coworker can follow projects across days. That creates a new operating question: when se...
Want to go deeper?
Get started today. Hire your first Teammate in minutes and put it to work on what you're reading about.
Get StartedWhat we shipped and what we learned.