Your AI Coworker Can Flag a Staffing Problem. It Cannot Be the Boss.

California signed a law on September 30 that says an employer cannot rely solely on an automated decision system to discipline or terminate an employee. The immediate lesson for every business building an AI coworker is not simply to add an approval button. It is to make sure a person can inspect the evidence, disagree with the recommendation, and own the decision.

The governor's announcement names SB 947 among a group of workplace-AI measures. The chaptered text of SB 947 says its employment-decision provisions become operative July 1, 2027. This is a California law with defined terms, not a universal rule for every employer today; ask employment counsel how it applies to your organization. But its design question is immediate: if an AI system influences a consequential decision about someone on your team, can you show where human judgment entered and what it actually did?

What California's employment-AI law changes—and what it does not

SB 947 addresses an automated decision system, or ADS: broadly, a computational process that produces a score, classification, or recommendation used to assist or replace discretionary decisions and materially affects people. It is not limited to conversational AI. The statutory definition also excludes ordinary tools such as calculators, databases, spam filters, and firewalls. Read the actual definition before labeling every spreadsheet or message draft a regulated system.

The law prohibits sole reliance on an ADS for disciplinary or termination decisions. Where an employer primarily relies on ADS output for such a decision, it requires a human to corroborate the decision using the underlying data or other relevant supporting information. If that output cannot be corroborated, or the human finds it inaccurate, incomplete, or misleading, the employer may not use it as the basis for discipline or termination. These are materially different tests from a manager clicking “approve” on a generated summary.

In the primarily-reliant scenario, the law also provides for a separate written postuse notice when the decision is communicated, a human point of contact, and an employee's ability to request a meaningful, objective description of their own data used by the system. That combination matters: the employee needs a route to challenge the inputs, not just an explanation of an opaque conclusion. The details, exceptions, and applicability belong in a review of the enacted bill text with counsel, rather than in a generic AI policy copied from another company.

This does not mean an AI coworker cannot help an operations or HR team. It can organize timelines, retrieve relevant records within its permitted scope, identify missing evidence, prepare questions, and flag inconsistencies. Those are preparation tasks. It should not quietly turn a productivity metric into a personnel verdict, or send a notice before a responsible person has made the decision.

The difference between assistance and authority

Suppose an operations Teammate reads a scheduling system and notices a pattern of missed shifts. Assistance means it produces a list of dates with source links, states which records are incomplete, and asks the manager to investigate. Authority means it labels an employee “unreliable,” recommends termination from a score it cannot explain, and queues the email for delivery. The latter collapses observation, interpretation, and action into one unreviewed path.

The legal threshold in California is specific, and a system that merely helps organize facts will not necessarily be treated the same way as one that principally drives the decision. Still, “we only used AI for a draft” is not an adequate operating rule if the draft becomes the de facto decision. Track how the recommendation influenced the human, not just which application generated the words.

A human signature is not the same as human judgment

The most dangerous review workflow is one designed to create the appearance of oversight. A manager receives a polished paragraph saying an employee failed to meet targets, a green approval button, and a deadline. The model's underlying data are not visible. The manager has no time to interview anyone or review context. The human signs, but the system decided.

What good looks like is slower at the decision boundary, not slower everywhere. The AI coworker assembles a case packet: original records, dates, data provenance, disputed items, missing context, and the policy provision the manager must assess. The manager can return the packet, request another source, record a contradictory explanation, or reject the recommendation entirely. The system cannot send a disciplinary communication or change an employee record through that review path.

Take a customer-support team in which a dashboard flags a representative for slow responses. A bad workflow treats response time as a proxy for performance and recommends discipline. A good workflow notices that the representative handles the most complex cases, that the queue changed during the measurement period, and that the dashboard omits reopened tickets. It presents these as questions, not excuses or a ready-made verdict. A human checks the source records and speaks to the people involved before any personnel decision.

This is seam design: assign repeatable evidence-gathering to the AI coworker and reserve interpretation of context, fairness, and consequences for the person accountable for the employee. A genuine human seam has time, access, authority, and a recorded outcome. Remove any one of those four and “human in the loop” becomes a label, not a control.

Give the reviewer a way to say no

A reviewer needs more than the ability to pause a process. They need an explicit reject action, a route to correct source data, and permission to close the case with no action. If a rejection only sends the same recommendation back tomorrow, the human is not in control. If the source is wrong, the corrected record should carry forward so the next report does not repeat the error.

For sensitive cases, separate fact-finding from decision authority. The Teammate that prepares the packet should not hold credentials that can edit personnel records or message the employee. An authorized person can initiate any downstream action through the business's normal process after review. This is trust architecture: make an unapproved write structurally unavailable instead of relying on a polite instruction not to use it.

Design an employment decision workflow that can be audited

Do not begin by asking which model writes the best HR memo. Begin with the route a concern takes through the organization. The following sequence is a practical design for a small business; it is not a substitute for legal advice or a claim that these steps alone satisfy SB 947.

1. Inventory the signals. List every AI-generated score, summary, alert, or recommendation that could influence discipline or termination. Include built-in features of scheduling, support, sales, and productivity software. A tool can influence decisions even if the company never calls it an AI coworker.
2. Name the decision owner. Assign one role with actual authority to decide and another place for escalation when that person is involved in the dispute. Record who may gather evidence, who may review it, and who may communicate the outcome.
3. Limit what the AI coworker can read and write. Grant only the records required to prepare a factual packet. Keep employee notices and personnel-system mutations outside that role's permissions. A read-only start is useful; our read-only AI coworker pilot describes how to test evidence quality before introducing action rights.
4. Require a source-backed packet. For each fact, record its originating system, time window, and an accessible source reference. Separate observed facts from inferences. Include missing records and alternative explanations. Do not generate a conclusory risk score simply because the interface has a place to put one.
5. Make corroboration an actual task. Give the reviewer the underlying records and independent context where applicable; require them to document what they checked and what changed their mind. If the evidence does not support the concern, stop. SB 947's corroboration rule in primarily-reliant cases makes this distinction especially salient for California employers.
6. Keep the communication step separate. A human decides whether any action is justified and follows the organization's personnel and notice process. If a California employer's use meets the law's primarily-reliant threshold after the operative date, counsel should verify the specific postuse notice and data-description requirements. Do not let a generic template stand in for that review.
7. Review corrections and near misses. Log when a human rejects an AI-supported finding, why the source failed, and whether the same error appears elsewhere. Reassess the workflow after tool changes, new data sources, and policy changes—not just after a complaint.

These steps create evidence that a reviewer can use before a decision. An audit trail assembled after the fact cannot restore an interview that never happened or undo the pressure of an automatic recommendation. For a broader operational treatment of accountable action, see our AI coworker approval workflow guide; personnel decisions require an even stricter boundary than routine purchasing or publishing approvals.

What to record, and what not to collect

A useful case record contains the original trigger, version of the workflow, sources consulted, contradictory information, reviewer identity, review date, decision, and any correction. It does not need to become a permanent warehouse of unrelated private conversations. Set retention and access rules with HR and counsel, and make sure the person preparing the case sees only material relevant to that case.

The review record should also show a path from source to decision. “AI suggested termination; manager approved” tells you almost nothing. “Scheduling alert covered these dates; two records were erroneous; manager checked the rota and interviewed the supervisor; no discipline” shows that the reviewer could independently alter the outcome. It is precisely the kind of evidence that helps a business discover a broken input before it affects another person.

Treat this as an intent-engineering problem, not a prompt problem

The wrong configuration says: “Monitor performance and make sure the team is accountable.” It sounds responsible but leaves the Teammate to invent a definition of performance, choose which data count, and decide what to do with an exception. A better role definition says: “On request, assemble verifiable attendance records for the named period; identify gaps or conflicts; do not score employees, propose discipline, edit personnel files, or contact employees; hand the packet to the designated manager.”

Even that instruction is not enough by itself. The account must lack personnel-write and outbound-messaging permissions, the handoff must identify a human owner, and the workflow must preserve source evidence. Better phrasing cannot compensate for an overpowered identity. Intent engineering translates the organization's purpose into permissions, escalation rules, and verification checkpoints that survive a model change.

California's September 30 signing is an unusually clear reminder of why that translation matters. The law does not reward a beautifully worded disclaimer. It distinguishes an automated output from corroborated human decision-making, and it gives the affected employee a route to understand the data behind certain decisions. The same design principle helps any small business, including one outside California: the more consequential the decision, the more independent the human review must be.

Begin with one workflow where errors are reversible and the decision owner can inspect the raw material. If you cannot articulate who can correct a bad record, who can veto the recommendation, and who speaks to the employee, do not connect that workflow to a personnel action. A useful AI coworker reduces clerical load while making the human decision clearer; it does not hide a management decision inside an automated pipeline.

FAQ

Q: Does California's SB 947 ban AI from HR entirely?
A: No. Its chaptered text sets specific restrictions on automated decision systems and discipline or termination, including a ban on sole reliance and added requirements where the employer primarily relies on system output. The provisions become operative July 1, 2027. Consult counsel about your use case and the law's definitions and exceptions.

Q: Is clicking an approval button enough human review?
A: Not if the reviewer cannot inspect and corroborate the evidence or change the outcome. In California's primarily-reliant scenario, the law requires a human to corroborate the decision using underlying or other relevant information; if the output cannot be corroborated or is inaccurate, incomplete, or misleading, it cannot be used as the basis for discipline or termination.

Q: Can an AI coworker summarize attendance records for a manager?
A: Yes, as a workflow design, evidence preparation is distinct from making a personnel decision. Verify the sources, restrict access to the relevant records, identify missing context, and leave any assessment or employee communication with the authorized human. Whether a particular product and use fall under SB 947 requires case-specific advice.

Q: What should a small business change first?
A: Inventory the AI outputs that can influence employment decisions, remove automatic write and send authority from evidence-gathering roles, and require a source-backed review packet with a named decision owner. Test whether that owner can reject the recommendation and correct faulty input.

Q: Does this apply outside California?
A: SB 947 is California legislation, not a nationwide mandate. Other employment, privacy, discrimination, and recordkeeping rules may apply elsewhere. Independent human judgment and a verifiable trail are sound operating practices regardless of jurisdiction, but local legal requirements need separate review.

If you're ready to configure AI coworkers with clear roles, human decision boundaries, and persistent operational context, explore Associates AI Teammates and choose a plan.