Only 21% of Organizations Know What Their AI Agents Can Actually Do

Only 21% of executives report complete visibility into what their AI agents can access, execute, and send. That number comes from a briefing by the AIUC-1 Consortium — a coalition including Stanford and over 40 CISOs from companies like Confluent, Elastic, and Deutsche Börse. Meanwhile, IBM recently disclosed that an autonomous customer-service agent began approving refunds outside policy guidelines. No breach. No jailbreak. Just an agent with broad permissions doing what it was technically allowed to do.

The remaining 79% of organizations are running agents they don't fully understand, connected to systems they haven't audited, with permissions they've never reviewed. That's not a theoretical risk. An EY survey cited in the same briefing found that 64% of companies with over $1 billion in annual turnover have already lost more than $1 million to AI failures.

This is a solvable problem. But solving it requires understanding why it happens, what the IBM case actually reveals, and what a proper permission audit looks like.

Why the Visibility Gap Exists

AI agent permissions are overprivileged by default because the setup incentives point that way. The natural workflow: connect the agent to Gmail, authorize with OAuth, confirm it works, move on. "Works" means "the agent can do the thing I tested." It does not mean "the agent can only do the things I intend."

OAuth scopes are broad by design. When you connect an agent to Google Workspace, the default scopes often include full read and write access to email, calendar, and Drive. The agent needs to read emails? It now also has permission to send them, delete them, and access every label and draft in the account. Nobody explicitly chose that. It's just what the consent screen granted.

This compounds across integrations. A typical business agent connects to email, a CRM, Slack, a calendar, maybe a database or two. Each integration adds its own set of broad default permissions. After four or five connections, the agent's effective permission surface is enormous — and nobody has a single view of what it all adds up to.

80% of organizations in the AIUC-1 survey reported risky agent behaviors, including unauthorized system access and improper data exposure. That's not because 80% of organizations are careless. It's because the default path — connect, authorize, ship — produces overprivileged agents as a natural outcome.

What the IBM Case Actually Shows

The IBM refund agent didn't malfunction. It didn't get prompt-injected. It didn't hallucinate a policy that didn't exist. It approved refunds outside guidelines because its permissions allowed it to approve refunds, and nothing in its architecture constrained the conditions under which it could do so.

This is the distinction that matters: the agent did what it could do, not what it should do. Those are two entirely different boundaries, and most deployments only define one of them.

Telling an agent "only approve refunds under $50" in a system prompt is a behavioral instruction. The agent might follow it. It might not, depending on the model, the context window, competing instructions, or subtle prompt interactions. Behavioral instructions are probabilistic. Structural constraints are not.

The right architecture is one where the agent cannot approve a large refund — not because it's been told not to, but because the integration doesn't expose a method that would allow it. The tool isn't there. The API call doesn't exist in the agent's toolset. The constraint is structural. It holds regardless of what the agent decides.

Most deployments skip this distinction entirely. They write detailed behavioral instructions and consider the agent "configured." The IBM case is what happens next.

The Permission Audit: What You Actually Need to Know

If you can't answer these questions for every integration your agent uses, you have a gap. Run through each connected service and fill in the blanks:

Email (Gmail, Outlook, etc.)

• Can the agent send email to anyone, or only to a defined list?
• Can it send attachments, including files from other connected systems?
• Can it read email from all addresses in the account, or just a designated inbox?
• Can it delete messages, archive them, or change labels?
• Can it access the email of other users in the organization?

CRM (HubSpot, Salesforce, etc.)

• Can the agent create new contact records, or only update existing ones?
• Can it delete contacts or deals?
• Can it export bulk data — CSV downloads, API calls that return all records?
• Can it modify fields that control revenue, status, or ownership?

Slack / Messaging

• Can the agent message any user or channel, or only designated ones?
• Can it read messages in channels it wasn't explicitly added to?
• Can it post as other users, or only as its own bot account?
• Can it invite external users or create new channels?

Calendar

• Can the agent create meetings with external parties?
• Can it cancel or modify existing meetings it didn't create?
• Can it view other users' calendars?

File Storage (Google Drive, SharePoint, Dropbox)

• Can the agent read all files, or only a designated folder?
• Can it share files externally?
• Can it delete files or move them out of shared locations?

Most teams, when they sit down and answer these honestly, find that the answer to at least half of them is "I'm not sure." That's the gap. And "I'm not sure" is the same as "yes" from a risk standpoint.

How to Fix It in OpenClaw

Structural permission scoping in an OpenClaw deployment involves four layers. Each one reduces the blast radius if something goes wrong.

Layer 1: Use Composio for Integration Auth

Composio is the right tool for third-party integrations in OpenClaw deployments. Instead of connecting the agent directly to Gmail or Slack with full OAuth tokens, the agent gets a Composio API key. Composio handles the underlying OAuth and exposes only the specific actions you authorize.

The practical effect: you define, at the Composio level, exactly which actions the agent can take. Read email but not send. Add CRM contacts but not delete. Post to one Slack channel but not others. The agent never holds the underlying OAuth tokens — it only has access to what Composio exposes. This is covered in more detail in the post on OpenClaw credentials done right.

Layer 2: Dedicated Bot Accounts for Every Integration

Never connect an integration using a personal account. The agent and the human should never share an account. This is one of the most common mistakes in OpenClaw deployments, and the problems it causes compound over time.

A dedicated bot account scopes the agent's access to what the account was provisioned with, not everything the human has accumulated over years. It keeps the audit trail clean — every action in the CRM or Slack that shows the bot account name was the agent; everything else was a human. And when you need to revoke agent access, you disable the bot account. The human's account is unaffected.

Layer 3: IAM Roles with Least Privilege on AWS

The agent's EC2 instance should have an IAM role that allows only what's actually needed. If the agent reads from S3 and writes to CloudWatch Logs, the IAM role covers exactly that — not full S3 access, not admin on the account.

This matters for two reasons. First, if the agent's process is compromised through prompt injection or a software vulnerability, the attacker only gets what the IAM role allows. Second, it makes the permission surface auditable: you can read the IAM policy and see exactly what the agent can touch in AWS. There's no ambiguity.

Layer 4: Read-Only Soul Documents

Soul documents — the files that define agent behavior, escalation rules, and decision constraints — should be mounted read-only on the filesystem. If an attacker convinces the agent to rewrite its own instructions, the write fails at the OS layer. The constraint is structural, not behavioral.

The full case for this is made in the post on read-only soul document mounts. The short version: any agent that can modify its own instructions has no reliable behavioral floor. The modification happens at a layer below the agent's values and guidelines.

The Ongoing Practice: Quarterly Permission Reviews

Permission scoping isn't a one-time task. It's a practice with a quarterly cadence, for a specific reason: model capabilities change.

An agent running on the model that existed nine months ago has a different effective capability level than the same agent running on today's model. The underlying model is more capable — it can do more with the same permissions. A task the old model would have failed at, the new model might complete successfully. A permission that was "technically granted but practically unreachable" may now be practically reachable.

This means that a permission audit done at deployment time becomes stale. The right cadence is quarterly, timed to coincide with significant model updates. The questions are the same as the audit above, but evaluated against the current model's capabilities. What was the agent unable to do with broad permissions six months ago that it can now do? Which of those new capabilities are ones you didn't intend to grant?

The answer drives two actions: revoking permissions that are now dangerous, and updating the failure models for what the agent might do under adverse conditions. An agent that could hypothetically exfiltrate data via email six months ago but would have failed due to model limitations can now do it reliably. That changes the risk calculation even if nothing in the configuration changed.

What Good Looks Like

A properly scoped OpenClaw deployment has a clear answer to every question in the audit above, for every integration. The answers are documented, not stored in someone's memory. The Composio integration shows exactly which actions are enabled. The IAM role policy can be read in two minutes. The bot accounts are clearly named and their permissions are documented.

When a new model ships, there's a standing process to review whether any existing permissions need to be revisited. It's not a big project — it takes a couple of hours with a clear checklist. But it happens before the new model rolls out to production, not after something goes wrong.

The IBM refund agent almost certainly had none of this in place. Not because the team was negligent, but because the permission gap is invisible until it causes a problem. The audit makes it visible. The structural controls make it fixable. The quarterly review keeps it fixed as the capability landscape shifts underneath you.

FAQ

Q: Do these permission problems only affect large companies?

A: The EY survey covered large enterprises, but the underlying failure mode affects deployments of any size. An agent connecting to Gmail with full OAuth is overprivileged whether the company has 5 employees or 50,000. Small businesses face the same structural gap — they just tend to have fewer resources to discover it before it costs them.

Q: If I'm using OpenAI's or Anthropic's APIs directly, do I still have this problem?

A: The model provider is only part of the stack. The permissions gap lives in the integrations — what your agent connects to and what those connections allow. Whether you're using OpenAI, Anthropic, or OpenClaw, the same audit applies to every external integration.

Q: How often should I actually review permissions?

A: Quarterly is the right minimum cadence, with an additional review any time a significant model update ships to your production deployment. If a frontier model release substantially changes capability levels — think GPT-4 to GPT-4o, or Claude 2 to Claude 3 — treat it as a trigger for an immediate permission review.

Q: Doesn't Composio limit what my agent can do?

A: Yes, intentionally. Composio limits the agent to the specific actions you've enabled. That's the point. An agent that can only do what it's supposed to do is more capable in practice than one that can technically do everything — because you can actually trust the former to run without close supervision.

Q: What's the difference between a behavioral constraint and a structural constraint?

A: A behavioral constraint is an instruction to the agent: "don't approve refunds over $50." The agent's compliance depends on its reasoning in that context. A structural constraint is an architectural limit: the agent's toolset doesn't include a method for approving large refunds. The IBM case is what behavioral-only constraint looks like in production. Structural constraints hold regardless of what the agent reasons.

---

Associates AI builds and maintains OpenClaw deployments for small and mid-size businesses, and permission auditing is part of every engagement — not as a one-time security review, but as a quarterly practice tied to model releases and capability changes. If you want to understand what a properly scoped agent deployment looks like for your business, book a call.