Photo by Julian Gojani on Unsplash

You know what you are holding. Two cards, face down, and you have looked at them enough times to stop looking. You know the rules of the game, which hands beat which, and roughly how often the deck cooperates. You know how many chips are on your side of the table and how many are in the middle. Three community cards are already turned, then a fourth, and you can read most of what they mean. You have even been watching the player across from you for an hour, and you think you know what that small pause before a raise usually means.

What you do not know is the river.

You do not know whether the card you need is still in the deck or already sitting in someone else’s hand. You do not know whether the read you have on your opponent is real or something they let you see. And you do not know which of your own habits they have been reading back. Every decision you make from here is made with real information, and none of it is complete.

That is the difference between a bet and a gamble. A gamble is a decision made without regard for the information available. A bet is a decision made because of it: a judgment that what you can see justifies putting something of value against what you cannot. Good players lose hands constantly. What separates them from bad players is not that they avoid uncertainty. It is that they know, at every point in the hand, what they are wagering and what has to be true for the wager to pay.

Enterprise AI is a bet. Almost every organization I know of has already made it, or is in the middle of making it, and most of them are making it for good reasons.

I should say plainly where I stand, because this is not the essay of an architect standing away from the table muttering that cards are dangerous. I use AI almost every day. I use it to draft and dismantle scripts, to pressure-test ideas, to argue with my own reasoning, to convert an argued point into prose for parts of these articles, and to find the hole in a design before someone else finds it in production.

I would not willingly give it up. I am already at the table, and I intend to stay there.

Which is precisely why I think it is worth being careful about what, exactly, is in the pot. Architects are paid, in part, to care about the cards nobody can see yet: what the organization is assuming about them, what it has already committed, and what happens to the system if the river is not the card everyone was hoping for. The enterprise AI bet has a hand, a stack, a board, and a set of opponents. It also has a dependency that very few of the people placing it have named, and the rest of this essay is an attempt to name it.

What’s in the Pot

The shallow version of the business case is that AI helps people write emails faster. Nobody is restructuring licensing budgets, rewriting operating models, or presenting to a board over faster email.

The real proposition is much larger, and it deserves to be stated at full strength before anyone argues with it.

The ambitious version is this: AI lets an organization extract more useful output from the same human labor, while reducing how much specialized expertise has to sit at every point in the process.

Historically, if a company wanted a hundred units of expert-level work, it needed something close to a hundred units of expert labor to produce it. Expertise is slow to develop, scarce, expensive, and mobile. It becomes a bottleneck, then a single point of failure, then a resignation letter. It cannot be scaled up for a demand spike or summoned on a Saturday. The enterprise AI bet is that the ratio can change, so that a hundred units of expert-looking work might come from thirty units of expert labor, forty of generalist labor, and a machine.

If that works, it is one of the most valuable propositions a business has been offered in a generation. The junior analyst no longer needs ten years of SQL to write the query. The project manager no longer needs deep technical fluency to understand the architecture. The finance employee builds the model they could not have built last year. The developer works outside their strongest language without losing a week to it. A small team operates like a large one.

There is a second layer underneath that one, and it may matter more.

A great deal of corporate productivity is lost not because nobody knows the answer, but because the person who needs it does not know who knows it, where it was written down, or which system holds it. Every organization runs on some version of “Ask Alice. She knows how that process works.” The promise of enterprise AI is that ‘Alice’ becomes searchable. Institutional knowledge stops living in a handful of heads and becomes explainable, retrievable, and reusable at scale. Handoffs shrink. Queues shorten. Decisions move closer to where the information originates.

And underneath that is process compression. A remarkable share of business process is simply a sequence of cognition: read something, understand it, classify it, compare it, decide what happens next, generate something, send it somewhere.

Those are exactly the steps current models and agents are increasingly good at. This is why the vocabulary of AI keeps drifting from productivity toward agency. The vendors are no longer describing a better tool for the employee. They are describing a participant in the work.

None of that is foolish; most of it is already partially true, and I am a daily beneficiary of the part that is. If I were an executive looking at that proposition, I would want to place the bet too.

But every version of the pitch contains a sentence with a clause missing. The sentence is: AI lets people produce work that previously required expertise they do not have. The missing clause is: provided someone can still tell whether the result should be trusted.

That clause is the hidden dependency in the hand. The economic case for AI is, at least in part, the case for no longer needing the person doing the work to know as much as they used to. That is the feature, and it is what the license is being bought for.

But every control the organization relies on to catch a bad result (review, testing, sign-off, judgment, the instinct that says that number looks wrong) was built on the assumption that the person doing the work did know. The return on the bet is paid by removing expertise from execution. The safety of the bet was paid for by that expertise being present.

Both are drawing on the same card. Most organizations have only counted it once.

Bought Like Software

The flop is the moment the hand stops being hypothetical. Three cards land, the pot grows, and the decision you made before you could see anything starts meeting reality. For most organizations, the flop is procurement, and the first thing it reveals is that the thing being bought does not behave like the category it is being bought in.

Enterprise AI is usually purchased like SaaS, introduced like a productivity feature, and capable of behaving like a new execution layer across the entire organization. That mismatch matters more than whether any particular model is safe.

A conventional SaaS product carries real risk, but its purpose is bounded. A CRM manages customer relationships. A ticketing platform manages tickets. A planning system plans. Because the purpose is bounded, the implementation can be reasoned about in familiar terms: what process it supports, who needs access, what data enters it, what it connects to, who owns it, who supports it, and what happens when it fails. Those questions are not always answered well, but they can be asked, and the answers, once found, have defined edges.

An AI assistant’s purpose, by design, is whatever the user can describe. That is close to the opposite of a bounded application, and it makes every one of those familiar questions strangely difficult to answer.

The economics are the first surprise. Traditional licensing gives you predictable marginal cost: so many seats at so much per month. AI adds a second dimension, which is how much inference or execution each seat consumes, expressed in tokens, credits, requests, or agent runs, depending on what the vendor has decided to meter.

I have written before about what that bill reveals about an organization, and I will not re-argue it here. The relevant point for this hand is narrower. A license can grant access to a capability without buying unlimited use of it, which means a successful rollout can discover that the capacity it purchased was never sized for success.

By the time that discovery arrives, users have built their work around the tool, and reducing consumption is no longer a budget decision. It is a decision to degrade processes that already depend on it.

The second surprise is the order of operations. Architects expect a sequence: requirement, architecture, security review, integration design, implementation, testing, production.

Enterprise AI frequently arrives in a different one: license, enable, connect, experiment, adopt, and then discover the architecture that resulted. The word doing the most damage in that sequence is connect. Connect SharePoint. Connect Outlook. Connect the CRM, the code repository, or the data platform.

From the user’s chair, it is a toggle. From an architect’s chair, a dozen questions just disappeared behind it. Which identity is used? Which permissions propagate? What gets indexed, cached, retained, or sent to inference? What can the agent invoke, as opposed to merely read? Which access model wins when the source system and the AI’s abstraction of it disagree?

Most of those questions have answers, and good vendors publish many of them. The trouble is that a connector is designed and maintained by the provider, configured in minutes, and rarely reviewed for what it permits to pass through. Nobody designed the resulting information architecture. People connected products, and an architecture emerged.

The third surprise is direction. Most enterprise technology moves from technical ownership outward. Someone identifies a need, IT evaluates options, architecture and security review them, procurement negotiates, and users are onboarded last. AI is marketed to the end user first, and it enters from the opposite direction: personal familiarity, employee demand, executive excitement, pilots, individual experimentation, organizational pressure, and finally, technical review.

The people who could evaluate it are downstream of the enthusiasm. By the time they are asked, the question is rarely “Should we adopt this?” It is “How quickly can you make the thing everyone already uses safe?”

Those are very different decision environments, and the difference is not neutral. A security team can reject an unfamiliar infrastructure platform before anyone depends on it. Rejecting an AI product after executives, analysts, and developers have already folded it into their work is a different conversation, conducted at a different political price. The cost of saying no rises with adoption, which means adoption itself becomes an argument for approval.

The governance decision is made by fait accompli, and the people formally responsible for it are asked to ratify a hand that was played before they sat down.

Then comes the workflow nobody designed. Somebody in finance (call her Susan) figures out that if she drops three spreadsheets into a folder, asks the assistant a particular question, runs the Python it gives her, and uploads the result to a shared site, a four-hour monthly close task takes twenty minutes.

That is genuinely valuable, and Susan deserves credit for it. She teaches four colleagues. Three months later it is a business process, with no source control, no dependency map, no tested recovery, no support model, and possibly nobody who understands what the generated code actually does. IT may not know it exists.

That is an old problem with a new accelerant. Shadow IT used to require someone technically adventurous enough to build it, and that requirement passively limited how much of it existed. AI supplies the technical adventurousness on demand. When Susan moves to another role, someone inherits a process with nobody left who remembers why it works, which I have argued elsewhere is not the same as having nobody responsible for it.

Each of these is survivable on its own. Together they describe the moment a player stops being able to fold cheaply.

Licenses expand. Processes reshape around the tool. Delivery expectations and headcount plans begin to assume the productivity gain. None of this is a mistake. It is simply what chips going into the pot look like from the inside.

The Friction Subsidy

The turn is where a hand changes character. The fourth card lands and what looked like a strong position becomes a different position entirely, with the same chips and the same cards in your hand. For enterprise AI, the turn is the point where the capability stops being a tool for the work people already do and becomes a means of doing work they could not do before. That is, of course, exactly what it was purchased for.

I have watched the following sequence play out more than once, and none of the people involved were doing anything they believed was wrong.

An employee wants to accomplish something slightly outside their normal competence. Not malicious, not even unusual: automate a report, move some files, pull data from a system that does not have a convenient export. Before AI, a series of small barriers stood between them and the outcome. They did not know how to write the script. They did not know which language or API to use. When it failed, they did not know why. When it failed because of a security control, they did not understand the control, did not know who administered it, and did not know how to phrase a request to change it in terms an administrator would take seriously.

AI removes those barriers one at a time. Write the script. Explain this error. What permission does this need? How do I find out who has the rights to change that setting? Draft an email asking them to enable it, and make it sound like I know what I’m talking about.

At no step does the model do anything malicious. It is being extraordinarily, admirably helpful. And in five exchanges it has converted I don’t know how to do this into here is an actionable path through every obstacle between you and the outcome, including a well-written, technically credible request that a busy administrator may simply approve.

This is the difference that matters most between AI and conventional software. An accounting application makes accounting easier. A planning system makes planning easier. Their blast radius is vertical: confined to a process. AI makes nearly any describable work easier, so its blast radius is lateral, extending into security, finance, data, legal, engineering, communications, and whatever else the employee can think to ask about.

Traditional applications scale a capability. AI scales capability itself.

Which brings us to the dependency I promised to name.

Every organization has an access model, and every access model has a gap between what it formally permits and what anyone actually does with those permissions. For most of the history of enterprise computing, that gap was managed, without anyone deciding to manage it, by friction. Exercising a permission fully required knowledge, effort, institutional context, and occasionally sheer stubbornness. Most people with Contributor access to a workspace never discovered most of what Contributor allowed. Most people who could technically query a dataset could not write the query. Most people who hit a security control turned around, because getting past it required knowing more than they knew.

That friction was a control. Nobody designed it, nobody documented it, and nobody budgeted for it, but it did real work in the security posture of nearly every enterprise. I think of it as the friction subsidy: the unrecorded portion of an organization’s security that is paid for by the difficulty of using what has already been granted.

AI withdraws the subsidy, and it does so organization-wide, at the same moment, for everyone who has been given access to the tool.

The consequence is a distinction most organizations never had to draw: the difference between permission and power.

Permission is what the access control says. Power is everything a person can realistically accomplish with that permission, given their knowledge, their tools, their persistence, and now an assistant that will research, generate, debug, and explain on demand.

For decades those two were correlated closely enough that enterprises could treat them as the same thing. AI stretches them apart. The ACL did not change. The job title did not change. The security groups did not change. What a given permission means in practice changed completely.

The same thing happens to information. An employee may be separately and legitimately allowed to read twenty different systems. That never meant anyone intended them to correlate all twenty, at machine speed, and ask novel questions across the result. Human retrieval was slow, and that slowness meant individually legitimate permissions rarely combined into anything larger. An agent that can traverse all twenty creates aggregate visibility, a capability no one ever granted and no access review ever evaluated. Authorization to access information has never been the same as authorization to make every inference that information makes possible. Privacy law has understood that for years. Most access models have not had to.

There is an honest corollary here, and it is the part of this argument I find most useful. If a permission was only safe because most people did not know how to use everything it allowed, then the permission was always too broad. The friction subsidy concealed it. AI did not create that exposure; it called it.

That is uncomfortable, but it is not alarming, and it reframes what governance in this environment is actually for. Some enterprise controls have been relying on a lack of individual capability without realizing it.

The work in front of most organizations is not to restore the friction, which cannot be done and would forfeit the very thing they bought. It is to find the places where friction was quietly doing a control’s job and put an actual control there.

The Black Wall

Most traditional systems an organization governs expose, at least in principle, an inspectable execution path. A SQL query has an execution plan. A pipeline has activities. An API call has a request and a response. A program has source code, infrastructure has configuration, and a business process has steps. Even sprawling distributed systems can be traced, given enough patience and enough logging.

Governance, audit, and support all rest on the same assumption: if something happens, we can find out how.

A language model offers something different. You submit a request, and something comes back. In between is what I have come to think of as the black wall. Enormous engineering happens on the other side of it, and none of what follows is a criticism of that engineering; the point is epistemic.

From the operator’s side, the relationship between instruction and outcome is no longer something you can inspect the way you inspect a plan or a stack trace. You can log the prompt, and you can record the retrieved sources. You can capture tool calls, set guardrails, and evaluate outputs. Those are real and valuable controls, but none of them restores the same inspectable relationship between instruction and execution that organizations are accustomed to governing.

For a chat assistant, that is a manageable property. The output arrives as text, and a person decides what to do with it. Agents change the shape of the problem. The pattern stops being input, wall, output, and becomes intent, wall, and a sequence of actions with consequences outside the wall. Observability becomes more important at exactly the moment it becomes less complete. For most of the history of engineering, the rule has been that as a system becomes more capable and consequential, we demand more visibility into it.

AI has given us systems that are dramatically more capable while introducing a region in which conventional visibility is structurally weaker. That should trouble architects, and it should trouble them more than any single model failure.

It also changes what authorization means. If I tell an agent to fix a permissions issue, what have I authorized? To identify the affected account? To modify a security group? To grant a role, remove a policy, alter a conditional access rule, open a firewall port?

A human colleague would understand the instruction as a goal and ask before doing anything drastic. An agent must translate the goal into actions, probabilistically, and then execute them under an identity that is authorized to do all of it. Traditional access control answers whether the identity was authorized to perform the action. The question that actually matters here is whether that particular interpretation of the instruction was authorized. “Fix the permissions issue” is not an ACL. Neither is “clean up the duplicates” or “make this faster.”

The security boundary is quietly moving from authentication of identity toward something closer to authentication of intent, and we do not yet have a mature mechanism for that.

The wall also blurs a boundary that security has relied on for decades: the separation between data and instructions. In conventional computing, a document is data. It does not tell the system what to do. To a model that reads it, a document is text, and text can contain instructions. A connected file, a retrieved web page, or a poisoned data source can influence what an agent does next. That is the substance of prompt injection, and it is not a bug in any particular product.

It is a property of systems that take direction in the same medium they take input.

The classic security triad still applies on this side of the wall, but each part shifts in a way worth naming once.

Integrity acquires a new layer. A database row can be correct while the model’s summary of it is wrong. A retrieval system can fetch exactly the right five documents, and the model can still produce a conclusion none of them support. The bits are intact, the records are intact, and the source systems are intact. The meaning was corrupted somewhere between the source and the reader.

I would call that semantic integrity risk, and I think “hallucination” undersells it badly. It makes the problem sound whimsical. An invented answer delivered into an operational process is an integrity failure, and the fact that it arrives fluent, formatted, and confident makes it harder to catch, not easier.

Availability becomes partly economic. The service can be fully online and still effectively unavailable, because a quota has been exhausted, a rate limit has activated, a budget control has stopped execution, or adoption has outgrown the purchased capacity. Availability used to ask whether the service could respond. It now also asks whether the organization can afford for it to respond at the rate the business has come to depend on.

And the platform can change underneath you. Poker at least offers the courtesy of a fixed deck. Enterprise AI does not. The provider can change the model, its reasoning behavior, its context limits, its safety policies, its tools, or its price, and a process that worked last quarter may still be technically available while behaving differently. It is a game in which the dealer occasionally introduces new cards while the hand is still underway.

Confidentiality I covered in the previous section. Aggregation is its real AI-specific form. The remainder is the familiar question of what crosses the wall, which brings us back to the poker table.

Part of a player’s exposure is information they reveal without realizing it. Enterprise users reveal a great deal this way. A question about customer complaints can pull email, attachments, CRM records, and shared files into a context window and send them to an external inference service. Nothing about that felt like a data transfer to the person who asked. They asked a question; the user’s mental model is conversational. The security reality, on the other hand, is computational. You may know exactly which cards you are holding and still be much less certain what the other side of the wall can see.

None of this is an argument against putting work behind the wall. External abstraction is normal; every organization runs on services it cannot inspect. The question is what consequential processes we are willing to place behind something we cannot fully interrogate, and what we build on our side of it to compensate.

Drawing Dead

In poker, a player is drawing dead when no card left in the deck can win the hand. They may not know it. They may be calculating outs, feeling good about the odds, and putting more chips in. But the cards they are counting on are already in someone else’s hand or already on the board, and the river cannot save them however it falls.

An organization can draw dead in exactly that way, and the mechanism is the missing clause from the pitch.

The common story about AI and expertise is that the technology lowers the expertise required to produce work. Sometimes that is simply true.

But producing something and validating it are different activities, and the story in the marketing deck tends to describe only the first.

When I ask a model to write a Python script, it takes seconds. If I understand Python, the environment the script will run in, the systems it touches, and the result I actually want, that is extraordinary leverage. An hour of work becomes ten minutes, most of which I spend reading what came back and correcting the parts that are wrong for my environment.

If I do not understand those things, I have received something that looks exactly like what an expert would have produced, and I have a harder problem than the one I started with. I now need to determine whether it is correct, and that may require precisely the expertise the tool was supposed to make unnecessary.

AI does not eliminate expertise. It moves where expertise is required, from creation to verification.

That shift is almost entirely absent from productivity rhetoric, and the reason is that the two halves are measured very differently. An organization can count faster development, more documents, shorter response times, more automation, and fewer people per deliverable.

It cannot easily count the assumptions nobody challenged, the edge case nobody recognized, the code nobody actually read, the security boundary nobody noticed had moved, or the analysis that was subtly and fluently wrong. The productivity gain arrives immediately and lands in a metric. The assurance cost arrives later, often much later, and lands in an incident.

The result is something I would call verification debt: work produced faster than the organization develops the capacity to determine whether that work can be trusted. Like any debt, it is not inherently bad. Borrowing against the future is often the right decision. But a business case evaluated during the period when verification debt is accumulating will look spectacular, because the borrowed value has arrived and the repayment has not.

Confusing production velocity with organizational capability is the most common way I can imagine for a sound AI program to become an unsound one without any single bad decision.

The deeper problem is what expertise was carrying that nobody wrote down.

A senior engineer does not merely know commands. They know not to run that job during month-end. They know that two systems can technically be connected and that there is a reason they are not. They know which table looks authoritative and is not, which field has not meant what its name says since an acquisition, and which permission will absolutely fix your problem and must never be granted.

They know because they have seen it fail. Competence is, in large part, accumulated exposure to consequences.

That knowledge is tacit, and it is exactly the knowledge AI is least able to supply. A model is extraordinarily good at explicit technique: the general, documented, widely written-about how. It has no access to the organization’s accumulated why not, because that part was never recorded. It lived in the people.

I have argued before that the most dangerous gap in institutional memory is the forgotten condition under which a rule stops being valid. AI widens that gap, because it lets work proceed without anyone present who would have remembered the condition at all.

Expertise is not merely knowing how to accomplish something. It is knowing which technically possible things should remain undone.

This is where the hand goes dead. The organization’s control environment is counting on a card: someone, somewhere in the process, who will recognize when the output is wrong. The business case is built on removing that card from the process, by design, because removing it is where the return comes from.

An organization that does both is drawing to an out it has already taken out of the deck. It may not know that. The odds may look fine, but the river still cannot save it.

There is a simple way to tell which side of that line a given use falls on, and it applies to individuals as much as to organizations.

Augmentation is asking AI to help with something you are capable of evaluating. Abdication is asking AI to do something you lack the competence to meaningfully evaluate.

The two interactions look identical on the screen. The prompt is the same, the interface is the same, and the output may even be the same. Architecturally, they are worlds apart, because in one case there is a competent judge on your side of the wall and in the other there is not.

The safest enterprise use of AI is not replacing expertise. It is multiplying the reach of people who already know where their expertise ends. The promise being sold is frequently larger: that expertise itself has become optional.

That is the version of the bet that is drawing dead.

Showdown

Eventually the river comes, the betting stops, and the cards are turned over. In poker, the showdown is the moment when everything that was uncertain becomes a fact and the pot goes to someone. In the enterprise, the showdown is the incident review.

Consider a plain one. Someone asks an assistant for a script to clean up a set of records. The code is syntactically correct, and it does exactly what it was written to do. It is also catastrophically wrong for the environment it runs in. It processes millions of rows one at a time instead of in batches, takes locks on a table that other processes depend on, and runs at 10:17 on a Tuesday morning.

Nothing about it is malicious, and nothing about it is broken in the narrow sense. It works, extremely poorly, and production goes down.

Now conduct the review. Who wrote the code? The model. Who asked for it? Alice. Who ran it? Perhaps Alice, perhaps a pipeline. Which identity authenticated the execution? A service account. Who approved the change? Perhaps Bob, who reviewed the pull request for correctness and did not think to evaluate its operational behavior. Who decided it was appropriate for production? That answer is already getting fuzzy. Who was responsible for making sure the environment could tolerate it? Now the review is getting somewhere.

The tempting mistake is to treat authorship as responsibility; they have never been the same thing. If an engineer copies technically valid code from a forum and takes down production, the forum does not become the incident owner, and a model generating the code instead does not change that. The useful question is not who wrote it. It is which control points should have prevented the outcome, and why each one did not.

Walk them and the answers start to separate. Alice ran code she was not competent to evaluate; that is hers. Bob approved a change without examining how it would behave under load; that is his. There was no environment in which to test it against realistic volume; that is organizational. Production permissions allowed arbitrary execution without a gate scaled to the risk; that is architectural. The organization actively encouraged people without engineering backgrounds to generate and run code against production systems without defining where that encouragement ended; that is a much larger organizational responsibility, and it was incurred long before Alice typed anything. If a provider made a specific representation that its generated code was safe for unattended production use, there may be provider responsibility too, depending on the facts and the contract.

Notice who never appears on that list as an accountable party: the model.

You cannot put it on a performance improvement plan. You cannot fine it, revoke its license, or ask it to testify about why it disregarded policy. It cannot compensate anyone. “The AI did it” is a perfectly accurate description of a mechanism. It identifies no one.

The question becomes sharper when the harm falls on someone outside the company.

Suppose a resident of the European Union submits a valid request to have their personal data erased under Article 17 of the GDPR. The regulation is not ambiguous about where responsibility for that request sits. Under Article 24, the controller must implement appropriate measures to ensure, and to be able to demonstrate, that processing complies. Under Article 28, a controller using a processor must use one that provides sufficient guarantees to protect the data subject’s rights.

In a typical enterprise arrangement, the organization may remain the controller for processing whose purposes and means it determines, while the AI provider acts as processor for processing performed on its behalf. The exact roles depend on the arrangement, but moving processing outside the organization does not move the obligation with it.

So the answer to that request cannot be: “We deleted the record from our systems, but some of it may persist in our AI provider’s prompts, conversation history, logs, indexes, embeddings, caches, or agent memory, and we do not know.”

To answer the request properly, the organization has to know where the person’s information went. Was it pasted into a prompt? Attached in a file? Indexed by a connector? Embedded in a vector store? Retained in telemetry? Used for model improvement under the service terms? Which of those count as personal data in context, which must be erased, which may lawfully be retained under an exception, how does the processor receive the instruction, and how does the organization verify it was carried out?

The black wall does not remove the obligation. It makes the black wall the controller’s governance problem.

California arrives at substantially the same place by a different route. The mechanics of the CCPA differ from the GDPR’s, and the two should not be treated as interchangeable. But California residents have rights to know, delete, and correct personal information held about them, and a deletion request goes to the business, which is responsible for directing its service providers to delete as well. “Our vendor has it now” is not the end of the business’s obligation. It is the reason the obligation includes the vendor.

You can outsource processing. You cannot outsource having chosen to process it.

Will a regulator accept that the AI did it? As a fact about the causal chain, it is relevant. As an exemption from responsibility, there is little reason to expect it. In 2023, U.S. regulators said plainly that existing consumer-protection and civil-rights law applies when automated systems are involved, and that there is no AI exemption from the laws already on the books. The EU AI Act assigns obligations to deployers of high-risk systems, including assigning human oversight to people with the necessary competence, training, and authority, and keeping logs.

Look at what every one of those frameworks does. It names providers, deployers, controllers, processors, and responsible persons. None of them transfers responsibility to the model.

Which makes attribution the part of this hand an organization most needs to play well.

Consider the log line most incident reviews would actually have: a service account named something like svc-ai-agent-prod changed a security group at 14:42:16. That is a fact, and it answers almost nothing. Who asked for it? What did they ask, in their words? Did the agent decide on its own that this change was necessary? What information informed that decision? Did anyone approve it? Was the person asking even entitled to request that operation?

Now try answering those questions six months later for an auditor, opposing counsel, a regulator, or a customer whose data was exposed.

A consequential AI transaction should leave behind enough evidence to reconstruct the chain from the authenticated person, through their instruction, the system and version that interpreted it, the context it used, the plan it produced, the tools and credentials it invoked, any human approval, the action actually executed, and the resources it affected. That is not a logging feature. It is evidence, and once something has gone wrong, it is the only thing that lets an organization show what it was responsible for and what it was not.

The farther execution moves from the person responsible for it, the stronger attribution must become. Agency can be delegated. Accountability cannot be delegated to something incapable of being accountable.

My Side of the Wall

It would be convenient to write this essay as if I were watching the table from a safe distance. I am not, and this essay is a reasonable place to show it.

This piece was developed in conversation with AI. That is not a confession; the footer of this site has said as much about everything published here. The ideas, the argument, and the structure are mine. The pressure-testing, some of the prose, and several of the sharper formulations arose from extended exchanges with two different models.

Some of the lines I like best in this essay arrived first on the other side of the wall. So it is fair to ask which use this was: augmentation or abdication.

The honest answer is that it was both, depending on the paragraph, and the boundary falls exactly where this essay says it should. When the model described how connectors propagate identity, how agents translate goals into actions, or why aggregation changes the meaning of a permission, I could evaluate every sentence. I have spent years building and governing the systems those sentences describe. When something was wrong, I knew it was wrong, and I said so. When it was vague, I knew what the specific version looked like.

In those sections, I was augmented. My judgment was on my side of the wall and the tool was multiplying its reach.

The regulatory sections are different. I know the shape of the GDPR and the CCPA well enough to work inside them as an architect, but I am not a privacy lawyer. When a model tells me what Article 28 requires of a controller, or what a California business owes its service providers, I can tell whether the claim is plausible. I cannot tell, on my own authority, whether it is precisely right.

In those portions, I must validate every claim before I hang an argument on it. If I skip that step and rely on the model anyway, it becomes abdication, and this essay becomes an example of its own argument, published under my name.

There is a second, more personal reason this argument matters to me. I am self-taught. I do not have a degree in any of the things I do for a living. Every piece of competence I have was paid for in friction: in queries that returned the wrong number for reasons that took a week to understand, in scripts that worked in development and failed in production, in permissions I did not know I should not have asked for.

The friction subsidy I described earlier is not an abstraction to me. It was the basis of my education.

Which puts me on both sides of this bet. AI offers people in my old position a faster path across the same ground, and I think, in some ways, that is a genuinely good thing. I probably would have taken it when I started. But the speed is in the producing, not in the learning, and I am not certain someone gets the latter from the former. I was made competent by exactly the consequences the tool now spares people from meeting.

I am not sure what replaces that, and I hear far more about how much faster the producing becomes than about what replaces the learning.

Pot Odds

A good player does not decide whether to call by asking whether they like their hand. They compare the price of staying in against the size of the pot and the odds of the cards they need. The same hand can be a clear call at one price and a clear fold at another.

That arithmetic is called pot odds, and it is the closest thing poker has to a governance model.

The enterprise version of that arithmetic is not caution; universal caution would forfeit most of what makes the technology worth having.

It is proportionality: the competence, oversight, and attribution required around a use of AI should scale with the consequence of being wrong and the difficulty of undoing it.

A mediocre internal email drafted by a model is cheap to be wrong about and trivial to reverse. A bad query against a development database is a lesson. A flawed assumption in an analysis that a competent analyst reviews before it reaches anyone is the system working as intended. Those uses deserve wide latitude, and organizations that strangle them with process are paying a real price for very little safety.

The price changes when the work touches production systems, privileged security controls, regulated data, material financial decisions, customer outcomes, legal rights, or anything that cannot be taken back. There, the question is no longer whether AI was involved. It is whether someone on the organization’s side of the wall can evaluate what came back, stop it before it becomes consequence, reconstruct what happened if they could not, and answer for it either way. Risk does not come from AI being involved. It comes from capability meeting authority, consequence, opacity, and irreversibility in the same place, with nobody positioned to judge the result.

I have written several pieces arguing for registers: a booking entry for exceptions, a record of what a model leaves out. This one does not need another. It needs one test, applied before the friction is withdrawn rather than after the incident:

Do not remove competence from a process unless you can name what replaced the judgment it carried.

If the answer is review, name the reviewer and confirm they can actually evaluate the output. If it is a technical control, name the control and the specific failure it prevents. If it is testing, name the environment and the conditions it reproduces. If it is attribution, name the evidence that will exist when someone asks what happened.

If the honest answer is that nothing replaced it, then the organization is not augmenting anyone. It is abdicating, at scale, and calling the difference productivity.

That test also tells an organization which parts of its bet it is actually managing and which it is merely making. Every enterprise AI program is wagering, whether or not anyone says so, that employees or someone near them will still recognize a wrong answer, that controls built for human-speed work will hold at machine speed, that enough of what happens behind the wall can be reconstructed when it matters, that the economics will hold once adoption is real and consumption is metered, that processes built around the tool will survive changes in price, model, or regulation, and that removing expertise from execution will not remove what expertise was carrying.

Some of those wagers are good ones. None of them are free, and most of them are being made by default rather than by decision.

There is one more poker concept worth borrowing, because it describes the most dangerous moment in any technology program.

A player becomes pot-committed when they have put so much into the middle that folding feels impossible, regardless of what the board now shows. It is a real phenomenon and a real trap, because commitment begins to feel like information.

Organizations become pot-committed the same way: The pilot became a program. The program became a dependency. Headcount assumptions changed. A process now runs on it, and sponsorship is public.

At that point, a security finding, an unexpected bill, or a regulatory question is no longer evaluated on its merits. It is evaluated against everything already in the pot.

Being committed to the pot is not evidence that the original read was correct. Organizations have to take risks; that is what a bet is.

The failure is not betting, but losing the ability to fold a hand that has turned against you because you can no longer tell the difference between what you have invested and what you know.

The River

So come back to the table.

You know what you are holding: a technology that is genuinely remarkable, that shortens the distance between an idea and its execution, and that makes capable people dramatically more capable.

You know the rules, or most of them. You know your stack and roughly what is already in the middle.

You have been watching the other players: the vendors, the regulators, your competitors, your own employees, and you have a read on all of them.

You do not know the river. Nobody does. You do not know whether the judgment your controls depend on is still in the deck, or whether you took it out yourself when you wrote the business case. You do not know what your users are revealing across the wall, or whether the dealer will change the cards before the hand is over.

The right response to that is not to leave the table; most organizations have already paid the blind. It is to play the hand as though you understood what you were wagering: know which cards you are drawing to, know whether they still exist, and know what you will do if the river does not fall your way.

And there is one more thing worth knowing before you call. If the bet pays off completely, and AI delivers everything its strongest advocates promise, so that every employee becomes dramatically more capable overnight, the governance problem does not get smaller. It becomes one of the most important architecture problems the enterprise has ever had.

The technology that makes competence less necessary for execution makes it more necessary for everything around execution.

That is not a reason to fold.

It is part of what we are betting.