Photo by FlyD on Unsplash

Author’s Note:

The workplace examples in this piece are generalized from requests I have reviewed and stories shared by friends, colleagues, and community members over the past several years.

Details have been altered, and none describes a specific person, team, or project.

~Dom

The Bug Wasn’t Where They Were Looking

On October 21, the PHP project released emergency patches addressing two remote code execution vulnerabilities, both rated 9.2 for severity, across its supported PHP 8.2 through 8.5 branches. The flaws were not in WordPress, Drupal, or the custom applications built on them. They were in PHP itself, in how the runtime manages memory and recovers from errors: machinery that every request passes through, in some configurations before the application’s own code has run at all. Researchers confirmed unauthenticated code execution against WordPress and Drupal installations in cases where the application’s own code had nothing wrong with it.

That is the uncomfortable part. A developer could have written careful, reviewed, and thoroughly tested code, done everything their discipline asks of them, and still shipped a server a stranger could take over. The weakness sat one layer beneath their work, in a foundation they had every reason to trust and no practical way to inspect.

The second uncomfortable part came out afterward. The PHP Foundation acknowledged that its security team had known about one of the underlying problems since 2024. A core developer found it while fixing a different flaw during the project’s first external security audit in years. The fix stalled for months, because every option carried a cost to compatibility across an ecosystem that runs much of the web, and the team lacked the people to settle it.

What turned a known issue into an emergency was a set of AI-assisted investigations in 2026 that showed, with proof, how it could be exploited. The weakness itself was known in at least one place, and largely invisible to the developers and operators who depended on that foundation.

Most developers do not debug their language runtime.

Most business leaders do not independently investigate the assumptions underneath their operating environment.

Both trust that someone else has made the foundation reasonably sound, and that trust is rarely naive; if everyone had to understand everything they depend on, far less would get built. Unfortunately, this also means that capable people routinely build functional things on top of weaknesses they have no reason to recognize.

And AI is rapidly increasing how much, and how quickly, any one person can build.

Every Business Has a Runtime

I see the organizational version of this regularly, usually in a review queue.

A SQL script came through a while back that did exactly what its author asked of it, but consumed enough shared compute capacity to throttle unrelated workloads for days. The logic was correct. What was missing was an understanding of what else depended on the resources it would consume.

Another request was more revealing. Someone asked IT to change a tenant-level setting so that an AI agent could use a feature it wanted for a task. The requester did not know there was a tenant, that it had an administrator, or that a setting at that level applies to everyone in the organization. The agent knew. It had identified the layer, determined that the layer was an obstacle, and drafted a request to move it.

Neither person was careless, exactly. Both were working inside the part of the system they could see, toward an objective that likely had a deadline.

Consider a more common case: Imagine that a project manager is designing a new supplier onboarding process with AI assistance. The assistant produces a coherent plan: roles, milestones, approvals, system integrations, expected efficiencies. The project manager can evaluate whether the plan meets the business objectives they were given, and it does.

What they may not know about is the informal supplier-validation step in Finance, the undocumented exception rules in Procurement, or the legacy identifiers the existing ERP still depends on. The assistant almost certainly doesn’t know either. The plan is plausible, and in some ways, the plausibility is the problem: it rests on assumptions whose importance is invisible from inside the planning exercise.

There is a difference between a risk that is unknown and a risk that is only unknown to the person doing the work. Finance may know precisely why a change to a supplier’s bank account requires independent verification. The control exists because impersonating a supplier to redirect its payments is one of the oldest frauds in accounts payable. The project manager may never have heard of it, and the assistant has no access to that institutional knowledge unless someone builds a way to tell it.

The weakness is not unknown to the organization overall, but it is unknown at the place where the decision is being made.

Security Learned to Remember

Cybersecurity faced this problem early, and at a scale that forced an answer. No discipline can expect every developer to independently discover every way software fails. There are too many weaknesses, too few specialists, and learning each lesson through an incident costs too much. So the field built a way to remember.

The mechanisms are easier to understand by what they do than by what they’re called. One catalogs recurring types of weakness in software and hardware, the kinds of mistakes that keep getting made. Another identifies specific, publicly disclosed vulnerabilities in specific products. A third gives technologies and products standard names, so a vulnerability can be matched to the systems that actually run the affected software. A fourth organizes the tactics and techniques adversaries have been observed using, so defenders can recognize behavior they have never personally encountered. These are, in order, CWE, CVE, CPE, and MITRE ATT&CK.

The acronyms, though familiar to nearly everyone in security, are not the achievement. The achievement is that a developer can learn about a weakness without ever having an incident around it, a security analyst can recognize an attack pattern without having faced that attacker, and an administrator can apply a protection based on a failure discovered by someone they will never meet.

The PHP case shows why the difference between those catalogs matters. The specific vulnerabilities were publicly disclosed in October, but the classes of weakness were familiar long before that.

One of the flaws involves internal state that outlives the memory it depends on, a use-after-free condition. Use-after-free has its own entry in the weakness catalog, CWE-416, and was a standard item in secure coding guidance long before this bug was written. The catalog couldn’t identify this particular vulnerability, but it documented the kind of problem investigators already knew could exist.

That distinction is what business planning mostly lacks. Organizations are not short on records of what went wrong before. Failure mode and effects analysis, risk registers, project retrospectives, and lessons-learned programs all exist, and have for a long time. Systematic organizational learning is not a new idea, and I’m not proposing that we reinvent it.

What remains underdeveloped is the delivery. Lessons get recorded as instances: what happened on that project, to that team, in that year. They are rarely generalized into classes that someone on a different project would recognize, and they are almost never delivered to the person making a decision at the moment the lesson may change the direction and outcome.

The problem is not memory, but the distance between where the memory is kept and where the decision is made.

Consulted Last

In my own organization, much of that memory currently lives with the people who review requests, including me, and experience elsewhere suggests that arrangement is common.

Over the past several years, the same patterns have arrived again and again. Static web applications, built quickly with AI assistance, submitted for hosting on SharePoint with an Excel workbook as the backend. The prototype looked right and worked for one person on one afternoon. Migration plans still wearing the formatting hallmarks of the assistant that generated them. A one-time diagnostic, complete with the model’s stylized output, left inside a scheduled production notebook, so that every trigger re-runs tests for a question that was settled weeks ago.

None of these are reasons to stop people from building, and most of them come from people who were solving real problems; some of the prototypes were genuinely good ideas. The issue is where the knowledge that would have changed them arrives. Someone in IT knows why a spreadsheet makes a poor concurrent database, why shared capacity is not free, and why diagnostic code should not survive into a scheduled job. But that knowledge reaches the requester at review: after the prototype is built, after it has been shown to a manager, after the requester has become attached to it.

That is, outside of an incident, the most expensive point at which a lesson can arrive. The rework is largest, the requester’s investment is deepest, and the conversation that follows feels less like advice than a veto.

So the honest version of my role in this story is that I am, often, a weakness catalog. An incomplete one, certainly, but a useful one for the systems I know. And I get consulted last.

What a Business Weakness Catalog Would Look Like

The answer is not another risk register. Every organization already has those, and most are written for one project, consulted at its start, and forgotten.

Imagine instead a curated library of known weakness patterns that describes classes rather than instances: the business equivalent of a catalog that makes use-after-free a recognizable class of weakness, rather than something every developer must independently discover. Each entry would describe a recurring failure mode, the kinds of initiatives and dependencies where it tends to appear, the evidence that suggests it may be present, and the controls that reduce it.

Entries need an organizing axis, and security already has a serviceable one. Confidentiality asks where information travels and who can see it, including any AI provider in the loop. Integrity asks who validates that the data, the rules, and the resulting process are correct. Availability asks whether the organization can operate and recover the thing without the person, or the model, that assembled it.

I’ve written about these three questions in the context of AI governance before. Here they serve a narrower purpose: describing the kinds of harm a weakness could cause. Finding relevant entries would require another set of characteristics: the type of initiative, the systems and dependencies involved, the business processes affected, and the observable signs that a known weakness may be present.

For the supplier onboarding project, imagine that the organization has already encountered a related problem during an earlier migration. A generalized entry, derived from that experience, might look like the hypothetical example below:

BW-017 – Integrity – Verification controls omitted during workflow replacement

• Applies to: Supplier onboarding, ERP migrations, workflow automation, master-data management.
• Known failure pattern: A replacement workflow reproduces the documented process but omits controls performed informally or outside the source system.
• Previous occurrence: An earlier migration reproduced the supplier creation and approval steps but missed an independent verification performed by Accounts Payable when banking information changed. The process passed functional testing because the undocumented step was absent from the original requirements.
• Observable indicators: Manual verification outside the source system, undocumented approval steps, different practices between regions, missing evidence of independent verification.
• Suggested controls: Walk through actual exceptions with process owners, document existing verification requirements, validate separation of duties, and test the replacement against representative changes to sensitive supplier information.
• Evidence: Approved control mapping, recorded exception tests, and confirmation from the responsible business owner.

Now imagine the project manager describing their initiative to an assistant with access to a library of entries like this one. Instead of producing a plausible plan on its own, the assistant could identify the few entries that likely apply, explain why, and suggest the specific questions to bring to Finance, Procurement, and IT while the plan is still a draft.

Given appropriate access to a project’s evolving documentation and decisions, it could also keep watching. ATT&CK catalogs adversary behavior, not project problems, but its central practice transfers: connecting something observed to a recognizable pattern. An approval role removed without announcement, a manual override that has become routine, or an exception process with no named owner. None of these proves a failure is underway, but each is a reason to ask a question while the answer is still cheap.

Some indicators are almost embarrassingly concrete. Leftover diagnostic output in production code, assumptions still presented as questions in an approved plan, or implementation notes describing decisions nobody remembers making. Even something as superficial as assistant-style formatting may be a reason to ask whether the work was reviewed, though it is hardly proof that it wasn’t. The concern is not whether AI was used, or whether its fingerprints remain. It is whether anyone with the necessary context examined the decisions that survived into the finished work.

Two kinds of record belong in this library. A catalog of weakness classes tells you what to look for. A history of actual failures tells you how the weakness has shown up in real situations, and what it cost. Security offers a useful parallel: CWE describes recurring classes of weakness, while CVE identifies specific, publicly disclosed vulnerabilities. An organization’s catalog would need to go a step further, connecting its generalized weaknesses to the incidents, near misses, and corrective actions from which they were learned. Those experiences are what keep the classes honest.

This is also different from the decision logs I’ve proposed elsewhere for recording exceptions and exclusions. Those record your own decisions so that you, or your successor, can reread them later. A weakness catalog moves knowledge between people who may have never met, and it is retrieved by context rather than reread by its author.

The point is not simply to remember what went wrong previously, but to allow that memory to travel. The benefit is not that a connected assistant suddenly knows everything. It is that the organization can connect what it has already learned to the people about to need it.

Knowing the Risk Is Not Managing It

A catalog of known weaknesses is incomplete by construction. It cannot anticipate every dependency, and a person or AI using it can select irrelevant entries, overstate minor risks, or miss circumstances that are genuinely new.

Security, again, knows this limit well. A vulnerability scanner that reports no findings has not established that a system is secure. It has established only that no issue was detected within the scanner’s visibility, configuration, and available detection rules. An AI-generated risk review that reports no serious concerns would mean almost exactly as much.

If the catalog is consulted as a checklist, it becomes a way to certify the absence of known problems, which is easy to mistake for the absence of problems.

Business weaknesses may also be considerably harder to classify than their technical counterparts. A use-after-free condition has a specific meaning. An initiative that fails because of unclear requirements, inadequate stakeholder engagement, or unrealistic timelines may involve several overlapping causes, none sufficient to explain the outcome alone. Likewise, two projects can exhibit the same warning signs and fail for entirely different reasons. Any useful catalog would need to preserve enough of the original context to distinguish a relevant precedent from a superficial resemblance, rather than reducing every failure to a tidy category.

Maintaining one is also real work; someone has to investigate actual failures, separate the generalizable lesson from the unusual circumstance, confirm that a proposed control actually helps, and retire entries when conditions change. Organizations already spend considerable effort investigating failures; they rarely turn the results into something the next team could recognize before living through it.

And, finally, knowing is not the same as acting. The PHP case provides a direct example: the PHP team had recognized signs of the underlying problem in 2024, but the issue remained unresolved until this October. And it stayed unresolved because fixing it was expensive, the trade-offs were contested, and there weren’t enough people to resolve them. A business weakness catalog will have the same limit. It can put the right question in front of the right person, but a catalog cannot make them, or their budget, answer it.

So the test of a lessons-learned program should focus less on whether the organization documented what went wrong, and much more on whether someone who later encounters the same conditions has a reasonable chance to recognize the risk before experiencing the consequence.

Neither a software vulnerability nor an organizational weakness becomes real at the moment someone discovers it. Both usually existed long before the incident that revealed them, and in many cases someone, somewhere, had already seen something like it.

We are not going to eliminate the unknown. Every system has dependencies nobody has mapped, and every plan rests on assumptions nobody thought to question. Not everything that goes wrong has happened before. But enough of it has that we should be much better at recognizing the patterns. Somewhere, another team may have already learned why that approach fails, or another architect may have discovered why those systems should not be connected.

Security spent decades learning to make that knowledge cumulative rather than forcing it to be rediscovered. As AI lets more people take on more ambitious work with less specialized understanding, the rest of the enterprise needs the same discipline. The same technology may also be the most practical way yet to deliver it.

The question was never whether someone knew. It was whether what they knew was in the room when the decision was made.