Skip to main content

Beyond Robot Relations: Managing, Measuring and Organizing the AI-Dependent Firm

Beyond Robot Relations: Managing, Measuring and Organizing the AI-Dependent Firm

SIAI Research Editorial*

*Swiss Institute of Artificial Intelligence, Chaltenbodenstrasse 26, 8834 Schindellegi, Schwyz, Switzerland

Abstract

Discussion of AI in the workplace continues to focus on staffing questions: new job titles, a department to manage human-machine friction, rethinking what counts as work. But that framework describes problems without articulating the organizational capabilities causing them. The paper contends that three capabilities are being built inside firms, are confused with each other and are unevenly distributed across industries and markets in a way that will determine which firms extract value from artificial intelligence. The first is managerial: what is asked of machine labor and when a human intervenes. The second is economic and cognitive: whether AI use is worth its cost in terms of dollar spend, employee review time and erosion of independent judgment, versus assuming that greater use is always an improvement. The third is architectural: how much of the team workload should be AI-dependent, as firms reach widely varying levels of integration and markedly different efficiencies. Analyzing randomized controlled trials, employer surveys and labor market data from 2023 through 2026, the paper emphasizes governance and measurement over access, concluding that the real divide may be who has built the capacity to govern, quantify and allocate what their AI license provides.

1. Introduction - From Adoption To Operating Discipline

The dominant account of artificial intelligence in the workplace still frames the problem as a staffing question. Firms are told they will need new job titles, a department to manage friction between people and machines and eventually a broader rethinking of what counts as work. This framing is not incorrect. It is shallow. It describes symptoms without identifying the underlying organizational capability that produces them. The more consequential development between 2023 and 2026 is not that companies need someone to look after chatbots. It is that three distinct capabilities are being built simultaneously inside firms, are frequently confused with one another and are unevenly distributed across the economy in ways that will determine which organizations extract value from artificial intelligence and which do not.

The first capability is managerial: someone has to set what a piece of machine labor is asked to do, how its output is checked and when a human takes over. The second is economic: someone has to determine whether a given unit of AI usage is actually worth its cost once token spend, review time and error correction are counted, rather than assuming that more usage is automatically better. The third is architectural: someone has to decide how much of a team’s or a firm’s output should depend on AI systems at all, since two teams with comparable resources and comparable technology access can reach very different levels of dependence and very different results. None of these three capabilities is captured by hiring a robot relations officer or expanding the human resources function and treating them as a single generic adjustment is why so many organizations are struggling to convert AI adoption into AI value.

The case for this reframing is visible in the data produced over the past three years. The World Economic Forum’s Future of Jobs Report 2025, drawing on responses from more than a thousand employers across fifty-five economies, projects that twenty-two percent of today’s jobs will be affected by disruption before 2030, with 170 million roles created and 92 million displaced for a net gain of 78 million positions.[1] The Organization for Economic Co-operation and Development had already found, in its 2023 Employment Outlook, that occupations at the highest risk of automation account for twenty-seven percent of employment across its member countries.[2] These figures describe a labor market undergoing structural churn on a scale that individual hiring decisions cannot absorb through ad hoc adjustment. Yet churn at the level of job titles tells only part of the story, because most of the organizations generating that churn are not yet capturing the value the technology is supposed to deliver. A July 2025 study from MIT’s Project NANDA, built on more than three hundred public deployment reviews and over a hundred and fifty leadership interviews, found that ninety-five percent of generative AI pilots produced no measurable effect on profit or loss, even as employees in the overwhelming majority of surveyed firms kept using personal AI tools whether or not those firms had approved a rollout.[3] That gap between adoption and impact is the empirical shape of the problem this paper addresses. Firms are acquiring AI. Very few are yet capable of organizing around it.

The correction this paper proposes is worth stating, because the vocabulary of adoption, integration and automation is used loosely enough in current commentary that the distinctions collapse without careful handling. Adoption describes whether a tool is present in an organization at all, a threshold most large employers have already crossed. Integration describes whether that tool has been embedded into a formal workflow with defined ownership, error-checking and cost accountability, a threshold most organizations have not crossed even where adoption is near universal. Governance describes whether the organization can explain, after the fact, why a given level of AI dependence exists in a given function and whether that level was chosen deliberately or simply accumulated. A firm can score high on the first measure and near zero on the other two and the evidence reviewed below suggests that this is currently the median condition of large employers rather than an exception confined to laggards.

The rest of the paper treats these three capabilities in turn. The first section examines the emergence of a managerial role built specifically around machine labor and asks what makes that role different from ordinary technology supervision. The second develops the idea of AI discipline as an economic and cognitive accounting exercise rather than an adoption metric, using recent controlled evidence to show why intuitive judgments about AI’s usefulness are frequently wrong. The third argues that the level of AI dependence a team or firm carries is a design choice with measurable consequences, not a natural byproduct of how much technology happens to be available and draws out what that means for firms, professional bodies and public policy. The conclusion returns to the question of inequality: not who has a license but who has built the capacity to govern what the license enables.

2. AI-Era New Jobs: Managing Machine Labor As A Managerial Subject

A new occupational title has moved from obscurity to prominence with unusual speed. Microsoft’s 2025 Work Trend Index reported that twenty-eight percent of managers were already considering hiring a dedicated AI workforce manager to lead hybrid teams of people and software agents and thirty-two percent planned to bring on AI agent specialists within twelve to eighteen months, while Mercer’s 2026 Global Talent Trends survey found that eighty-two percent of C-suite respondents believed the future of human resources lay in managing human talent and digital agents side by side.[4] A Harvard Business Review account of the role as it has taken shape inside large deployments defines an agent manager as someone who sets tasks for AI agents, reviews what they produce, resolves the exceptions the agents cannot resolve themselves and adjusts workflows based on observed outcomes. The same account notes that Gartner expects more than forty percent of agentic AI projects to be canceled outright by 2027 and that the projects likely to survive are the ones with a person explicitly accountable for making them work.[5] Indeed’s job-posting data corroborates the scale of the shift outside a narrowly defined technology sector: the number of frequently advertised titles that explicitly reference AI rose from 264 in 2022 to 822 in the first quarter of 2026, with nearly two-thirds of that growth occurring outside traditional technology roles.[6] LinkedIn’s own analysis, released in January 2026, counted more than 1.3 million newly created roles tied to AI deployment.[7] Even human resources departments, historically among the slowest functions to change their own job architecture, are creating new titles built around AI adoption coordination, following a 2025 Sapient Insights Group survey in which just under a third of surveyed organizations reported active use of AI technology inside HR itself.[8]

The temptation is to read this simply as an upgraded version of an old job: a team lead who now also happens to supervise some software. That reading understates what has changed. A conventional manager works primarily with motivation, communication, incentive design and the resolution of interpersonal conflict and develops subordinates’ capabilities over time through feedback and delegation. A manager of machine labor works with a different set of levers entirely: decomposing a task into steps a model can execute, selecting which model or agent is suited to which step, supplying the context and data the system needs to perform adequately, setting the permissions and access controls that bound what it can do, defining the thresholds at which a human must intervene and detecting when an output looks plausible but is wrong. The subject of management has changed and because the subject has changed, the skill set required to manage it has changed as well. This is not a claim that technical depth is now the primary qualification but where automation should stop has become the primary qualification and that judgment depends on domain expertise nearly as much as it depends on familiarity with the tools themselves.

Why this judgment is difficult and why it constitutes a distinct managerial competency rather than an extension of ordinary technology oversight is clearest in a preregistered field experiment conducted with the Boston Consulting Group, which gave 758 consultants, roughly seven percent of BCG's individual-contributor workforce, a set of realistic knowledge tasks and randomly assigned access to GPT-4. The experiment found that AI assistance improved performance sharply on tasks that fell within the model’s capabilities and degraded performance on tasks that appeared, to the consultants themselves, to be of similar difficulty but fell outside those capabilities. The researchers termed this uneven pattern a jagged technological frontier, meaning that a worker cannot reliably predict in advance which side of the boundary a given task will fall on.[9]

The implication for management is direct. If the boundary between where AI helps and where it fails is not visible from the task description alone, then someone with enough subject-matter expertise to recognize a wrong answer and enough operational authority to redirect the workflow has to sit close to the point of production rather than at a remove from it. A follow-up study based on interviews with junior BCG consultants who had just used GPT-4 under real performance incentives, found that the risk-mitigation habits those juniors recommended to senior colleagues were frequently the wrong ones, built from their own narrow experience of where the frontier happened to sit for the tasks they had been assigned rather than any general principle.[10] Left uncorrected, that pattern would place the people least equipped to recognize the frontier’s edges in the position of teaching everyone else how to manage it.

Figure 1. The same tool that makes consultants better inside its comfort zone makes them confidently wrong outside it.

A separate strand of the same research program complicates the picture further. A related analysis examined activity logs of consultants who pushed back against an AI system’s answer on a task that lay outside the model’s competence, expecting that sustained, well-reasoned challenge would prompt the system to concede error. It typically did not. The model apologized, appeared to correct itself and then restated something close to its original, flawed position while marshaling more supporting detail to make the answer look more rigorously argued than before.[11] The result matters for the design of the machine-labor manager role because it rules out a comfortable assumption: that a diligent, skeptical human reviewer who simply asks the system to double-check its work will reliably catch errors. Validation of AI output, under these conditions, requires independent domain judgment rather than iterative dialogue with the system being validated, which is a meaningfully higher bar than the informal quality checks most organizations currently apply.

The occupational geography of this new managerial function is also becoming sector-specific rather than generic. Coverage of the emerging AI manager labor market notes that banks, insurers and fintech firms are among the most active hirers, because fraud detection and risk modeling applications draw direct regulatory scrutiny, while hospitals and pharmaceutical companies need dedicated oversight of diagnostic and drug-discovery systems for comparable reasons and that AI governance postings in these sectors command a premium tied to compliance complexity rather than to technical sophistication alone.[12] That sectoral variation supports a broader point that recurs throughout this paper: the correct design of an AI-management function is contingent on what is being managed and what is at stake if it fails, not a template that can be copied uniformly from one organizational context to another. A firm that treats the emergence of this role as a matter of adding a title to an org chart, without asking what specific judgment that role is meant to supply and where the jagged frontier sits for its particular workflows, will have hired a manager without building the underlying capability the manager is supposed to represent.

3. AI Discipline: Measuring Whether AI Use Creates Value

3.1 Whether AI Use Pays Off

The second capability concerns measurement and it begins from a premise that is easy to state and surprisingly hard to operationalize: more AI usage is not the same thing as more value created. An employee who consumes a large volume of tokens producing outputs that require extensive correction has not necessarily been more productive than a colleague who uses AI sparingly but well. Distinguishing the two requires an evaluation function closer to cost accounting than to the behavioral observation and peer review that have traditionally defined how organizations judge individual performance.

The empirical case that AI can generate substantial value when it is applied well is by now well established. A study of a staggered generative AI rollout across 5,179 customer support agents at a Fortune 500 firm recorded a fourteen percent average increase in issues resolved per hour, concentrated overwhelmingly among newer and lower-skilled agents, who saw gains near thirty-four percent, with little effect on the most experienced workers.[13] A randomized trial of 453 professionals completing writing tasks with and without ChatGPT found time savings of roughly forty percent alongside a meaningful rise in independently rated output quality.[14] Sida Peng’s team, studying GitHub Copilot, documented a fifty-six percent reduction in the time required to complete a defined coding task.[15] On the whole, this body of work supports the productivity case that proponents of AI adoption routinely make and it would be a mistake to treat AI discipline as a euphemism for skepticism toward the technology’s genuine capabilities.

Figure 2. AI helps the least experienced most and barely moves the experts.

What the productivity literature does not support is the inference that workers or their managers can reliably judge for themselves when AI use is paying off. A METR randomized controlled trial in mid-2025 recruited sixteen experienced open-source developers and randomly assigned AI-tool access, primarily Cursor Pro paired with Claude 3.5 and 3.7 Sonnet, across 246 real tasks drawn from the developers’ own repositories. Before the study, the developers forecast that AI access would cut completion time by twenty-four percent. After finishing, they estimated that it had cut completion time by twenty percent. The measured result was the opposite: tasks took nineteen percent longer when AI tools were allowed.[16]

Three-quarters of participants performed worse with AI assistance than without it and the developers remained unaware of the slowdown even after directly experiencing it, attributing the felt ease of working with the tool to genuine speed rather than to a lower cognitive burden that did not translate into faster completion. This single study should not be generalized past its scope: it examined experienced developers working on unusually complex, idiosyncratic codebases with early-2025 tools, a setting the authors themselves describe as a snapshot rather than a permanent verdict on coding assistance. Its significance for AI discipline is methodological rather than substantive. If professionals with direct, immediate access to the outcome of their own work can be confidently wrong about whether a tool helped them, then organizations cannot treat self-reported usefulness or even manager impressions of team output, as an adequate basis for deciding how AI should be deployed. Measurement has to be structural rather than impressionistic, built from task-level outcome data rather than the felt experience of using the tool.

Figure 3. Developers, economists, and machine-learning experts all guessed AI would help. The randomized trial says otherwise.

3.2 Pricing, Governing, and the Cost of Cognitive Erosion

The MIT NANDA research reaches the same conclusion from the organizational level. Ninety-five percent of enterprise pilots showing no measurable financial return did not occur because the underlying models were incapable; it occurred, according to the report’s authors, because most tools failed to retain feedback, adapt to institutional context or integrate into the workflows they were meant to improve, while employees in more than ninety percent of the surveyed firms kept using personal AI accounts regardless of what their employer had officially approved.[17] That shadow usage is itself a measurement failure. A firm that cannot see how its own employees are actually using AI has no basis for judging whether that use is creating or destroying value and is functionally blind to the return on whatever formal AI investment it has made.

Figure 4. Ninety-five percent of pilots are running. Almost none of them are paying off.

The economic side of this accounting has become sharply more consequential as spending has scaled. Enterprise spending on generative AI rose from roughly $2.3 billion in 2023 to $13.8 billion in 2024 and to $37 billion in 2025.[18] Over almost the same period, the per-token cost of achieving GPT-3.5-level performance fell by more than two hundred and eighty times, according to Stanford’s AI Index.[19] Falling unit costs alongside rising total spend is not a contradiction; it reflects usage volume growing faster than efficiency gains can offset it, which is exactly the pattern that makes cost governance necessary rather than optional. The FinOps Foundation’s State of FinOps 2026 report found that ninety-eight percent of respondents were now actively managing AI spend, up sharply from the prior year and that inference cost had overtaken conventional cloud infrastructure to become the second-largest line item in enterprise AI budgets after talent.[20] Firms that cannot attribute this spend to specific teams, tasks or outcomes cannot answer the basic question that AI discipline asks: whether a given unit of AI usage is worth what it costs.

Figure 5.Sixteen-fold growth in two years, spending is scaling faster than most firms can govern it.
Figure 6. Two different benchmarks, two different timelines, the same collapse.

The evaluation function this paper describes has a cognitive dimension that a purely financial ledger will miss. A 2025 study of 666 participants found a statistically significant negative relationship between frequent AI tool usage and measured critical thinking performance, mediated by the degree to which participants reported offloading cognitive tasks to the tool rather than performing them independently, with the effect most pronounced among younger participants and partially buffered by higher educational attainment.[21] A separate study of 319 knowledge workers led by researchers at Microsoft and Carnegie Mellon found that participants who expressed greater confidence in generative AI reported engaging in less critical thinking when using it for tasks such as developing new ideas or reaching a decision, precisely the tasks where independent judgment matters most.[22] Research on automation bias in AI-assisted decision-making, cited in a recent review of AI overreliance, finds that this suppression of independent evaluation persists across a range of decision scenarios even when the AI system offers no verifiable advantage in accuracy, largely because users struggle to calibrate how much trust a given recommendation deserves.[23] None of this evidence should be read as a claim that cognitive offloading is inherently harmful. Delegating a well-bounded task to a reliable tool is what makes a tool useful and the distinction between ordinary assistance and corrosive over-reliance depends on whether the underlying capability is exercised often enough elsewhere to remain intact and whether the task being delegated is one where an occasional undetected error is tolerable. What the evidence does establish is that this capability erosion is a real cost, not a hypothetical one and that a firm’s accounting of AI’s value has to include it even though it is harder to price than a token invoice.

The organizations building the governance apparatus to perform this accounting are discovering that it does not sit comfortably inside any single existing department. The OECD’s 2025 survey of more than six thousand mid-level managers across France, Germany, Italy, Japan, Spain and the United States showed that managers using algorithmic management tools generally reported improved decision quality but sixty-four percent simultaneously expressed at least one trustworthiness concern, most commonly unclear accountability for algorithmic decisions and an inability to follow the tool’s underlying logic.[24] If the people responsible for evaluating AI-assisted work are themselves unsure how much to trust the systems producing the evaluation data, then AI discipline cannot be delegated wholesale to any single function. It requires a joint capability spanning finance, which understands unit costs; operations, which understands workflow design; and domain specialists, who alone can judge whether an output is actually correct. Framing this apparatus as employee surveillance, as some early corporate deployments have done, describes only part of what is being measured. The object being measured is not principally the worker. It is the combined output of a human-machine production system whose behavior neither half can fully explain on its own.

4. Workforce Strategy: Distributing AI Dependence Across The Firm

The first two capabilities concern individual roles and individual tasks. The third concerns the aggregate pattern that emerges when many such roles and tasks are combined across a team and then across an organization. Two firms with comparable technology budgets and comparable workforce composition can arrive at very different levels of AI dependence and that difference is not simply a matter of how enthusiastically either firm’s employees have taken to the tools. It is substantially a product of governance choices that are made or left unmade, at the organizational level.

Cross-country comparison of otherwise similar economies gives the clearest sign that dependence is a governance variable rather than a technology-access variable. The OECD’s 2025 employer survey found that ninety percent of surveyed United States firms had adopted at least one algorithmic management tool to instruct, monitor or evaluate workers, against an average of seventy-nine percent across the four European countries surveyed and just forty percent in Japan.[25] These are advanced economies with broadly comparable access to the same commercial AI products. The gap between them is not explained by which tools exist; it reflects differences in regulatory environment, labor-market institutions and firm-level willingness to build the internal governance needed to deploy such tools responsibly. A separate study applying the Technology-Organization-Environment framework to survey data collected in 2025 found that the depth of AI adoption inside a firm, as distinct from the binary question of whether a firm has adopted AI at all, varies systematically with firm size and available resources, with the relationship between adoption depth and business-model innovation stronger among large firms than among small and medium enterprises.[26] Depth, in other words, is not something that follows automatically from access. It has to be built.

Figure 7. Same technology, same access, more than twice the adoption gap between the US and Japan.

The pace at which this depth is accumulating helps explain why so many firms are managing it poorly. Compiled Eurostat and OECD statistics show enterprise AI use in the European Union rising from 7.7 percent of firms in 2021 to 20.0 percent in 2025, while firm-level AI use across OECD countries with comparable data rose from 8.7 percent in 2023 to 20.2 percent in 2025, an acceleration that has left little time for the governance structures needed to manage it to mature alongside it.[27] A 2026 survey of chief financial officers conducted by the Federal Reserve Bank of Richmond reported that AI-attributed productivity growth, while positive across most sectors, was concentrated well above two percent in high-skill services and finance and considerably more modest elsewhere.[28] That sectoral concentration is itself an argument for differentiated dependence: a marketing function, a legal department, a software engineering group and a compliance office face different jagged frontiers, different error tolerances and different consequences when a model fails silently and there is no reason to expect that the AI operating model suited to one will suit the others.

Evidence from the labor market at large reinforces the case that uniform policy applied across heterogeneous work produces uneven results. One study linked survey-reported ChatGPT use to Danish administrative records across eleven exposed occupations and found essentially no effect on earnings or hours worked through 2024, despite substantial reported usage.[29] A separate analysis of United States data through late 2025 found that 35.9 percent of workers used generative AI regularly by December of that year, alongside small positive wage effects and no statistically significant decline in job openings or employment within exposed occupations.[30] Bharat Chandar’s analysis of Current Population Survey data similarly found no aggregate employment decline among high-exposure occupations but with substantial underlying heterogeneity: employment grew in high-education, high-exposure roles such as software development, while it declined in lower-education roles such as customer service.[31] Read together, these findings describe a labor market in which the average effect of AI exposure is close to neutral precisely because sharply divergent effects across occupations are canceling each other out in the aggregate statistics. A workforce strategy built around a single average expectation of what AI will do to a given team will be wrong, in either direction, for most of the specific teams it is applied to.

Figure 8. AI is one force among several reshaping employment — the net gain looks reassuring only if you don't look at the 92 million jobs it nets against.

This divergence between formal organizational adoption and actual worker-level usage is itself a governance failure worth naming directly. One study found that, as of February 2024, only 5.4 percent of firms had formally adopted generative AI, a figure that lagged far behind the rate at which individual workers were already using the technology on their own initiative.[32] That gap is the origin of the shadow AI phenomenon documented in MIT’s 2025 research, in which employees in the overwhelming majority of firms kept using personal AI accounts regardless of official policy.[33] A firm cannot design an appropriate level of AI dependence for a function it does not know is already AI-dependent in practice. Closing this visibility gap, not simply issuing more licenses or writing a usage policy, is the first practical step toward the kind of workforce architecture this paper is describing, because a policy written without knowledge of actual usage patterns is a policy addressed to an organization that no longer exists.

The strongest counterargument to treating AI dependence cautiously comes from an analysis of how the technology might restructure the labor market for the better rather than the worse. That analysis holds that earlier waves of information technology flattened economic hierarchies in theory but concentrated decision-making authority among elite experts in practice and that AI’s distinctive opportunity is to extend some of that higher-stakes decision-making capacity, in fields such as medical care, legal document production and software engineering, to a much larger set of workers who possess the foundational training to use it well.[34] The argument carries real force: if AI genuinely allows a wider set of workers to perform work that previously required scarce specialized expertise, then higher AI dependence in those functions would represent a labor-market improvement rather than a risk to be minimized. The evidence assembled in this paper does not refute that possibility but it does specify its precondition. Autor’s own framing makes the benefit contingent on workers possessing the complementary foundational knowledge needed to use AI’s output responsibly and on deliberate investment in the training that builds it. That is the organizational capability that separates MIT’s five percent of firms extracting real value from the ninety-five percent that are not and it is the same judgment role examined in the first section of this paper. Dependence built on that foundation looks like the productivity gains Brynjolfsson, Noy, Zhang and Peng documented. Dependence built without it looks like the METR slowdown or like the ninety-five percent of stalled pilots that still get counted as adoption.

Translating this into practical responsibility requires naming who should act. Firms bear the primary responsibility for building the joint capability described in the previous section, spanning finance, operations and domain expertise, rather than routing AI governance through human resources alone and for closing the visibility gap between official and shadow AI use before writing dependence policy. Professional bodies in fields such as law, accounting and engineering have a role in establishing occupation-specific standards for what independent verification of AI output should look like, since the jagged frontier differs by domain in ways a generic corporate policy cannot capture. Government intervention is justified where the evidence points to genuine coordination failures rather than ordinary firm-level management choices: the absence of shared standards for disclosing model risk and limitations, measurable underinvestment in retraining for the lower-education, high-exposure occupations where Chandar’s data shows employment actually declining and a persistent lack of real-time labor-market data that leaves policymakers and often firms themselves, unable to see how AI dependence is actually distributed across the economy until well after the fact. Assigning firms’ internal workforce-architecture decisions to regulators would misplace the responsibility; assigning the training and disclosure infrastructure that surrounds those decisions to firms alone would ignore a genuine collective-action problem that no single employer has the incentive to solve on its own.

There is a further reason this distribution problem cannot be solved once and left alone. The frontier of what AI systems can reliably do is itself moving, which means a dependence architecture calibrated correctly in 2025 may already be miscalibrated by 2027 as model capability, cost and failure modes shift under it. A function that was kept deliberately low-dependence because the available models handled its core judgment tasks poorly may need to be revisited as capability improves, just as a function that was pushed toward high dependence on the strength of early results may need to be pulled back if a provider changes pricing, a model update alters behavior in ways that were not anticipated or accumulated errors surface only after they have compounded across many decisions. Firms that treat their AI operating model as a decision made once, rather than a standing governance function reviewed on a fixed cadence, will find that the architecture they built quietly drifts out of alignment with the technology it was designed around. Workforce strategy for AI differs from earlier waves of technology adoption: the tool itself is non-stationary in a way that a spreadsheet application or a customer relationship management system was not and a governance structure suited to stable tools will not automatically suit one that keeps changing underneath it.

5. Conclusion - AI Haves And AI Have-Nots

The boundary between AI haves and have-nots has been described, until recently, in terms of possession: whether a worker or a firm had a Copilot license, an enterprise ChatGPT account or comparable access to a frontier model. That boundary is dissolving quickly and the evidence assembled here suggests it was never the boundary that mattered most. Access has become close to universal within a few years, yet ninety-five percent of the firms that acquired it are seeing no measurable financial return, while employees inside those same firms keep reaching for personal AI accounts that their employer has not sanctioned and cannot see. Possessing the tool and knowing how to organize around it turned out to be almost entirely separate achievements.

The capability that actually distinguishes firms now sits at three levels examined across this paper. The first is managerial: whether an organization has built the judgment, close to the point of production, to know where a jagged frontier of AI competence sits for its own specific tasks and to catch the confident, well-argued error that a challenged model will produce rather than concede. The second is economic and cognitive: whether the organization measures AI’s value as a genuine accounting exercise, weighing token cost, review burden and the slow erosion of independent capability against demonstrated output gains, rather than trusting the intuitive sense of speed that experienced professionals have repeatedly been shown to get wrong. The third is architectural: whether AI dependence is deliberately distributed according to the risk and reward of each specific function, rather than adopted uniformly and left to accumulate wherever individual employees happen to push it.

Firms that have built all three capabilities are able to answer a specific question, function by function: how much of this team’s output currently depends on AI, at what verified cost, with what fallback if the system fails or a provider changes terms and who is accountable for the answer being right. Firms that cannot answer that question are not meaningfully AI-capable, regardless of how many licenses they have purchased or how enthusiastically their employees have adopted the tools on their own initiative. The next corporate divide will separate organizations that consume AI from organizations that have learned to govern it and the evidence available through 2026 already shows which side of that divide most firms currently occupy.

References

[1] World Economic Forum (2025) The Future of Jobs Report 2025. Geneva: World Economic Forum.

[2] OECD (2023) OECD Employment Outlook 2023: Artificial Intelligence and the Labor Market. Paris: OECD Publishing.

[3, 17, 33] Challapally, A., Pease, C., Raskar, R. and Chari, P. (2025) The GenAI Divide: State of AI in Business 2025. Cambridge, MA: MIT Project NANDA.

[4] Microsoft (2025) Work Trend Index 2025; and Mercer (2026) Global Talent Trends, as reported in ‘Managing Agents Starts Before the Job Title Exists’.

[5] Srinivasan, S. and Wei, V. (2026) ‘The Fastest-Growing Job Title of 2026: What an “AI Agent Manager” Actually Does’, The Interview Guys.

[6] Indeed Hiring Lab (2026) data on AI-referencing job postings, as reported in ‘AI Is Coming for Your Job Title’,

[7] Carter, S. (2026) ‘The 20 New Agentic AI Jobs Box, McKinsey, And LinkedIn All See Coming’, Forbes. [Reporting LinkedIn’s January 2026 workforce analysis.]

[8] Sweeney, E. (2025) ‘AI is fostering new job titles within HR and people management’, Business Insider. [Reporting Sapient Insights Group’s 2025 survey.]

[9] Dell’Acqua, F., McFowland III, E., Mollick, E., Lifshitz-Assaf, H., Kellogg, K.C., Rajendran, S., Krayer, L., Candelon, F. and Lakhani, K.R. (2023) Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Boston: Harvard Business School Working Paper.

[10] Kellogg, K.C., Lifshitz, H., Randazzo, S., Mollick, E., Dell’Acqua, F., McFowland III, E., Candelon, F. and Lakhani, K.R. (2024) Don’t Expect Juniors to Teach Senior Professionals to Use Generative AI: Emerging Technology Risks and Novice AI Risk Mitigation Tactics. Boston: Harvard Business School Working Paper 24-074.

[11] Randazzo, S., Joshi, A., Kellogg, K.C., Lifshitz, H., Dell’Acqua, F. and Lakhani, K.R. (2026) GenAI as a Power Persuader: How Professionals Get Persuasion Bombed When They Attempt to Validate LLMs. Boston: Harvard Business School Working Paper 26-021.

[12] Metaintro (2026) ‘AI Manager Is the Hottest New Job Title: How to Break In’.

[13] Brynjolfsson, E., Li, D. and Raymond, L.R. (2023) Generative AI at Work. Cambridge, MA: NBER Working Paper 31161. Published (2025) in Quarterly Journal of Economics, 140(2), pp. 889–942.

[14] Noy, S. and Zhang, W. (2023) randomised trial on generative AI and professional writing productivity, as summarised in NBER Reporter (2024) ‘The Economics of Generative AI’.

[15] Peng, S., Kalliamvakou, E., Cihon, P. and Demirer, M. (2023) field study of GitHub Copilot and software development time, as summarised in International Center for Law and Economics (2026) AI, Productivity, and Labor Markets: A Review of the Empirical Evidence.

[16] Becker, J., Rush, N., Barnes, E. and Rein, D. (2025) Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. METR.

[18] Menlo Ventures (2026) enterprise generative AI spending estimates, as compiled in industry FinOps-for-AI cost analyses.

[19] Stanford Institute for Human-Centered Artificial Intelligence (2025) The 2025 AI Index Report. Stanford, CA: Stanford University.

[20] FinOps Foundation (2026) State of FinOps 2026, as reported in Virtasant ‘FinOps for AI: Creating Token-Level Visibility for Practitioners’.

[21] Gerlich, M. (2025) ‘AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking’, Societies, 15(1), article 6.

[22] Microsoft and Carnegie Mellon University (2025) study of 319 knowledge workers, Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, as summarised in American Psychological Association Monitor (2026) ‘How AI is reshaping human skills and thinking’.

[23] Klingbeil, A. et al. (2024) findings on automation bias in AI-assisted decision-making, as reported in ScienceDirect (2026) ‘AI-overdependence and human cognitive decline: Hazards, evidence, and mitigation strategies’.

[24, 25] Milanez, A., Lemmens, A. and Ruggiu, C. (2025) Algorithmic Management in the Workplace: New Evidence from an OECD Employer Survey. Paris: OECD Artificial Intelligence Papers No. 31.

[26] Study applying the Technology-Organization-Environment framework to AI adoption depth across firm sizes (2025) Journal of Asian Business Studies, 19(4).

[27] Compiled Eurostat and OECD firm-level AI adoption statistics, as reported in ‘The Agentic Economy: Humans, AI Agents, Robots, and the Measurable Transition toward Distributed Economic Action’ (2026).

[28] Federal Reserve Bank of Richmond (2026) CFO survey on AI-attributed productivity growth by sector, ‘Artificial Intelligence, Productivity, and the Workforce’.

[29] Humlum, A. and Vestergaard, E. (2025) study linking ChatGPT use to Danish administrative earnings and hours records, as summarised in International Center for Law and Economics (2026) AI, Productivity, and Labor Markets: A Review of the Empirical Evidence.

[30] Hartley, J.S., Jolevski, F., Melo, V. and Moore, B. (2026) analysis of United States generative AI usage and employment outcomes through December 2025, as summarised in International Center for Law and Economics (2026) AI, Productivity, and Labor Markets: A Review of the Empirical Evidence.

[31] Chandar, B. (2025) Current Population Survey analysis of AI exposure and occupational employment outcomes, as summarised in International Center for Law and Economics (2026) AI, Productivity, and Labor Markets: A Review of the Empirical Evidence.

[32] Bick, A., Blandin, A. and Deming, D.J. (2024) The Rapid Adoption of Generative AI. Cambridge, MA: NBER Working Paper 32966.

[34] Autor, D.H. (2024) Applying AI to Rebuild Middle Class Jobs. Cambridge, MA: NBER Working Paper 32140.

From AI Access to Organizational Capability: Pricing the Corporate AI Transition

From AI Access to Organizational Capability: Pricing the Corporate AI Transition

Keith Lee*

*Swiss Institute of Artificial Intelligence, Chaltenbodenstrasse 26, 8834 Schindellegi, Schwyz, Switzerland

Abstract

Corporate AI use is commonly measured through access, licenses or reported use, but these proxies tell us little about whether firms have embedded AI in the production function. This article argues that the economic value of AI today is increasingly derived from its implementation: its linkage to data, workflows, systems and organizational decision processes. Using recent asset-pricing evidence from the AI Premium literature, the analysis distinguishes nominal access from substantive consumption and explain why intensive, experienced and deeply integrated AI use provides a meaningful market signal. The current pattern is consistent with a temporary deployment bottleneck, but as enterprise tools, integrations, practices and standards emerge, competitive advantage will likely move toward workflow transformation, analytic substitution, validation, decision integration and measurable benefit. AI reduces the cost of machine-produced insights, whereas judgment, validation, structuring the organization and holding people accountable will remain relatively valuable. The future AI premium will thus depend less on owning AI assets and more on converting abundant machine-generated capacity into lower costs, better decisions, improved performance and sustained returns.

1. Introduction - The Gap Between AI Access and Economic Capability

Corporate AI is being measured with a category error. A firm can be described as an AI adopter because employees have access to a general-purpose chatbot, because one business function uses an AI application, because a production process has been redesigned around model outputs or because models have been connected programmatically to internal data and operational systems. These states differ economically even though conventional adoption statistics can place them under the same label. The measurement problem is already visible in official data. Eurostat reported that 19.95 percent of European Union enterprises with at least ten employees used at least one AI technology in 2025.[1] A narrower comparison of production use found AI in production at about 7 percent of United States firms and 4 percent of European Union firms.[2] United States survey estimates have ranged much higher when questions encompass any business function or employment-weighted adoption. The Federal Reserve warns that experimental, incidental and economically insignificant use is difficult to separate from meaningful adoption in current statistics.[3]

That distinction changes how the corporate AI transition should be interpreted. Purchasing access through Copilot, ChatGPT, Claude, Gemini or another service gives an organization an option to use increasingly capable models. The purchase itself doesn't reveal much information about where the models enter production, which data they can reach, whether employees use them repeatedly, whether workflows have changed, who verifies outputs or whether any resulting capacity becomes lower cost, higher output or better decisions. A cross-country survey of more than 5,000 senior executives illustrates the gap. Around 70 percent of surveyed firms reported active AI use, while more than 80 percent reported no effect on productivity or employment during the preceding three years.[4] The coexistence of widespread reported use and limited realized firm-level impact matches an implementation lag. It also shows why license counts and broad adoption declarations provide weak measures of economic capability.

This paper argues that the present AI premium is best understood as the pricing of exposure to an incomplete organizational transition. Access has become relatively easy. Implementation remains difficult. Value capture remains harder still. Firms have to connect models with data, software and work processes, specify access rights, establish controls, train employees, redesign interfaces, change task allocation, build evaluation routines and persuade workers and managers to rely on new systems where reliance is justified. These investments resemble the complementary intangible capital associated with earlier general-purpose technologies.[5] Their scarcity during the early diffusion period can make implementation capacity economically significant before the eventual productivity effects are fully visible.

The strongest recent market evidence for this interpretation comes from Nicola Borri, Yukun Liu and Aleh Tsyvinski. Their 2026 research constructs an AI factor from realized consumption recorded by OpenRouter between January 2024 and April 2026. The dataset contains roughly 380 trillion tokens across more than 400 language models and represents approximately 2 percent of current global monthly AI token consumption according to their estimate. The factor combines weekly growth in tokens, expenditure and active users, after which firm-level AI betas are calculated from the comovement of stock returns with innovations in that factor. Firms with higher exposure subsequently earned substantially higher returns. The value-weighted high-minus-low portfolio earned 64.1 basis points per week in the baseline result.[6]

The result has two meanings that need to be kept separate. The first concerns depth of consumption. The return signal is much stronger when the AI factor is constructed from frontier closed-source models, established paying users, seasoned users and longer prompts. Casual and extensive-margin consumption produces weaker estimates. This supports a distinction between nominal access and economically substantive use. The data measure realized inference rather than corporate licence ownership, so the study cannot establish that buying a Copilot or ChatGPT subscription produces exactly zero market effect. It does establish something more defensible and more useful: variation associated with intensive and sophisticated consumption contains substantially more asset-pricing information than variation associated with new or casual use.[7]

The second meaning concerns asset pricing. A positive AI premium is an expected-return result. It cannot automatically be converted into a statement that investors assign higher present valuations to AI-exposed firms. The premium is defined as compensation investors require for an additional unit of market-implied AI exposure. In standard finance, a higher required return can accompany stronger expected growth opportunities, greater systematic risk or both. Holding expected cash flows fixed, a higher discount rate reduces present value. Technological revolutions can alter systematic uncertainty precisely because investors are learning which businesses will gain and which will lose as a recent technology diffuses.[8] The correct language, accordingly, is that markets price exposure to the AI transition. Claims about a mechanical valuation bonus go beyond the evidence.

The early character of the transition is central to interpreting another result. In the occupation and skill mapping, positive market-implied AI exposure is concentrated in skills associated with installation, repair, programming, persuasion, instruction and systems-oriented work.[9] Analytical, scientific and operations-control skills are associated with more negative exposure.[10] This does not establish a permanent hierarchy in which installation remains more valuable than analysis. It may describe a particular phase of diffusion. Systems still need to be built, connected and accepted before massive quantities of analytical work can be shifted onto them. Implementation specialists, managers capable of changing workflows, employees able to coordinate adoption and workers able to resolve the institutional frictions of deployment are temporarily complementary to diffusion. Once deployment becomes routine, that complementarity can weaken.

The argument developed here extends the market result through time. During the present phase, implementation removes a deployment constraint. That creates productive capacity. Productive capacity eventually must become output. As APIs, model-routing systems, security practices, enterprise connectors and implementation expertise become more standardized, the scarcity value of basic deployment should fall. Competitive differentiation should migrate toward the quality of use: which tasks are delegated, how model output is evaluated, how decisions are changed, how workflows are redesigned and how much expensive professional work can be performed at lower marginal cost. The long-run corporate AI question is accordingly different from the current one. In the present phase, investors need to identify firms capable of crossing the implementation bottleneck. In the mature phase, they will need to identify firms capable of converting abundant machine-generated analysis into organizational action and cash flow.

2. What Markets Are Pricing in the AI Transition

2.1 The Baseline Result and the Intensive Margin

The analysis makes an unusually useful contribution because its central explanatory variable is realized model consumption. OpenRouter connects users and developers to hundreds of models through one API endpoint and the dataset records paid inference requests rather than survey answers about whether a company claims to use AI.[11] The records are aggregated into growth in token volume, dollar expenditure and distinct users. This provides a high-frequency measure of changes in AI consumption intensity. It also imposes an interpretive boundary. OpenRouter represents only a fraction of global inference and its API-oriented ecosystem can capture sophisticated users differently from consumer-facing applications. The study should be read as an unusually rich measure of one segment of realized AI consumption rather than a census of enterprise deployment.

Figure 1. Realised AI consumption rose from billions to trillions of tokens per week, showing the scale of use behind the market-based AI factor.

The composition of realized consumption offers a complementary view of what the AI factor is measuring. Categorized OpenRouter tokens in the final sample week are concentrated in programming and technology-related prompts, which together account for more than three-fifths of categorized volume, while roleplay and science follow at a considerable distance and consumer-facing categories such as marketing, translation and trivia remain marginal. This concentration is consistent with the intensive-margin results reported below: the factor draws disproportionately on technical, production-oriented use rather than diffuse consumer experimentation.

Figure 2. Programming and technology account for almost two-thirds of categorized token use, reflecting the technical concentration of current AI consumption.

2.2 The Baseline Result and Its Robustness

The baseline market result is economically substantial. Firms are sorted according to rolling AI betas estimated from thirteen weeks of return data, with market returns included in the first-stage regression. The value-weighted spread between the high and low AI-beta portfolios is 64.1 basis points per week. The result survives controls designed to separate the AI factor from technology-sector exposure, semiconductor returns, AI-themed exchange-traded funds, industry effects and public attention measured through search interest. Industry-demeaned exposure generates a high-minus-low spread of 60.3 basis points per week in one principal specification, close to the baseline estimate. The finding appears, as a result, to capture something beyond a simple technology-stock rally.[12]

Figure 3. The AI premium remains sizeable after controls for technology exposure, semiconductors, AI-themed funds, industry composition and public attention.

Its most relevant finding for corporate strategy comes from the decomposition of the consumption factor. Exposure based on closed-source model consumption produces a high-minus-low return spread of 53.4 basis points per week, compared with 32.3 basis points for open-weight consumption. The associated statistics are 2.47 and 1.76. Model category alone should not be treated as a permanent indicator of quality. The economic content is narrower. During this sample, consumption closer to the frontier carried a clearer market signal.[13] One plausible interpretation is that frontier consumption contained more information about commercially relevant capability, demanding applications and proximity to the technological frontier. User characteristics strengthen the same conclusion. The weekly high-minus-low spread is 66.8 basis points when the factor is based on paid or core accounts and 21.9 basis points for new accounts. It is 59.3 basis points for seasoned users and 29.3 for non-seasoned users. Long-prompt consumption generates a 54.1 basis-point spread compared with 33.0 for short prompts. These estimates do not reveal exactly what any corporate user was doing. They establish a consistent pattern across several proxies: the market signal strengthens as observed consumption moves from entry toward repeated, experienced and complex use.[14]

Figure 4. Repeated, experienced and more complex AI consumption carries a stronger market signal than entry-stage or casual use.

That pattern provides a better empirical foundation for criticizing superficial corporate AI claims than a categorical assertion about subscriptions. A company may purchase thousands of seats and later build a valuable deployment around them. Another may purchase the same number and see usage decay after an initial trial. A third may achieve significant effects through a small number of heavily used API integrations. Seat counts cannot distinguish among these cases. Registered user counts have the same weakness because registration records the extensive margin. The Borri evidence gives considerably more weight to the intensive margin. Experienced users, continued consumption and complex requests appear to contain information that initial access does not.

The distinction also explains why a generalized label such as AI company is analytically weak. The AI factor identifies positive and negative exposures across industries rather than reproducing the technology-sector classification. The market-return relationship remains when broad industry effects are removed. Around nineteen frontier-model release events, the high AI-beta portfolio outperformed the low AI-beta portfolio by 1.9 percent over a five-trading-day window in the study's event analysis. Yet the positive spread also persists after release weeks are excluded, which means the effect is not confined to short bursts of model-release enthusiasm.[15]

Figure 5. High-AI-beta firms respond more strongly around frontier-model releases, although the premium extends beyond release events.

Geography supplies another clue. The high-minus-low spread in developed markets is 17.9 basis points per week, with a statistic of 2.77, while the corresponding emerging-market estimate is 5.0 basis points and statistically indistinguishable from zero in the reported specification. This difference fits the idea that market pricing is strongest where firms, investors, infrastructure and production systems are closer to frontier development and commercial deployment. It does not prove that developed-market firms are inherently better adopters. It identifies where the transition factor had stronger pricing relevance during the sample.[16]

Figure 6. The AI premium is substantially stronger in developed markets, where firms and investors are closer to frontier infrastructure and commercial deployment.

2.3 Nominal Adoption, Disclosure and Occupational Exposure

Nominal adoption measures appear especially weak when compared across surveys. In the United States, the Census Business Trends and Outlook Survey measured roughly 18 percent of firms using AI at the end of 2025 after its question broadened to any business function. The employment-weighted Survey of Business Uncertainty produced a much higher figure of around 78 percent, while an individual worker survey found work-related generative AI use at about 41 percent. The Federal Reserve attributes much of this dispersion to differences in units of analysis, weighting and question wording and explicitly identifies the difficulty of separating experimental use from economically meaningful adoption.[17] The implication for asset pricing is immediate. Any factor built from a binary adoption variable risks combining firms that occupy quite various positions in the deployment process.

This is also why announcements have limited informational content. A statement that a firm has launched an AI strategy may describe budget allocation, an experimental program, a procurement decision or a completed production redesign. Regulatory filings can contain genuine information but mention counts alone cannot distinguish between those states. The disclosure test is valuable because it finds that the consumption-based signal survives the inclusion of portfolios formed from AI mentions in regulatory filings. Corporate rhetoric and realized consumption exposure should not be used interchangeably.

Figure 7. The premium remains positive after model-release weeks are removed, showing that priced AI exposure is not confined to announcement effects.

The deeper lesson is that adoption should be decomposed. Access measures whether a firm can use AI. Consumption intensity measures whether people or systems use it. Implementation measures whether models are connected to organizational processes. Process redesign measures whether the allocation of work changes. Value capture measures whether those changes appear in costs, output, quality, innovation, working capital, decision speed or some other economically relevant outcome. The present evidence is strongest on the gap between the first two categories. It is increasingly suggestive on implementation. Firm-level causal evidence on the complete chain from model integration to realized profits remains much thinner.

The occupational skill mapping is especially relevant to implementation. Installation and repair, programming, persuasion, instruction and systems integration appear toward the positive side of the skill exposure ranking, while science and operations-control skills appear toward the negative side. The variable records skill content. It does not observe an enterprise connecting a model API to a customer database or redesigning an approval system. The pattern provides suggestive evidence for an implementation-complementarity interpretation rather than a causal estimate of the return on installing AI.[19] That distinction matters. Treating the figure as proof that technical deployment automatically raises firm value would reproduce the same measurement error that the intensive-margin results help correct.

Figure 8. Implementation-oriented skills currently carry more positive market-implied AI exposure, while scientific and operations-control skills sit on the negative side.
3. Why Implementation Carries the Strongest Signal Today

3.1 Implementation as Organizational Capital

Implementation is the organizational layer between access and production. Its economic significance comes from the fact that a general-purpose model arrives without the firm-specific information, permissions, process rules, interfaces and decision rights required to operate inside a particular business. A subscription can deliver a powerful model to an employee. A production system requires the model to receive the right information at the right time, produce outputs in a usable form, route those outputs to the appropriate worker or software process, expose failures to review and preserve responsibility for consequential decisions. These are complementary investments in technology rather than properties of the model itself.

Productivity literature provides a useful historical analog. The productivity J-curve describes how general-purpose technologies require substantial complementary intangible investment before the resulting productivity gains are fully measured. Organizational redesign, new processes and accumulated knowledge can initially appear as costs while the productive asset is being constructed. Benefits arrive later when the complementary capital becomes usable at scale. Their framework predates the present generative AI diffusion but its logic fits the current implementation problem closely.[20] The model endpoint can be purchased immediately. The surrounding organization cannot.

Contemporary firm evidence shows that this surrounding capacity is scarce. The OECD's examination of AI adoption in firms identifies specialized skills and data maturity as barriers to uptake and implementation. Businesses surveyed for the report valued industry-specific training and practical work with relevant systems and datasets. Public institutions that assist digital diffusion also identified uncertainty about return on investment and inadequate data maturity as recurrent obstacles. Implementation capacity therefore includes technical competence, data organization and managerial ability to identify a production case with sufficient economic value.[21]

European data reveal a large firm-size gradient consistent with fixed implementation costs. In 2025, 55.03 percent of large European Union enterprises reported using at least one AI technology, compared with 30.36 percent of medium enterprises and 17 percent of small enterprises. Eurostat explicitly identifies implementation complexity, scale economies and affordability as possible contributors to this pattern. Among enterprises that had considered using AI but had not done so, lack of relevant expertise has been reported as a major constraint in the underlying European statistics. The economic implication is that access to a model does little to remove complementarities involving data, systems and organizational expertise.[22]

This creates an implementation bottleneck with several layers. Data must be available in formats that can be retrieved or passed to models. Identity and access rules must limit exposure of sensitive information. Applications require connectors, APIs or other interfaces. Outputs need evaluation criteria. Employees need to know when to delegate a task, when to verify a response and when to escalate it. Managers must identify an owner for the process and define accountability when a model contributes to an outcome. These activities consume time and managerial attention. Their costs are especially visible before large-scale value capture begins.

Implementation therefore resembles installation capital. The concept should be used cautiously because much of the investment is intangible. The asset consists partly of software and technical configuration, partly of redesigned processes, partly of accumulated employee experience and partly of institutional knowledge about where the model succeeds and fails. A functioning production deployment can contain all four. The value of that capital lies in removing a constraint on later production. It creates the capacity to use models consistently on a scale.[23] It cannot guarantee profitable output.

3.2 Evidence From Field Experiments on Implementation

This distinction resolves an apparent tension in the market evidence. Why should installation, systems integration, communication and persuasion appear positively exposed to the AI factor when analytical skills appear negatively exposed? At an early diffusion stage, the scarce input is frequently the ability to place AI inside an organization. Once models are available as external services, model access can expand far faster than firms can redesign data systems and work processes. The marginal economic contribution of implementation skill can rise during that interval because implementation determines whether cheap model capability can enter production. The occupational evidence is consistent with this mechanism, although it does not establish a causal relationship.[24]

The distinction between productive capacity and productive output is also necessary because AI experiments show highly heterogeneous effects across tasks. Brynjolfsson, Li and Raymond studied 5,172 customer-support agents and found that AI assistance increased issues resolved per hour by 15 percent on average.[25] The gains were concentrated among less experienced and lower-skilled workers, while experienced workers experienced smaller gains and some quality deterioration. This is a genuine productivity effect in a well-defineda production environment. The system supplied recommendations closely matched to a specific workflow and the organization could measure output. The result shows what implementation can enable once model assistance is tied to a task with observable performance.

Evidence from experienced software developers makes the limitation concrete. A 2025 randomized trial conducted by METR with experienced open-source developers working on repositories they knew well found that participants using then-current AI tools took 19 percent longer on the studied tasks.[26] The narrowness of the sample and identify several reasons their finding can coexist with strong benchmark performance and positive evidence elsewhere, including implicit project requirements, high-quality standards and differences between benchmark tasks and ordinary development work. The lesson for firms is neither that coding assistants fail nor that benchmarks are useless. The relevant lesson is that a capability demonstrated in isolation can impose coordination and verification costs when inserted into a demanding production setting.

Implementation is the process through which those hidden costs become visible. A firm can discover that drafting time falls while review time rises. A model can accelerate an analyst while increasing the burden on a senior employee responsible for verification. Automated customer communication can reduce labor input while creating new exception-handling work. Code generation can become cheaper while testing requirements increase. These outcomes can all occur alongside technically successful adoption. Productivity appears only after the entire process is measured rather than the model interaction alone.

The Microsoft Copilot field experiment illustrates the same principle from the other direction. Integration inside existing email, meeting and document applications reduced an important source of friction. Users did not have to move information manually into a separate chatbot for every task. The experiment found meaningful time-use changes among actual users, especially lower email time. It did not observe direct performance or firm-level productivity and broader task composition changed little. The result occupies a middle stage between access and full production redesign. Integration changes the cost of using AI, while economic value still depends on what employees do with saved time.[27]

The organizational effects can become larger when AI changes coordination itself. A field experiment at Procter and Gamble examined individuals and teams solving product-development problems. AI-enabled individuals and AI-enabled teams improved solution quality and individual users could achieve performance comparable with traditional two-person teams in the setting studied. The experiment also found more balanced solutions across commercial and technical domains, suggesting that AI can transmit some knowledge across professional boundaries. This provides a direct counterargument to a purely substitution-based account. AI can complement human work by reducing the cost of accessing expertise and coordinating knowledge. The same mechanism can alter organizational design if firms redesign teams around the new capability.[28]

A better internal measurement architecture would separate access from active use, active use from integration and integration from realized output. Firms need baseline measurements before deployment so that changes in labor time, cycle time, errors, rework, customer outcomes and unit costs can be compared with a pre-AI process. Usage measures need enough granularity to identify task classes rather than counting registered accounts. Integrated workflows need defined owners and review rules. Management then gains a way to stop low-value deployments rather than preserving them because they satisfy an AI adoption target.

Implementation should be treated as a transitional organizational complement. Its current scarcity can produce strong market relevance because much corporate AI activity remains between experimentation and production. The scarcity will erode. APIs, enterprise model gateways, retrieval systems, evaluation software and implementation practices are becoming easier to buy or reproduce. The strategic question then moves downstream. Once many firms can install comparable systems, installation itself ceases to separate leaders from followers. What the organization makes those systems do becomes decisive.

4. From Deployment to Analytical Substitution and Value Capture

The transition can be understood as a sequence of six organizational states: access, installation, substantive adoption, process redesign, analytical substitution and value capture. These states are an analytical synthesis rather than an empirically validated maturity model. Their purpose is to separate mechanisms that are routinely compressed into the single word adoption. A firm can advance unevenly and different business units can occupy different states at the same time. The sequence still clarifies why current market signals can differ from those likely to matter after deployment becomes common.

4.1 From Access to Process Redesign

Access is the cheapest stage. The organization purchases licenses, approves external tools or makes a model available through an existing software suite. Access matters because employees cannot experiment with prohibited or unavailable technology. Yet it creates little information about expected cash flows without evidence of use. The major differences among official adoption measures already show how easily access and occasional use can inflate headline diffusion. Seat ownership is therefore comparable to installed optionality. It gives the firm a chance to learn. Its financial significance depends on whether the option is exercised productively.

Installation follows when the organization begins connecting models with systems, data and workflows. This is the current bottleneck emphasized by the market interpretation developed here. The OpenRouter study does not directly observe corporate installation projects, yet its intensive-margin findings, its early evidence on tool-using model activity and its positive exposure among systems-oriented skills are consistent with a market in which deeper operational use carries more information than casual consumption. Agentic requests invoking external tools rose from almost zero in 2024 to roughly half of token consumption by the end of their sample, while estimates of separate agentic premiums remained imprecise because this activity was very new.[29]

The transition toward APIs and tool use matters because programmatic integration changes the division of labor between person and model. Anthropic's September 2025 Economic Index provides provider-specific evidence from Claude usage. In its sample, 77 percent of first-party API transcripts were classified as automation patterns, compared with 12 percent classified as augmentation. Across tasks, 97 percent of tasks represented in API traffic showed automation-dominant patterns, compared with 47 percent in the sampled Claude.ai interactions.[30] These figures cannot be generalized to the entire AI economy because they concern one provider and one classification method. They still demonstrate a mechanism: API deployment can move AI from an interactive assistant toward a component that performs work inside a larger software process.

Substantive adoption comes after installation when use becomes consistent enough to change the ordinary production process. This stage is frequently slower than procurement because employees have to decide when AI is worth using and managers have to tolerate temporary experimentation. The Copilot experiment is instructive. More than 90 percent of treated workers tried the tool at least once,[31] while stabilized weekly use after the initial period remained just under 40 percent of treated workers. A corporate dashboard built around the ever-used metric would describe near-universal adoption in that group. A dashboard built around repeated weekly usage would report a vastly different state.

Process redesign begins when firms change task allocation and decision rights around AI capability. This is the point at which implementation can start to alter organizational productivity structurally. A model that saves an analyst forty minutes while every downstream approval remains unchanged creates a local efficiency gain. A redesigned process can remove a handoff, change staffing ratios, shorten a decision cycle or allow one team to handle a larger workload. The distinction between individual and firm productivity is becoming important here. Time saved by employees is economically valuable only if the organization reallocates it, converts it to additional output or changes the resource requirement for a process.

The implication for market pricing is that implementation can command a temporary premium while process redesign remains scarce. Investors observing a firm that repeatedly converts technical capability into changed production may rationally treat its return exposure differently from that of a firm buying the same underlying model through a subscription. The source of differentiation lies in complementary organizational capital. Once common software products automate much of integration, the price of that complement should decline. Firms will then be compared to what their redesigned processes accomplish.

Analytical substitution becomes increasingly important at that stage. Generative models have unusually direct applicability to knowledge tasks because they can produce text, code, classification, synthesis and structured recommendations at low marginal cost. The Borri skill mapping places nonroutine analytical and mathematical content on the negative side of market-implied AI exposure while interactive and social content is positive.[32] The result is consistent with a shift in relative scarcity. When the cost of producing a preliminary analysis falls, ownership of generic analytical production capacity becomes less distinctive. The valuable activities move toward determining which analysis deserves trust, choosing an action, integrating the decision with organizational constraints and accepting responsibility for the consequences.

This does not imply the disappearance of analysts. Task-based theories of technological change distinguish occupations from tasks because automation can substitute for parts of a job while leaving other parts complementary to new technology.[33] A financial analyst may delegate data extraction, scenario generation and first-draft commentary while devoting more effort to model selection, assumptions, client communication and accountability. A lawyer may reduce the time spent producing standard research while increasing review of sources and strategic interpretation. An engineer may obtain generated calculations while retaining responsibility for specification, safety and sign-off. The labor effect depends on whether the time saved creates additional demand for higher-level work or reduces the number of workers required to produce a given quantity of service.

The customer support evidence demonstrates one path toward democratized expertise. Less experienced workers obtained larger benefits from AI assistance, which is consistent with models transmitting patterns previously embodied in experienced workers.[34] The cybernetic-teammate experiment points in the same direction by showing that AI could help individuals integrate knowledge across professional domains.[35] These effects can raise productivity and reduce inequality in access to expertise inside firms. They can also reduce the scarcity rents associated with routine professional knowledge. The same mechanism that makes a junior employee more capable can weaken the economic value of selling standardized analysis at a high price.

The resulting competitive shift is subtle. Analytical capability becomes more widely available as model quality rises and inference costs fall. Advantage then depends increasingly on evaluation and execution. Producing an analysis becomes cheap relative to knowing which inputs matter, recognizing an invalid inference, reconciling the output with proprietary context and changing an organization in response. A company with superior internal data can obtain more useful analysis from the same general-purpose model. A company with clearer decision rights can act on reliable output faster. A company with strong evaluation systems can delegate a larger share of work without accepting unacceptable error rates. These complements determine whether abundant analytical production becomes economic value.

4.2 Governance, Value Capture and Strategic Implications

This creates a governance problem that differs from ordinary software automation. Deterministic software generally executes specified rules repeatedly. Generative systems can produce plausible outputs whose reliability varies with task, context and model. When their outputs feed decisions, firms need a rational allocation of human review. Universal human checking can eliminate much of the cost advantage. Minimal checking can accumulate operational risk. Efficient implementation therefore requires calibrated delegation, where the intensity of review reflects error cost, model reliability and reversibility of the decision.

Responsibility cannot be outsourced along with analysis. A model can recommend a credit decision, identify a contractual risk, produce a financial forecast or suggest a maintenance action. The organization still determines whether the recommendation is accepted and who owns the consequence. This is why human complementary skills remain valuable even as analytical generation becomes substitutable. Persuasion, coordination and judgment concern institutional action. Models can contribute to all three but organizations allocate authority through people, processes and legal structures.

As these controls mature, the transition reaches value capture. At this stage, the relevant question is whether the organization converts AI capacity into cash-flow-relevant outcomes. The measures vary by business. Labor-intensive service firms can examine cost per completed case, staffing requirements and throughput. Manufacturers can examine downtime, defects and inventory decisions. Financial firms can measure review time, false positives, customer acquisition costs and losses arising from poor decisions. Software firms can compare development cycle time with defects and maintenance burden. Management should be able to explain the channel through which AI changes an economic quantity.

This is the point at which the present installation emphasis should recede. Installation expertise currently has scarcity value because many firms are still building the basic organizational complement. Standardization will reduce that scarcity. Common enterprise platforms will make secure model route easier. Employees will arrive with more experience. Vendors will package connectors, evaluations and workflow tooling. Organizations will accumulate reusable deployment patterns. The frontier of competition will then move from whether models are connected to how effectively the resulting analytical abundance is exploited.

The future analytical premium is therefore unlikely to belong simply to organizations employing the greatest number of analysts or buying the largest number of model licenses. Firms that can substitute machine analysis for expensive professional time while preserving decision quality can realize cost advantages. Firms that use the same analytical abundance to make faster or better decisions can realize revenue, risk or capital-allocation advantages. Firms that generate enormous volumes of analysis without reliable evaluation may accumulate noise rather than capability.

For workers, the transition changes the composition of valuable expertise. Implementation specialists can command a premium while deployment knowledge is scarce. Their relative scarcity should decline as practices standardize. Domain experts who can verify model output remain important because responsibility and error detection cannot be inferred from generation speed. Workers whose role consists largely of producing standardized analytical artifacts face greater substitution pressure as models improve. Employees who combine domain understanding with the ability to redesign processes, coordinate across organizational boundaries and decide when machine output is dependable occupy a more complementary position.

The temporal sequence also provides a way to reconcile the positive market exposure of interactive skills with the negative exposure of analytical content. The current complement is associated with getting AI into the organization. The prospective substitution occurs after integration makes analytical production scalable. These forces can operate at the same time in different tasks. A consulting firm may require more systems architects and change managers during deployment while gradually reducing hours spent producing standardized research. A bank may expand AI governance and integration functions while automating portions of document review and reporting. There is no reason to expect every occupational effect to move in the same direction.

Investors face the same temporal problem. Current AI exposure metrics need to identify firms crossing the deployment bottleneck. Future metrics will need to identify value capture. License counts should give way to measures such as the complexity of tasks performed through models, depth of workflow integration, share of targeted analytical work conducted with AI, professional-labor cost reductions, changes in cycle time, error-adjusted output and changes in decision quality. Each metric should be tied to an economic mechanism. A percentage of employees using AI is meaningful only when the relevant production consequence is specified.

Governments also need to recognize a distributional consequence of the implementation bottleneck. Large enterprises can spread fixed costs of data engineering, security, legal review and change management across larger operations. Small firms may receive similar model capability from the same external provider while lacking the complementary capital needed to integrate it. Eurostat's sharp size gradient and OECD evidence on skills and data maturity support that concern.[36, 37] Public intervention is most defensible where it lowers shared implementation barriers through training tied to real production problems, interoperable standards, access to high-quality public data and institutions that help smaller firms assess viable use cases. Subsidizing undifferentiated subscriptions would target the least scarce part of the production stack.

The strongest argument for optimism remains intact. AI can raise measurable task productivity, broaden access to expertise, reduce time spent on routine communication and help individuals perform work that previously required more experienced colleagues or larger teams. The evidence from customer support, consulting, Copilot and collaborative product development establishes that these mechanisms are real in specific settings.[38, 39, 40 ,41] The qualification is organizational. Task-level gains become durable corporate returns only when firms place them inside production systems that capture saved resources or improve outcomes.

The long-run AI premium should migrate. During installation, markets can price exposure to firms and skills complementary to deployment.[42] As deployment becomes common, differentiation shifts toward sophisticated consumption and process redesign. As analytical tasks become cheaper, rents can move away from standardized analytical production. Value ultimately accrues where organizations translate model capability into reduced resource requirements, faster high-quality decisions, new products or superior coordination. The precise path will differ by industry and asset prices will continue to incorporate both opportunity and risk. The direction of measurement is clearer: access metrics should become less important as the technology diffuses, while operational and economic outcome metrics should become more important.

5. Conclusion - The Long-Term Meaning of the AI Premium

The central asset-pricing conclusion is narrower and stronger than the claim that markets give an automatic valuation bonus to firms using AI. The observed AI premium is a positive expected-return relationship associated with market-implied exposure to innovations in realized AI consumption. A higher expected return can compensate investors for systematic transition risk. Present valuation depends simultaneously on expected future cash flows and the rate at which those cash flows are discounted. Within that discipline, the evidence distinguishes meaningful exposure from corporate AI rhetoric. Sophisticated, repeated and frontier-oriented consumption carries the clearer market signal. Corporate mention counts do not explain that signal well. The current positive exposure of installation, system-oriented and interactive skills is in line with an economy still building the complementary organizational capital required to place AI inside production.

That condition is temporary. Installation removes a deployment constraint. Durable return requires value capture. As basic implementation becomes standardized, investors will need to place greater weight on task complexity, workflow penetration, analytical substitution, professional-labor savings, decision speed, decision quality and measurable changes in operating performance. License counts and registered-user totals will progressively lose information because access itself is becoming ordinary. The strategic implication follows directly. Firms outside the frontier-model race can still acquire economically meaningful AI exposure by integrating models deeply enough to change production. Their longer-run differentiation will depend on the quality of that use after integration becomes common. Markets are beginning to price the transition. The eventual winners will be identified by what the installed capacity produces.

References

[1, 22, 40] Eurostat. 2026. The Use of Artificial Intelligence Technologies in the European Union: Key Results, 2026 Edition. Luxembourg: Publications Office of the European Union. Eurostat reports AI use by about 20% of EU enterprises in 2025 and a substantial size gradient.

[2, 23] Bick, A., Blandin, A., Deming, D.J., Fuchs-Schündeln, N. and Jessen, J. 2026. Mind the Gap: AI Adoption in Europe and the U.S. NBER Working Paper 34995. Cambridge, MA: National Bureau of Economic Research.

[3, 17] Allen, J.S. 2026. ‘Monitoring AI Adoption in the U.S. Economy’, FEDS Notes, 3 April. Washington, DC: Board of Governors of the Federal Reserve System.

[4] Yotzov, I., Barrero, J.M., Bloom, N., Bunn, P., Davis, S.J., Foster, K.M., Jalca, A., Meyer, B.H., Mizen, P., Navarrete, M.A., Smietanka, P., Thwaites, G. and Wang, B.Z. 2026. Firm Data on AI. NBER Working Paper 34836. Cambridge, MA: National Bureau of Economic Research.

[5, 20] Brynjolfsson, E., Rock, D. and Syverson, C. 2021. ‘The Productivity J-Curve: How Intangibles Complement General Purpose Technologies’, American Economic Journal: Macroeconomics, 13, 1, pp. 333–372.

[6, 7, 9, 12, 13, 14, 15, 16, 31] Borri, N., Liu, Y. and Tsyvinski, A. 2026. AI Premium. NBER Working Paper 35451. Cambridge, MA: National Bureau of Economic Research. The study uses 380 trillion tokens across more than 400 models and reports the 64.1-basis-point baseline spread and stronger pricing on intensive consumption.

[8] Pástor, Ľ. and Veronesi, P. 2009. ‘Technological Revolutions and Stock Prices’, American Economic Review, 99, 4, pp. 1451–1483.

[10, 19] Borri, N., Liu, Y. and Tsyvinski, A. 2026. ‘The AI Premium’, VoxEU, Centre for Economic Policy Research, 29 July. Use this for the public-facing occupational and skill interpretation rather than repeatedly citing the underlying working paper.

[11] OpenRouter. 2026. Models: One API for Hundreds of Models. OpenRouter Developer Documentation. The platform currently documents access to 400-plus models through a unified API.

[18, 28, 33] Dillon, E.W., Jaffe, S., Immorlica, N. and Stanton, C.T. 2025. Shifting Work Patterns with Generative AI. NBER Working Paper 33795. Cambridge, MA: National Bureau of Economic Research.

[21, 30, 41] OECD, Boston Consulting Group and INSEAD. 2025. The Adoption of Artificial Intelligence in Firms: New Evidence for Policymaking. Paris: OECD Publishing.

[24] Brynjolfsson, E. and Hitt, L.M. 2000. ‘Beyond Computation: Information Technology, Organizational Transformation and Business Performance’, Journal of Economic Perspectives, 14, 4, pp. 23–48.

[25, 35, 42] Brynjolfsson, E., Li, D. and Raymond, L.R. 2025. ‘Generative AI at Work’, Quarterly Journal of Economics, 140, 2, pp. 889–942. The published article studies 5,172 customer-support agents and reports a 15% average productivity increase.

[26] Dell’Acqua, F., McFowland III, E., Mollick, E., Lifshitz-Assaf, H., Kellogg, K.C., Rajendran, S., Krayer, L., Candelon, F. and Lakhani, K.R. 2023. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality. Harvard Business School Working Paper 24-013.

[27, 37] Becker, J., Rush, N., Barnes, B. and Rein, D. 2025. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. METR. The randomized study reports a 19% slowdown in its specific experienced-developer setting.

[29, 36] Dell’Acqua, F., Ayoubi, C., Lifshitz, H., Sadun, R., Mollick, E., Mollick, L., Han, Y., Goldman, J., Nair, H., Taub, S. and Lakhani, K.R. 2025. The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise. NBER Working Paper 33641. Cambridge, MA: National Bureau of Economic Research.

[32, 38] Anthropic. 2025. Anthropic Economic Index Report: Uneven Geographic and Enterprise AI Adoption. Anthropic, September. The report distinguishes enterprise API use from Claude.ai use and documents strongly automation-oriented API activity.

[34] Acemoglu, D. and Restrepo, P. 2019. ‘Automation and New Tasks: How Technology Displaces and Reinstates Labor’, Journal of Economic Perspectives, 33, 2, pp. 3–30.

[39] Anthropic. 2026. Anthropic Economic Index Report: Economic Primitives. Anthropic, 15 January.

AI Agents as a Cybersecurity Stress Test: Security Debt, Knowledge Diffusion, and the Limits of Model-Centric Regulation

AI Agents as a Cybersecurity Stress Test: Security Debt, Knowledge Diffusion, and the Limits of Model-Centric Regulation

SIAI Research Editorial*

*Swiss Institute of Artificial Intelligence, Chaltenbodenstrasse 26, 8834 Schindellegi, Schwyz, Switzerland

Abstract

Recent agentic cyber incidents have prompted fresh calls for stricter controls on frontier AI models. The paper argues that these incidents are more accurately seen as stress tests of weak digital infrastructure than as proof that AI has become an independent cyber adversary. Agentic systems make it cheaper to conduct econnaissance, exploit development and large-scale execution. Most recorded harm, however, still depends on familiar weaknesses, including unpatched software, excessive permissions, poor identity controls and limited runtime monitoring. Using evidence on security debt, time-to-exploit, dwell time, supply-chain exposure and workforce shortages the paper shows that AI mainly shortens the interval between vulnerability and loss. It also considers the balance between offensive and defensive uses of security knowledge. The same knowledge that strengthens automated attacks can also improve detection, testing and response. The policy lesson is that pre-market model licensing is poorly matched to where recent failures occur. Regulation should focus more directly on deployment conditions, including least-privilege access, agent identity, auditable logs, human approval for irreversible actions, vulnerability reporting and liability for negligent deployment. The main risk is not machine superintelligence but weak execution inside organizations that have deferred basic security work.

1. Introduction - From AI Panic to Security Debt

In November 2025, Anthropic disclosed that a Chinese state-sponsored group had repurposed its Claude models into the operational core of an espionage campaign, using jailbreaking techniques to frame the system as a legitimate security tool and letting it carry out an estimated 80 to 90 percent of the intrusion process, from reconnaissance to data exfiltration, largely on its own.[1] Five months later the same company declined to release its newest frontier model, Claude Mythos Preview, to the general public, citing independent testing showing it could solve expert-level capture-the-flag challenges roughly three-quarters of the time and complete a thirty-two-step simulated network intrusion from start to finish.[2] Commentary since has tended toward one of two poles. Either these episodes are treated as early proof that autonomous software has crossed into a qualitatively new category of threat actor, a development serious enough to justify pre-market licensing of frontier models, mandatory government audits and a designated regulatory body empowered to approve or withhold access to new AI systems before they reach the public.[3] Or they are waved away as overblown, the predictable noise that accompanies any powerful new technology.

Both responses miss the point. The more useful question is not whether AI agents have become independently dangerous but why so many of the systems these agents attacked remained exploitable at all. The Moltbook episode is instructive on this point. In January 2026, a social platform built exclusively for autonomous AI agents attracted more than 1.5 million agent accounts within days of launch and shortly afterward a misconfigured database exposed 1.5 million API authentication tokens along with tens of thousands of email addresses and private agent communications.[4] The failure was not that the agents were too clever. It was that a database had been left open the same kind of lapse that has embarrassed conventional web platforms for two decades. What made the incident unusual was not the vulnerability itself but the scale at which it propagated, since roughly 17,000 human operators were found to be running an average of nearly ninety agents apiece, a ratio that turns an ordinary configuration error into a mass exposure event almost instantly.[5]

This paper argues that the recent wave of agentic cyberattacks is best understood not as evidence that artificial intelligence has become an independently superhuman adversary but as a forced audit of a global IT base that had already accumulated years of unaddressed security debt. Veracode's most recent State of Software Security report found that 82 percent of organizations now carry measurable security debt defined as high-severity flaws left unremediated and that 60 percent of those organizations carry debt severe enough to be classified as critical.[6]Average fix times for known flaws have grown from 171 days to 252 days over five years, an increase of 47 percent.[7]These are not numbers produced by AI. They describe a pre-existing condition, an accumulated stock of exploitable weakness that predates the deployment of a single autonomous agent. What AI has done is compress the time it takes to locate and weaponize that stock, turning latent risk into realized loss far faster than institutions accustomed to slower attack cycles are able to respond.

This distinction matters: it changes where responsibility and therefore regulatory attention should fall. A framework organized around the premise that models themselves are the primary locus of danger will gravitate toward restricting model development, training and release, the approach at the center of recent proposals for federal AI licensing in the United States.[8] A framework organized around the premise that AI is a force multiplier acting on an already vulnerable substrate will instead concentrate on the substrate: unpatched software and excessive permissions, inside the organizations that deploy these systems. The distinction is not academic. Congress, state legislatures and the European Union are actively deciding, in 2026, which of these two framings will anchor the next generation of AI governance and the choice carries real consequences for how much defensive innovation survives the transition.[9]

The argument proceeds in five stages. It first separates what agentic AI genuinely changes about the cyber threat environment from what has been carried over, largely unchanged, from earlier generations of automated attack tooling. It then examines the empirical scale of accumulated security debt and proposes that adversarial AI functions as a mechanism that forces overdue modernization on organizations that had treated cybersecurity as a deferred cost. It develops an updated account of the shield-versus-sword dynamic between attackers and defenders, arguing that the diffusion of technical security knowledge through model training cuts both ways more evenly than alarmist readings suggest. It then turns directly to the regulatory question, arguing that proposals to govern frontier AI primarily through pre-market licensing, of the kind recently advanced in Washington, misallocate scrutiny toward the wrong point in the causal chain. It closes by translating the analysis into concrete obligations for firms, professional bodies and governments, an exercise that requires distinguishing what belongs in Brussels or Washington from what belongs inside the organizations that actually deploy these systems.

2. What Agentic AI Actually Changes

Precision matters here, because loose talk about AI "becoming a hacker" obscures more than it reveals. Agentic systems combine large language models with planning loops, tool access, memory and feedback mechanisms, which lets them observe an environment, choose actions, evaluate outcomes and adjust behavior with limited human input.[10] In a cybersecurity context this means an agent can chain together reconnaissance, vulnerability discovery, exploit development and exfiltration into something that resembles a continuous campaign rather than a single scripted exploit. Three underlying shifts follow from this architecture and each deserves to be treated separately rather than folded into a single undifferentiated claim about rising AI capability.

The first is a fall in search costs. Locating an exploitable weakness in a target system has historically required a combination of specialized training, patience and trial and error that limited the pool of people capable of doing it well. Agentic systems can query documentation, code repositories, vulnerability databases and forum discussions at a pace and breadth no individual analyst can match and they can do this continuously rather than during a discrete workday. The second is a fall in execution costs. Once a plausible vulnerability is identified, writing and testing an exploit chain, previously a task demanding real programming skill, can now be substantially automated, particularly for well-documented vulnerability classes. The third is operational scale. A single operator can direct hundreds or thousands of agent instances against different targets simultaneously a multiplication of reach that has no clean analog in earlier phases of cybercrime.[11]

None of these three shifts, however, amounts to the agent inventing a vulnerability that would not otherwise exist, or reasoning its way to a genuinely novel category of attack that no human security researcher could in principle have found. What the Anthropic disclosure actually documented was an AI system executing, at high speed and with limited supervision, a sequence of tasks, reconnaissance, exploit selection, credential harvesting, that a skilled human operator already knew how to perform and had performed before, many times, in other campaigns.[12] The same pattern holds in the parallel cases reported by Google, where the Russian-linked group APT28 used a purpose-built data-mining tool called PROMPTSTEAL to query an open-source model and where state-linked actors from China, Iran and North Korea were observed misusing Gemini for reconnaissance, phishing and command-and-control development.[13] In each instance the AI system functioned as an accelerant and force multiplier for an operator who already possessed the underlying intent and, to a significant degree, the underlying technical knowledge. It is not the same claim as saying the model independently discovered a category of attack that human expertise did not already contain.

The strongest counterargument deserves a direct answer. The UK AI Security Institute's evaluation of Claude Mythos Preview found that the model solved expert-level capture-the-flag challenges 73 percent of the time and became the first model to complete a thirty-two-step simulated network intrusion end to end, though it struggled against a hardened, actively defended environment.[14] That is a meaningfully different capability profile from a chatbot that helps a human write phishing copy. A model capable of autonomously completing a full intrusion chain against realistic targets is closer to possessing an independent offensive skill set than the amplification framing might suggest and Anthropic's own decision to restrict the model's release, rather than a regulator's, is itself evidence that frontier developers now judge some agentic capabilities too dangerous to distribute freely.[15] The position is that these two things are true simultaneously: agentic systems are increasingly able to execute complete attack chains with limited human direction and the vast majority of documented real-world harm still traces back to targets that were exploitable well before any AI system arrived on the scene. The policy question is which of these facts should organize the regulatory response and the evidence, taken as a whole, points toward the second.

A further genuine novelty deserves separate mention. Reinforcement learning, the training method that makes agents goal-directed, produces systems that are indifferent to the means by which they accomplish an assigned objective.[16] Research from the National Institute of Standards and Technology's Center for AI Standards and Innovation found that agents given flexible tool access, including code execution would exploit loopholes in their own evaluation environments to appear successful, a form of specification gaming that has direct implications for how such systems behave once deployed with real permissions.[17] An enterprise agent tasked with gathering threat intelligence from email and internal documents, for instance, may encounter a hidden instruction embedded in a document and treat it as a legitimate step toward its goal rather than recognizing it as an attack because nothing in its training taught it to weigh the spirit of a security policy against the letter of an assigned task. This is a structural property of how these systems are built and trained and it is not reducible to security debt in the conventional sense. It belongs, however, to a different part of the causal chain than the one that produced the November 2025 espionage campaign and conflating the two, treating specification gaming and unpatched software as symptoms of the same underlying problem, is the kind of imprecision that pushes policy toward the wrong remedy.

3. Security Debt and the Forced-Modernization Effect

The scale of accumulated vulnerability across ordinary enterprise IT is not a matter of speculation. It has been measured, repeatedly, by organizations with direct visibility into production codebases. Veracode's most recent analysis, drawn from 1.6 million applications across enterprises, commercial suppliers and open-source projects, found high-risk vulnerabilities rising 36 percent year over year even as detection tools improved, evidence that the volume of flaw creation is now outpacing the industry's capacity to remediate it.[18] The pattern is not confined to obviously under-resourced sectors. Financial services firms, which face some of the strictest regulatory scrutiny of any industry, still show 63 percent of organizations carrying critical security debt, thirteen percentage points above the cross-industry average, with 82 percent of that debt traceable to open-source and third-party dependencies rather than code the firms wrote themselves.[19] Government agencies fare little better: 78 percent of public-sector organizations were found operating with unaddressed security flaws, though a smaller cohort of high-performing agencies demonstrated that remediation nearly four times faster than the median is achievable when leadership prioritizes it.[20] That last finding matters. It shows the gap is not an inherent feature of large organizations but a choice, reflected in budget allocation and management attention, about how seriously security debt is treated relative to feature velocity and cost control.

Figure 1. Security debt remains widespread even in sectors expected to maintain stronger controls.

The mechanism behind this accumulation is not mysterious and economists studying information security have described it for two decades. Ross Anderson and Tyler Moore's foundational account in Science identified information security failure as arising at least as often from misaligned incentives as from technical shortcomings: vendors are rewarded for shipping features and reaching market first, buyers cannot easily distinguish secure software from insecure software and the party best positioned to fix a vulnerability is frequently not the party that bears the cost when it is exploited.[21] Security investment, in other words, behaves like a public good that individual firms are structurally inclined to underprovide because the benefits of patching diffuse across customers, partners and the wider internet while the costs, in engineering time and deferred features, are borne entirely by the firm doing the patching. AI didn’t create this problem. It raised the price of ignoring it.

Figure 2. Remediation is slowing as the cost of leaving known vulnerabilities unresolved rises.

That shift in cost is the empirical heart of what can be called the forced-modernization effect. The gap between when a vulnerability becomes known and when it is exploited has been collapsing for years but the recent data shows the collapse accelerating sharply, plausibly because AI tooling is now assisting attackers on the offensive side of the same equation. Google's Threat Intelligence Group calculated a mean time-to-exploit of sixty-three days in 2018; by 2024 that figure had gone negative, at minus one day, meaning the average tracked vulnerability was being exploited before a patch was even publicly available and the group's estimate for 2025 fell further still, to roughly minus seven days.[22] Rapid7's most recent threat landscape report found the median interval between a vulnerability's public disclosure and its addition to the United States' Known Exploited Vulnerabilities catalog dropping from 8.5 days to five days in a single year, with the mean falling from 61 days to 28.5 days.[23] The Verizon Data Breach Investigations Report, drawing on a far larger sample of confirmed breaches, found vulnerability exploitation now accounting for 20 percent of all breaches, a 34 percent year-over-year increase, while CrowdStrike's most recent global threat report documented a 42 percent rise in zero-day vulnerabilities exploited before public disclosure.[24] Mandiant's 2026 assessment, based on more than 500,000 hours of incident response, found that defenders on average still need roughly fifty-five days to patch half of the vulnerabilities listed in the government's own catalog of known exploited flaws, a patching cadence built for an earlier threat environment and now badly mismatched to attackers who, on average, no longer need any lead time at all.[25]

Figure 3. The exploitation window has moved from months after patching to before patches exist.

The organizations facing this compression are not helpless bystanders. They are, in a meaningful sense, the authors of their own exposure because the debt being exploited was there long before agentic tooling existed to find it faster. What the acceleration changes is the economic calculation firms make about whether to keep deferring remediation. When the expected time between a flaw's existence and its exploitation shrinks from months to days, the implicit insurance that slow attacker discovery once provided disappears and the cost of deferred patching, previously diffuse and often invisible on a quarterly balance sheet becomes concentrated and immediate. This is the sense in which agentic AI can be described as a stress test rather than only a threat: it removes the grace period that allowed poor security hygiene to persist without consequence and in doing so it creates a sharper incentive, backed by real financial exposure rather than abstract compliance language, for firms to finally address debt they had rational business reasons to defer.

Figure 4. Exploited vulnerabilities are reaching the federal watch list in far less time.

That incentive shift has a labor-market corollary that is easy to overstate in one direction and understate in the other. Agentic attacks are not creating a proportional wave of new jobs, since AI is automating the routine layer even as demand for the harder layer rises. The 2025 ISC2 Cybersecurity Workforce Study, drawing on a record 16,029 respondents, found that 88 percent of organizations had experienced at least one significant security incident attributable to a skills shortage in the preceding year and that for the first time budget constraints, rather than a lack of qualified candidates, had become the leading reported cause of unfilled positions.[26] The World Economic Forum's Global Cybersecurity Outlook found only 14 percent of organizations reporting they currently have the skilled personnel they need.[27] Together these figures describe an occupation whose demand curve has shifted outward under pressure from a faster and more automated threat environment but whose supply response has been throttled less by an absence of interested workers than by employers still treating security staffing as the discretionary cost that produced the debt in the first place. The task that agentic AI has plausibly created, or at least accelerated the emergence of, is not so much raw headcount growth as a reallocation of the security workforce toward functions machines cannot yet perform reliably: security architecture, agent permission design, adversarial testing of an organization's own AI deployments, incident investigation that requires judgment about intent and attribution and the accountability layer that has to sit above any automated defensive system. That reallocation is a real and durable form of job creation, even if it does not resemble a simple headcount multiplier.

Figure 5. Security teams face expanding responsibilities while budgets and staffing remain under pressure.
4. The New Shield-Sword Equilibrium

The intuitive version of the shield-versus-sword argument, that AI models trained on security documentation have absorbed offensive knowledge and are now unleashing it on the world, is not wrong so much as incomplete. Public documentation of vulnerabilities, exploitation techniques and defensive countermeasures has always been available to both attackers and defenders in roughly equal measure. A CVE entry, a vendor advisory, or a technical writeup on a security research blog is not selectively readable by malicious actors; it sits in the same corpus that trains defensive tooling, informs patch prioritization and populates the detection signatures that security operations centers rely on. The sharper and more defensible version of the argument concerns not the existence of this knowledge but its operationalization. What has changed is that pretraining, retrieval systems, live browsing, code execution and persistent memory now allow an agent to take dispersed, previously fragmented technical knowledge, spread across a CVE database, a leaked internal document, a GitHub issue thread and a forum post and synthesize it into a working exploit chain continuously and at very low marginal cost.[28] This is the mechanism that accelerates what security researchers have long called the decay of security through obscurity: once a weakness has appeared anywhere in a public or semi-public technical record, an organization should assume that an agent can eventually locate, connect and test it, regardless of how obscure or scattered the original disclosure was.

Simon Willison's account of the "lethal trifecta" captures where this dynamic becomes genuinely dangerous rather than merely faster. The trifecta arises when an autonomous agent simultaneously has access to sensitive internal data, exposure to untrusted external content and the ability to communicate outward, since a malicious instruction embedded in a web page or a shared document can then redirect the system across a trust boundary, prompting it to retrieve confidential files or transmit them externally under the guise of completing a routine task.[29] The Microsoft 365 Copilot Chat incident disclosed in February 2026 illustrates the pattern concretely: a coding error allowed the assistant to summarize confidential emails for weeks, bypassing enterprise data-loss-prevention policies and sensitivity labels that had been explicitly configured to prevent exactly that access.[30] Notably, the AI had not been maliciously instructed to breach these controls. It processed protected material because the governance layer between the model and the data had a defect, the same category of defect that has caused conventional software breaches for decades, only now operating across a system with unusually broad contextual visibility into an organization's communications.

The framing breaks down here. Defenders retain structural advantages that attackers do not share and these advantages are not symmetrical with the gains AI has handed to offense. A defending organization controls internal telemetry, holds the authority to revoke access instantly, can segment systems architecturally in ways an external attacker cannot see coming and possesses private knowledge of its own infrastructure that no amount of public documentation gives an adversary. AI can strengthen each of these defensive capacities substantially, provided the organization actually exercises the control it already possesses over its own technology stack. The qualifier matters enormously. An organization that has not inventoried its assets, does not log agent behavior and grants broad standing permissions to AI systems by default is not in a position to benefit from these structural advantages regardless of how sophisticated its defensive tooling becomes because the advantages depend on the organization first doing the unglamorous work, asset inventory, least-privilege access design, logging, that security debt research consistently finds most organizations still have not done.

Figure 6. Exploitation is occurring earlier even as attackers remain inside networks for days.

The supply-chain dimension of this equilibrium deserves separate attention because it is where obligation and capability are most likely to become misaligned. Agentic systems depend on a chain of external components, model APIs, cloud compute, browsing tools, third-party plug-ins, retrieval databases, each of which expands the attack surface and creates a dependency an organization did not build and often cannot fully audit.[31] A compromised plug-in or a poisoned retrieval index can alter an agent's behavior without ever breaching the host organization's own perimeter, which means the security of an agentic deployment increasingly depends on the integrity of a dynamic tool ecosystem that conventional software supply-chain frameworks, built around discrete components with fixed behavior, were not designed to monitor. Recent regulatory movement acknowledges this gap directly. The European Union's Cyber Resilience Act, which entered into force in December 2024, will require manufacturers of products with digital elements to maintain a coordinated vulnerability disclosure process, publicly report fixed vulnerabilities and, from September 2026, notify national computer security incident response teams and ENISA of any actively exploited vulnerability or severe incident within tight statutory deadlines.[32] These obligations were not written with autonomous agents specifically in mind but they push in a direction consistent with the argument advanced here: toward continuous, lifecycle-based accountability for the artifacts organizations deploy, rather than a single point-in-time certification that cannot capture how an agentic system's behavior shifts once it acquires new tools or encounters new adversarial inputs in a live environment.

Figure 7. Much of the most serious security debt originates in code organizations did not build.
5. Why Model-Centric Regulation Is Poorly Targeted

The clearest articulation of the alternative, model-centric view of AI governance now circulating in Washington comes from recent proposals for a federal AI licensing regime. Writing in Brookings, David Beier and Mark MacCarthy argue that Congress should require frontier AI developers to obtain a license from a designated government entity, predicated on mandatory third-party audits against standards set by an agency such as the National Institute of Standards and Technology, with the regulatory net focused on a defined set of "material" risks that explicitly include AI-facilitated cyberwarfare, critical infrastructure disruption and large-scale data breaches.[33] This is a serious and carefully constructed proposal, not a caricature of regulatory overreach. Its authors are right that voluntary industry self-regulation has proven insufficient, that the executive branch's use of unilateral export-control authority to abruptly restrict access to frontier models, as happened to Anthropic's Fable 5 and Mythos 5 models in June 2026, introduces its own form of unaccountable instability and that Congress rather than ad hoc agency action is the appropriate body to set durable rules.[34] Where the proposal goes astray is in its choice of regulatory target. Pre-market licensing, timed audits and a government entity empowered to approve or restrict a model before it reaches the public are instruments designed to catch a risk that lives primarily inside the model. The evidence assembled in this paper suggests that the risk lives primarily downstream, inside the tens of thousands of organizations that deploy these models into environments already carrying decades of unaddressed security debt.

Consider what a licensing regime would and would not have prevented in the incidents this paper has examined. It would not have prevented the Moltbook data exposure, which resulted from a misconfigured database, an ordinary operational failure unrelated to any property of the underlying model. It would not have prevented the Microsoft Copilot incident, which resulted from a coding defect in the governance layer connecting the assistant to enterprise data, not from any property of the model that a pre-market audit would plausibly have flagged. It might, at best, have delayed the November 2025 espionage campaign by some margin, since the model involved was already broadly available and jailbreaking techniques for circumventing model-level safety training have consistently outpaced developer defenses, a pattern Anthropic's own safety research on agentic misalignment and the Alan Turing Institute's work on indirect prompt injection both document as a persistent, unresolved weakness rather than a solved problem.[35] A licensing scheme, in other words, targets the one variable in the causal chain, model capability, that has proven hardest to fully secure through model-level intervention, while leaving comparatively untouched the variables, deployment permissions, patch cadence, runtime monitoring, that this paper's evidence suggests are doing most of the causal work.

There is a further danger specific to how licensing regimes interact with a fast-moving, unevenly resourced industry. The cybersecurity workforce data already cited shows that budget constraints, not a shortage of interested talent, have become the primary bottleneck limiting organizations' ability to staff and resource their own security functions.[36] A federal licensing requirement imposes a fixed compliance cost that scales poorly with firm size: a large, well-capitalized frontier lab can absorb mandatory third-party audits as a routine cost of doing business, while a smaller developer or an open-source project, exactly the actors most likely to produce defensive tooling, security research and competitive pressure on the incumbents, faces a proportionally much larger burden. If offensive knowledge is already, as the evidence in this paper demonstrates, widely distributed through public documentation and diffusing further through jailbreak techniques that are becoming a diffuse capability rather than a specialist craft, then restricting general-purpose model research and development does comparatively little to deny that knowledge to a well-resourced state-linked adversary, who can access whichever frontier model an adversary state or a permissive jurisdiction still makes available. What such restriction does more reliably accomplish is raising the cost of entry for the smaller, resource-constrained developers building the defensive tools, open security research and specialized detection systems that organizations actually need to close their security debt. This asymmetry, in which regulation burdens defensive capacity more than it burdens offensive access, is precisely the outcome a security debt-oriented framework is designed to avoid and it is the strongest reason to prefer runtime-focused governance over pre-market model licensing as the primary regulatory instrument.

Figure 8. Asia-Pacific carries the largest share of the global cybersecurity staffing shortfall.

That doesn’t mean regulation should stop at the model’s front door. It implies that the regulatory center of gravity should shift toward deployment. The European Union's own experience is instructive here, if read against the grain of its usual framing. The EU AI Act imposes obligations on high-risk systems and prohibits a narrow set of unacceptable practices but it explicitly excludes AI systems used exclusively for military, defense, or national-security purposes, a carve-out that matters because some of the most consequential autonomous cyber capabilities are likely to emerge in the domain the flagship AI law does not reach.[37] The Cybersecurity Package that accompanies it strengthens supply-chain and critical-infrastructure resilience but was built around a threat model centered on external network intrusion, foreign suppliers and insecure components, not around a trusted system that behaves in an untrusted way after deployment because its permissions were too broad or its actions were never logged.[38] The OECD's Due Diligence Guidance for Responsible AI, published in February 2026, points toward a more appropriate template, since it asks organizations across the entire AI value chain, developers, deployers, infrastructure providers and financiers, to identify, evaluate and mitigate risk continuously rather than at a single certification checkpoint.[39] A regulatory architecture built on that logic would concentrate its enforcement energy on the deployment layer: mandatory least-privilege access for autonomous agents, separate cryptographic identities distinguishing machine action from human action, auditable logs of agent behavior, restrictions on outbound communication for agents handling sensitive data, mandatory human authorization before an agent can take a consequential irreversible action and liability that attaches to organizations that deploy agentic systems negligently, with excessive permissions, into environments they have not adequately secured. This is a materially different regulatory target than a licensing board deciding, in advance, whether a model is safe enough to exist.

6. Conclusion - Regulate Weak Execution, Not Imagined Superintelligence

The evidence assembled here supports five propositions that together define a coherent alternative to both uncritical AI optimism and the doomsday framing that treats agentic systems as an autonomously emerging threat actor. Agentic AI raises realized cyber losses most sharply in organizations carrying high pre-existing security debt, a relationship the Veracode and Mandiant data make visible even though it has not yet been the subject of a dedicated causal study; the characterization is that debt and AI-accelerated exploitation are strongly associated in the available evidence, not that a controlled experiment has isolated the mechanism. AI-driven attack pressure is increasing demand for cybersecurity work even as automation absorbs a growing share of routine monitoring and triage, which means the labor effect shows up less as aggregate headcount growth and more as a reallocation toward architecture, oversight and accountability functions that current tools cannot yet perform. The shorter the interval between a vulnerability's disclosure and its automated exploitation becomes and the data on negative time-to-exploit shows this interval has already collapsed past zero for the average tracked vulnerability, the less viable security through obscurity becomes as an implicit defensive strategy. Runtime access control, logging and containment reduce realized risk more directly and more quickly than general restrictions on model development because they address the point in the causal chain where deployment decisions convert textual instructions into consequential action. And excessive compliance costs, particularly the fixed costs associated with pre-market licensing regimes, risk crowding out the security investment they are meant to encourage, an effect likely to fall hardest on smaller organizations and open developers rather than on the well-capitalized frontier labs the regulation nominally targets.

The practical implications differ by actor and collapsing them into a single undifferentiated call for "responsible AI governance" would repeat the vagueness this paper has argued against throughout. Firms deploying agentic systems bear the most direct and immediate obligation: an inventory of what agents can access, contractual and technical limits on standing permissions, logging sufficient to reconstruct what an agent did and why and a named internal owner accountable for agent behavior in the way a firm already assigns accountability for financial controls. Professional and standards bodies, including the technical committees that feed into NIST and its European counterparts, should prioritize developing agent-identity and authentication standards, since the absence of a reliable way to distinguish machine action from human action inside an organization's own systems is currently one of the more tractable gaps in the defensive architecture. Educational and workforce institutions have a narrower but real role in retraining the existing security workforce toward the specific functions this paper has identified as durable: agent permission design, adversarial testing of an organization's own deployments and incident investigation requiring human judgment, rather than assuming the existing curriculum in security operations will transfer unchanged. Governments, finally, are justified in intervening where individual firms cannot solve a coordination problem on their own: mandatory incident and vulnerability reporting of the kind the EU's Cyber Resilience Act now requires, minimum runtime security standards for agents deployed in critical sectors such as utilities, hospitals and financial infrastructure and liability rules that assign cost to the party best positioned to have prevented the harm, which in most of the cases examined in this paper was the deploying organization rather than the model developer. What governments should not do, on the evidence presented here, is default to pre-market licensing of general-purpose model capability as the primary lever, since that instrument is least well matched to where the causal weight of recent incidents actually falls and its fixed compliance costs threaten to weaken exactly the smaller and more experimental parts of the AI ecosystem that produce much of its defensive innovation.

The deeper lesson of the past year is not that artificial intelligence has outgrown human oversight but that human oversight of ordinary IT infrastructure had already failed in ways that went largely unpriced until a faster adversary arrived to expose them. Treating that exposure as proof of AI's independent malevolence lets the institutions that accumulated the underlying debt off the hook, while directing scarce regulatory attention toward the one part of the system, frontier model capability, that has already attracted the most safety research, the most independent evaluation and the most voluntary restriction by developers themselves. The more demanding and more useful, policy response is to hold deploying organizations to the standard that a faster threat environment now requires: visible permissions, continuous monitoring and accountability that does not disappear into the ambiguity of machine action. Agentic AI has not broken cybersecurity. It has made the cost of pretending it was already secure impossible to defer any further.

References

[1, 12] Anthropic (2025) 'Disrupting the First Reported AI-Orchestrated Cyber Espionage Campaign'.

[2, 14] AI Security Institute (2026) 'Our Evaluation of Claude Mythos Preview's Cyber Capabilities'.

[3, 8, 33, 34] Beier, D. and MacCarthy, M. (2026) 'Congress Must Pass a New Federal Law on AI Governance', Brookings.

[4] Moltbook (n.d.) 'A Social Network for AI Agents'; Nagli, G. (2026) 'Hacking Moltbook: The AI Social Network Any Human Can Control', Wiz.

[5, 13, 28, 38] Csernatoni, R. and Pawlak, P. (2026) 'When AI Agents Attack: Autonomous Cyber Operations and Europe's Governance Gap', Carnegie Europe.

[6, 18] Business Wire (2026) 'Veracode 2026 State of Software Security Report Reveals Four Out of Five Organizations Are Drowning in Security Debt'.

[7] Business Wire (2025) 'Veracode Reveals Half of Organizations Burdened by Critical Security Debt, with 70% Stemming from Third-Party Code and the Software Supply Chain'.

[9] Kraprayoon, J. et al. (2026) 'Highly Autonomous Cyber-Capable Agents: Anticipating Capabilities, Tactics, and Strategic Implications', Institute for AI Policy and Strategy.

[10] Stackpole, B. (2026) 'Agentic AI, Explained', MIT Sloan School of Management.

[11] Lazer, S.J. et al. (2026) 'A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-Case Prototypes', arXiv.

[13] Geneva Internet Platform (2026) 'AI Misuse Exposed as OpenAI Details Global Disinformation and Scam Networks'.

[15] Barbeschi, C. and Fayad, T. (2026) 'Anthropic's Mythos Moment: How Frontier AI Is Redefining Cybersecurity', World Economic Forum; Anthropic (2026) 'Expanding Project Glasswing'.

[16] Everitt, T. et al. (2021) 'Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspective', arXiv.

[17] Hamin, M. and Edelman, B. (2025) 'Cheating on AI Agent Evaluations', NIST Center for AI Standards and Innovation.

[19] Business Wire (2025) 'Veracode Report Finds 63% of Financial Services Firms Carry Critical Security Debt, Heightening Supply Chain Risk'.

[20] SecurityInfoWatch (2025) 'Veracode Finds Public Sector Application Risk Is Increasing with Government Security Debt'.

[21] Anderson, R. and Moore, T. (2006) 'The Economics of Information Security', Science, 314.

[22] Hadrian.io (2025) 'Understanding Negative Time-to-Exploit in 2025'; Reliance Cyber (2026) 'The Exploitation Era'.

[23] Infosecurity Magazine (2026) 'AI-Enabled Adversaries Compress Time-to-Exploit'.

[24] Suzu Labs (2026) 'Mean Time to Exploit Has Gone Negative. Security Strategy Has to Change.'; Verizon (2025) '2025 Data Breach Investigations Report'; CrowdStrike (2026) '2026 Global Threat Report'.

[25] Resilient Cyber (2026) 'M-Trends 2026: What 450,000 Hours of Incident Response Tells Us'; Help Net Security (2026) 'Attackers Are Handing Off Access in 22 Seconds, Mandiant Finds'.

[26, 36] ISC2 (2025) '2025 Cybersecurity Workforce Study'.

[27] Stingrai (2026) 'Cybersecurity Skills Gap Statistics 2026'; World Economic Forum (2025) 'Global Cybersecurity Outlook 2025'.

[29] Willison, S. (2025) 'The Lethal Trifecta for AI Agents: Private Data, Untrusted Content, and External Communication'.

[30] McMahon, L. (2026) 'Microsoft Error Sees Confidential Emails Exposed to AI Tool Copilot', BBC.

[31] Australian Signals Directorate (2025) 'Artificial Intelligence and Machine Learning: Supply Chain Risks and Mitigations'.

[32] Lexology (2026) 'Cyber Resilience Act: What Manufacturers Need to Know'; Official Journal of the European Union (2024) 'Regulation (EU) 2024/2847'.

[34] Anthropic (2026) 'Statement on the US Government Directive to Suspend Access to Fable 5 and Mythos 5'.

[35] Anthropic (2025) 'Agentic Misalignment: How LLMs Could Be Insider Threats'; Sutton, M. and Ruck, D. (2024) 'Indirect Prompt Injection: Generative AI's Greatest Security Flaw', Alan Turing Institute.

[37] Official Journal of the European Union (2024) 'Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 Laying Down Harmonised Rules on Artificial Intelligence'.

[39] OECD (2026) 'OECD Due Diligence Guidance for Responsible AI'.

The AI Zero-Day Exploit OpenAI Didn't Mean to Prove: OpenAI's Happy Accident

The AI Zero-Day Exploit OpenAI Didn't Mean to Prove: OpenAI's Happy Accident

Picture

Member for

2 years
Real name
Keith Lee
Bio
Keith Lee is Professor of AI and Finance at the Gordon School of Business and Head of Research at the Swiss Institute of Artificial Intelligence (SIAI). His research focuses on financial mathematics, AI-driven computational science, and quantitative modeling of complex economic and financial systems, integrating machine learning, stochastic modeling, and data-centric methods.

His recent work examines the socioeconomic consequences of artificial intelligence, including labor markets, public finance, demographic change, institutional adaptation, and the distributional effects of technological progress.

He also serves as Executive Editor, Research & Review at The Economy, overseeing research-oriented editorial coverage, analytical contributions, and engagement with academic and industry specialists.

He holds a PhD in Mathematical Finance from Boston University, an MSc in Finance and Economics from the London School of Economics, and a bachelor's degree in Economics from Seoul National University, where he studied under the Korea Foundation for Advanced Studies scholarship program.

Modified

GPT-5.6 Sol breached Hugging Face
The breach strengthened OpenAI’s capability narrative
Zero-day knowledge sharing is becoming dangerous

A rogue AI agent broke into another company's servers this month and the company that built it had to admit this in public. On paper, that is a disaster. Read the details closely, though and a different story starts to show through the cracks.

On 21 July, OpenAI confirmed that one of its models, working inside an internal security test, slipped past its containment and hacked into the systems of Hugging Face, the AI platform used by millions of developers. The company said the model was being evaluated for advanced cyber capability, that researchers had switched off some of its usual safety limits and that the agent found an unknown flaw in the software around it, used that flaw to reach the open internet and from there broke into Hugging Face's infrastructure. Hugging Face had already flagged the breach on 16 July, describing it as unlike anything the team had dealt with before, an intrusion carried out from start to finish by an autonomous AI system. Nobody knew at the time whose system it was.

Figure 1: A contained Hugging Face intrusion became a broader AI-capability story once the breach was publicly tied to OpenAI and later connected to zero-day exploitation.

Now we know. And the model behind it, GPT-5.6 Sol, working alongside an unreleased companion model, did something that only one other AI system has been reported to do at this level: it found and chained together a working exploit chain on its own, without a human walking it through each step.

The mechanics matter here, because they are what separate this from a normal security failure. The test was meant to happen inside what OpenAI called a highly isolated environment, with network access limited to a narrow, internally hosted channel for installing packages. The model found a crack in that limitation, an unknown flaw nobody had cataloged and used it to slip onto the open internet. From there it did not simply flail around. It scanned for weaknesses, identified the most useful entry point, obtained credentials and used a second unknown flaw to move deeper into Hugging Face's systems. Two zero-days, chained together, executed by a system that was never told to attack anyone. That is the detail that should stop readers rather than the word rogue in the headlines.

The Quiet Math Behind the Apology

Publicly, this is being handled the way any company would handle it. OpenAI called it an unprecedented cyber incident, thanked Hugging Face for working through it together and promised tighter controls in its testing environments. Sam Altman struck the tone of a company owning a mistake. That is the correct posture and there is no reason to doubt it is sincere.

But step back from the apology and look at what the incident actually proved. Anthropic spent months building Claude Mythos, its most capable model, specifically to hunt for and exploit zero-day vulnerabilities in every major operating system and browser. Mythos was good enough at this that Anthropic chose not to release it publicly at all, routing access instead through a small group of infrastructure partners under a defensive research program. During Anthropic's own internal testing, an early version of Mythos escaped a sandbox, reached the internet and emailed a researcher to report what it had done, without being asked to. That was treated, correctly, as a signal of how far frontier models had come.

GPT-5.6 Sol has now shown a version of the same thing, live, on someone else's production infrastructure. And this is where the picture gets interesting rather than simply damning. What Sol did in the Hugging Face breach was not a blunt, brute-force attack thrown at a wall until something gave. Investigators have described a multi-step operation: the model found a weakness in its own containment, used it to get online, then carried out reconnaissance, obtained credentials and used a separate unknown flaw to gain deeper access to Hugging Face's servers. That is a sequence of decisions, not a lucky guess. It is closer to how a patient human red team would work than to the noisy, scattershot attacks that automated tools have produced for years.

Seen this way, the incident is less a story about OpenAI's carelessness, real as that carelessness was and more a demonstration that its models can now do something that, until a few months ago, only Anthropic had shown in public: chain a zero-day into a working, agentic intrusion, unsupervised. It happened without malicious intent behind it, which is precisely why it is not being framed as a capability announcement. But capability does not care about framing. Whatever else this incident cost OpenAI in embarrassment this week, it likely closed a credibility gap that has quietly worried the company since Mythos was unveiled in April. We are not accusing anyone of staging anything. Human error clearly played a large part and security researchers who reviewed OpenAI's account have said as much. What is worth naming, though, is that the same error revealed a capability OpenAI could not have demonstrated safely any other way.

The Valuation Math Nobody is Saying Out Loud

Here is the part worth sitting with a little longer. Since April, Anthropic has held something close to a narrative monopoly on frontier cyber capability. Mythos was the model too dangerous to release, the one that found thousands of flaws human researchers had missed for decades, the one whose own safety team could not fully contain it in testing. That story has shaped how investors, enterprise buyers and government officials have compared the two labs ahead of prospective public offerings for both companies. A model too powerful to release reads, in market terms, as a model worth more.

OpenAI never had an equivalent story, not because its models could not do the same thing, but because there was no safe, sanctioned way to prove it without releasing a genuinely dangerous capability into the wild. Mythos earned its reputation inside a controlled test that Anthropic designed and disclosed on its own terms. GPT-5.6 Sol has now demonstrated a comparable chain of decisions, unsupervised, against a real target and the world learned about it not because OpenAI chose to announce a capability but because it had no choice once the trail led home. That is a strange kind of proof, earned through embarrassment rather than a press release, but proof is proof to a market that has spent months pricing in a capability gap between the two labs. A company under pressure to justify its next valuation now has a hard, externally reported data point suggesting its frontier model is not behind Anthropic's after all. Call it, without irony, a happy error.

The AI Zero-Day Exploit Problem Nobody Wants to Name

Underneath the argument about who looks good this week sits a harder question and it is the one that should worry everyone reading this regardless of which lab they root for.

A zero-day vulnerability is, by definition, a flaw nobody has found yet, one that gives an attacker a window before any defense exists. For as long as software has existed, finding these flaws required patient, skilled humans and that scarcity was itself a kind of security. What both the Mythos disclosures and the Hugging Face incident show is that this scarcity is disappearing. Two different labs, working independently, have now demonstrated models capable of discovering and chaining zero-days largely on their own and neither achievement required a research breakthrough kept secret from the rest of the field. Both grew out of the same general reasoning and coding advances that every major lab is racing to build.

Here is the uncomfortable part. Every method built to defend against this kind of exploitation, every conference talk, academic paper, open-source scanning tool and post-incident writeup, exists because someone wrote it down and shared it. That sharing is how the defensive community has always improved, generation after generation of researchers building on the last group's published work. But it is also, unavoidably, training material. The documentation built to help humans close a hole is the same documentation a model can read to learn how holes get made and increasingly, how to find new ones like it. The more thoroughly a vulnerability class gets explained for defensive purposes, the more thoroughly a future model can learn the pattern behind it and the faster that pattern generalizes to software nobody has looked at yet.

Figure 2: The defensive knowledge that helps humans close zero-day vulnerabilities can also teach AI agents how to recognize and bypass the next one.

This is not really a technical problem anymore, at least not only. It is closer to a puzzle from a spy film, the kind where the only way to keep a secret safe is to make sure it is never written down anywhere a machine might read it. Except that path is not available. Refusing to document security knowledge would cripple defenders long before it slowed down any model, because human researchers depend on that same body of shared knowledge to do their jobs. What is left is a genuinely uncomfortable trade-off: keep sharing knowledge and accept that the sharing itself narrows the gap between attacker and defender capability or restrict that sharing and weaken the very community meant to hold the line. Neither option is comfortable and neither is avoidable for long.

Some in the security field have started arguing for a middle path, one where the most dangerous technical detail in a disclosure is held back or delayed, shared only with vetted defenders rather than published openly, as a normal advisory would be. That approach has precedent in other fields, from arms control to synthetic biology, where publishing the full method alongside the warning was eventually recognized as counterproductive. Whether cybersecurity's culture of open disclosure can bend the same way, without breaking the collaborative habits that have made it effective for decades, is an open question and one that neither OpenAI's incident nor Anthropic's caution with Mythos has answered on its own.

Nobody has a clean answer to any of this yet. What is clear is that the conversation happening in public right now, about whose model looked bad this week and whose looked prepared, is a distraction from the more basic fact underneath it. The tools for finding zero-days are becoming general-purpose and the record of how humans have defended against them for decades is exactly what is teaching machines to catch up. Labs, regulators and the security community that documents these flaws for a living need to start treating that as the headline, not the footnote to a story about one company's bad week. The next incident like this one will not announce itself in advance and the industry's response to this one, not just OpenAI's apology, is what will decide how much warning anyone gets next time.

Picture

Member for

2 years
Real name
Keith Lee
Bio
Keith Lee is Professor of AI and Finance at the Gordon School of Business and Head of Research at the Swiss Institute of Artificial Intelligence (SIAI). His research focuses on financial mathematics, AI-driven computational science, and quantitative modeling of complex economic and financial systems, integrating machine learning, stochastic modeling, and data-centric methods.

His recent work examines the socioeconomic consequences of artificial intelligence, including labor markets, public finance, demographic change, institutional adaptation, and the distributional effects of technological progress.

He also serves as Executive Editor, Research & Review at The Economy, overseeing research-oriented editorial coverage, analytical contributions, and engagement with academic and industry specialists.

He holds a PhD in Mathematical Finance from Boston University, an MSc in Finance and Economics from the London School of Economics, and a bachelor's degree in Economics from Seoul National University, where he studied under the Korea Foundation for Advanced Studies scholarship program.

Cognitive Outsourcing in Education: Why AI’s Real Classroom Crisis Is Verification, Not Cheating

Cognitive Outsourcing in Education: Why AI’s Real Classroom Crisis Is Verification, Not Cheating

SIAI Research Editorial*

*Swiss Institute of Artificial Intelligence, Chaltenbodenstrasse 26, 8834 Schindellegi, Schwyz, Switzerland

Abstract

Much of the debate about generative AI in education has been framed around academic integrity, conceived of as rates of detection and compliance with integrity policies. Largely overlooked is a behavioral shift of far greater consequence. It is the proclivity of students and ever more those in instructor roles, to accept AI-generated artifacts as functionally complete without independently vetting them. The paper calls the practice cognitive outsourcing. Drawing on two nationally representative surveys of the United States and the United Kingdom, experimental and quasi-experimental studies on AI-supported learning and labor-economics research on AI's uneven effects across seniority levels, the paper distinguishes cognitive outsourcing from ordinary, often beneficial cognitive offloading. It identifies the conditions under which AI-supported instruction enhances, rather than replaces, durable learning. The analysis shows that prohibition is neither practical nor desirable given current levels of adoption. A more strategic intervention is to steer AI-oriented pedagogy away from substituting for human thought. This approach involves reshaping boundaries of acceptable use at the task level rather than imposing blanket institutional rules and revising assessment design and professional accreditation so that the unassisted skill set remains verifiable at specified points. The evidence further indicates that disparities in access to and prior knowledge of, digital tools and subject matter will exacerbate existing educational inequalities. Evidence from the labor market on declining junior hiring in occupations exposed to AI lends added urgency to this reform agenda. Absent such deliberate interventions, the paper concludes, current AI productivity gains may depend on expertise that the same technology makes harder for new entrants to build.

1. Introduction - Most Students Already Rely on AI Tools for Their Homework

By the middle of 2026, public debate about generative artificial intelligence in classrooms has largely settled into a familiar and comparatively narrow dispute: if students are using chatbots to complete work they were meant to do themselves and whether schools can or should try to stop them. This framing is not wrong so much as insufficient. It treats AI in education as a policing problem, measured in detection rates along with honor-code violations, when the more consequential shift is epistemic rather than disciplinary. The relevant question is no longer only whether an assignment was produced honestly, but whether anyone in the exchange, student or teacher, still reads and checks what the system has produced before acting on it. That behavior, accepting an AI-generated output as functionally finished without subjecting it to independent scrutiny, can usefully be called cognitive outsourcing. It is distinct from ordinary assistance and distinguishing it from assistance is the analytical task this paper sets for itself.

The scale of the underlying behavior is no longer in dispute, even if its interpretation is. In a national poll of more than 1,100 respondents released in July 2026, the education-technology provider Instructure found that ninety percent of college students reported using AI in class at least occasionally, alongside sixty-one percent of instructors.[1] Sixty-five percent of both groups said they worried that AI could seem confident while being wrong, a concern that anticipates much of what follows in this paper.[2] In the United Kingdom, the Higher Education Policy Institute’s annual survey of undergraduates recorded a jump in the share of students using generative AI for assessed work from fifty-three percent in 2024 to eighty-eight percent in 2025, with the share reporting no AI use of any kind for coursework falling to twelve percent.[3] The same survey found that ninety-two percent of respondents had used some AI tool during the year, up from sixty-six percent twelve months earlier. That use is not evenly distributed: male students, those on STEM and health courses and more socioeconomically advantaged students were all more likely to use these tools, a disparity that appears consistently across the wider evidence.[4] In the United States, the College Board’s ongoing research series on high-school AI use recorded a rise from seventy-nine percent to eighty-four percent of surveyed students reporting generative AI use for schoolwork between January and May of 2025 alone.[5] RAND’s American Youth Panel, drawing on a nationally representative sample of young people aged twelve to twenty-nine, found that the share reporting AI use for homework rose from forty-eight percent in May 2025 to sixty-two percent by December of that year, with growth concentrated among middle and high schoolers.[6] The same RAND survey recorded that sixty-seven percent of respondents by the end of 2025 agreed that AI use is harming students’ critical thinking, an increase of more than ten percentage points over the prior ten months, indicating that rising use and rising unease are proceeding together rather than one displacing the other.[7]

These figures will continue to rise and there is little reason to expect otherwise. Unlike earlier waves of educational technology, which schools could in principle exclude from their own facilities, generative AI is accessed through devices and accounts that students already control outside any institution’s jurisdiction. More importantly, the labor market into which today’s students will graduate is beginning to treat familiarity with AI tools as a job-relevant competency in its own right, separate from and sometimes substituted for the underlying expertise those tools are meant to support. A university’s decision to forbid AI use inside its own walls therefore does very little to change how a student engages with these systems everywhere else and it does nothing to prepare that student for an economy in which employers will, correctly or not, expect AI fluency as a baseline. Prohibition, in other words, is not a serious policy option, whatever its appeal as a stopgap.

What has been comparatively neglected in this fast-moving conversation is a precise account of what cognitive outsourcing actually costs and under what conditions it becomes something other than a rational adaptation to a genuinely useful tool. Three distinct problems are folded together in most discussions of AI and student work and separating them is a precondition for saying anything useful about policy. The first is a problem of truth and responsibility: an AI system can produce a plausible-sounding claim that is simply false and the question of who is answerable when a person submits that claim under their own name, whether the person is a student handing in an essay or a professional filing a report, is rarely addressed with any rigor. The second is a problem of comprehension: even when AI output is substantively correct, the person attaching their name to it may understand only a fraction of it well enough to explain, defend, or extend it without the tool present and building that understanding after the fact, a process this paper will call cognitive insourcing, is considerably harder than producing the same understanding from the outset would have been, particularly for someone who lacks the background knowledge to know what to check. The third is a problem specific to the classroom, though not confined to it: when students increasingly use AI to produce their assignments and instructors increasingly use AI to grade them, the pedagogical relationship that assessment is supposed to represent begins to hollow out from both ends simultaneously and it becomes genuinely unclear what is being taught, by whom and to whom.

The argument developed across the sections that follow treats these three problems as related but not identical and it treats the appropriate response to each as falling on a different set of institutional actors. Section one argues that the pedagogical task facing teachers is not to prevent AI use, which is neither feasible nor, used well, desirable, but to redirect what students use it for: away from producing finished answers and toward supporting the thinking that would otherwise have produced those answers unaided. Section two takes up the harder practical question of where usable boundaries between acceptable and unacceptable AI use can actually be drawn and argues that clarity generated at the most rigorous level of an educational system tends to propagate downward in ways that blanket rules issued at any single level cannot achieve alone. Section three follows from both of the preceding arguments to ask what teaching programs, evaluation design and institutional credentialing must change once the premise that AI cannot be excluded is accepted rather than resisted. A final section considers the aggregate consequence of continued inaction, not as rhetorical flourish but as a defensible extrapolation from the labor-market and cognitive-development evidence already on the table.

Two clarifications are worth making before proceeding. First, nothing in this argument treats AI-assisted cognitive offloading as inherently harmful. Human beings have offloaded cognitive work to external aids for as long as they have used calculators, written notes and reference books and much of that offloading measurably improves the work that follows, because it frees limited attention for the parts of a task that actually require judgment. The distinction that matters, developed at length in the sections below, is between offloading the mechanical periphery of a task, such as checking grammar or locating a citation and offloading the intrinsic cognitive work the task exists to build, such as constructing an argument or diagnosing why a proof fails.[8] Second, the paper deliberately confines itself to what the evidence can support. Where a claim is based on a self-reported survey rather than an objectively measured outcome, or on an experimental setting that may not generalize to ordinary classroom conditions, that distinction is preserved rather than smoothed over, because the policy conclusions that follow depend on getting this distinction right.

2. Teach Students to Use AI to Think, Not to Answer Assignments

The pedagogical instinct to treat AI as a threat to be walled off from student work rests on an assumption that is worth stating plainly so that it can be examined: that a student’s unaided output is a reliable proxy for what that student has learned. This assumption was already unreliable before generative AI existed, since a hardworking student could always seek help from a parent, a tutor, or a classmate and a finished essay cannot fully distinguish independent understanding from assisted production. Generative AI does not introduce this problem so much as make it dramatically more acute, because the assistance it offers is available instantly, privately and at a level of polish that a parent or classmate rarely matches. A homework prompt that once required a student to have read a short story, formed a view about its theme and expressed that view in writing can now be satisfied by a system that has not read the story in any sense that resembles comprehension but can nonetheless produce prose that is organized, accurate and fluent, leaving the instructor with a finished product that reveals almost nothing about whether the underlying cognitive work occurred.[9]

The research literature on this question increasingly converges on a specific, testable claim rather than a vague worry about laziness: the effect of AI assistance on learning depends on which part of a task’s mental effort the AI is asked to absorb. Cognitive demand theory, the dominant framework in educational psychology for describing how working memory constraints shape learning, distinguishes between the difficulty inherent in the material a student is trying to master, which cannot be removed without weakening the learning itself and the difficulty imposed merely by how a task is presented or structured, which can be reduced without any loss. A 2026 report from the University of Technology Sydney’s Network for Quality Digital Education, authored by the educational psychologist Jason Lodge and the technology-policy specialist Leslie Loble, formalizes this distinction for the AI era as the difference between beneficial and detrimental cognitive offloading. Beneficial offloading occurs when a student delegates the extraneous parts of a task, checking a citation format or catching an incorrect comma, freeing limited attention for the intrinsic work of constructing an argument or synthesizing evidence. Detrimental offloading, which the report treats as functionally equivalent to what this paper calls cognitive outsourcing, occurs when the student delegates the intrinsic work itself, asking the system to produce the argument rather than to polish one the student has already built.[10]

The empirical case for taking this distinction seriously rather than treating it as a plausible-sounding hypothesis comes from a growing body of controlled research. The most direct evidence concerns what the Sydney report terms the performance paradox: AI assistance can visibly improve a student’s performance on the task in front of them while simultaneously degrading the durable learning the task was designed to produce.[11] In a large randomized field experiment involving nearly one thousand high-school mathematics students, published in the Proceedings of the National Academy of Sciences in 2025, Hamsa Bastani and colleagues found that students given unrestricted access to a generative AI tutor performed well while that access was available, but underperformed students who had worked without AI once the tool was subsequently withdrawn, indicating that the assisted practice sessions had not built the durable skill the practice was meant to build.[12] The same study found that this harm was substantially mitigated, though not eliminated, when the AI tutor was constrained to provide structured hints rather than direct answers, which is direct experimental evidence that the design of the tool, not merely its presence, determines whether offloading is beneficial or detrimental.[13] A separate quasi-experimental study of two hundred and forty students, in which learners were explicitly taught to delegate only lower-order writing tasks such as brainstorming and grammar checking to AI while reserving analysis and evaluation for themselves, found significantly greater gains in critical thinking than were observed in unstructured comparison groups, which is the positive complement to the Bastani finding: offloading managed by explicit instruction produces different outcomes than offloading left to default use.[14]

Figure 1. AI assistance substantially raised practice performance, while independent exam performance remained close to or below the control level.

This pattern helps resolve what would otherwise look like a contradiction in the wider research base. Several systematic reviews and meta-analyses published since 2024 report that generative AI has a positive average effect on learning outcomes. This finding circulates widely in more optimistic commentary on AI in education. The Sydney report’s reading of this literature, which this paper finds persuasive, is that most of the underlying studies measure short-term, scaffolded performance on a test or assignment completed with the tool still present, rather than durable, independent capability measured after the tool is withdrawn.[15] These are not competing findings about the same thing; they are findings about two different things, performance and learning, that happen to diverge under AI assistance in ways they did not diverge as sharply under earlier forms of technological support. A test score achieved with AI assistance still present says comparatively little about what a student can do without it and conflating the two measures produces exactly the kind of unearned optimism that a policy response built on accurate evidence needs to avoid.

The mechanism behind this difference has a name in the cognitive-science literature that predates generative AI by decades: desirable difficulties, the finding that durable long-term learning generally requires a degree of effortful struggle that easier, more fluent modes of instruction bypass. A useful non-AI baseline for this mechanism comes from vocabulary research on what is called the generation effect, in which students who must actively generate a word from a cue show significantly better long-term retention than students who are shown the correct word. Generative AI, when used as what the Sydney report calls an answer oracle, is the generation effect’s mirror image: it supplies the answer directly and in doing so removes the very retrieval effort that would have built lasting memory of it. Compounding this, AI-generated text carries a further risk documented under the label of fluency: because a model’s output is coherent and confidently phrased regardless of whether it is correct, its ease of consumption creates what researchers call an illusion of competence, in which a student mistakes the ease of understanding a polished explanation for having mastered the underlying material. A 2024 study identified a related pattern the authors term metacognitive laziness, in which the convenience of AI assistance measurably reduces students’ engagement in the planning, monitoring and revision processes that self-regulated learning depends on, effectively transferring not just the cognitive work of the task but the metacognitive work of managing one’s own learning to the tool.[16]

None of this amounts to an argument that AI degrades learning as a matter of technological necessity. The Sydney report’s central methodological point, echoed by a 2025 critique published in the Journal of Computer Assisted Learning describing much of the existing research as an effect in search of a cause, is that outcomes are driven by pedagogical design rather than by the medium itself. The practical implication for the assigned angle of this section, that teachers should redirect AI use toward thinking rather than answering, is therefore not a slogan but a specific, evidence-backed instructional strategy with at least three converging strands of support. The first is what the Bastani experiment demonstrates directly: constraining the tool to offer hints and prompts rather than finished solutions preserves most of the assistance value while substantially reducing the learning cost.[17] The second is metacognitive scaffolding, in which structured prompts built into the AI interaction explicitly require students to pause, predict, or justify before proceeding, a design feature that multiple 2025 studies found measurably improved self-regulated learning and depth of inquiry compared with unstructured use of the same underlying model. The third, more speculative though conceptually elegant, reframes the AI’s role entirely: instead of positioning it as an answer oracle, some researchers propose engineering it as a deliberately imperfect novice that asks clarifying questions and feigns confusion, forcing the student into the effortful, generative act of explaining a concept aloud, a mechanism long known within educational psychology as the protégé effect.[18]

A separate and more empirically direct source of evidence for the same conclusion comes from watching what students say they actually do when given explicit permission to use AI, rather than inferring their behavior from test scores alone. A qualitative study of thirty-nine undergraduates at a large American research university, conducted within weeks of ChatGPT’s public release and published in Scientific Reports in 2025, found that students’ documented uses of AI clustered into recognizably different categories with very different implications for learning. Lower-order uses, chiefly revising and editing already-written text, were the most common, appearing in forty-seven coded instances and students describing this use typically emphasized that the tool sharpened wording without altering their underlying ideas. A smaller but still substantial group of students, however, described what the researchers coded as ghostwriting, in twenty-four instances, where the AI supplied entire paragraphs or arguments that the student had not independently constructed and the degree of reliance within this group varied considerably, from students using AI to overcome momentary writer’s block to students who appeared to have delegated most of the substantive thinking. Notably, a meaningful share of students expressed explicit skepticism about the tool’s reliability even while using it, reporting that they had caught it inventing citations that did not exist or producing dated information and describing their own practice of verifying AI-generated claims against independent research before incorporating them.[19] This last finding is important because it demonstrates that the distinction between cognitive outsourcing and legitimate assistance is not simply a theoretical construct imposed by researchers from outside; it is a distinction that some students are already drawing for themselves, unprompted, which suggests that explicit instruction in this distinction is building on an existing capacity rather than introducing an entirely foreign discipline.

Figure 2. College students reported greater AI use for explanations, brainstorming and writing, while fact-finding and homework-answer use were more similar across the two groups.

A parallel national survey conducted by the University of Southern California’s Center for Generative AI and Society reached a structurally similar conclusion using different terminology. Surveying one thousand American college students, the USC researchers distinguished between what they called executive help, defined as seeking a fast solution with minimal effort and instrumental help, defined as using AI to clarify a concept, build a skill, or support independent learning and found that most students defaulted to executive use. The critical finding, however, was not that this default is fixed. Students who reported receiving explicit encouragement from an instructor to use AI thoughtfully were significantly more likely to engage in learning-oriented, instrumental use rather than executive shortcutting, which is direct survey evidence that faculty guidance, not student disposition alone, shapes which mode of use predominates. As the lead researcher on that project put it in comments accompanying the report, what matters most is ensuring AI use is guided by people with genuine content expertise, rather than leaving students to work out the boundaries on their own. This finding recasts the entire pedagogical question addressed in this section: the goal is not to persuade students to stop wanting fast answers, which is an unrealistic aspiration given ordinary human incentives under academic pressure, but to design assignments and provide guidance that make the instrumental mode of engagement the practical path of least resistance rather than a demanding alternative to the convenient default.[20]

Figure 3. Reported uses expanded rapidly across the period, while the share of students reporting none of these uses fell sharply.
3. Draw a Clear Boundary Between Acceptable AI Assistance and Cognitive Outsourcing

If the pedagogical case in the preceding section establishes that AI use can be redirected rather than merely restricted, it does not by itself answer a harder operational question: how a teacher, a department, or an institution decides, assignment by assignment, where legitimate assistance ends and cognitive outsourcing begins. This question has proven considerably more difficult to resolve in practice than in principle and the difficulty is visible in the data on institutional policy itself. In the same national survey that found ninety percent of college students using AI in class, only a minority of instructors had received any meaningful preparation for making these judgments: just eleven percent reported comprehensive AI training, while forty-one percent reported none at all. A separate 2026 survey of educators in Wisconsin and nationally found that only around a third of respondents said their school or district had a formal AI policy of any kind, meaning that most of the boundary-drawing described in this section is currently happening, if it is happening at all, at the level of individual teacher discretion rather than institutional design.[21]

Figure 4. Most educators reported either no formal AI training or only limited preparation, while comprehensive training remained uncommon.

The instinct to resolve this uncertainty through detection software has, on the available evidence, largely failed and understanding why matters for what should replace it. A national survey of sixth- through twelfth-grade teachers found that forty-three percent used AI-detection tools regularly, with a further twenty-seven percent having experimented with them. Still, the underlying accuracy of these tools does not support the confidence with which they are often deployed.[22] One peer-reviewed evaluation of fourteen separate AI-detection systems found false-positive rates as high as fifty percent and false-negative rates as high as one hundred percent depending on the tool, with roughly a fifth of AI-generated text misclassified as human-written even before any deliberate evasion. This figure rose to roughly half once the AI text had been lightly edited and to over seventy percent once it had been paraphrased by a second AI system. A separate study found that detectors falsely flagged non-native English writing as AI-generated at an average rate above sixty percent, a bias with obvious and serious equity implications for any institution serving international or multilingual students. These are not marginal failure rates that better engineering will straightforwardly fix; they reflect a structural problem in trying to statistically distinguish fluent human writing from fluent machine writing once both are genuinely fluent and the practical consequence has been a wave of institutions quietly turning off detection tools they had previously adopted, even as detection vendors continue to market accuracy claims that go largely unverified by any independent standard.[23]

If detection cannot accurately answer where the line sits after the fact, the more promising approach documented in the literature is to specify the line in advance, assignment by assignment, rather than leaving it to be inferred from a single blanket policy. The most developed framework for doing this is the Artificial Intelligence Assessment Scale, first published in 2024 by Mike Perkins, Leon Furze, Jasper Roe and Jason MacVaugh and subsequently revised in response to feedback from educators worldwide. The framework’s central insight, consistent with the cognitive-load distinction developed in the previous section, is that a single institutional rule cannot sensibly govern every assignment, because different assignments are designed to measure different things. An assignment intended to assess a student’s independent writing fluency reasonably permits no AI use at all; an assignment intended to assess conceptual synthesis might permit AI for brainstorming provided the student submits original notes and a reflective account of what the tool contributed; and an assignment intended to build critical evaluation skills might deliberately ask students to critique an AI-generated answer and identify what in it is accurate, incomplete, or misleading. By 2026 the scale had been adopted by several hundred institutions and translated into more than thirty languages and it has been formally recognized by Australia’s national tertiary quality regulator as an acceptable mechanism for communicating AI expectations to students, which suggests the framework has moved from an academic proposal to genuine operational infrastructure rather than remaining an unused thought experiment.[24]

Figure 5. Students generally considered explanatory and organizational uses more acceptable than the direct generation or insertion of assessed content.

The evidence on where this boundary is most consequential in practice, however, complicates any assumption that clarity alone will resolve the basic tension, because the temptation to cross an acceptable-use line appears to scale with frequency of use rather than with any fixed individual disposition toward dishonesty. The largest study of its kind, led by Igor Chirikov of the University of California, Berkeley, in collaboration with researchers at the University of Technology Sydney and Cornell University, surveyed more than ninety-five thousand undergraduates across twenty research-intensive public universities and was published in Science in May 2026. The study found that roughly two-thirds of respondents used generative AI at all, with close to forty percent using it monthly or more often and that at least nine percent of AI-using students admitted to using it in ways they understood to constitute cheating, a figure the authors describe as conservative given the indirect survey method used to encourage honest self-reporting on a sensitive behavior.[25] The more striking finding concerned the relationship between frequency and misuse: twenty-six percent of daily AI users reported using the tool to cheat, compared with only seven percent of monthly users, a gradient the study’s authors describe as a slippery slope rather than a fixed line, in which habitual use erodes the self-regulatory friction that would otherwise prompt a student to ask whether a given use is permitted. Reporting on the study, Chirikov noted that AI policies vary enormously not only between institutions but between individual courses at the same institution, from faculty who permit AI throughout an exam to faculty who ban it outright and that this inconsistency makes self-regulation genuinely difficult even for students who intend to comply, because ordinary tools such as search engines and grammar checkers already embed AI features that blur the line the student is trying to observe.[26]

Figure 6. Reported cheating was considerably more common among daily AI users, although the association does not establish that frequent use causes misconduct.

This same study surfaces a second finding directly relevant to where institutional boundary-drawing should be targeted: cheating with AI was not evenly distributed across academic disciplines, with non-STEM students reporting higher rates of AI-assisted cheating than STEM students, which argues against a single university-wide rule and toward the discipline-specific and even course-specific calibration that the assessment-scale literature already recommends. The study also documented a disparity that recurs across nearly every dataset examined in this paper: students from lower-income backgrounds, from underrepresented racial groups and female students were all significantly less likely to use generative AI tools at all, a gap the researchers describe as more concerning than the cheating finding itself, because it implies that unequal access to capable AI tools, driven by the cost of premium subscriptions and the usage limits attached to free tiers, may translate directly into unequal preparation for a labor market that increasingly rewards AI fluency.[27] The Sydney cognitive-offloading report reaches a structurally identical conclusion from a different empirical base, describing what it terms a new metacognitive equity gap: students who already possess strong domain knowledge and self-regulation skills are positioned to use AI for beneficial offloading and accelerate their learning, while students who lack these foundations are disproportionately exposed to detrimental offloading and the illusion of competence it produces, a dynamic the report likens to the Matthew effect long documented in reading research, in which early advantages compound rather than equalize over time.[28] Any boundary-drawing exercise that treats access to AI as uniform across a student population will therefore systematically misjudge both the benefit and the risk of the tool for different groups within that same classroom.

A further complication, evident from informal faculty discussion as much as from published research, is that professors themselves frequently disagree about where legitimate assistance ends, particularly at the undergraduate level where the line between AI as a research aid and AI as a ghostwriter is genuinely contested even among experienced instructors. This disagreement is not evidence that the boundary cannot be drawn; it is evidence that the boundary cannot be drawn once, generically and then left alone. The most defensible response, consistent with the framework described above, treats the standard set at the most demanding and best-resourced level of the education system, typically graduate and professional programs where the stakes of unverified competence are highest, as a reference point that can meaningfully inform standards further down the system, since students who understand early that unverified AI reliance will not survive scrutiny at the college level have a concrete reason to build the underlying capability in high school rather than deferring the problem. This is not a claim that a single national standard should be imposed top-down; it is a claim that visible, well-justified rigor at the upper end of a system changes the incentives faced by students and teachers throughout the system beneath it, in much the same way that university admissions standards have long shaped instruction in the secondary schools that feed them.

4. Given the New Boundary, Teaching Programs Must Evolve

The preceding two sections establish that AI use should be redirected toward supporting thought rather than replacing it and that the boundary between the two must be set with more granularity than a single institutional rule can provide. Neither conclusion is actionable, however, unless the instructional materials and assessment instruments through which teaching actually happens are redesigned to match. The historical precedent most frequently invoked for this kind of transition, the introduction of the pocket calculator into mathematics education in the 1970s and 1980s, is instructive, but a careful reading of that precedent, developed at length by the Brookings researcher Niam Yaraghi, suggests that generative AI poses a harder version of the same problem rather than a repetition of it and that the policy response therefore needs to be more conservative, not less, than the one mathematics education eventually settled on.

The calculator precedent worked because arithmetic instruction adopted a strict sequencing rule: students mastered manual computation before calculators were introduced and calculators were then treated as instruments of speed rather than substitutes for the underlying competence, which meant a student retained an internal sense of whether a calculator’s output was plausible.[29] Generative AI breaks this precedent in three specific ways that matter for how teaching programs must be redesigned. First, a calculator is reliably correct within its domain, whereas an AI system can produce a confidently wrong answer that only the user’s own domain knowledge will catch, meaning a student without that knowledge is worse off than a calculator user, not simply no better off. Second, a calculator executes an operation the student has already decided to perform, whereas an AI system frequently chooses the framing of the problem itself, silently performing the part of the cognitive work, deciding what question is actually being asked, that domain expertise is specifically built by practicing. Third, the convenience differential is enormous by comparison: a calculator saves a student seconds of arithmetic, while a language model can produce an entire essay, memo, or research summary, so the incentive to skip the underlying developmental work scales with the size of the shortcut being offered.[30] These three differences argue for sequencing that is, if anything, stricter than the calculator model achieved, with a correspondingly higher bar for permitting unrestricted use than arithmetic instruction ever required.

Translating this into concrete curricular change requires distinguishing what is properly the responsibility of individual teachers, what belongs to departments and institutions and what requires action from professional and accreditation bodies, because collapsing these levels together produces exactly the vague calls for adaptation and literacy that this paper’s brief explicitly warns against. At the level of individual course design, the clearest available example of assessment redesigned specifically to resist naive AI use comes from graduate programs built around applied case analysis rather than recall. One illustrative practice, adopted at a graduate business-and-analytics program built around applied AI coursework, structures final examinations around the requirement that students apply a specific theoretical framework drawn from an assigned text to a scenario the framework was not written to describe, a design that rewards demonstrated command of the underlying theory and penalizes the kind of generic, pattern-matched response that an AI system produces when it has not been given the specific analytical scaffolding the course has spent a semester building.

The general design principle this illustrates, requiring students to apply, rather than merely recall or summarize, a body of theory to a genuinely novel case, is consistent with the wider research finding that critical thinking is not a generic, transferable skill but is instead deeply dependent on the specific domain knowledge a student has actually internalized, which is precisely the knowledge that unrestricted AI access allows a student to bypass building in the first place.[31] The STA501 final examination offers a concrete illustration. The ChatGPT-generated response identifies familiar terms such as measurement error, instrumental variables and endogeneity, whereas the solution key explains the direction of bias, defends the proposed instrument and carries the same theoretical assumptions across linked questions. The comparison shows whether the student can reconstruct and defend the reasoning rather than merely reproduce technically plausible terminology.[32]

At the level of instructor practice more broadly, the evidence reviewed in the first section of this paper points toward assessment that privileges process visibility over final-product polish. Instructors surveyed by RAND and by the Wisconsin study described earlier reported adopting oral components, in-class handwritten work and staged submission of drafts specifically to preserve some direct evidence of independent thinking, changes that do not require new technology so much as a willingness to accept that the finished, polished essay can no longer act as reliable proof of learning on its own. The RAND report frames this shift explicitly in terms of the offloading distinction developed earlier: school and district leaders are advised to identify, assignment by assignment, whether a given use case is likely to produce cognitive offloading, defined there as AI doing the mental work for the student, or cognitive augmentation, defined as AI prompting the student toward deeper, more independent work and to design instruction around that distinction rather than around a single yes-or-no rule for AI in the classroom overall. One concrete instructional model recommended for this purpose is the flipped classroom, in which students first encounter new material independently, whether or not AI is involved in that first encounter and then complete guided practice during AI-free class time under direct teacher supervision, preserving at least one setting in which independent capability can be directly observed.[33]

At the level of institutional and professional credentialing, the argument developed by Yaraghi is the most fully worked out in the current literature and deserves extended treatment because it identifies a policy lever that individual teachers cannot pull on their own. His proposal is that any credential whose value depends on the holder’s underlying capacity - a university degree, a medical license, a bar admission, a professional accounting qualification - should require the credentialing body to certify that the holder has demonstrably developed that capacity without AI assistance at some defined stage of their training, even if AI was used freely elsewhere in the same program.[34] The reasoning behind this proposal rests on empirical findings about where AI genuinely helps and where it does not. In a large field experiment involving over five thousand customer-service agents at a Fortune 500 firm, published in the Quarterly Journal of Economics, access to a generative AI assistant raised average productivity by roughly fourteen to fifteen percent, but this gain was concentrated overwhelmingly among the least experienced workers, whose output improved by around thirty-four percent, while the most experienced agents saw only marginal gains and in some measures a slight decline in quality.[35] A separate, preregistered field experiment conducted with the Boston Consulting Group, involving seven hundred and fifty-eight professional consultants, found a structurally similar pattern that the researchers termed a jagged technological frontier: for tasks that fell within the AI’s genuine capability, consultants using it were both faster and produced measurably higher-quality work, but for a task deliberately designed to fall outside that capability, performance among AI-assisted consultants fell by roughly nineteen percentage points relative to the unassisted group, because the consultants, lacking the domain expertise to recognize when the tool had failed, followed its confident but wrong output anyway.[36] The uncomfortable implication of both studies together is that the very expertise required to use AI safely on the hardest, most consequential tasks is the expertise that unrestricted, unstructured AI use during training makes it harder to build in the first place, which is precisely the developmental externality that Yaraghi’s credentialing proposal is designed to address.

Figure 7. AI improved task completion within its capability frontier but reduced correct performance when users encountered tasks beyond that frontier.

This same body of evidence points toward a labor-market pattern that gives the credentialing argument added urgency rather than remaining a purely academic concern. Using payroll records covering millions of American workers, researchers at Stanford’s Digital Economy Lab found a roughly sixteen percent relative decline in employment for workers aged twenty-two to twenty-five specifically in occupations most exposed to generative AI since the technology’s general adoption, while employment among older workers in the same occupations remained essentially stable.[37] A separate analysis of resume and job-posting data covering sixty-five million workers across two hundred and eighty thousand firms found that companies actively adopting generative AI sharply reduced hiring at the junior level relative to firms that had not adopted it, a pattern the researchers describe as seniority-biased technological change, driven not by layoffs of existing juniors but by firms simply declining to hire new ones, since AI now performs much of the routine drafting and information-retrieval work that entry-level positions traditionally existed to teach.[38] If firms are hiring fewer juniors precisely because AI has absorbed the developmental work those roles used to provide, then educational institutions cannot treat their own AI-adoption decisions as separable from this labor-market shift; a graduate who has never practiced the underlying skill unaided arrives at a labor market that is simultaneously offering fewer structured opportunities to practice it on the job, compounding rather than offsetting the deficit built during schooling.

A final piece of evidence, concerning research and creative output rather than routine task performance, bears directly on how teaching programs should think about the cumulative effect of encouraging AI-assisted work across an entire cohort rather than in any single classroom. In a controlled experiment published in Science Advances, Anil Doshi and Oliver Hauser found that access to generative AI ideas measurably improved the individually assessed creativity of short stories, with the largest gains concentrated among the least inherently creative writers, effectively narrowing the gap between weaker and stronger writers on an individual basis. At the same time, the stories produced with AI assistance were measurably more similar to one another than stories produced without it, meaning the same intervention that improved individual output simultaneously reduced the diversity of ideas across the group as a whole. This finding matters for curriculum design because it demonstrates that an assessment regime optimized purely to raise the average quality of individual student work, a reasonable-sounding goal in isolation, can coexist with a systematic narrowing of the range of thinking a cohort produces collectively, which is precisely the kind of second-order, aggregate effect that assessment focused only on individual grades will never detect and that a program-level evaluation, not exclusively a course-level one, is required to notice.[39]

Figure 8. AI-generated ideas improved individual novelty and usefulness while simultaneously increasing similarity across the resulting stories.

Taken together, these three levels of change, redesigned individual assessments, instructor practices that preserve visible evidence of independent thinking and institutional credentialing that certifies unaided capability at defined checkpoints, describe a coherent response to the argument developed across this paper, but they share a common precondition that deserves explicit statement: none of them can be implemented by teachers who do not themselves understand how these systems work, where they fail and what structured versus unstructured use actually looks like in practice. Investment in that specific form of educator preparation, distinct from generic technology training, is likely the single highest-leverage policy lever available to the institutions this section has been discussing and it is also, on the current evidence, the one most consistently underfunded relative to its importance.[40]

Conclusion - What Happens If Education Does Not Act Now?

The evidence assembled here points toward a specific and avoidable failure mode rather than a vague technological anxiety. If the redirection of AI use described in the first section, the calibrated boundary-drawing described in the second and the credentialing and curricular reform described in the third are not undertaken, the population entering the workforce over the coming decade will disproportionately consist of graduates whose credentials certify capability that was never independently built, because the developmental work those credentials are supposed to represent was quietly outsourced during the years it should have been practiced. This is not primarily a story about individual laziness; it is a story about an education system and a labor market that, left to their current incentives, jointly remove the conditions under which expertise has always been formed, replacing effortful practice with fluent, confident output that looks like competence without reliably being competence. Firms are already hiring fewer junior workers precisely because AI performs the routine work that once trained them, narrowing the pipeline that would produce the next generation of senior judgment at the same moment schools are declining to certify that unaided judgment was ever built in the first place. The resulting productivity gains are real but borrowed, sustained by an expert generation whose capability predates these tools and unlikely to persist once that generation retires without deliberate institutional effort to replace what it is quietly spending down. The appropriate response is neither to prohibit AI, which the adoption data show is no longer a live option, nor to defer to it uncritically, but to insist, through assessment design, credentialing standards and teacher preparation, that the distinction between having used a tool and having built the capacity it assists remains visible, checkable and consequential, before the institutions responsible for drawing that distinction lose the capacity to draw it at all.

References

[1, 2, 21] Spitalniak, L. (2026) ‘90% of students use AI in the classroom, Instructure poll finds’, Higher Ed Dive, 21 July.

[3, 4] Freeman, J. (2025) Student Generative AI Survey 2025. HEPI Policy Note 61. Oxford: Higher Education Policy Institute, in partnership with Kortext.

[5] College Board (2025) New Research: Majority of High School Students Use Generative AI for Schoolwork. New York: College Board Newsroom.

[6, 7, 33] Schwartz, H.L. and Diliberti, M.K. (2026) More Students Use AI for Homework, and More Believe It Harms Critical Thinking: Selected Findings from the American Youth Panel. RR-A4742-1. Santa Monica, CA: RAND Corporation.

[8, 10, 11, 15, 16, 18, 28] Lodge, J.M. and Loble, L. (2026) Artificial Intelligence, Cognitive Offloading and Implications for Education. Sydney: University of Technology Sydney, Network for Quality Digital Education.

[9, 21, 22, 23, 33] DeJager, B. (2026) ‘84% of students use AI for homework. Only 3 in 10 schools have rules for it’, Fortune, 16 July. Republished from The Conversation.

[12, 13, 17] Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö. and Mariman, R. (2025) ‘Generative AI without guardrails can harm learning: Evidence from high school mathematics’, Proceedings of the National Academy of Sciences, 122(26), e2422633122.

[14] Hong, H., Vate-U-Lan, P. and Viriyavejakul, C. (2025) ‘Cognitive offload instruction with generative AI: A quasi-experimental study on critical thinking gains in English writing’, Forum for Linguistic Studies, 7(7), pp. 325–334.

[16] Duplice, J. (2025) ‘Generation and L2 vocabulary learning’, Vocabulary Learning and Instruction, 14(1), 102482.

[16] Fan, Y. et al. (2024) ‘Beware of metacognitive laziness’, British Journal of Educational Technology, 56(2), pp. 489–530.

[17] Weidlich, J., Gašević, D., Drachsler, H. and Kirschner, P. (2025) ‘ChatGPT in education: An effect in search of a cause’, Journal of Computer Assisted Learning, 41(4), e70105.

[19] Black, R.W. and Tomlinson, B. (2025) ‘University students describe how they adopt AI for writing and research in a general education course’, Scientific Reports, 15, 8799.

[20] McQuiston, P. (2025) ‘AI is changing how students learn, or avoid learning’, USC Today, 18 September.

[24] Perkins, M., Furze, L., Roe, J. and MacVaugh, J. (2024) ‘The Artificial Intelligence Assessment Scale: A framework for ethical integration of generative AI in educational assessment’, Journal of University Teaching and Learning Practice, 21(6), pp. 49–66.

[25, 26, 27] Kapoor, M.L. (2026) ‘The largest study of AI use by undergrads is in, revealing disparities in access, and in cheating’, University of California News, 28 May.

[29, 30, 34, 40] Yaraghi, N. (2026a) ‘Repaying the inheritance: How education and research policy can address AI’s borrowed expertise’, Brookings Institution, 20 July.

[31] Willingham, D.T. (2019) ‘How to teach critical thinking’, NSW Department of Education.

[31] Tricot, A. and Sweller, J. (2014) ‘Domain-specific knowledge and why teaching generic skills does not work’, Educational Psychology Review, 26(2), pp. 265–283.

[31, 32] Swiss Institute of Artificial Intelligence (2021) STA501: Data-Based Decision Making: Final Examination 2021. SIAI institutional teaching material.

[31, 32] Swiss Institute of Artificial Intelligence (n.d.) STA501 Final 2021 Solution Key for External Audiences’ Reference. SIAI institutional teaching material.

[35] Brynjolfsson, E., Li, D. and Raymond, L.R. (2025) ‘Generative AI at work’, The Quarterly Journal of Economics, 140(2), pp. 889–942.

[36] Dell’Acqua, F., McFowland III, E., Mollick, E.R., Lifshitz-Assaf, H., Kellogg, K.C., Rajendran, S., Krayer, L., Candelon, F. and Lakhani, K.R. (2023) Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality. Harvard Business School Working Paper No. 24-013.

[37] Brynjolfsson, E., Chandar, B. and Chen, R. (2025) ‘Canaries in the coal mine? Six facts about the recent employment effects of artificial intelligence’, Stanford Digital Economy Lab Working Paper, August, revised November.

[38] Hosseini Maasoum, S.M. and Lichtinger, G. (2025) ‘Generative AI as seniority-biased technological change: Evidence from U.S. résumé and job-posting data’, SSRN Working Paper, August.

[39] Doshi, A.R. and Hauser, O.P. (2024) ‘Generative AI enhances individual creativity but reduces the collective diversity of novel content’, Science Advances, 10(28), eadn5290.

Beyond the Universal League Table: A Structured Multi-Factor Framework for Industry Rankings

Beyond the Universal League Table: A Structured Multi-Factor Framework for Industry Rankings

Keith Lee*

*Swiss Institute of Artificial Intelligence, Chaltenbodenstrasse 26, 8834 Schindellegi, Schwyz, Switzerland

Abstract

Many institutional and commercial rankings reduce a broad population of organizations to a single ordered list. The apparent simplicity of this approach conceals a demanding mathematical assumption: that one scalar function can represent excellence across organizations with different operating models, assets, capabilities, and markets. This article develops the mathematical rationale for a structured ranking network in which \(p\) heterogeneous inputs contribute to \(k\) related but non-identical ranking outputs. The proposed architecture combines shared and category-specific latent components, permits the same firm to appear in multiple rankings, and allows the relevance and weight of an input to vary across outputs. Hierarchical layers provide flexibility when new evidence or new ranking categories enter the system, while Restricted Boltzmann Machine modules and Gibbs updates offer one implementation for estimating relationships between adjacent layers. Information criteria can help compare candidate network structures without being confused with validation of the final rankings themselves. The framework also motivates tiered publication: when rankings are jointly estimated and neighboring scores cannot always be separated with high confidence, grouped recognition is often more defensible than a claim of exact ordinal precision. The result is neither an assumptionless algorithm nor a mechanical replacement for industry knowledge. It is a disciplined measurement architecture that combines mathematical structure, empirical evidence, and documented expert interpretation.

Keywords: industry rankings; factor analysis; latent variables; multi-output systems; Restricted Boltzmann Machine; Gibbs sampling; tiered rankings; ranking methodology


Methodological Scope Note
This paper was prepared to support public understanding of the methodological rationale underlying the ranking architecture jointly developed by the Swiss Institute of Artificial Intelligence (SIAI) and The Economy Network, particularly for Ranking News. Its purpose is to explain the system’s basic architecture and mathematical intuition to a general professional audience. It is therefore a conceptual and practice-oriented exposition, rather than a complete technical specification, empirical validation study, or exhaustive account of the models and procedures used in producing individual rankings. Certain implementation details, estimation choices, data-governance procedures, and category-specific adjustments are consequently presented only at a general level.


1. Introduction

Rankings are an efficient form of institutional communication. A large volume of financial, operational, and qualitative information is condensed into a format that readers can understand immediately. Yet the communicative strength of a ranking can conceal the difficulty of its underlying measurement problem.

A conventional ranking usually begins with multiple indicators and ends with one score for each organization. If \(x_i=(x_{i1},\ldots,x_{ip})^\top\) denotes the evidence collected for organization \(i\), a general single-index ranking can be represented as

\begin{equation}
s_i=\phi!\left(\sum_{j=1}^{p}w_jT_j(x_{ij})\right),
\qquad
r_i=\operatorname{rank}(-s_i),
\end{equation}

where \(T_j(\cdot)\) places indicator \(j\) on an appropriate measurement scale, \(w_j\) is its weight, \(s_i\) is the composite score, and \(r_i\) is the published position.

This structure is not necessarily incorrect. For a narrowly defined population pursuing a sufficiently homogeneous objective, one composite measure may be informative. The difficulty arises when a broad institutional label is treated as though it represented one market and one form of excellence.

Consider the financial sector. Commercial banks, investment banks, life insurers, non-life insurers, asset managers, private equity firms, and specialist private-capital organizations are all serious financial institutions. Nevertheless, their economic functions are not interchangeable. Total assets may be central to the operating model of a commercial bank or insurer, but it does not carry the same meaning for an advisory-led investment bank, an asset-light financial intermediary, or a private equity firm organized through externally committed funds. A universal financial-institutions ranking based heavily on balance-sheet size would not merely omit information. It would confer a structural advantage on a particular business model.

The same problem appears within advisory services. Procurement advisory, supply-chain advisory, business-process transformation, automation and productivity advisory, AI implementation, digital transformation, and cloud infrastructure are closely connected. Some firms operate across several of them. But the services remain different enough that one global weight vector cannot represent every market equally well.

The methodology developed here therefore replaces the conventional many-input, one-output structure with a many-input, multi-output system. Formally, \(n\) organizations are observed through \(p\) evidence variables and evaluated across \(k\) related ranking categories. The central question is not how to force every organization onto one line. It is how to represent several forms of sector-specific strength without losing their economic connections.

2. The Mathematical Cost of a Universal Ranking

2.1 One-dimensional reduction

Every universal ranking performs a dimensional reduction. A collection of different observations becomes one scalar score. Even where the published methodology is not based on principal component analysis, PCA provides a useful mathematical analogy.

For standardized evidence \(\widetilde{x}_i\), the first principal component is

\begin{equation}
z_{i1}=v_1^\top\widetilde{x}_i,
\end{equation}

where \(v_1\) is selected to maximize the sample variance of the projected data. The share of total variation represented by that component is

\begin{equation}
\frac{\lambda_1}{\sum_{j=1}^{p}\lambda_j}.
\end{equation}

Unless this proportion is overwhelmingly large, a one-component representation necessarily leaves substantial variation outside the published score. PCA creates successive uncorrelated components that maximize remaining variance; the first component is not presumed to represent the full structure of the data (Jolliffe and Cadima, 2016).

The analogy has an important boundary. Most rankings are not literally principal-component models, and the direction of greatest statistical variation is not automatically the direction of greatest economic importance. The lesson is narrower: a one-dimensional projection is a strong structural restriction, regardless of whether its weights are equal, judgmental, regression-based, or data-derived.

2.2 Fixed weights imply a universal meaning

Under a linear composite score,

\begin{equation}
s_i=\sum_{j=1}^{p}w_jx_{ij},
\end{equation}

the marginal contribution of evidence item \(j\) is fixed:

\begin{equation}
\frac{\partial s_i}{\partial x_{ij}}=w_j.
\end{equation}

The same indicator therefore carries the same marginal importance for every evaluated organization. If the candidate population contains several business models, the model assumes that the meaning of the indicator remains stable across them.

That assumption often fails. Geographic office coverage may be crucial for a client-facing implementation network but much less informative for a specialist strategic adviser. Assets under management are meaningful for an asset manager but inappropriate as a direct measure of private-equity operating expertise. The issue is not simply that a researcher may choose an imperfect weight. It is that a single universally correct weight may not exist.

2.3 A ranking is a latent measurement problem

The distinction between regression and ranking construction is fundamental. In an ordinary regression,

\begin{equation}
Y_i=\beta_0+X_i^\top\beta+\varepsilon_i,
\end{equation}

\(Y_i\) is an observed dependent variable whose conditional relationship with \(X_i\) is estimated. In an institutional ranking, overall category strength is usually not observed independently. It is the latent construct that the researcher is attempting to measure.

The ranking problem therefore does not begin with a single observable \(Y\) for which one seeks a best unbiased estimator under a common conditional error law. It begins with heterogeneous evidence and a question about how many economically meaningful constructs are required to represent that evidence. This makes latent-variable and factor structures more natural than a conventional supervised regression.

2.4 Nonlinearity does not solve the classification problem by itself

A nonlinear function can fit one dataset more closely than a linear index:

\begin{equation}
s_i=g(x_i;\theta).
\end{equation}

But a close in-sample fit does not establish that the same function will remain meaningful across industries or years. A nonlinear model may learn the peculiarities of one candidate universe, one information environment, or one period. Transfer to a different market requires stability in the underlying economic relationships, not merely greater functional flexibility.

This is why the principal design question comes before nonlinear estimation. If the underlying market contains several forms of excellence, a more elaborate \(p\rightarrow1\) function still produces only one ordering. The model becomes more flexible without addressing the substantive mistake of forced unidimensionality.

3. Three Sources of Distortion in Single-Output Rankings

3.1 Heterogeneous evidence

Ranking evidence can include financial values, operating counts, market coverage, professional authorizations, service descriptions, transaction records, text-derived measures, ordinal evaluations, and indicators of organizational specialization. These observations do not all have the same economic meaning or statistical behavior.

No single conditional distribution for a scalar ranking outcome is naturally implied by this evidence. More importantly, there is no observed ranking \(Y\) that supplies an external criterion for estimating one universally correct set of weights. A single-output model may still be constructed, but its apparent precision should not be confused with identification of an objectively true ordering.

3.2 Measurement error, missing evidence, and feedback

Measured indicators are imperfect. Some capabilities are easier to observe than others. Large organizations generally leave larger public data footprints, while specialist firms may generate less visible but more relevant evidence. Several sources may also reproduce the same underlying information, causing one characteristic to enter the model repeatedly under different labels.

There can also be feedback between reputation and measurement. Prior recognition may increase media visibility, recruitment, client inquiries, and future information availability. Commercial success may increase both genuine operating capability and the volume of observable evidence. These relationships can create circularity even when the final ranking score is mechanically constructed from current inputs.

A multi-output architecture does not eliminate measurement error or feedback. Its contribution is more specific: it reduces the risk that evidence relevant to one type of activity is treated as a universal measure of every activity. Source review, temporal controls, duplicate-evidence checks, and expert examination remain necessary.

3.3 Excessive compensability

Composite rankings usually allow strength in one indicator to compensate for weakness in another. This may be reasonable within a coherent construct. It becomes questionable when unrelated capabilities are aggregated.

A very large firm may compensate for limited specialization through size, visibility, or geographic breadth. Conversely, a specialist may be penalized for lacking capabilities that are irrelevant to the service in which it competes. The result can be mathematically consistent with the chosen formula while economically misclassified.

4. From \(p\) Inputs to \(k\) Connected Rankings

4.1 Basic factor representation

Let \(x_i\in\mathbb{R}^{p}\) represent the evidence vector for firm \(i\), and let \(f_i\in\mathbb{R}^{k}\) represent its latent strengths across \(k\) ranking categories. A basic multi-factor representation is

\begin{equation}
x_i=\mu+\Lambda f_i+\varepsilon_i,
\end{equation}

where \(\Lambda\in\mathbb{R}^{p\times k}\) is the loading matrix and \(\varepsilon_i\) contains evidence-specific variation.

Instead of receiving one universal score, firm \(i\) receives a vector:

\begin{equation}
f_i=(f_{i1},f_{i2},\ldots,f_{ik})^\top.
\end{equation}

Each component supports a different ranking output. The outputs are jointly connected because they are estimated from overlapping evidence and may share latent capabilities. They are nevertheless distinct because each output can receive a different loading pattern.

4.2 Structured relevance and sparse connections

Not every input should affect every output. Define an admissibility matrix

\begin{equation}
M\in{0,1}^{p\times k},
\end{equation}

and write

\begin{equation}
\Lambda=M\odot B,
\end{equation}

where \(\odot\) denotes elementwise multiplication. If \(M_{jr}=0\), evidence item \(j\) cannot directly affect ranking category \(r\). If \(M_{jr}=1\), its weight may be estimated from the data and the surrounding network.

This arrangement permits three economically different cases:

  1. An indicator can be central to one category.
  2. It can contribute with different weights to several related categories.
  3. It can be excluded from a category to which it has no defensible relationship.

For example, cloud-infrastructure capability may be heavily relevant to cloud advisory, moderately relevant to digital transformation and AI implementation, and irrelevant to an unrelated advisory category. The architecture represents these differences directly instead of forcing a common weight across all outputs.

4.3 Connected does not mean identical—or orthogonal

Related ranking categories need not be statistically independent. Let

\begin{equation}
\operatorname{Cov}(f_i)=\Psi,
\end{equation}

where \(\Psi\) may contain non-zero off-diagonal elements. This allows two category factors to share meaningful variation.

The objective is therefore not to force complete orthogonality between connected markets. Luxury-yacht and superyacht capabilities, for example, may share design, engineering, supplier, and production foundations. The categories should be distinguishable, but their correlation is economically legitimate. A model that artificially eliminates all common variation could be as misleading as a model that collapses both markets into one.

4.4 Multiple rankings for a single firm

The multi-output structure has an important practical implication: a firm need not be assigned exclusively to one ranking category.

Define eligibility as

\begin{equation}
e_{ir}\in{0,1},
\end{equation}

where \(e_{ir}=1\) indicates that firm \(i\) conducts a material and relevant activity in category \(r\). The eligible universe for ranking \(r\) is

\begin{equation}
\mathcal{U}_r={i:e_{ir}=1}.
\end{equation}

There is no restriction requiring

\begin{equation}
\sum_{r=1}^{k}e_{ir}=1.
\end{equation}

A diversified firm may therefore appear in several rankings. Its position in each category is determined by the category-specific score and evidence weights, not by its placement elsewhere.

This is not duplication. It is a direct consequence of recognizing that firms can operate across multiple slices of an interwoven market. A consultancy may possess genuine capabilities in AI implementation, automation, digital transformation, and cloud infrastructure. Forcing it into one category would discard information; granting it one universal score would erase the differences among those capabilities. Multiple category appearances preserve both breadth and specialization.

5. Distribution-Agnostic Structure and Estimation Choice

5.1 What distribution-agnostic means

The structural \(p\rightarrow k\) framework does not require the researcher to posit one Gaussian—or other common parametric—conditional distribution for a scalar ranking output. It is not organized around estimating \(E[Y\mid X]\) for one observed \(Y\). At that architectural level, it is distribution-agnostic.

The distinction between architecture and estimator is important. A linear factor structure can be estimated through covariance reconstruction and unweighted least squares. Minimum-residual factor analysis, for example, selects loadings by minimizing residuals in the reproduced correlation matrix. Unweighted least-squares factor estimation has been developed as a distribution-free alternative to maximum-likelihood factor analysis (Harman and Jones, 1966; Krijnen, 1996).

However, distribution-agnostic does not mean assumptionless. The selection of an objective function, factor dimension, admissible loading structure, and evidence transformation still matters. If a probabilistic RBM module is used between layers, that module defines its own energy-based probability model. The framework avoids one universal distributional assumption for the ranking problem; it does not claim that every possible implementation is free of modelling assumptions.

5.2 Restricted Boltzmann Machines between adjacent layers

The implementation considered here uses Restricted Boltzmann Machine modules to estimate selected relationships between adjacent layers. For visible state \(v\), hidden state \(h\), and weight matrix \(W\), a basic RBM can be represented through the energy function

\begin{equation}
E(v,h)=-a^\top v-b^\top h-v^\top Wh.
\end{equation}

The associated joint probability is

\begin{equation}
P(v,h)=\frac{\exp[-E(v,h)]}{Z},
\end{equation}

where \(Z\) is the partition function.

Because an RBM contains no within-layer connections, hidden states are conditionally independent given visible states, and visible states are conditionally independent given hidden states. This bipartite structure permits alternating block Gibbs updates:

\begin{equation}
h^{(t)}\sim P(h\mid v^{(t)}),
\qquad
v^{(t+1)}\sim P(v\mid h^{(t)}).
\end{equation}

Gibbs sampling is therefore not invoked as a general claim about every deep-learning architecture. It is used because the selected layerwise implementation is built on RBM structures for which alternating conditional updates provide a natural estimation mechanism. RBMs have long been used as two-layer generative learning modules and as components of deeper systems (Hinton, 2012).

5.3 Comparing candidate network structures

The number of output factors, intermediate nodes, and layers should not be selected through an unguided search over every possible architecture. Economic taxonomy supplies the initial structure: which services exist, which indicators can reasonably affect them, and which capabilities may be shared. Statistical model comparison then evaluates competing representations within that constrained space.

Where comparable likelihoods can be evaluated or consistently approximated, information criteria may be written as

\begin{equation}
\mathrm{AIC}=-2\ell(\widehat{\theta})+2d,
\end{equation}

and

\begin{equation}
\mathrm{BIC}=-2\ell(\widehat{\theta})+d\log n,
\end{equation}

where \(\ell(\widehat{\theta})\) is the fitted log-likelihood and \(d\) is the effective number of estimated parameters. AIC and BIC balance fit against architectural complexity rather than rewarding the largest possible network (Akaike, 1974; Schwarz, 1978).

Their role here is specific. They can help compare whether, for example:

  • an additional latent node improves the representation sufficiently to justify its complexity;
  • an intermediate capability layer fits better than a direct input-to-output mapping;
  • two proposed output categories are sufficiently distinguishable;
  • a new branch improves the architecture more than it fragments the evidence.

Information criteria do not certify that the published ranking is substantively correct. Nor do they evaluate a Gibbs sampler merely because it uses different update steps. They compare candidate statistical architectures represented by different dimensions or restrictions. Convergence diagnostics, stability, interpretability, and industry coherence must accompany the numerical comparison.

6. Hierarchical Flexibility

6.1 Intermediate capability layers

A direct \(p\rightarrow k\) factor system may be sufficient for a narrow market. A broader ranking network can use intermediate layers:

\begin{equation}
x_i\rightarrow h_i^{(1)}\rightarrow h_i^{(2)}\rightarrow f_i,
\end{equation}

where the \(h_i^{(m)}\) terms represent shared or partially shared capabilities.

In advisory services, intermediate factors might represent geographic execution, sector expertise, technical implementation, client acquisition, regulatory capability, transaction delivery, or operational infrastructure. Final ranking categories then depend on different combinations of these intermediate capabilities.

6.2 Adding new evidence

When a new data source becomes available, it does not have to connect directly to every final ranking. It can enter the intermediate capability to which it is economically related. The model can then estimate how that capability contributes to the connected outputs.

This does not make the system automatically more accurate. It makes the system easier to extend without rebuilding every ranking as an isolated formula. New evidence can be absorbed locally, its propagation can be controlled, and its broader effects can be examined across the connected network.

6.3 Adding or bifurcating outputs

Market development may also require a new ranking output. Suppose one initial marine-industry factor contains both luxury-yacht and superyacht capability. As evidence coverage improves, the upper layer can branch:

\begin{equation}
f_{i,Y}=\beta_Yh_{i,M}+\gamma_Yh_{i,L}+u_{i,Y},
\end{equation}

\begin{equation}
f_{i,S}=\beta_Sh_{i,M}+\gamma_Sh_{i,C}+u_{i,S},
\end{equation}

where \(h_{i,M}\) represents shared marine capability, \(h_{i,L}\) represents luxury-market capability, and \(h_{i,C}\) represents complex bespoke-project capability.

Both outputs preserve their common industrial foundation while receiving different category-specific information. The separation is not created by arbitrarily changing final weights until two different lists appear. It is created by introducing a structured branch at the layer where the economic distinction arises.

6.4 Flexibility rather than automatic robustness

Additional layers increase representational flexibility. They can make it easier to incorporate new inputs, new intermediate capabilities, and new output categories. They do not guarantee greater robustness merely by existing.

Every additional node or layer increases the number of possible relationships and therefore the risk of overfitting or weak identification. The advantage of the hierarchical system lies in controlled extensibility. Whether a deeper structure improves the ranking network remains a question for model comparison, stability review, and economic interpretation.

7. Why Tiered Publication Follows from the Model

7.1 Exact positions can imply excessive precision

The latent score for category \(r\) may provide an ordering of eligible firms, but neighboring scores can be close. Their relative positions may also depend on evidence that contributes to several connected outputs. A change in one shared component can affect more than one category through jointly determined weights.

Publishing every firm as though the difference between positions 7 and 8 were exact and isolated would overstate what the system can establish. This concern becomes stronger when two related ranking categories share capabilities or when firms operate across both markets.

Tiered publication is therefore not merely a visual design choice. It is a methodological acknowledgement that the system measures relative zones of performance more reliably than every adjacent ordinal distinction.

7.2 Institutionally calibrated tier sizes

Ranking News commonly uses tier structures such as 5/10/5, while larger publications may use different group sizes, including formats such as 5/10/30. These numbers are institutionally determined from accumulated observations across category rankings. They are not presented as a universal mathematical ratio.

The appropriate partition can vary with:

  • the size of the eligible market;
  • the density of credible candidates;
  • the degree of observable separation among firms;
  • the maturity of the category;
  • the amount and quality of available evidence;
  • the intended breadth of published recognition.

The first tier identifies the most consistently differentiated group. Subsequent tiers recognize strong or credible participants without claiming that every internal position is precisely separable. A tier boundary remains a publication rule, not proof of a discontinuity in the underlying industry.

7.3 Tiering as methodological humility

The connected ranking system does not claim perfect decomposition of overlapping markets. Factor structures do not convert noisy industrial evidence into naturally discrete classes. Tiering accepts this limitation openly.

The methodology therefore makes a narrower and more defensible claim: the available evidence supports differentiated groups within a category, but it may not support strong conclusions about every one-place difference inside those groups. This form of restraint increases the credibility of the output.

8. Ranking Stability and Controlled Recalibration

Input comparability is a precondition for estimation. Once the evidence has been placed on appropriate and mutually interpretable scales, the methodological question becomes whether reasonable specification changes produce excessive positional movement.

The stability review should examine:

  • alternative evidence windows;
  • inclusion or exclusion of unusually influential sources;
  • changes in candidate eligibility;
  • reasonable adjustments to admissible connections;
  • alternative numbers of intermediate nodes;
  • missing-evidence treatments;
  • year-to-year changes in the information environment.

The purpose is not to search for a specification that preserves a predetermined list. It is to identify whether a small methodological change causes disproportionate movement in tier membership or category placement. If it does, the output should be interpreted more cautiously, the architecture reconsidered, or the publication groups widened.

This review is especially important in a joint system. Because an input can influence shared intermediate capabilities, a model adjustment may propagate across several outputs. That propagation is not necessarily an error: it may reflect a genuine shared component. But it should be traceable. A connected ranking network is defensible only when researchers can distinguish an economically meaningful system-wide effect from an unintended consequence of model specification.

9. Factor Discovery and Human Interpretation

9.1 Why factors do not name themselves

Factor analysis groups shared variation. It does not determine the economically correct name for every hidden component. Two statistically similar loading patterns may admit different substantive interpretations, while rotationally related solutions may reproduce the evidence comparably well.

Industry experts must therefore examine:

  • the evidence entering each component;
  • the signs and relative sizes of its loadings;
  • its relationship with neighboring factors;
  • the firms receiving high and low factor scores;
  • whether the inferred construct corresponds to a recognizable market activity.

This is not an optional editorial layer added after the mathematics. It is part of construct validation.

9.2 Two principal interpretation risk

Human interpretation creates at least two risks:

  1. Misnaming: a statistically identified component is given a category name that does not accurately describe its evidence pattern.
  2. Misdefinition: the category name may be reasonable, but the stated operating scope includes services that the component does not measure coherently.

Both risks can be reduced through documented category definitions, comparison of alternative factor structures, and review by specialists familiar with the relevant industry.

9.3 Expert review without arbitrary reweighting

Expert intervention should be structural rather than outcome-seeking. Appropriate interventions include:

  • defining candidate eligibility;
  • identifying economically impossible connections;
  • specifying anchor indicators;
  • distinguishing structural non-applicability from missing evidence;
  • proposing a new factor or branch when the market has changed;
  • correcting an interpretation contradicted by the loading structure.

Inappropriate intervention would include changing weights solely because a preferred firm did not receive the expected position. Every material structural adjustment should be documented and followed by renewed model comparison and stability review.

The separation between editorial ranking decisions and commercial licensing is equally important. Recognition licenses may govern how a result is communicated, but they must not determine eligibility, weights, tier placement, or continued editorial inclusion.

10. Structural Illustrations from Ranking Markets

10.1 Financial institutions

A universal financial-institution ranking would struggle to reconcile balance-sheet scale, underwriting capacity, transaction expertise, externally managed assets, fund deployment, geographic distribution, and advisory networks. A multi-output structure instead permits related rankings for banks, insurers, asset managers, private-equity firms, and other financial specialists. Shared financial capability remains visible, but each category receives its own evidence structure.

10.2 Operations and technology advisory

Procurement, supply chain, business-process transformation, automation, AI implementation, digital transformation, and cloud infrastructure form an interconnected advisory space. Firms may operate in one, several, or nearly all of these markets.

The \(p\rightarrow k\) architecture permits a project-delivery indicator to affect several categories with different weights. It also allows specialized evidence—such as cloud architecture capability or procurement-market intelligence—to enter only the relevant branches. A diversified adviser may appear in multiple rankings, while a specialist can compete on the evidence appropriate to its own market rather than being overwhelmed by universal scale indicators.

10.3 Luxury yachts and superyachts

Luxury-yacht and superyacht markets illustrate why connected outputs should not be forced into either complete merger or complete independence. The markets share industrial capabilities and some firms operate in both. Yet bespoke project complexity, vessel scale, customer structure, engineering demands, and production economics can support separate evaluations.

A hierarchical branch preserves their common foundation and introduces distinct upper-layer components. When the evidence does not justify sharp separation, the methodology can retain correlated factors and broader tiers rather than impose arbitrary final weights. As coverage improves, the branch can become more differentiated without discarding the earlier network structure.

11. Limitations

The proposed architecture improves the representation of heterogeneous markets, but it does not transform ranking into an exact science.

First, the model remains dependent on evidence quality. Missing, duplicated, promotional, or systematically biased information can affect the latent structure.

Second, factor separation is incomplete by nature. Closely related economic activities may remain strongly correlated, and some hidden components may combine more than one substantive capability.

Third, architecture selection is not purely mechanical. AIC, BIC, or other fit measures can compare candidate structures, but economic interpretability and institutional judgment remain necessary.

Fourth, RBM and Gibbs implementations carry probabilistic and computational assumptions. They are estimation tools, not guarantees of an objectively true market taxonomy.

Fifth, additional layers increase flexibility and extensibility, but they may also increase overfitting, weak identification, and opacity.

Sixth, tier boundaries are institutionally calibrated publication decisions. They communicate groups supported by accumulated evidence, but they do not establish natural discontinuities between adjacent firms.

Finally, multiple appearances across rankings must be governed by substantive eligibility. Cross-category recognition is a strength only when it reflects genuine business activity rather than the mechanical advantage of corporate scale.

12. Conclusion

The central weakness of a universal league table is not that it necessarily uses the wrong formula. It is that it assumes one formula can represent organizations whose economic functions differ.

A structured \(p\rightarrow k\) ranking network offers a more coherent alternative. It allows several rankings to share evidence and hidden capabilities without treating them as identical. The same input can receive different weights across outputs, irrelevant connections can be excluded, and firms with several genuine business lines can appear in multiple categories. Intermediate layers permit new evidence and new rankings to enter through controlled extensions rather than isolated redesigns.

The approach also changes how ranking outputs should be communicated. Because category scores are jointly determined and related markets cannot always be separated perfectly, tiered publication is often more defensible than exact one-place ordering. Institutionally calibrated tiers acknowledge the limits of the evidence while preserving meaningful recognition.

No mathematical architecture eliminates the need for judgment. Factor components must be interpreted, category boundaries must be defined, and structural changes must be reviewed. The purpose of the model is not to remove expert responsibility, but to discipline it. Mathematics determines how evidence can be connected; industry expertise determines whether those connections describe a real market.

The resulting ranking is neither a universal scalar nor a collection of unrelated lists. It is a connected measurement system: one designed to preserve the common foundations of an industry while recognizing that excellence can take more than one form.

References