When it comes to adopting artificial intelligence (AI), for many governments change cannot come soon enough. “We must move fast and take action to win the global race”, said Keir Starmer, the U.K.’s Prime Minster, when announcing a national “blueprint to turbocharge AI” in 2025. The research and advisory firm Gartner predicts that in 2028, 80% of governments will deploy AI agents to automate routine decision making. Even when taking such forecasting with the advisable grain of salt, the “agentic state” is undeniably on the rise. Just recently, the German government launched its “Agentic AI Hub”, connecting AI startups with public administration to build autonomous systems that can accelerate bureaucratic procedures. The promise of agentic AI adoption lies in cost-efficient governance that reduces bureaucracy, allows faster, pro-active and tailored public service, delivered round-the-clock.
Agentic AI systems go beyond supporting public servants with information, draft texts, or indicators; they can make decisions on their own. Does their anticipated roll out by governments mean that human bureaucrats will be replaced by machines?
Automation Requires Oversight
Yes, because continuing to involve humans in fulfilling tasks that AI agents can handle on their own slows down procedures. Hybrid systems that causally require human action – performed by a human in the loop – to bring about a decision create a bottleneck for the efficiency gains that governments expect from implementing AI. In addition, finding humans to place them inside of decisional loops will become harder. The public sector is particularly challenged by the ageing of Western societies’ working population: the median age in public administrations across OECD countries is in some cases already over 50. Agentic systems could partially offset the losses in the human labour pool.
No, because public-service AI systems have already been deployed to autonomously decide, for example, if citizens committed welfare fraud – with devastating results. In 2013, the U.S. state of Michigan launched an automated system called MiDAS to detect fraudulent claims for unemployment insurance. As documented by Giest and Klievink, the system was fully automated and enabled significant staff reductions. MiDAS led to a fivefold increase in cases of detected fraud and the issuance of $57 million in fines. However, many citizens accused of fraud appealed the decisions. An audit found that in only eight percent of the appeals actual fraud was detected. Based on the audit, a law was passed that bans solely automated fraud determinations and requires human involvement. In addition, new roles were created to monitor the system. By now, many normative frameworks for responsible AI use include human oversight as a key risk mitigation mechanism. The EU’s AI Act, for example, requires in Article 14 that high-risk AI systems are effectively overseen by natural persons to mitigate risks to health, safety, and fundamental rights. Many of the use cases that are deemed high-risk under the AI Act fall within the public sector. AI regulation thus secures human labour in bureaucracies.
This leaves us with two forces that will determine the future of humans in the public sector: the instrumental rationale of replacing human labour with agentic AI and the normative demand for humans actively overseeing AI. These forces only seemingly pull in opposite directions. They are conditioning each other and will bring about significant changes to the occupational profiles of civil servants and the epistemic nature of the citizen-state relationship.
Overseeing Agents Instead of “Seeing” Citizens
In Seeing Like a State, James C. Scott identifies the legibility of society as a core concern for modern statecraft. The pre-modern state, as Scott writes, “knew precious little about its subjects, their wealth, their landholdings and yields, their location, their very identity”. Since then, public officials have come up with standardised measures and metrics to record and monitor even highly complex social practices. The introduction of permanent last names and population registers, the grid-based design of cities, and the simplification of agriculture by reducing crop variety are in Scott’s view all attempts at making land and people legible for public authority. Once legibility increased, Törnberg et al. write, statistics became the “science of statecraft” of the modern state.
Today, governments push AI to innovate their modes of governance yet again. With AI agents, the seeing of the state will turn into an overseeing. Humans will be less constitutive of administrative decisions and retreat to a corrective form of public agency. Instead of keeping bureaucrats in the decisional loop, governments will place them on the loop of otherwise autonomous systems to monitor their functioning and intervene, if necessary, to mitigate risks. Consequently, bureaucrats will become less directly involved in citizens’ affairs: when automated systems handle routine cases, Giest & Klievink note, public servants will be needed to deal with exceptions and hard cases that require additional information.
Public servants will face a different and quite likely, more difficult job. In her “Ironies of Automation”, Lisanne Bainbridge notes how automation is supposed to make the human’s job easier but instead makes it harder. Automated systems such as AI systems are often more complex than other technologies; overseeing them requires higher skill levels while at the same time, peoples’ skills might decline if machines take over tasks that humans previously performed. In the agentic state, the skills required from bureaucrats will, to a significant degree, be defined by the risk-mitigation function of oversight. This function is primarily focused on the human-AI relationship and much less on the human-to-human relationship between bureaucrats and citizens. Humans will be visible to the state’s employees primarily as difficult cases that the AI cannot handle well.
A Constitutionalist View on the Agentic State
From a constitutionalist perspective, we want to understand what happens to public authority when public servants cease to be constitutive participants in administrative decision-making and instead assume a corrective role. One consequence of the shift from seeing to overseeing is that it will likely increase the legibility of bureaucratic action itself. The tools and mechanisms of human oversight aim at making AI systems legible to human decision-makers. Many states never had the appetite for scrutinising their human bureaucrats’ decisions to a comparable degree, albeit plenty of evidence of human fallibility and proneness to biases. The agentic state can thus be seen as a further step in the long-term process of rationalisation in the public sector. While the relationship between state and citizen will become more instrumental, the overseeing of the state will largely remain invisible for citizens – unless oversight leads to palpable interventions or fails to intervene when necessary. The agentic state is well-advised to thoroughly consider who should receive citizens’ blame for its failures. Blaming an AI system is pointless, blaming its oversight personnel can be unfair, considering the difficulty of the job and the limited influence they have on the system design itself.
Elsewhere, I have argued that the legitimacy of public authority partially depends on institutional design and its epistemic reliability. If governments implement agentic AI with human oversight on the loop and these systems are more reliable (i.e., make fewer mistakes, are less biased, etc.) than hybrid systems or bureaucrats working without AI, then this could, from an epistemic point of view on public authority, be the more legitimate institutional design. However, while epistemic considerations matter, they cannot by themselves justify public authority: institutions should be designed to be reliable, yet they still require procedural and egalitarian legitimation.
That point becomes acute in the agentic state. It is easy to compare human performance to machine performance when benchmarks are available – for example, when we know what constitutes an error in an output because we can meaningfully assess its accuracy. However, many optimisation targets and risk thresholds embed contestable judgments about fairness, rights, and the acceptable distribution of errors. Overseeing AI requires value choices that defy quantitative benchmarks and must remain tied to democratic institutions that can produce the kind of procedural and egalitarian legitimation that instrumental reasons alone cannot provide. The importance of keeping humans working in the agentic state thus goes beyond mere risk-mitigation. Human agency enables public authority to be accountable and responsive to genuine human reason. Our democratic community remains the source of legitimacy for any socio-technical engineering of the institutional design of the state.
Johann Laux
Johann Laux is a Departmental Research Lecturer in AI, Government & Policy at the Oxford Internet Institute, University of Oxford, and a Fellow at GovTech Deutschland. He researches the legal, ethical, and social implications of emerging technologies such as AI. At Oxford, he is the principal investigator of the Emerging Laws of Oversight project which investigates how human oversight of AI systems can be implemented effectively.
