Automation has changed what it means to interact with state administration on both sides. Those who administer and those who are subject to administration find themselves in systems that were not designed with their humanity as a central concern and that privilege efficiency, speed, savings, or the performative notion of innovation instead. Humanity remains a residual safeguard, and so ultimately, the automated state poses a question that is existential rather than technical: What kind of humans can we be within institutions that act upon people, without people?
Reflecting on the topics raised by the speakers in the “Humanity in the Automated State” lecture series and the blogpost symposium essays, we question what we know and what we should know about how humanity is experienced in the automated state. We do not offer a synthesis so much as a reading: an attempt to trace, across the full range of contributions, where the human keeps getting displaced, what that displacement costs, and what questions it leaves unresolved. We organise this reflection around three sites of encounter with the automated state — those who administer it, those who are subject to it, and the relationship between the two — before asking what, if anything, can be done.
Humanity within the Automated State: The Displaced Administrators
In theory, automated decision making in state administration should make the working life of civil servants easier, freeing up time and cognitive resources to focus on the cases that most require human judgment and care.
In practice, that is far from always being the case: roles shift, workload changes and responsibilities get reassigned without necessarily reflecting the expertise and intuition that are developed in and are crucial to providing administrative services. Where they do not get replaced, civil servants are moved from the core to the periphery of decision-making. With that, the relevance of their contextual knowledge, insights, and discretion in handling the case also diminishes, raising the question of how important the human touch is in exercising judgement, and whether it is possible to learn to be human and take that humanness into account when making decisions about other humans.
Addressing these issues is not purely a legal challenge, it is also a behavioural, organizational, and interpersonal. What goes away when a civil servant’s role is not to exercise judgement but monitor an automated process? What does this loss, if any, do to those who make the decisions and those who receive them? How can the law require that automated systems take administrators’ humanity into account, and should it? And if the role of the individual acting on behalf of the state is redefined by the technologies the state uses, what happens to our thinking about who should be held accountable – and to how the accountable persons experience their accountability? If no one – or nothing truly is or can be accountable anymore, how should the legal system adapt?
Humanity versus the Automated State: The Alienated Subjects
When decisions are automated, the recognition of being human, or its absence, becomes a crucial experience for those on the receiving end. It shapes how a decision lands, whether one feels like the decision was made with a concern for you, or as something that had happened to you. Being heard, being able to tell one’s story, receiving an explanation are part of what makes a decision legitimate.
When those are taken away, people do not experience the absence the same ways. Some of the differences come down to individual resources. Those who do not have the language to operate in automated systems, the means to afford legal support, or the literacy to navigate these systems carry a heavier burden. But other constraints are not about individual capacity at all. Citizenship determines who is surveilled and to what degree. Social pressures exerted by the state and its contractors can limit how much a person can react to being observed, recorded, and acted upon by the automated state, regardless of how articulate or resourceful they might otherwise be.
But then, not everyone may experience the removal of the human as a loss at all. On the one hand, for those whose encounters with human administrators have been marked by discrimination, hostility, or outright persecution, automation can feel like a form of protection, a system that at least does not look at you the way a person can. On the other hand, those who are consistently assured that automation is more efficient, more objective, fairer, may experience the change as progress rather than loss.
While the state becomes impersonal, the pressing questions are zeroing in on the individual experiences. How can one gather the resources to push back, and on what terms? What would it mean to contest a decision if you cannot access the reasoning behind it, and neither can the person communicating it to you? When vulnerability is structural, can procedural rights do what we expect them to? How can the law protect those who have no other choice but be subject to the automated state?
Humanity across the Automated State: By the Machine, of the Machine, for the Machine?
The relationship between citizen and state has always been unequal, and automation raises an even more unsettling question of whether the relationship exists at all anymore. Explanation, contestation, accountability as the things that made the exercise of power into the relationship are becoming harder to reach. The legal concepts built to hold the relationship together, such as the duty to give reasons or the right of appeal, were designed for human judgement and are now stretched across processes they were never meant to or envisioned to fit. The problem goes deeper than legal categories struggling to keep up. The technical features of automated systems are not neutral as they are built by those who may never be subject to them and who have never been required to imagine themselves as such. This matters because the ability to imagine oneself being the one who is subject to a decision is arguably a condition of democracy itself.
The rearrangement of the state-individual relationship calls into question a reflection on who codes the rules and how their consequences are experienced. If the two are different groups of people, what happens when a system encounters a situation that it is incapable of recognising? Can legal concepts designed for human judgement be remade to fit the processes that are not following the human experience? Should the law start from the system, or from the people it acts upon and those displaced by it? Does the state that runs these systems remain one where people on either side of the decision-making process retain meaningful space to be present, to contest, to be heard, and to influence what happens to them?
Patch up or Rebuild from Scratch?
The automation of the state is reshaping what it means to administer and to be administered, and the humans on both sides are feeling it before the law has worked out what to do. The concepts, safeguards, and relationships meant to keep public power in check were built for a different kind of encounter between people. Can they be stretched to fit what is happening now, or does something new need to be built? And if so, built by whom, for whom, and with what version of humanity and the state in mind?
Daria Morozova
Daria Morozova is Assistant Professor at the Department of Business Studies at Leiden Law School. Her research explores what it means to be human at work, examining how people perceive their capacities in the artificially intelligent world, what makes people successful in changing environments, and how the ways we speak about the world shape our experiences.
Melanie Fink
Melanie Fink is Associate Professor of European Law at Leiden University. Her research explores the intersection of digital public administration and EU law from an individual rights perspective.
