The introduction of automated decision-making (ADM) systems by government organisations has, over the last five decades, fundamentally altered the relationship between citizens and public authorities. About twenty-five years ago, Bovens and Zouridis identified the constitutional implications of this transformation that needed (and still need) addressing. While ADM facilitates millions of citizens to receive health care benefits, study loans, income support or other forms of social benefits, the arrival of the system-level bureaucracy comes with risks like bias, arbitrariness, rigidity, and a shift of discretionary powers to system designers. Additionally, an overarching risk that looms large in the age of ADM is the opaqueness, or black box-character of decision-making systems.
Since the first ADM systems were introduced in government decision-making, the question of transparency and explainability has consistently remained relevant. Yet, while administrative legal requirements typically focus on transparency and explainability of individual decisions, a shift to a system-level bureaucracy demands systematic insight into the structural and automated execution of the law. This post aims to identify several risks posed by rule-based ADM and poses one potential method of mitigating these risks: knowledge modelling. It first defines rule-based systems and identifies several associated risks. These risks, and the necessity to mitigate them, are subsequently illustrated by two court rulings: AERIUS from the Netherlands, and BOSCO from Spain. Finally, it gives a brief overview of ways in which knowledge modelling techniques could mitigate these risks, concluding with several examples of (mostly European) projects that are currently working on precisely that.
The main risks of rule-based systems
In short, rule-based systems use rules and data to generate outcomes, whereas case-based systems use data and outcomes to generate rules. The majority of ADM systems in place for the initial distribution of social benefits, aim to translate laws into executable software code. In other words, these tend to be rule-based systems as opposed to case-based systems. Even though rule-based systems have limitations (e.g. they are static and lack ‘intelligence’), they function exclusively according to their pre-programmed instructions and specifications. This makes them suitable for automating the execution of laws (i.e. producing initial welfare benefit decisions) and arguably less suitable for other administrative responsibilities like fraud detection. For that, more advanced models are used, and a different legal framework applies (with a focus on anti-discrimination and data protection, compare relevant cases like SCHUFA, Dun & Bradstreet, or the Dutch case of Syri).
In addition to the issues raised by Bovens & Zouridis, complex rule-based systems can lead to numerous problems. Firstly, legal professionals and software developers often do not speak the same language. It can be challenging for a software developer to keep in mind the intent of the lawmaker and it can be challenging for legal professionals to check whether the developer accurately translated this into software. Secondly, especially smaller scale government organisations may choose to procure software projects as they lack in-house expertise. Next to being costly, procuring software creates a dependence on private entities, for instance when the law is changed or when updates and improvements are warranted. Next to that, transparency and reasoning-giving can be especially challenging when a government organisation has outsourced software development. In addition to a lack of expertise, the lack of open-source software and potential non-disclosure agreements can hinder technical transparency (e.g. sharing of source code). Thirdly, the complexity of rule-based systems is not only problematic for government agencies. Individuals who are met with automatically produced decisions need to be able to understand how they were produced to challenge them effectively. Fourth, and finally, even courts may lack the necessary expertise to properly assess the legality of automatically produced decisions.
What do courts say?
To get a grip on complex rule-based ADM, the Dutch highest administrative court ruled in 2017 that semi-automated decision-making with a calculator called AERIUS carries the risk of not being transparent and verifiable due to a lack of insight into its functionality and input data. The calculator, developed by the Dutch government, is a complex rule-based tool to determine nitrogen emissions of proposed projects. The court concluded that the complexity, opaqueness, and information asymmetry can constitute a violation of the right to a fair trial (inequality of arms, article 6 ECHR). Thus, to be able to assess the legality of administrative decisions, governments must, in individual procedures, disclose these choices and data assumptions proactively (or as later added, upon request) and in a manner that is full, timely, and appropriate.
Similarly, in September 2025, the Spanish Supreme Court issued a landmark decision on the rule-based BOSCO algorithm. The tool and its source code, created by the government and used by energy suppliers to assess energy subsidy eligibility for citizens, had to be disclosed following a public information request. In its ruling, the Supreme Court held that digital democracy warrants algorithmic transparency for public oversight, and that the constitutional right of access to public information outweighs intellectual property (IP) rights and potential security risks of publishing the source code. While rooted in the relevant legal provisions, outcomes of the tool were virtually incontestable due to a lack of information and reason-giving.
These examples demonstrate a need for algorithmic transparency for public scrutiny and contestability. Without, democratic principles and the right to a fair trial are undermined, not to mention the impact on public trust in government agencies and their decision-making processes. Yet even after AERIUS, agencies struggle to fulfil their obligation to offer the necessary insight. The scope and extent of the transparency requirements is not clear, and not every agency possesses the expertise required to offer it, resulting in insufficient reason-giving and inability to demonstrate procedural or substantive due care (as required by the General Administrative Law Act). Doing so can furthermore conflict with contractual obligations and IP-rights. In the case of BOSCO, the ruling also raises questions about which ADM systems fall under the obligation to disclose, and about the impact on public-private software collaboration (after a legislative change France now includes a similar obligation, yet some source codes were ruled not necessary to disclose). But what more can courts do than to command algorithmic transparency when they cannot assess the legality of decisions without it? What becomes of the phrase Da mihi facta, dabo tibi ius if the facts (and how they were inserted, tweaked, and calculated) are unclear?
The responsibilities for the tools used to execute the law lie with the relevant executive agencies. Are they capable of offering meaningful insight into complex rule-based ADM systems? I believe so. Let me explain why and how.
Knowledge modelling, in this day and age?
The idea of knowledge modelling and representation stems from early research in artificial intelligence and lead to the development of logic programming in the 1970s, which in turn was used as a methodology for trying to develop legal expert systems (programs aimed at emulating human expert behaviour to automate or assist in decision-making requiring expertise). The overarching assumption of these initiatives was the idea that domain knowledge could be represented in simple independent rules and facts, allowing computers to make inferences. For legal expert systems, this turned out to be easier said than done. In addition, the idea that law is static and reducible to simple logic statements was met with understandable criticism. Consequently, by the end of the millennium, much of the attention for the development of legal expert systems had ebbed.
Yet when juxtaposed with existing ADM systems, it seems that some areas of law have, after all, been reduced to software code. Even though the legal expert system has been forgotten, the practice of legal knowledge modelling is still ongoing, although not everywhere. Knowledge can be modelled in ontologies, knowledge graphs, taxonomies, and languages. For example, since 2009, the Dutch Tax Authority developed its own ‘Controlled Natural Language’, called RegelSpraak (based on the idea of Business Rule Management and its RuleSpeak language). This is a natural language with restricted grammar and vocabulary and clear instructions on the meaning of words. Legal rules are gathered and translated, and these translations are subsequently verified, validated and tested. This has some significant advantages. Firstly, the natural language is human-readable, meaning that software developers and legal professionals can collaborate on translations. Secondly, the language is machine-readable, enabling the Tax Authority to generate executable software code directly from these rules (see ALEF). Thirdly, these rules are technology-independent, meaning that the content or formulation of the rules is independent from the means of execution (manual or automated, and which software or hardware used) and that they are adaptable to changes in law. Fourthly, rules are linked with their legal sources, and the outcomes can be traced through every requirement and decision-tree node.
Going back to the mentioned problems of complex rule-based ADM systems, this type of knowledge modelling can offer some much-needed solutions. It enables collaboration from different disciplines; implementing changes requires little technical expertise, thereby making procurement a costly and unfavourable option, especially given that software code can be automatically generated. If agencies share the same methodology, expertise can furthermore be exchanged between them. Even without an extensive analysis of the requirements of transparency, explainability, and reason-giving (for that, see this paper), being able to demonstrate how a decision was produced and which datapoints have resulted in which outcomes, accompanied by the relevant legal sources and the used rule translations is a massive leap forward in giving insight into the functionality and outcomes of ADM systems. The lack of precisely that kind of meaningful insight results in problems of incontestability and lack of democratic oversight.
Next to RegelSpraak, there are numerous other examples of ‘Rules as Code’ tools currently being developed. Other examples include eFLINT, a domain-specific language for formalizing norms as executable specifications, with a corresponding ontology; Blawx, a drag-and-drop tool for digitizing legal knowledge; Catala, a domain-specific programming language that can generate software from legal texts; and Publicodes, a platform-based domain-specific language with a focus on explainability (here is an informative assessment of different methods). Furthermore, Regelrecht and SPRIN-D, together with Rulemapping Group, illustrate how the development of LLMs can assist such tools.
The worrying implications of the arrival of system-level bureaucracies remain as pertinent as ever, as it is the system – more so than the individual administrative decision – that dictates the legal position of individuals. Meanwhile, the administrative legal framework has lagged behind, which confronts us with the fact that contesting or assessing the legality of individual decisions is fruitless when the underlying system is complex, opaque, and undisclosed. That is why it is so important to think of new ways to open the black box and draw attention to long overdue structural changes to automated administrative decision-making.
Robbert Bruggeman
Robbert Bruggeman a doctoral candidate at the Department of Constitutional and Administrative Law at Leiden University. His research focuses on the intersection of digitalization within executive agencies and the pursuit of People-Centred administrative law.
