Date/Time
Date(s) - 18/03/2026
5:00 pm - 6:30 pm
Categories
March 18, 17-18:30 (CEST): Ecologies of evaluation: Sites and registers of machine learning evaluation in a public sector context
This talk develops the concept of “evaluation ecologies” to theorize how machine learning (ML) systems are assessed in public sector contexts. Through a conceptual analysis supported by case studies of ML deployments in Danish higher secondary education and Dutch psychiatric clinics, we demonstrate how evaluation practices extend beyond technical assessment to encompass complex negotiations of power, expertise, and accountability. Drawing on theoretical perspectives from Science and Technology Studies (STS) and building upon Halpern and Mitchell’s (2023) work on experimental governance and Amoore’s (2020) analysis of cloud ethics, we advance “evaluation ecologies” as a framework for understanding how ML assessments unfold through multiple, often contradictory registers. The Danish case reveals how legal and ethical registers can override technical considerations, leading to an algorithm’s termination despite its technical robustness. Conversely, the Dutch case demonstrates how practical utility and policy alignment can become dominant forces despite professional reservations about algorithmic predictions. Our analysis demonstrates that ML evaluation in public institutions is shaped not only by technical metrics but by complex interactions between professional discretion, legal frameworks, ethical considerations, and political objectives. We argue that successful ML implementation requires attention to these broader ecologies of evaluation, where different registers—technical, legal, ethical, and professional—compete for influence. This ecological perspective contributes to debates about algorithmic governance by showing how local contexts and professional expertise remain crucial despite what Amoore (2023) terms the “dissolution of the situatedness of data.” The article advances both theoretical understanding of ML evaluation politics and practical insights for public sector implementation, suggesting that evaluation frameworks must evolve beyond rules-based approaches to account for the complex socio-political contexts in which ML systems operate.
Helene Friis Ratner is a full professor of organization studies and technology at The Technical University of Denmark, DTU Management. Combining science and technology studies with organization studies, she researches how digital data infrastructures, data visualizations, and AI applications transform welfare organizations. Her research is published in journals such as Big Data & Society, AI & Society, and Organization Studies. She is currently co-PI of the Algorithms, Data and Democracy project (VELUX foundations) and REPAI – Responsible AI for Value Creation (Grundfos) as well as chief scientist in Denmarks National Centre for AI in Society (CAISA).
Nanna Bonde Thylstrup is an Associate Professor on the Promotion Programme in Modern and Digital Culture. She is PI of Data Loss: The Politics of Disappearance, Destruction and Dispossession in Digital Societies (DALOSS) funded by the European Research Council. The project is premised on the idea that datafication is inherently conditioned by loss, and that this loss can also be generative. Rather than framing loss retroactively as something that can be ‘fixed, patched or recovered’, then, DALOSS investigates loss as actively constituted through social, political, and aesthetic relations.
The invited talk by Nanna Thylstrup and Helene Ratner takes place online and is open to everyone.