Tobias Krafft

• The advancing digitalization has led to the widespread use of algorithmic decision-making (ADM) systems in various sectors of our lives. While these systems have proven beneficial to optimizing workflows and decision-making processes, their use in assessing and classifying individuals can have significant impacts on the respective lives. Public stakeholders studying these adverse effects often face the challenge of examining ubiquitous, but opaque ADM systems from the outside. In critical sectors, like healthcare, consumer protection, or safety, the impact of algorithmically governed platforms is technologically complex to address within traditional research modalities, which poses a major obstacle to oversight. Therefore, (1) barely any analyses can be performed by researchers operating outside the field of computer science, because of the necessary technical knowledge. (2) It is also unclear whether an analysis of opaque ADM systems without knowledge of internal functions is even possible and sufficient, in order to investigate the cause of a negative outcome that raised suspicions. So how do we bridge the gaps of knowledge between those experts trying to analyze the potential dangers of ADM systems and those computer science researchers accustomed to employing methodologically grounded tools like black-box analyses? How can we empower non-computer science stakeholders like patient and consumer advocates, charities/NGOs, regulators and centres of social science and biomedical research to undertake more complex investigations into the ADM systems that shape their respective fields? Which information and access is required to be able to carry out appropriate analyses at all? This dissertation focuses on targeting those questions by presenting a process model for investigating ADM systems as a black box, building on traditional black-box analysis approaches. The model is both accessible and adaptable for non-computer science domain experts and has been developed, tested, and discussed in multiple studies, each based on a different use case in different disciplines. Depending on the possible access to an ADM system, the presented process model can be used to investigate black-box systems with regard to a variety of questions. Additionally, it shows regulating authorities indirectly how costly external analyses are without direct access provided by the platform operators. The model developed in this dissertation empowers expert actors within diverse fields of civil society to find answers to questions with far-reaching social and societal consequences, like whether Google's search results aim for political manipulation, or whether medical advertisements try to exploit the susceptible part of the population. The limitations of the process model are presented and discussed from social, technical and legal perspectives.Understanding and addressing these limitations is essential for conducting effective and reliable black-box analyses. Furthermore, the studies show, that there are limitations that cannot be solved by methods of black-box analysis alone and therefore the political implications of this research are significant, particularly in the context of increasing interest and regulatory efforts in enhancing the transparency and accountability of algorithmic systems. The process model aligns with these initiatives and provides concrete guidelines for promoting black-box analyses. Additionally, the recommendations for further research and political action highlight the need for strengthened rights of data subjects, establishment of suitable interfaces for investigation, legal certainty for black-box analyses, and a watchdog approach for continuous monitoring and evaluation of ADM systems. As society continues to grapple with the challenges and opportunities presented by ADM systems, the insights and methodologies presented in this dissertation contribute to a more comprehensive and critical understanding of these systems. By fostering interdisciplinary collaboration and promoting a distributed approach to analyzing ADM systems, this research aims to shape a society that can navigate the complexities of algorithmic decision-making while safeguarding fundamental rights and values.