When Machines Decide – Are EU countries prepared?

Profiling job applicants based on the contents of their personal email inboxes in Finland, deciding which patients get treatment in the public health system in Italy, sorting the unemployed in Poland, automatically identifying children vulnerable to neglect in Denmark, detecting welfare fraud in the Netherlands, credit scoring systems in many EU countries – the range of applications of automated decision-making (ADM) has broadened to almost all aspects of daily life. Many of them can provide valuable benefits to citizens, but their use also poses risks of unjust discrimination, intrusive monitoring or increased inequality.

For political decision makers it is almost impossible to keep track of which systems are used where, and for what purposes. At the level of the EU Commission and Parliament, and also in Member States, there is active discussion of issues raised by ADM. This discussion mostly centres around criteria to evaluate and govern such systems, i.e. using ethical guidelines. This is to be welcomed. At the same time, evidence of concrete uses of automated decision-making is crucial to develop realistic and actionable ideas to deal with the challenge. The report presents more than 60 examples from twelve countries, as well as an overview of relevant stakeholders and the political debate focusing on ADM.

The countries surveyed vary widely in meeting this challenge:

• Funding
  While in Spain (GDP 1,437 billion USD) government hands out a mere 4 million Euros via the Activa Industria 4.0 programme to support 400 companies to "advance their digital transformation and improve their competitiveness by adopting new enabling technologies", in Sweden (GDP 276 billion USD) one private foundation alone awarded 100 million Euros to two universities through the "Wallenberg Autonomous Systems and Software Program" to develop machine learning, AI, and the mathematical apparatus behind them.

• Experts commissions, civil society activities
  In Germany and the UK, in addition to a number of government and parliamentary commissions, data protection authorities, business associations and NGOs look into the consequences of ADM. In countries like Poland, Slovenia or Italy, automated decision-making systems are used in practice but almost invisible in the political debate.
  
  
• Legislation and oversight
  In France a law mandates all branches of government to make their algorithms transparent but no one complies. In Finland, the Non-Discrimination Ombudsman brought a case to a tribunal that banned a company from using a certain credit scoring model because the tribunal deemed it discriminatory, threatening a 100,000 Euro fine for non-compliance.

The report for the first time not only shows the pervasiveness of ADM systems, it also reveals how varied and inconsistent are the efforts to deal with them in different countries. Policy makers and civil society should use the opportunity of these results to compare the situation in their countries with others and put their own approaches to the test.

"Ideas of machine-created 'artificial super-intelligence' are all the rage right now, but practically irrelevant," says Matthias Spielkamp, AlgorithmWatch’s executive director and editor of the report. "What’s instead crucial to understand are the current challenges to our societies, like 'predictive analytics' used for forecasting human behaviour, be it in elections, criminal activity, or of minors. We urgently need to ensure that our institutions, regulation and oversight procedures are up to these challenges."

With respect to the recommendations of the report, Ralph Müller-Eiselt, our digitization expert, argues "to close the policy-gap between the Member States of the European Union, Europe needs to join forces and speak with one voice when it comes to setting standards for automated decision making." He sees a particular necessity in empowering public administration: "Administrations need to catch-up and adapt to these new policy challenges in order to put new technology at the service of society."