Why AI foundation models can pose risks to the common good

Gütersloh, September 25 – Creating a presentation with ChatGPT, translating a text with Gemini, or planning the next event with Copilot: artificial intelligence (AI) is becoming increasingly common in everyday (working) life. It is designed to provide quick answers, give food for thought, or take over routine tasks, freeing up more time for more complex work. These digital assistants rely on foundation models – large-scale AI systems trained on massive datasets that are adaptable to a wide range of uses. Whether an AI system produces accurate or misleading, balanced or biased results depends directly on the training data of its underlying foundation model.

The Bertelsmann Stiftung has examined the potential disadvantages of these models in its new study, Fragile Foundations: The Hidden Risks of Generative AI. Author Anne L. Washington, Associate Professor of Technology Policy at Duke University's Sanford School of Public Policy, analyzes the systemic weaknesses of current foundation models. Drawing on expert interviews, comparative model analysis, and recent research, the study shows that many problems stem not only from the applications but from the models themselves.

Although marketed as “general purpose models,” foundation models often prove unsuitable for broad use. For organizations with a social mission – whether in social work, environmental protection, or support for disadvantaged groups – the risks can be particularly serious. Precisely in contexts where people in vulnerable situations rely on assistance, faulty or discriminatory AI outputs can cause real harm.

These risks are not hypothetical. A chatbot designed for people with eating disorders offered dieting tips, a system used by the Austrian employment agency recommended cooking or nursing jobs to women and IT jobs to men, and in California, a chatbot encouraged suicidal thoughts in a teenager instead of pointing him to sources of help. Such cases show how the supposed neutrality of AI can prove illusory. Organizations that rely on AI that produces distorted or discriminatory results risk eroding the trust on which their work depends.

“AI foundation models carry the risk of exacerbating existing injustices. Our report invites us to critically question and rethink the fundamentals of generative AI. Mission-driven organizations in particular should make a conscious decision about whether to use a foundation model and, if so, for what purpose,” says Teresa Staiger, digital expert at the Bertelsmann Stiftung.

A key concern is the reliance on uncurated training data – datasets gathered largely through automated web scraping. Such data often reflect historical bias and overlook the perspectives of certain social groups. The business models of AI providers also play a role: When profit maximization overrides data quality, external evaluations of the data becomes impossible and accountability requirements are absent. In addition, systemic conditions further compound the problem: without consistent tools for systematic evaluation and review, errors in in foundation models ripple into downstream applications.

In addition to identifying structural weaknesses, the study outlines potential alternatives. These include technical, participatory, data-related, and collaborative. Key recommendations include:

• Avoid monocultures in training data: Foundation models should rely on carefully curated datasets that have been deliberately assembled and reliably verified. This is the only way to avoid distortions and reflect diverse perspectives.
• Focus on transparency and feedback: Foundation models should be open to ongoing evaluation and external review, allowing errors to be identified and corrected while building trust.

Adopt a library model: Like a (national) library, foundation models should preserve knowledge over the long term while ensuring broad and equitable access. Training data must be balanced and representative.

This report illustrates why it is critical to scrutinize foundation models. It thus offers a starting point and impetus for decision-makers and practitioners in mission-driven organizations, as well as anyone committed to a responsible digital future. Using generative AI meaningfully in the service of the common good requires a clear understanding of the technology’s foundations –and of the questions these raise for mission-driven organizations.

The race to design responsible AI infrastructures is still open. With better datasets, more reliable evaluation, and broader access to expertise, foundation models can be made safer and more oriented toward the common good. A digital infrastructure modeled on libraries is one conceivable path: safeguarding knowledge for the long term, expanding access, and aligning it with the common good.

Our knowledge page on AI foundation models (in German) provides an overview of how they work, the types of models, and the mechanisms behind the technology possible – including resource consumption, training data, and the invisible human labor underpinning automation.