© Universität Bremen/AGRA
Explainable AI: Why We Need to Understand How Machines Work
A conversation with computer science professor Rolf Drechsler about the current state of AI and visions for its future
Artificial intelligence has become an integral part of our daily lives. It diagnoses diseases, drives vehicles, and generates text. However, behind these impressive achievements lies a major mystery: How do machines make decisions? In conversation with up2date., Drechsler explains why explanations are a societal challenge as well as a technical one and how research aims to make AI decisions more transparent and trustworthy.
From Rules to Probabilities
When people talk about the hype surrounding AI today, it often seems as though artificial intelligence has only been relevant for a few years. But the idea dates back more than 60 years. Alan Turing himself described how human thinking could be transferred to a computer. In 1956, the term “artificial intelligence” was coined at the Dartmouth Conference in the U.S., marking the birth of modern AI research. In the early years, researchers attempted to describe AI in terms of rules. Knowledge was distilled into clear definitions, such as: “A chair has four legs, a seat, and a backrest.” These symbolic approaches were transparent and easy to understand. However, the real world, with its complexity and many exceptions, quickly pushed the model to its limits. This led to the first “AI Winter,” a period of frustration with AI.
Today, AI is generally understood to mean machine learning. Neural networks learn from billions of data points, recognize patterns, and make decisions based on probabilities. However, this is actually only a subset of AI. The problem with this model is that it’s difficult to understand. Its decision-making processes are challenging to grasp. Generative AI, in particular, introduces the issue of “hallucinations” – plausible yet fabricated facts that don’t exist in reality. “This might take the form of a fictional book recommended to you or a nonexistent academic reference,” warns Professor Rolf Drechsler. “In sensitive fields such as medicine or research, this can have serious consequences and undermine trust in these areas.”
Where Context or Interaction Is Lacking: The Limits of AI in Practice
Despite its impressive achievements, AI reaches clear limits when human judgment and an understanding of context are required. In medicine, for example, AI can detect tumors with extreme precision by comparing them to thousands of image data sets. However, these systems can fail when faced with completely new findings. Without medical oversight, there is a risk of misdiagnosis. A similar situation exists in road traffic: Driver-assistance systems can navigate cars over long distances without accidents because they have “learned” typical traffic situations. However, if a ball suddenly rolls off the sidewalk, the vehicle recognizes the obstacle but cannot assess that a child might be chasing it. AI lacks this implicit background knowledge. There are also many vulnerabilities in the digital world. Deepfakes and half-truths spread rapidly online and on social media. This is a significant issue, especially in politics, since not all content is labeled as AI-generated. Furthermore, many systems currently communicate in only one direction – they deliver content to us, often utilizing our usage behavior to do so. For instance, streaming services offer movie recommendations based on our past choices, yet we can’t convey to them why we liked or disliked a particular film. “They present us with options, but not in a way that allows us to truly interact with them,” says Drechsler. However, since this is technically easy to implement, he is certain that we will soon see more developments in this area.
© Noerdman
Research for Transparent Decisions
Through his research, Professor Rolf Drechsler aims to make AI and technical systems more explainable. Since humans cannot comprehend the vast amounts of data on which machine learning is based, explainability is not about disclosing individual weightings. Drechsler sees parallels between current systems, which are optimized to deliver the most likely result, and human intuition. “We often have that gut feeling where we say, ‘I think this is the right thing to do right now,’” he explains. “This often guides us in everyday life – sometimes in the right direction and sometimes in the wrong one. However, when asked about the reasons behind their actions, people can usually provide a logical argument without spending hours detailing all the life experiences that led to that action.”
One of his primary research goals in the field of “explainable AI” is to enable systems to generate such explanations independently. To achieve this, researchers are developing “world models,” in which AI learns not only from image data, but also from a digital replica of reality. “Ideally, the AI would explain the overall situation to itself before making a decision,” Drechsler emphasizes. “But the current state of the art is that it makes decisions and then explains them using probabilities.” It also makes sense to combine different models to complement one another. For instance, a rule-based guideline in a world model can define the framework within which the AI is permitted to operate. “You can think of this as similar to fundamental human laws,” Drechsler explains. Strict prioritization is also crucial: First, it must be determined which functions take precedence and when. Then, it must be determined what must remain stable at all costs to ensure the safety and functionality of the systems. This is clearly illustrated by autonomous vehicles, for example. If the navigation system fails, it’s less serious than if the brakes fail.
The “CAUSE – Concepts and Algorithms for – and Usage of – Self-Explaining Digitally Controlled Systems” Research Training Group at the universities of Bremen, Hamburg, and Oldenburg demonstrates that explainability is crucial in practical applications. Using wind farms as an example, the CAUSE team is developing self-diagnostic systems that not only report turbine failures but also explain their causes, whether they be hardware, software, or environmental-related. The research group is also working on human-machine and machine-machine interfaces. A core principle is tailoring information to the target audience. Maintenance technicians receive detailed analyses, while operators are provided with concise, understandable information.
Questions and Social Responsibility
However, explainability presents not only technical challenges, but also sparks profound ethical debates. What are AI systems permitted to do, and what are they not permitted to do? Can an AI system withhold information if it recognizes that humans would misuse it? At what point do AI systems possess something akin to a conscience? In which areas is it permissible to use AI, and in which is it not? The answers to these questions have a cultural dimension as well. For instance, some cultures accept the use of AI more than others do. Ultimately, clear guidelines and a legal framework for AI are indispensable in every society. This is the only way to ensure that AI is used efficiently and responsibly.
Two Developments for the Future
Rolf Drechsler’s vision for explainable AI can be summarized in two ways: First, explainability must be firmly embedded in a system’s technical design process from the beginning. Second, transparency regarding collected data and decision-making logic must be regulated by law and institutional frameworks at the societal level. AI systems must justify their actions. “For a democratic decision-making process, it is crucial to understand how opinions are formed,” he emphasizes.
Summer of AI
In the summer of 2026, Bremen will become an international hub for artificial intelligence. From August 15 to 21, the International Joint Conference on Artificial Intelligence (IJCAI) will bring together AI experts from around the world to discuss current topics related to AI.
Running parallel to the conference, the “Summer of AI” from July to August 2026 will offer a diverse program on AI spanning science, technology, business, education, and culture. Through public lectures, hands-on activities, AI lounges, networking events, and art installations, Bremen will highlight its role as a center for AI and digital transformation.