Checking the AI that checks chemical processes

In a chemical plant, trouble often begins quietly. A pressure value drifts. A temperature curve no longer follows its usual rhythm. A sensor reports something that may be harmless – or the early sign of a serious process failure. Detect such anomalies too late, and the consequences can affect people, the environment, and production safety. Detect them too often, and false alarms can stop processes unnecessarily, wasting time, material, and money.

This narrow corridor between missed warnings and false alerts is where the DFG Research Unit FOR 5359 – “Deep Learning on sparse chemical process data” is working. Coordinated at RPTU Kaiserslautern-Landau, the research unit has now entered its second funding phase. Together with colleagues from Kaiserslautern, Prof. Daniel Neider has secured almost 500,000 euros for his subproject.

FOR 5359 addresses a problem that makes chemical processes a difficult test case for artificial intelligence. Deep learning often thrives where data is abundant. In chemical engineering, however, relevant data can be sparse, proprietary, expensive to obtain, or simply rare because critical events do not occur frequently – and should not occur frequently. At the same time, many processes run under stable conditions for long periods, producing measurements that look deceptively similar.

The research unit brings together expertise from computer science, machine learning, visualization, mathematical modelling, optimization, thermodynamics, and chemical engineering. Partners and contributors include RPTU Kaiserslautern-Landau, TU Dortmund University, the Fraunhofer Institute for Industrial Mathematics ITWM, TUM Straubing, and a Mercator Fellow from the University of California, Irvine.

Daniel Neider’s role begins with a second, crucial question. If neural networks are meant to detect anomalies in safety-critical chemical processes, how can we know whether the neural networks themselves are reliable?

Neider is Professor for Verification and Formal Guarantees of Machine Learning at the Department of Computer Science at TU Dortmund University and a member of the Research Center Trustworthy Data Science and Security (RC Trust). His subproject, “Verification of Anomaly Detectors”, focuses on the formal verification of neural networks used in process engineering. In simple terms, it asks how AI systems can be checked with mathematical rigour before they are trusted in demanding real-world settings.

In the new funding phase, the project will extend verification methods to more complex neural network architectures, including generative models. It will also lower the barrier for applying verification tools by developing an interactive assistant based on large language models, designed to support experts from process engineering and machine learning. A third strand aims to identify the causal sources of errors in neural networks – and to develop ways of correcting them systematically.

For RC Trust, the project touches the core of what trustworthy AI means in practice. Reliability is not something that can be added after a system has been built. It must be examined, tested, explained, and improved.

In chemical processes, this is more than a technical question. It is a question of safety, responsibility, and confidence in systems that may one day help prevent accidents before humans can see them coming.