When language models remember too much

Large language models learn from enormous amounts of information. But what happens when they remember things they should not – copyrighted books, personal information, or data someone has asked to be deleted?

These questions shaped Prof. Ivan Habernal’s keynote at an international Pre-EMNLP Symposium at TU Darmstadt. In “The dark side of LLMs: On anonymity, privacy, copyright infringement, unlearning, and safety,” he showed why protecting information in language models is rarely a matter of applying one technical fix.

What does an LLM actually remember?

Copyright is one example. Researchers may try to determine whether a particular book was part of a model’s training data. Membership inference can provide evidence, Ivan Habernal explained, but not definitive proof – especially when researchers have little insight into a model’s original training data.

Similar problems arise with machine unlearning, which aims to remove the influence of selected information without retraining an entire model. Such interventions can work, but they may also change model behavior elsewhere, including aspects of safety.

Anonymization poses another challenge. If an LLM can identify a person from anonymized text, the obvious conclusion is that the anonymization failed. But if the model already encountered the original text during training, it may simply be recalling what it already knows.

That makes reliable evaluation difficult: public data may already be part of training sets, synthetic data has limitations, and truly private data is difficult to share.

Ivan Habernal also joined an international fireside chat on methodological priorities for AI for Science, broadening the discussion to a larger question: how much should researchers delegate to AI systems whose limitations they may not fully understand?

A stop on the way to EMNLP

The symposium took place on October 22–23, 2026, at TU Darmstadt under the theme “The Future of Science: AI-First, Trustworthy, Human-Steered.” It was co-organized by the Zuse School ELIZA, the ELLIS Unit Darmstadt, and the Ubiquitous Knowledge Processing (UKP) Lab at TU Darmstadt.

TrustHLT was also represented shortly afterwards at the Natural Legal Language Processing Workshop (NLLP 2026) during EMNLP in Budapest. Two papers co-authored by Ivan Habernal were presented there:

“Legal Experts Disagree With Rationale Extraction Techniques for Explaining ECtHR Case Outcome Classification” by Mahammad Namazov, Tomáš Koref, and Ivan Habernal, and “What Drives an LLM’s Legal Reasoning? Measuring Faithfulness at the Concept Level” by Yassine Thlija, Lena Held, and Ivan Habernal.

Note: A recording of the symposium is available on YouTube. Ivan Habernal’s keynote starts at approximately 7:30:00.