# Data contamination

> When test questions leak into a model’s training data, making its scores look better than they should.

Source: https://www.superintelligence.com.au/glossary/data-contamination/

Data contamination is what happens when a model has already seen the answers to a test during training. Modern models learn from huge collections of text gathered from the internet, and test questions and their answers are often posted online too. If they end up in the training material, the model may recall them rather than work them out.

The everyday comparison is a student who finds the exam paper in advance. A high mark tells you about their memory, not their understanding. With AI it is harder to spot, because the training collections are enormous and often not published in full.

Contamination matters because benchmark scores are a main way of judging progress. If some of the scores are inflated, comparisons between models become unreliable. Test builders respond in several ways: keeping questions private, writing new ones regularly, or checking for overlaps with training text. None of these is perfect, and it is often hard for outsiders to tell how much contamination there has been. Researchers also disagree about how much it distorts the overall picture of progress.

Related: https://www.superintelligence.com.au/glossary/benchmark.md, https://www.superintelligence.com.au/glossary/evaluation.md, https://www.superintelligence.com.au/glossary/benchmark-saturation.md, https://www.superintelligence.com.au/glossary/pre-training.md