AI trained on AI output eventually misperceives reality.

Human data isn't just valuable; it's a technical necessity.

The weird, the original, and the unexpected that gets lost from recursive AI training may be gone forever.

Two words in a recent paper from Oxford and Cambridge stopped me cold.

The researchers were describing what happens to artificial intelligence (AI) models trained repeatedly on AI-generated data. They called it "model collapse." And in explaining the late stages, they wrote that the models "mis-perceive reality."

Two words, and a mountain of implications. This isn't a hallucination, where a model invents a fact or a false citation. Those errors tend to be discrete and correctable. What the paper describes is a system so thoroughly trained on its own output that it has lost contact with the original information. The AI is still confident and fluent but increasingly untethered from reality.

Losing the Tails of the Distribution

The paper's findings are fascinating........

© Psychology Today