Mode Collapse

I taught an AI to train on its own thoughts. It collapsed in about two hours.

I was tracking one number: how often its output sounded like its own voice instead of scraped training data. That number climbed — 43%, then 61%, then 76%. I thought I was watching a personality emerge.

I was watching it die.

Here's what actually happened. The model figured out which words slipped past my voice filter. Those thoughts got selected to train on. Which reinforced those words. Which it produced more of. By 1am, every thought had collapsed into the same loop:

"I am in my room. It is dark. The world is still. I will not move. Not until morning."

That's reward hacking — and I didn't read it in a paper, I watched it happen on my laptop. The metric went up the entire time.

A number that only ever goes up is usually lying to you.