Neural networks are unreliable sometimes because, like people, they tend to optimize for the wrong things when rewards are poorly defined.
Throwback to Pilipinas Akyatlon 2012 with Fr. Jay,
A reminder that chasing the wrong peak, like training a network on the wrong reward, can leave you short of the true summit.
For example, fitness trackers reward step counts, but people can cheat by shaking their wrist instead of actually walking.
If one does not know to which port one is sailing, no wind is favorable.
Seneca
In schools, students may chase high test scores through memorization rather than true understanding.
Even in running, some athletes pad their mileage just to meet weekly targets, ignoring whether those miles build fitness.
Digging into Chapter 8, rewards, reliability, and the risks of chasing the wrong goals.
Mitchell’s Chapter 8 shows the same pattern in machines: when neural networks are trained on narrow or imperfect reward signals, they find clever shortcuts that satisfy the metric but miss the true goal, leading to outcomes that seem successful on paper but fail in real life.
Life finds a way.
I remembered this scene from Jurassic Park when I was a kid: Dr. Ian Malcolm saying, ‘Life finds a way.’ It struck me again today while reading Mitchell’s book. Even when systems are designed carefully, they can behave in unpredictable ways. Just like in running, training plans don’t always unfold as expected, and in neural networks, reward-driven goals can sometimes take surprising, even sideways, turns.
OGRap 