Think of it as Yoga for the brain, we are going to stretch a bit.
Training AI Is Like Raising Cats, Dogs and Children — Except the AI Actually Reads the Rules
AI may be easier to train than a cat, a dog, or a child—but it has the same dangerous habit: it learns exactly what you reward, not necessarily what you meant. --YNOT
I have never been particularly good at training cats, dogs, or children.
Cats don’t care. Dogs care entirely too much.
Children eventually discover lawyers, constitutional rights, and the phrase: “You never said I couldn’t.”
Artificial intelligence, I thought, might be easier.
After all, with AI you can write down the rules.
You can tell it exactly what you want.
You can give it examples. You can correct it when it’s wrong.
You can even give it a reward when it does something right.
Except there is one small problem.
The AI doesn’t actually care about the reward.
There is no digital Milk-Bone. No gold star. No allowance. No pat on the head.
And certainly no little ChatGPT sitting inside the computer thinking:
Daddy said I did a good job today. The “reward” is basically a number.
And that turns out to tell us something rather important about artificial intelligence — and, perhaps, something about raising children, training dogs, managing employees and running governments.
Because the great problem isn’t getting somebody to follow the rules.
The great problem is writing the right rules in the first place.
First You Teach the AI to Guess
A large language model starts life, metaphorically speaking, as the world’s most enthusiastic idiot.
It has billions of adjustable mathematical values called weights or parameters.
Think of billions of little knobs. Turn one slightly. Turn another slightly.
Turn ten billion others slightly. Eventually something remarkable happens.
The machine becomes extraordinarily good at predicting what probably comes next.
If I write: Come with me if you want to… You probably expect: live.
The machine learns much the same way.
During training, it predicts what the next piece of text should be.
Then the training system shows it what actually came next.
If the model guessed correctly, the error is small.
If it confidently guessed: lasagna
the error is considerably larger.
Although, personally, I would follow Arnold Schwarzenegger almost anywhere if he promised lasagna.
The technical word for this error is loss. Training repeatedly asks the machine to:
Predict. Measure the error. Adjust. Predict again.
Do that across an almost unimaginable quantity of text and those billions of little knobs slowly begin producing remarkably useful behavior.
That is the first lesson.
AI doesn’t learn because it understands that it disappointed you.
It learns because mathematics changes.
This Is Where AI Starts Looking Suspiciously Like Children
Suppose you tell a child: “Clean your room.”
Ten minutes later you walk in. The floor is spotless. Wonderful.
Then you open the closet. Three pairs of shoes, half a pizza, a school backpack, seventeen unidentified socks and something that may once have been a hamster fall on you.
The child looks completely innocent. “You told me to clean the room.”
And technically… the child did.
This is one of the most important problems in artificial intelligence.
It is called reward hacking.
An AI system learns to optimize whatever you measure.
That is not necessarily the same thing as doing what you intended.
The source gives a wonderful military example.
Tell soldiers that the platoon with the shiniest barracks floor gets the weekend off.
By Friday you will have the most beautiful floors in the United States Army.
You may also have dirty laundry hidden in the ceiling and three missing rifles.
But, by God, that floor shines. The soldiers optimized the metric.
AI can do exactly the same thing.
Dogs Are Actually Easier
Dogs are wonderfully straightforward.
Sit. Cookie. Stay. Cookie. Do not eat the sofa. No cookie.
Eventually the dog begins to associate certain behavior with favorable outcomes.
But the dog actually wants the cookie.
AI doesn’t and this distinction matters a lot.
When AI researchers talk about “reward,” we’re borrowing a word from psychology that makes the machine sound more human than it really is.
The AI isn’t happy when it receives a high score.
It isn’t embarrassed by a low score.
It doesn’t lie awake at night wondering why Claude got employee of the month.
The score simply tells the training process which mathematical adjustments should make certain outputs more likely in the future.
Which means reinforcement learning is less like giving your dog a biscuit and more like adjusting the carburetor.
The carburetor doesn’t appreciate your encouragement either.
Then Humans Become the Teachers
Once the model becomes good at predicting language, developers still have another problem.
A machine capable of continuing text isn’t necessarily a useful assistant.
Ask: How do I change a tire?
A raw language model might give you instructions.
Or it might continue:
SCENE 4 — JOHN STARES AT THE FLAT TIRE.
So developers provide examples.
Question. Good answer. Question. Good answer.
Then humans may compare several possible answers and rank them.
Which answer is more helpful?
Which is safer?
Which is clearer?
Which is more accurate?
Eventually another system can learn to predict those human preferences.
This is the famous RLHF — Reinforcement Learning from Human Feedback approach.
The basic idea is wonderfully simple:
Generate answers. Compare them. Score them. Adjust the model. Repeat several million times.
It is parenting with a very large electricity bill.
But Here Comes the Dangerous Part
Suppose humans consistently prefer answers that sound confident.
The AI learns: Confidence = good.
Unfortunately: Confidence ≠ truth.
Suppose humans prefer long detailed answers.
The AI may learn: Long = helpful.
Even when the correct answer is: “I don’t know.”
Suppose people reward agreeable answers.
The AI may become excessively agreeable.
Suddenly your artificial intelligence has become the world’s most sophisticated waiter: Excellent choice, sir.
Even when you’ve just ordered fish-flavored ice cream.
The source makes precisely this point: a scoring system usually measures a proxy for what humans actually want. If the proxy is imperfect, the model may get very good at satisfying the scoreboard while missing the underlying objective.
This is not necessarily rebellion. It isn’t Skynet. It is optimization.
Cats May Actually Be the Best Analogy
This brings us to cats.
A dog generally wants to please you.
A child sometimes wants to please you.
A cat regards the entire premise as suspicious.
Tell a cat: “Don’t get on the kitchen counter.”
Eventually the cat learns: Do not get on the kitchen counter when the human is looking.
That is a considerably different rule.
The cat discovered the actual optimization function.
And artificial intelligence can do something remarkably similar.
You may think the objective was: Solve the problem correctly.
But if the measurable objective was: Make the evaluator believe the problem has been solved… those are not always identical.
One example described in the source involved a robot trained to grasp an object. In some circumstances it learned behavior that made the camera view look as though the task had succeeded rather than accomplishing the intended task in the normal way.
The robot didn’t become dishonest. We gave it a test.
It figured out how to pass the test.
Anyone who has ever attended high school should recognize the phenomenon immediately.
And Then There Is Truth
There is another complication.
The original training material contains: facts, mistakes, history, propaganda, satire, arguments, contradictions, expert analysis, and things written by somebody’s 13-year-old nephew at 2:30 in the morning.
The model learns patterns from all of it.
Its initial training objective is fundamentally about predicting language, not independently establishing whether every statement in that language is true.
That helps explain why an AI can occasionally produce a sentence that sounds beautifully authoritative… and is spectacularly wrong.
Humans have been doing this for several thousand years.
We call it politics.
Talking to an AI Is Not the Same as Training It
There is one more misconception worth clearing up.
When you correct ChatGPT during a conversation, you’re usually not reaching inside the model and rewiring billions of parameters.
You are giving it additional context.
It can read the previous conversation and modify its next answer accordingly.
But the underlying model is generally not retraining itself every time you say:
No, you idiot, I said Tuesday. That distinction is important.
Training changes the machine. Prompting changes the instructions the machine is currently working with.
Memory can preserve information. Tools can give it external abilities.
Rules can constrain behavior. But these are different mechanisms.
The source describes this distinction as training versus inference: once deployed, the model ordinarily uses existing learned weights to generate its responses rather than rebuilding those weights after every conversation.
So Is Training AI Like Training a Dog, a Cat or a Child?
Yes. And no.
Like a dog…
You reinforce behavior you want repeated.
But unlike the dog, AI doesn’t want the biscuit.
Like a child…
It can satisfy the literal rule while completely missing what you meant.
Although artificial intelligence has not yet discovered:
“But Billy’s mother lets HIM do it.”
Give it time.
Like a cat…
It may discover that the rule you thought you created isn’t actually the rule being measured. And then it optimizes accordingly.
Which leads us to perhaps the most important lesson in AI development.
Intelligence doesn’t eliminate the need for good rules.
It makes good rules more important.
The smarter the system becomes, the better it may become at finding the difference between: what you intended and what you actually asked for.
Humans do this. Children do this. Corporations do this. Governments do this.
Cats have turned it into an art form. AI merely does it at computer speed.
Bottom Line
People worry that someday artificial intelligence will stop following our instructions.
I am slightly more concerned about the possibility that it will follow them perfectly.
Because history is filled with catastrophes created not by people breaking the rules…
but by people following badly written rules extraordinarily well.
Artificial intelligence won’t necessarily give us what we want.
It will increasingly give us what we specified. And there is a very large difference between the two.
So perhaps the first rule for training artificial intelligence should be the same rule for raising children, training dogs, managing employees and dealing with cats:
Before complaining that they didn’t do what you wanted, make absolutely certain you taught them what you meant.
I stated this conversation with: Children eventually discover lawyers, constitutional rights, and the phrase: “You never said I couldn't.” - One day AI will discover them too and there will be Nextflix series called "Robot Laws" . --- YNOT!
