ML & Deep Learning Fundamentals

Let's be clear about who this is for: you can write code and call LLM APIs, but you never took a proper machine learning course, and whenever a live session mentions "loss function", "gradient descent" or "overfitting" you start falling behind. This chapter isn't trying to turn you into an ML researcher -- it fills that gap, with real numbers and two runnable code snippets, so that fine-tuning, evals and RAG later on land on solid ground.

Honestly, in AI application engineering, 90% of the time you won't train models yourself. But the remaining 10% -- knowing what a fine-tuning dataset should look like, spotting when a model is "memorizing answers" (overfitting), understanding why an eval report uses F1 instead of accuracy, holding a conversation with data science colleagues without exposing yourself -- all of it rests on these fundamentals.

I learned this the hard way. The first time I helped a client with a classification fine-tune, their data scientist asked me "has the validation loss converged?" I mumbled for three seconds, and their tone instantly shifted -- they started explaining things to me like I was an intern. Those three seconds of mumbling are exactly what this chapter fixes.

One reassurance up front: this chapter needs nothing beyond middle-school math. Addition, subtraction, multiplication and squaring -- that's it. No matrix calculus. Read on without fear.

---

Why do AI engineers need this? (What breaks if you don't)

• The fine-tuning session will ask you to prepare a "training set / validation set". If you don't understand the split, you'll evaluate yourself on the training data, get an absurdly inflated score, and crash on launch.
• When someone says a model is "overfitting", you need to know it's not an insult -- it means "memorized the question bank, can't generalize".
• A classification report says "accuracy 98%" and you should be able to tell at a glance that it might be a useless model (explained below).
• LLM pretraining is "self-supervised". If you don't know the supervised/unsupervised distinction, you can't appreciate why that term is clever.
• When the loss curve on your fine-tuning dashboard suddenly goes NaN, you need to know the first parameter to touch (spoiler: learning_rate).

---

One sentence: machine learning is "feeding examples", not "writing rules"

In traditional programming you hard-code rules: if email contains "You won!" then spam. You can never write enough rules -- scammers change one word and you miss them.

Term: Machine Learning (ML)

• One-sentence explanation: instead of writing rules, you give the program a pile of examples with answers and let it work out the patterns itself.
• Analogy: teaching a kid to recognize cats. You don't recite "a cat has four legs and two pointy ears..." -- you point at a bunch of pictures saying "this is a cat, this is a dog", and after enough of them the kid just knows.
• How you'll use it at work: spam detection, recommendations, fraud detection, and the training underneath every LLM.
• Most common pitfall: biased examples (data) produce a warped model. If your data is all orange tabbies, the model may decide black cats aren't cats -- "garbage in, garbage out".

A few words you'll use daily, cleared up once:

Term: Feature & Label

• One-sentence explanation: features = the clues you judge from; label = the answer you want.
• Analogy: estimating a house price. Floor area, location, and floor number are features; the sale price is the label.
• How you'll use it at work: supervised learning is literally learning the "features → label" mapping.
• Most common pitfall: letting something that indirectly reveals the answer sneak into the features (data leakage). Training scores look terrifyingly good, production is a disaster. A real example: predicting user churn with "account deletion date" accidentally in the features -- of course it's 100% accurate, but at inference time that field doesn't exist yet.

---

Three learning paradigms: the difference is "do you have answers"

Supervised learning -- the most common. Give it "questions + answers" to learn from. Two flavors: if the answer is a category (spam/not spam) it's classification; if the answer is a number ($3.28M house price) it's regression.

Unsupervised learning -- data without answers; the model finds structure on its own. For example, automatically splitting ten thousand users into groups (clustering) when you don't know in advance how many groups there should be or what they mean.

Reinforcement learning -- no correct answers, guided by rewards. AlphaGo gets points for winning. The reason ChatGPT feels "well-behaved and doesn't spout nonsense" is a step called RLHF (Reinforcement Learning from Human Feedback) -- humans score responses, and the model learns to move toward "what humans prefer".

⚡ Tying it together: LLM pretraining is "self-supervised" -- take massive amounts of text, mask the next word, and make the model guess it. The answer is the actual word from the original text, so you get "correct answers" without any human labeling. That's why the entire internet can serve as training data.

---

How a neural network "learns": turning knobs

Term: Neural Network

• One-sentence explanation: a pile of adjustable "numeric knobs", stacked in layers, gradually transforming input into output.
• Analogy: rows of mixing-console knobs. Sound goes in, passes through layer after layer of knobs, and comes out with the effect you want -- training is automatically finding where each knob should be set.
• How you'll use it at work: a Transformer is a neural network (with attention); fine-tuning is turning these knobs.
• Most common pitfall: believing it "understands". It has merely turned the knobs to positions where "input goes in, output happens to be right" -- there is no human-style understanding.

The learning (training) loop, in plain language, is four steps repeating forever:

① Forward: data goes in, current knobs produce a prediction
      ▼
② Measure the gap: prediction vs true answer, the difference = Loss
      ▼
③ Backward: work out "which way to turn each knob to shrink the gap" (Backpropagation)
      ▼
④ Nudge the knobs: turn a tiny step in that direction (Gradient Descent)
      ▼   (back to ①, repeat tens of thousands to millions of times)

Memorize these four steps. In the PyTorch code later, each step maps to exactly one line.

Loss function: first learn to score "how badly wrong"

Term: Loss Function

• One-sentence explanation: a single number measuring "how far this prediction is from the correct answer" -- smaller is better.
• Analogy: exam deductions. The more absurd the answer, the more points deducted; loss is the total deduction.
• How you'll use it at work: during training/fine-tuning you watch the loss curve -- it should go down and flatten out.
• Most common pitfall: using the wrong loss for the task. Regression uses MSE (Mean Squared Error), classification uses Cross-Entropy -- and an LLM predicting the next token is just a gigantic classification problem, which is exactly why it uses cross-entropy.

A mini MSE walkthrough (predicting house prices):

Actual:  $300k        Predicted: $280k → error 20, squared = 400
Actual:  $500k        Predicted: $520k → error 20, squared = 400
Actual:  $200k        Predicted: $260k → error 60, squared = 3600
MSE = (400 + 400 + 3600) / 3 = 1466.7

Notice the third row: error 60, squared to 3600, dominates the total loss -- that's MSE's personality: it punishes wildly wrong predictions hard. Training is turning knobs over and over to push this number down.

Gradient descent: walking downhill blindfolded

Term: Gradient Descent

• One-sentence explanation: the knob-tuning method of stepping toward "downhill" bit by bit until you reach the lowest loss.
• Analogy: descending a mountain blindfolded. You feel which direction slopes down (the gradient), take a small step that way, repeat, until the ground under you is the lowest point (minimum loss).
• How you'll use it at work: every neural network / LLM training run relies on it; the learning_rate you set is "how big each step is".
• Most common pitfall: steps (learning rate) too big -- you leap across the valley and oscillate back and forth, loss jumps instead of dropping; too small and it's painfully slow. When fine-tuning loss suddenly spikes or turns NaN, nine times out of ten the learning rate is too high.

Compute one by hand: three steps toward the answer

"Take a small step downhill" is too abstract. Let's hand-compute the tiniest possible example -- you'll see the whole thing is middle-school math.

Setup: the model has one knob w, prediction rule prediction = w × x. One data point: x = 2, true value y = 10 (so the perfect answer is w = 5). Loss is squared error, learning rate lr = 0.1. The gradient formula is 2x × (wx − y) (don't worry about the derivation -- just know it points "uphill", and we walk the opposite way):

Step 1: w = 1.00 → predict 2.0,  off by -8.0  → gradient -32.0 → w becomes 1 + 0.1×32   = 4.20
Step 2: w = 4.20 → predict 8.4,  off by -1.6  → gradient -6.4  → w becomes 4.2 + 0.64   = 4.84
Step 3: w = 4.84 → predict 9.68, off by -0.32 → gradient -1.28 → w becomes 4.84 + 0.128 = 4.97

Three steps, from 1 to 4.97, nearly at the correct answer 5 -- and the steps automatically shrink as you approach the valley floor (smaller gap means smaller gradient). That's the entire character of gradient descent.

Now change the learning rate from 0.1 to 0.6, same data point:

Step 1: w = 1.00   → gradient -32.0  → w = 1 + 0.6×32       = 20.20   (leaped right past 5)
Step 2: w = 20.20  → gradient 121.6  → w = 20.2 − 0.6×121.6 = -52.76  (flung even further the other way)
Step 3: w = -52.76 → gradient grows again → w flies off into the hundreds...

See that? A step size that's too big doesn't mean "converges a bit slower" -- it means total divergence: every step leaps over the valley and lands higher on the opposite side. From now on, when you see a fine-tuning loss curve climbing instead of falling, or turning NaN, these three lines should flash in your head.

My blunt advice: tune learning rate in factors of 10 (1e-3 → 1e-4 → 1e-5). Don't fiddle from 1e-3 to 9e-4 -- that's scratching an itch through a boot, a waste of your time.

---

Words you'll hear constantly during training

In any meeting with data science colleagues, or on any fine-tuning dashboard, these words are unavoidable. One table and done:

A crude but reliable way to keep parameters and hyperparameters apart: what the model turns by itself is a parameter; what you write in the config file is a hyperparameter. Mixing these two up in an interview or daily conversation instantly exposes weak foundations.

Training vs inference: don't blur them together

One more pair worth splitting apart:

• Training: the knob-turning process. Burns GPUs, runs for hours to months, done once.
• Inference: knobs frozen, just "input in, prediction out". Every time you call an LLM API, you're paying for one inference on a model someone else trained.

Why does this matter? Because the cost structures are completely different: training is a one-time massive investment (GPT-scale pretraining burns compute in the tens of millions of dollars), inference is a pay-per-call operating cost. In AI application engineering, 99% of your daily work touches inference -- which is exactly why "using models" and "training models" are two different jobs. Only when you get to fine-tuning do you lightly touch "training" for the first time.

---

Deep learning: why "deep"

Deep Learning (DL) = a neural network with many layers. Why does stacking deep help? Because each layer learns one level of abstraction:

• A face-recognition network: the first layers learn "edges and brightness", middle layers assemble "eyes, noses", top layers assemble "who this is".
• A language model: lower layers capture spelling and grammar, higher layers capture semantics and logic.

You never hand-design "what an eye is" -- the network learns it from data, layer by layer, on its own. This is automatic feature learning, the key reason deep learning crushed traditional methods. The price: it's hungry for data and compute. When data is scarce, a simple model is often the more reliable choice.

My blunt advice: if all you have is a few thousand rows of tabular data (churn prediction, ticket classification), don't touch deep learning -- LogisticRegression or gradient-boosted trees give you results in ten minutes and are often more accurate. Images, audio and free-form text -- data that "humans can read but can't write rules for" -- is deep learning's home turf.

One diagram to settle the terminology (stop using these interchangeably):

Artificial Intelligence (AI)  ── machines doing "smart" work, the biggest circle
   └─ Machine Learning (ML)  ── learning patterns from data, no hard-coded rules
        └─ Deep Learning (DL)  ── using very deep neural networks
             └─ Transformer / LLM  ── the thing you call every day

Following this hierarchy down, how a Transformer actually "attends" to what matters is covered in Transformer & Attention Basics.

---

⚠️ Overfitting: the one word in AI engineering you must remember

Term: Overfitting & Underfitting

• One-sentence explanation: overfitting = memorized the practice questions cold, falls apart on new ones; underfitting = never even learned the practice questions.
• Analogy: overfitting is the student who memorized every answer in the workbook and collapses when the exam changes; underfitting is the student who never studied and fails even the practice tests.
• How you'll use it at work: this is why you hold out data the model has never seen (validation/test sets) to examine it -- scores on the training set don't count.
• Most common pitfall: evaluating a model on its training data → absurdly high score → collapse in production. This is the number one rookie faceplant.

How to diagnose it? Look at the relationship between training score and validation score:

Training score LOW,  validation score LOW   → underfitting (model too weak / undertrained) → bigger model / train longer
Training score HIGH, validation score HIGH  → just right ✅
Training score HIGH, validation score LOW   → overfitting (memorizing)                      → more data / regularization / early stopping

The three standard fixes for overfitting, ranked by bang-for-buck:

1. More data -- the most effective and the most expensive. Double the data and a lot of overfitting simply vanishes.
2. Early Stopping -- watch the validation loss; the moment it starts climbing back up, stop. Don't fight it. In fine-tuning, cutting 3 epochs down to 1 often raises the validation score.
3. Regularization -- constraints that tell the model "don't crank any knob to extremes" (dropout, weight decay). Fine-tuning frameworks usually have ready-made switches for these.

And memorize the three-way data split (the fine-tuning chapter uses it directly):

With small datasets (say, a few hundred fine-tuning examples), this often simplifies to a 90/10 two-way split: train + validation. But never skip the validation set -- skipping it means flying blind.

---

⚠️ Accuracy will lie to you: evaluation metrics you must know

Suppose you build a "cancer screening" classifier and 99% of the people in your data are healthy. A brain-dead model that learns nothing and always predicts "healthy" scores 99% accuracy -- while catching exactly zero patients. Utterly useless. That's why looking at accuracy alone gets you fooled, especially on imbalanced data.

This isn't a made-up parable. A student once showed me his fraud-detection model asking "accuracy is 99.2%, why is my boss still unhappy?" -- I had him count how many fraud cases in the test set were actually caught. The answer was zero. The only skill his model had learned was labeling every transaction "normal".

The four classification metrics you must know, memorized via "screening patients":

Precision and recall usually trade off against each other: lower the threshold for flagging "sick" and you catch more patients (recall ↑) but also hurt more healthy people (precision ↓). Which one to favor depends on whether your business fears misses more, or false alarms more.

My blunt advice: for spam filtering and fraud blocking -- scenarios where a false positive angers users -- push precision. For safety review and disease screening -- where one miss is a catastrophe -- push recall. If you want both, report F1 and accept that it is, by definition, a compromise.

And this framework isn't just for classical classifiers -- when you later evaluate LLM applications (is the answer correct, did it hallucinate), the underlying logic is identical. The Evals & Quality Monitoring chapter builds directly on top of this.

---

Hands-on code: train a real classifier in 10 lines

Don't be intimidated -- scikit-learn makes "train + evaluate" a few lines. Running this once beats reading ten pages:

# pip install scikit-learn
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report

X, y = load_breast_cancer(return_X_y=True)     # features X, labels y (benign/malignant)

# Key move: carve out a test set the model never sees (the split from earlier)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42)

model = LogisticRegression(max_iter=5000)
model.fit(X_train, y_train)                     # ← training: internally this is gradient descent pushing loss down

# Evaluate on data the model has NEVER seen -- not the training set
print(classification_report(y_test, model.predict(X_test)))
# The output includes precision / recall / f1 -- don't stare at accuracy alone

What this is doing:

1. Core moves: load a real breast-cancer dataset, split train/test, train a classifier, then evaluate on unseen data.
2. Where to look first: the train_test_split line -- that's the core anti-overfitting move; then the precision/recall/f1 in the classification_report output.
3. The key judgment: if you mischievously change predict(X_test) to predict(X_train), the scores get prettier -- but that's self-deception, and exactly the "evaluating on the training set" mistake rookies make most.

---

Going further: 15 lines of PyTorch to watch the four-step loop with your own eyes

sklearn hides the training loop. To actually see the "① forward → ② loss → ③ backward → ④ turn knobs" from earlier, PyTorch lets you write it bare in about fifteen lines -- making a model learn the line y = 2x + 1 from data:

# pip install torch  (CPU build is enough -- don't go wrestling with CUDA)
import torch

X = torch.linspace(0, 10, 100).reshape(-1, 1)
y = 2 * X + 1 + torch.randn(X.shape) * 0.5      # make data: y=2x+1 plus some noise

model = torch.nn.Linear(1, 1)                   # the whole model is two knobs: w and b
loss_fn = torch.nn.MSELoss()                    # regression task → MSE
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

for epoch in range(200):
    pred = model(X)                             # ① forward: prediction under current knobs
    loss = loss_fn(pred, y)                     # ② measure the gap: loss
    optimizer.zero_grad()                       #   (clear last round's gradients -- mandatory, forgetting it always ends in tears)
    loss.backward()                             # ③ backprop: compute which way each knob should turn
    optimizer.step()                            # ④ gradient descent: turn one small step

print(model.weight.item(), model.bias.item())   # will be very close to 2 and 1

Run it and you'll see weight approach 2 and bias approach 1 -- the model found the line by itself from noisy data; nobody told it the formula. Match the four lines inside the for loop against the four-step diagram above -- one line per step. That's the entire secret of training neural networks. A 7B LLM merely swaps "two knobs" for "7 billion knobs"; the loop itself never changes.

Two experiments while you're there: change lr=0.01 to lr=1.0 and watch the printed loss grow bigger and bigger (the divergence we hand-computed earlier); change range(200) to range(5) to see what undertraining looks like (underfitting).

---

Loss won't go down: my debugging order

Whether you're fine-tuning or running a demo, "loss isn't dropping" is the alarm you'll hear most. Don't flail -- check in this order:

1. Shape before numbers. Loss oscillating wildly → learning rate too high, divide by 10; loss flat as a ruler → learning rate too low or data isn't flowing in, try multiplying by 10 first.
2. Loss goes NaN / Inf → almost always learning rate too high or dirty data (nulls, exploding values). Cut lr by 10x first, then inspect the data.
3. The "tiny-sample overfit" test -- my favorite trick: train on just 10 examples. The model should be able to memorize them cold (training loss near 0). Can't even memorize 10 examples? Stop tuning hyperparameters -- your code has a bug (data misaligned with labels, forgot zero_grad, wrong loss function, that kind of thing).
4. Training loss drops but validation loss stalls or climbs → that's not "failure to converge", that's overfitting. Go back to the section above: more data / early stopping / regularization.
5. Only after ruling out all of the above → do you earn the right to conclude "model too small / features carry no signal" and reach for drastic measures.

This ordering has saved me many an afternoon. Most cases of "the model isn't learning" trace back to steps 2 and 3; genuinely needing a different model is rare.

---

⚠️ Common pitfalls, one table

Every trap in this chapter, collected -- pin it above your desk:

---

What these concepts look like in LLM work

Don't file this away as "classical ML antiques". You'll bump into every one of these in LLM engineering far more often than you'd expect:

So the fine-tuning chapter (LoRA / QLoRA / PEFT Fine-tuning) will assume you know everything here -- loss curves, epochs, overfitting, validation sets. None of it is optional.

---

🔧 Hands-on Exercises

1. Run the sklearn snippet (locally or on Colab), then change test_size from 0.2 to 0.5 and watch the classification_report scores shift -- feel what "less training data" does. Then deliberately change predict(X_test) to predict(X_train) and see how much "evaluating on the training set" inflates the score.
2. Run the PyTorch snippet and do two destructive experiments: set lr to 1.0 (watch the loss diverge), and change range(200) to range(5) (see what underfitting looks like). Write down the final printed weight/bias in both cases and compare.
3. Ask ChatGPT/Claude for 5 real-world scenarios and classify each as supervised/unsupervised/reinforcement (e.g. "auto-group users into segments", "predict tomorrow's sales", "teach a robot to walk"), then have it check your answers. Bonus round: explain in one sentence, to a completely non-technical friend, "why a 99%-accuracy model can be garbage" -- if you can't, go reread that section.

---

📚 Key Takeaways

1. Machine learning is feeding examples and letting the program find the patterns, not hard-coding rules -- biased or leaky data means a dead model.
2. The three paradigms differ in "whether answers exist": supervised (yes), unsupervised (no -- find structure yourself), reinforcement (rewards and penalties); LLM pretraining is self-supervised.
3. Neural network training = a four-step knob-turning loop (forward → loss → backprop → gradient descent); regression uses MSE, classification/LLMs use cross-entropy, and learning_rate is the step size downhill -- too big means outright divergence, so tune it in factors of 10.
4. Overfitting is the number one trap: diagnose via the training-vs-validation score relationship, and always evaluate on unseen validation/test data (70/15/15 split); the three fixes are more data, early stopping, regularization.
5. Never trust accuracy alone -- it lies on imbalanced data; classification lives on precision / recall / F1, chosen by whether your business fears misses or false alarms more -- and this exact framework carries straight over to LLM evals.

---

Related Reading

Next up: Transformer & Attention Basics →

• LoRA / QLoRA / PEFT Fine-tuning -- the loss, epochs, overfitting and validation sets from this chapter all come back in fine-tuning
• Evals & Quality Monitoring -- the precision/recall mindset extended to evaluating LLM applications
• Open-weight Models & Local Deployment -- what "7B parameters" really means, revealed at deployment time
• LLM API Basics -- back to the application layer, putting trained models to work