Probability Basics

The language of uncertainty — how likely is something to happen?

Key Points

• Probability always sits between 0 and 1. A coin flip is 0.5 — exactly in the middle.
• If there's a 30% chance of rain, there's automatically a 70% chance it won't rain. They always add up to 1.
• When two events don't affect each other (like rolling two dice), you multiply their probabilities to find the chance both happen.
• When two events can't happen at the same time (like rolling a 3 and a 5 on one die), you add their probabilities.
• Almost everything in data science — model confidence, error rates, test results — is built on these basic rules.

What it is

Probability is just a number between 0 and 1 that tells you how likely something is to happen. A 0 means it will never happen. A 1 means it always happens. Everything in between is a maybe. Think of it like a confidence score — when a spam filter says an email is 87% likely to be spam, that's probability at work. Every prediction a data science model makes is really just a probability dressed up as an answer.

Why it exists

Nothing in the real world is 100% certain. A model predicting whether someone will buy a product can't just say yes or no with certainty — it has to say 'there's a 73% chance.' Probability gives us a shared language for talking about uncertainty in a way that's measurable and consistent. Without it, we'd have no way to say how confident a model is, how reliable a test is, or how likely we are to be wrong.

What it measures

Probability measures how often something would happen if you repeated a situation many times. Flip a fair coin 10,000 times and heads shows up about 5,000 times — so the probability is 0.5. In machine learning, it measures how confident a model is. When a model says P(spam) = 0.87, it's saying: out of every 100 emails that look like this, about 87 would actually be spam.

Intuition

Example

A spam classifier looks at an email and outputs P(spam) = 0.87.

The model isn't saying 'this is spam.' It's saying 'this looks 87% like spam to me.' Whether it actually gets marked as spam depends on a threshold — usually if P > 0.5, it's flagged. But you can move that threshold. If you're building something where missing a spam email is really costly, you might lower the threshold to 0.3. If false alarms are the bigger problem, raise it to 0.7. The probability gives you flexibility — the threshold is the decision.

Formulas

When to use

• Whenever you need to put a number on how likely something is — model predictions, error rates, outcomes
• Use the multiplication rule when events don't affect each other, like rolling two separate dice
• Use the complement rule when it's easier to calculate the opposite — 'what's the chance it doesn't happen' — then subtract from 1
• Use probability to decide classification thresholds — at what confidence level should a model take action?

When not to use

• Don't multiply P(A) × P(B) if the two events are connected — drawing a card and then drawing another without putting the first back changes the odds each time
• Don't assume a probability of 0.5 means exactly half — flip a coin 10 times and you might get 3 heads. Probability is about the long run, not individual trials
• Don't fully trust model output probabilities without checking — a model saying 0.9 doesn't always mean it's right 90% of the time on your specific data

Common mistakes

• Using the multiplication rule when events aren't independent — drawing two aces from a deck without replacing the first card affects the second draw
• Mixing up P(A given B) with P(B given A) — the chance of having a disease given a positive test is very different from the chance of a positive test given you have the disease
• Writing off rare events as impossible — a 1% chance still happens 1 in every 100 times, which adds up fast in large datasets
• Ignoring how common something already is — a test that's 99% accurate can still produce mostly wrong results if the thing it's testing for is very rare

In practice

Probability is the foundation underneath most of what data science does. Classification models (like logistic regression and Naive Bayes) don't just predict a label — they predict a probability for each class and then pick the most likely one. Hypothesis testing uses probability to decide if a result is real or just noise. When you evaluate a model using ROC curves, AUC, or log-loss, you're really asking: how good are its probability estimates? Getting comfortable with probability is what makes all of those concepts click.

Related concepts