Python for Data Science

The one language to rule them all, and the four libraries you'll actually use.

Key Points

• NumPy: fast number crunching on arrays. The engine under almost everything else.
• pandas: DataFrames. Like Excel in code. You'll load, clean, filter, group, and merge here.
• Matplotlib and Seaborn: plotting. Histograms, scatter plots, bar charts, heatmaps.
• scikit learn: all the standard ML algorithms, like regression, classification, clustering, plus tools for preprocessing and evaluation.
• SQL is still essential. Most data lives in databases, and you'll use Python to connect and query them.

What it is

Python won. It's the most popular language for data science because it's readable, has a huge library ecosystem, and lets you go from exploring to production without switching tools. The core stack: NumPy for number crunching, pandas for data manipulation, Matplotlib and Seaborn for charts, and scikit learn for machine learning. You don't need to be a software engineer. But you do need to be comfortable with basic Python.

Why it exists

Before Python, data science was split across multiple tools: R for stats, MATLAB for matrices, Java for speed, Excel for quick looks. Python unified it. Now you can explore a CSV in pandas, visualise it, train a model, and deploy an API, all in one language.

Intuition

Example

One line of pandas: `df.groupby('region')['sales'].mean()` gives average sales per region. In pure Python without pandas, that's about 10 lines of loops and dictionaries.

Libraries save you from reinventing the wheel. You focus on the analysis, not the boring implementation details.

When to use

• Use Python when you need to go beyond Excel or SQL: custom calculations, ML, advanced charts.
• Use it when your data is too big for memory based tools, though pandas has limits. Then try Polars or Spark.
• Use it for reproducibility. A Python script runs the same way every time, unlike clicking around in a spreadsheet.

Common mistakes

• Writing slow pandas code, like using `iterrows()` instead of vectorised operations. Learn to think in columns, not loops.
• Not using virtual environments. Different projects need different library versions. Mixing them will break things.
• Trying to learn every library at once. Master pandas first, then scikit learn, then add others as needed.

Notes

• Jupyter Notebooks are the standard for exploration. But for production, you'll write `.py` scripts and functions. Both matter.

Related concepts