Data science

(extracting useful insight from real-world data using statistics and computation)

What is data science?

Data science combines statistics, programming, and subject-matter knowledge to extract useful insight from real, often messy data. Where the probability and statistics articles cover the mathematical theory, data science is about applying that theory to actual, imperfect, real-world data.

The data pipeline

A typical data project moves through several stages: collecting the raw data, cleaning it (fixing errors, removing duplicates, handling missing values), analyzing it (looking for patterns), visualizing it (turning numbers into charts people can actually understand), and communicating the results clearly.

Why cleaning matters: a concrete example

Suppose a store's sales records accidentally include the same transaction twice, due to a system glitch. If left uncleaned, that duplicate inflates the average sale amount, since the same transaction gets counted twice when computing the mean (as defined in the introductory statistics article) — a genuine, incorrect distortion, not a hypothetical one, and exactly the kind of error a data scientist's cleaning step is specifically meant to catch before drawing any conclusions from the data.

Why visualization reveals what raw numbers can hide

A striking, well-documented illustration: statistician Francis Anscombe constructed four different data sets in 1973, each with nearly identical summary statistics (the same mean, the same variance, the same correlation) — numbers alone would suggest all four look the same. But plotting them reveals four completely different patterns: one is a clean straight-line relationship, one is a clear curve, one is a straight line with a single outlier throwing it off, and one is a vertical cluster with one extreme point. This is a real, verifiable fact about those specific data sets, not just a claim — and it's exactly why data scientists insist on visualizing data directly, rather than trusting summary numbers alone.

Why data science matters

Nearly every data-driven decision — which product to recommend, which medical treatment shows promise, which policy is working — depends on correctly extracting a genuine signal from real data that is messy, incomplete, and easy to misread if handled carelessly.