The ghost of random chance
A p-value tells you how likely your data would be if there were no real effect, helping you decide if your results are statistically significant.
Imagine you're trying to see if a coin is rigged. You flip it 10 times and get 9 heads. The p-value is like the chance that you'd see such a lopsided result just by random luck if the coin were fair. If that chance is tiny, you might suspect the coin is actually biased.
P-values are the gatekeeper for scientific claims in fields like medicine, psychology, and economics—they decide whether a new drug works or a marketing strategy is effective. You encounter them in research papers, news headlines about studies, and even in A/B testing for websites.
The most common mistake is thinking the p-value tells you the probability that your hypothesis is true. In reality, it only tells you the probability of seeing your data (or something more extreme) assuming the null hypothesis is true, not the truth of the hypothesis itself.
In null hypothesis significance testing, the p-value is the probability of obtaining test results at least as extreme as the observed results, under the assumption that the null hypothesis is correct. It measures the strength of evidence against the null hypothesis, with lower values indicating stronger evidence for an effect. A common threshold is 0.05, but this is arbitrary and not a magical cutoff.