Evaluating Your Binary Classifier: Metrics That Matter

A straightforward overview of sensitivity, specificity, and their counterparts

Binary classification is an important task for statisticians, data scientists, data analysts, and machine-learning practitioners. There are many ways to evaluate the quality of a binary classifier; here are several common metrics and the links to their Wikipedia entries:

• sensitivity and specificity

• precision and recall

• positive predictive value and negative predictive value

Note that

• sensitivity and recall mean the same thing

• precision and positive predictive value mean the same thing

If you are unfamiliar with these concepts, you should read those 3 Wikipedia articles carefully and digest them thoroughly.

These concepts are easy to define in words and mathematical formulas, but they are difficult to grasp intuitively. I STRONGLY encourage you to stare at the following diagram to absorb the intuition behind sensitivity and specificity.

Image courtesy of FeanDoe via Wikimedia Commons

In this diagram,

• the small circles are the data

• the black circles are the positive cases

• the white circles are the negative cases

• the oval is the classifier

• the circles within the oval are the cases that the classifier deems to be positive.

There is an analogous image for precision and recall.

Image courtesy of FeanDoe via Wikimedia Commons