The Categorical Distribution

A discrete probability distribution modelling 2 or more categories

The categorical distribution is a discrete probability distribution that describes the probability of each possible value for a random variable with a fixed number of categories. For example, the categories could be

• the 2 sides of a coin. (If you roll a fair coin, the the probability of getting each side is 1/2. This would be a special case called the Bernoulli distribution.)

• the 6 faces of a die. (If you roll a fair die, then the probability of getting each side is 1/6. This would be a special case called the discrete uniform distribution.)

• 3 possible colours like orange, blue, and grey.

• 4 possible fruits like apples, oranges, peaches, and bananas.

As with any probability distribution, all probabilities within a categorical distribution must

• be non-negative.

• sum to equal 1.

Here is an example: Imagine you have a bag of candies with three different colours: orange, blue, and grey. If you randomly pick one candy, the categorical distribution could tell you the probability of picking each color.

For instance:

• Orange: 0.5 (50% chance)

• Blue: 0.3 (30% chance)

• Grey: 0.2 (20% chance)

You can use the categorical distribution to model this situation.

Categorical distributions are fundamental in statistics and machine learning. They model situations where outcomes are categorical rather than numerical. This can be useful in fields like marketing (customer preferences), medicine (disease diagnosis), natural language processing (word prediction), and many more.