Discrete and Continuous Distributions

Both the Bernoulli and Uniform distributions previously covered are classified as discrete probability distributions because their outcome spaces are defined on a countable number of events. Discrete probability distributions are commonly defined on integers, meaning they do not to take on fractional values.

In contrast to discrete probability distributions, there are continuous probability distributions. Continuous distributions are defined on outcome spaces of uncountable values. You can think of uncountable values as the continuous range of values, including rational and irrational numbers, along a number line. A continuous random variable can take on any value between any other two values included in the outcome space.

When defining a continuous probability distribution, it is not possible to set nonzero probabilities for each event that all sum to 1. Roughly speaking, this is because there are an infinite number of events, meaning that regardless of how small you make the probabilities, they will sum to a large quantity. Because of this, the probability that a continuous distribution will take on an exact value is always zero.

Recall that we characterize discrete probability distributions by specifying the probability of each possible event. Because continuous distributions assign a probability of zero to every event, we need another way to characterize continuous probability distributions.

This is where probability density functions (PDF) come into the picture. PDFs give relative likelihoods of exact values, rather than their absolute values, which are all zero. This means the PDF lets you compare the likelihoods of specific values with each other, but it does not give the actual probability any of those values. Importantly, the area under the PDF curve is always equal to 1.

Here is a plot of the PDF of a continuous random variable that follows what is called a "normal distribution":

PDF for the normal distribution with a mean of 0 and a standard deviation of 1.

Note that the values of the PDF for each value of X do not sum to 1, but still show which values of X are more likely than others. For instance, X=0 is more likely than X=2.

Mathematically defining the PDF requires learning about the cumulative density function (CDF), which will not be covered in this section. Fully defining the PDF and CDF requires high school level calculus; as you may have noticed, the requirement of the area under the PDF curve being equal to 1 can be checked by ensuring the integral of the PDF is equal to 1. You are highly encouraged to pursue these concepts beyond this section.