The Poisson, Exponential, and Uniform

September 20, 2018

Until recently, I had trouble remembering the pdf of the Poisson distribution. The expression

didn’t make much sense to me. I knew the Poisson could be derived as the limit of the binomial distribution, but that didn’t help my memory much.

Here’s an alternative method of deriving the Poisson’s pdf that I personally find much more intuitive. Suppose we have a Poisson process with parameter , and let the hits of this process occur at Define as the wait time between hits:

And note that . It’s well known that . The pdf of the Poisson at is equal to the probability of getting exactly hits in the first unit of time. We both need the th point to be within the first unit of time, and the th to be after the first unit of time. Writing this out as math and using we get:

We can compute this by integrating over all combinations of (; to be considered later) that satisfy . For each combination, we compute the likelihood of those being drawn and of drawing a that satisfies .

So in fact we’ve been integrating a constant this whole time. All that’s left to do now is compute which corresponds to the volume of an -dimensional simplex. The answer is guessable (a 2D simplex has area half; 3D, a sixth; dimensions, probably ) but it can also be derived by looking at the problem back in space rather than space. A valid set of leads to ; each is in the range and they must be in ascending order. Consider the unit -dimensional hypercube where each point corresponds to a choice of satisfying . We’d expect of the volume of our hypercube to be occupied by arranged in ascending order because are ways to arrange items, and by symmetry they’re equally likely. The hypercube itself has volume 1 so our integral evaluates to .

Putting everything together, we get as desired. I guess the TL;DR is that the Poisson’s pdf can be thought of as “the exponential distribution, evaluated at , raised to the th power, divided by

More abstractly…

It’s tempting to conclude from our constant integrand that spacing points according to an exponential distribution leads to a uniform distribution of points. This clearly isn’t true though, because uniform distributions are on bounded intervals while the exponential distribution isn’t. The following procedure, however, will generate points following a uniform distribution:

• Start at zero
• Repeat times:
• Get a sample from
• Move right by
• If we’re right of 1, clear everything and restart the procedure
• Place a point on the current position
• Get a sample from
• Move right by
• If we’re not right of 1, clear everything and restart the procedure

In fact, we can show that the actual distribution of (the exponential, conditioned on the procedure not restarting) matches the distribution of space between samples:

Where the top integral is over and the bottom is over . As shown earlier, the integrand for either integral is (the same) constant, so our expression simplifies to a ratio of volumes. We know the volume of the simplex defined by is . The shape defined by is also a simplex, but in dimensions and scaled down by a factor of . So, its volume is . Dividing the two, we get