3.4    Moment Generating Function

In cases where we know a formula for the p.d.f., can often find all moments at once in a convenient way!

Def:  Let X be a discrete random variable.  Then the moment generating function of X is the function of the variable t defined as

m_X(t) = E(e^tX)
• the moment generating function is the expected value of the function  e^tX
• the variable t is just a parameter (auxiliary variable), whose use will become clear
• the moments of X are hidden inside the function m_X(t)!

AP example; recall that the p.d.f. for the score Z of a randomly selected student was given by table
What's the moment generating function of the random variable Z?
Well,
m_Z(t)  =  E(e^tZ)  = 
=   e^t (.15)  +  e^2t (.20)  +  e^3t (.40)  +  e^4t (.15)  + e^5t (.10)
Notice that this is a function of the variable t.
 

Flip a coin until you get a tail; let N = the number of flips. Then we found previously that the p.d.f. was  f (n) = (1/2)ⁿ. Find the moment generating function.
As above,
 m_N(t)  =  E(e^tN)  = 
= 
=   e^1t * (1/2)¹   +   e^2t * (1/2)²  +  e^3t * (1/2)³  + ...
=   (e^t * 1/2)¹   +   (e^t * 1/2)²  +  (e^t * 1/2)³  + ...
But this is another geometric sum, with first term  a = e^t/2  and ratio  r = e^t/2, whose value is thus
So
m_N(t)  = 

Theorem
If m_X(t) is the m.g.f. of X, then the moments of X can be found as

  E(X^k)   = 

i.e., to find the kth moment, take the kth derivative of the moment generating function and evaluate at t = 0.

ex:

Example above; by the theorem,
E(N)  =   =    (by the quotient rule)
=    =    =   2.
 
This is the mean of N.
 
E(N²)  =     = 
=      (by quotient rule)

=    =   6
 

From this we can find the variance of N:
var(N)  =  E(N²) - E(N)²   =   6 - 2²  =  2.
 
 

• gives easy way to compute mean & variance
• can identify the type of a random variable by looking at its moment generating function, as stated by the following:

Theorem If two random variables have the same moment generating function, then they have the same p.d.f .
This will be useful to us in the future, when we're trying to determine what type of random variable we get when we look at specific combinations of other random variables!
 
 
 

1. the moment generating function of the random variable  cX  is
m_cX(t) = m_X(ct)
2. the moment generating function of the random variable  X + c  is
m_X+c(t) = e^ct m_X(t)
3. Let  X1  and  X2  be independent random variables with moment generating functions

m_X1(t)  and  m_X2(t).  Then the moment generating function of the random variable  X1 + X2  is
m_X1+X2(t)  =  m_X1(t) * m_X2(t)

The Geometric Distribution

1. have an infinite series of trials; on each trial, result is it is either success (s) or failure (f). (Such a trial is called a Bernoulli experiment.)
2. the trials are independent, and the probability of success is same on each trial. (The probability of success in each trial will be denoted  p,  and the probability of failure will be denoted  q;  thus  q  =  1 - p.)
3. X represents the number of trials until the first success.

ex:

Consider the "flip a coin until you get a tail" experiment above; then N (= # of flips until a tail occurs) is a geometric random variable, with  p = 1/2:  N counts the number of trials until the first success.
 

f(x)  =  pq^x-1  =  p (1 - p)^x-1,    x = 1, 2, 3, ...

m_X(t)  = 

We can thus use the moment generating function to find the moments, and hence the mean and variance and standard deviation:

E(X)  =    =      (from the quotient rule)
 
=   =   1/p      (using the fact that   p = 1 - q )
 
E(X²)  =    = 
 
=     = 
var(X)   =   E(X²) - E(X)²  =   q/p²

m  =  1/p
var(X)  =  q/p²
(so the standard deviation is   s  =    )

Dice game; pick a number from 1 to 6, then keep rolling until get that value. Let X = total number of rolls needed to achieve this. Then X is a geometric random variable: it counts the number of trials until the first success.
The probability of success on each trial is here  p = 1/6. Thus from the above, the expected number of rolls until the desired number appears is  E(X)  =  1/p  =  1/(1/6)  =  6. (This is pretty much what we would have anticipated!) The variance is  var(X) = q/p²  =  (5/6)/(1/6)²  =  30, so the standard deviation is  s  = 5.48  , which indicates that the value of X will usually fall in the range 6 +- 5.48; thus we should not be surprised if the number of rolls needed to get our number is as few as 1 or as many as 12.

Consider the Pennsylvania daily number lottery, discussed before, in which the probability of winning on any given day is 1/1000. Now let N be the number of times you play before winning the first time. Then N is a geometric random variable, since it's counting the number of trials until the first success, with  p = 1/1000.  Thus the expected number of plays is  E(N)  =  1/p  =  1/(1/1000)  =  1000;  thus you should expect to have to play 1000 times before winning! Since you would then be down $1000 (since it costs $1 to play), and would only recoup $500 for winning, this isn't such a great situation. Notice that this agrees with our previous results, in which we determined that, on average, you should expect to lose $.50 each time you play.
 
 
 

The cumulative probability function  F(x)  of a geometric random variable  X  with probability of success  p  is
F(x)  =  1 - q^x   =   1  -  (1 - p)^x
 (This follows by summing the values of the p.d.f., and using the formula for the value of a finite geometric sum.)

ex:

For the lottery example above, what's the probability that you'll win in a year (312 days, not counting Sundays) or less?
Want  P(N <= 312)  =  F(312)  =  1  -  (1 - .001)³¹²   =   1  -  .999³¹²   =   1  -  .732  =  .268;
thus there's only about a 1 in 4 chance that you'll win in a year if you play every day!