Exercises in EM, Expectation Maximization Algorithm
Flury, Bernard and Zoppe
I am going to explain how to obtain equation (2) of the paper “Exercises in EM” by:
Flury, Bernard and Zoppe, Alice. Exercises in EM. The American Statistician, Vol. 54, №3, August 2000.
In equation (2) of the paper, we see the Expected value for two cases (two categories of lamps (light bulbs) ) are obtained as:
A brief explanation about the notation of exponential distribution:
We know exponential distribution as the :
is the rate or the probability of “failing/expiring” lamps over the unit of time, and
is the probability of a lamp to be alive until time t.
So we can say the probability of expiring a lamp at time t, when exactly expiration happens at t, is the multiplication of two probabilities, I mean:
We can replace λ with 1/θ and reach this new notation:
Let’s go back to the paper:
First part of Eq. 2:
The first part of equation (2) says the expectation value (lifetime) of those lamps that are still alive after the moment t becomes:
It is easy to guess. It is due to memoryless property of exponential distributions.
Second part of Eq. 2:
I have explained this part using two methods.
For the case when the lamps are failed/expired before time t, we have:
X_i is the time of the ith lamps. So the expectation of the X_i for this case becomes:
Where the integrand itself is a conditional probability:
and we can write the integral in this way:
Explaining the denominator:
We know the probability of a lamp to be alive until time t is
p = exp(-t/θ)
Since these kinds of lamps are expired/failed before time t, the probability becomes
Explaining the Integrand:
The integrand is :
Where we can write for each part:
and therefore we obtain:
Now we have this new integral:
and finally we reach:
We consider total cases, I mean E_i=0 and E_i=1, in other words I consider both categories of lamps, those which are expired before and those which are still alive after time t.
So the expectation value of X_i , over the whole range of time, becomes:
According to the law of total expectation, the left side of the above equation can be written as:
So we have:
We can substitute the values we already know, as:
and finally obtain the expectation of those lamps that are expired before time t :