Exercises in EM, Expectation Maximization Algorithm

How to obtain equation (2) of the paper “Exercises in EM” by:

Flury, Bernard and Zoppe, Alice. Exercises in EM. The American Statistician, Vol. 54, №3, August 2000.

Reference paper link

In equation (2) of the paper, we see the Expected value for two cases (two categories of lamps (light bulbs) ) are obtained as:

A brief explanation about the notation of exponential distribution:

We know exponential distribution as the :

exp(-λt)

Where

λ

is the rate or the probability of “failing/expiring” lamps over the unit of time, and

exp(- λt)

is the probability of a lamp to be alive until time t.

So we can say the probability of expiring a lamp at time t, when exactly expiration happens at t, is the multiplication of two probabilities, I mean:

λ exp(-λt)

We can replace λ with 1/θ and reach this new notation:

Let’s go back to the paper:

First part of Eq. 2:

The first part of equation (2) says the expectation value (lifetime) of those lamps that are still alive after the moment t becomes:

t+θ

It is easy to guess. It is due to memoryless property of exponential distributions.

Second part of Eq. 2:

I have explained this part using two methods.

Method I:

For the case when the lamps are failed/expired before time t, we have:

X_i is the time of the ith lamps. So the expectation of the X_i for this case becomes:

or:

Where the integrand itself is a conditional probability:

and we can write the integral in this way:

Explaining the denominator:

We know the probability of a lamp to be alive until time t is

p = exp(-t/θ)

Since these kinds of lamps are expired/failed before time t, the probability becomes

(1-p)

Explaining the Integrand:

The integrand is :

Where we can write for each part:

and therefore we obtain:

Now we have this new integral:

and finally we reach:

Method II:

We consider total cases, I mean E_i=0 and E_i=1, in other words I consider both categories of lamps, those which are expired before and those which are still alive after time t.

So the expectation value of X_i , over the whole range of time, becomes:

According to the law of total expectation, the left side of the above equation can be written as:

So we have:

We can substitute the values we already know, as:

and finally obtain the expectation of those lamps that are expired before time t :