# Exercises in EM, Expectation Maximization Algorithm

## Flury, Bernard and Zoppe

I am going to explain how to obtain equation (2) of the paper “Exercises in EM” by:

Flury, Bernard and Zoppe, Alice. Exercises in EM. The American Statistician, Vol. 54, №3, August 2000.

In equation (2) of the paper, we see the Expected value for two cases (two categories of lamps (light bulbs) ) are obtained as:

We know exponential distribution as the :

`exp(-λt)`

Where

`λ`

is the rate or the probability of “failing/expiring” lamps over the unit of time, and

`exp(- λt)`

is the probability of a lamp to be alive until time t.

So we can say the probability of expiring a lamp at time t, when exactly expiration happens at t, is the multiplication of two probabilities, I mean:

`λ exp(-λt)`

We can replace λ with 1/θ and reach this new notation:

Let’s go back to the paper:

# First part of Eq. 2:

The first part of equation (2) says the expectation value (lifetime) of those lamps that are still alive after the moment t becomes:

`t+θ`

It is easy to guess. It is due to memoryless property of exponential distributions.

# Second part of Eq. 2:

I have explained this part using two methods.

# Method I:

For the case when the lamps are failed/expired before time t, we have:

X_i is the time of the ith lamps. So the expectation of the X_i for this case becomes:

or:

Where the integrand itself is a conditional probability:

and we can write the integral in this way:

We know the probability of a lamp to be alive until time t is

`p = exp(-t/θ)`

Since these kinds of lamps are expired/failed before time t, the probability becomes

`(1-p)`

The integrand is :

Where we can write for each part:

and therefore we obtain:

Now we have this new integral:

and finally we reach:

# Method II:

We consider total cases, I mean E_i=0 and E_i=1, in other words I consider both categories of lamps, those which are expired before and those which are still alive after time t.

So the expectation value of X_i , over the whole range of time, becomes:

According to the law of total expectation, the left side of the above equation can be written as:

So we have:

We can substitute the values we already know, as:

and finally obtain the expectation of those lamps that are expired before time t :

Written by