Exercises in EM, Expectation Maximization Algorithm

Flury, Bernard and Zoppe

I am going to explain how to obtain equation (2) of the paper “Exercises in EM” by:

Flury, Bernard and Zoppe, Alice. Exercises in EM. The American Statistician, Vol. 54, №3, August 2000.

Reference paper link

Image for post
Image for post

In equation (2) of the paper, we see the Expected value for two cases (two categories of lamps (light bulbs) ) are obtained as:

Image for post
Image for post

We know exponential distribution as the :

exp(-λt)

Where

λ

is the rate or the probability of “failing/expiring” lamps over the unit of time, and

exp(- λt)

is the probability of a lamp to be alive until time t.

So we can say the probability of expiring a lamp at time t, when exactly expiration happens at t, is the multiplication of two probabilities, I mean:

λ exp(-λt)

We can replace λ with 1/θ and reach this new notation:

Image for post
Image for post

Let’s go back to the paper:

First part of Eq. 2:

The first part of equation (2) says the expectation value (lifetime) of those lamps that are still alive after the moment t becomes:

t+θ

It is easy to guess. It is due to memoryless property of exponential distributions.

Second part of Eq. 2:

I have explained this part using two methods.

Method I:

For the case when the lamps are failed/expired before time t, we have:

Image for post
Image for post

X_i is the time of the ith lamps. So the expectation of the X_i for this case becomes:

Image for post
Image for post

or:

Image for post
Image for post

Where the integrand itself is a conditional probability:

Image for post
Image for post

and we can write the integral in this way:

Image for post
Image for post

We know the probability of a lamp to be alive until time t is

p = exp(-t/θ)

Since these kinds of lamps are expired/failed before time t, the probability becomes

(1-p)

The integrand is :

Image for post
Image for post

Where we can write for each part:

Image for post
Image for post

and therefore we obtain:

Image for post
Image for post
Image for post
Image for post

Now we have this new integral:

Image for post
Image for post

and finally we reach:

Image for post
Image for post

Method II:

We consider total cases, I mean E_i=0 and E_i=1, in other words I consider both categories of lamps, those which are expired before and those which are still alive after time t.

So the expectation value of X_i , over the whole range of time, becomes:

Image for post
Image for post

According to the law of total expectation, the left side of the above equation can be written as:

Image for post
Image for post

So we have:

Image for post
Image for post

We can substitute the values we already know, as:

Image for post
Image for post

and finally obtain the expectation of those lamps that are expired before time t :

Image for post
Image for post

Written by

Web geek, Self-taught full-stack web developer, Learning Python, Laravel, Vuejs, UX/UI design, Nuclear Physicist PhD

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store