Problem-1

Question

Let \(\displaystyle X_{1} ,X_{2} ,\cdots\) be a sequence of independent, exponentially distributed random variables, each with mean \(\displaystyle 1\). Let \(\displaystyle N\) be a Poisson random variable with parameter \(\displaystyle \lambda\) that is independent of each \(\displaystyle X_{i}\). Compute the expectation of \(\displaystyle X_{1} +\cdots +X_{N^{2}}\).

Answer

We have \(\displaystyle E[ X_{i}] =1\). Let \(\displaystyle Y=X_{1} +\cdots +X_{N^{2}}\). Since \(\displaystyle X_{i}\) and \(\displaystyle N\) are independent, we have \(\displaystyle E[ X_{i} \ |\ N] =E[ X_{i}] =1\). Using the linearity of expectation, the conditional expectation \(\displaystyle E[ Y\ |\ N]\) can be computed to be:

\[ \begin{aligned} E[ Y | N] & =E[ X_{1} +\cdots +X_{N^{2}} | N]\\ & =E[ X_{1} | N] +\cdots +E[ X_{N^{2}} | N]\\ & =N^{2} \end{aligned} \]

Note that \(\displaystyle E[ Y\ |\ N]\) is itself a random variable, namely \(\displaystyle N^{2}\). Using the law of total expectation. \(\displaystyle E[ Y] =E[ E[ Y|N]]\). This outer expectation is over the Poisson distribution:

\[ \begin{equation*} \begin{aligned} E[ Y] & =E\left[ N^{2}\right]\\ & =\text{var}( N) +E[ N]^{2}\\ & =\lambda +\lambda ^{2} \end{aligned} \end{equation*} \]

We have used the fact that \(\displaystyle \text{var}( N) =E\left[ N^{2}\right] -E[ N]^{2}\) and the fact that the mean and variance of \(\text{Pois}(\lambda)\) is \(\lambda\).

Simulation

Turning this problem into a simulation involves the following steps in a single run of the simulation:

• Sample \(N\) from \(\text{Poisson}(\lambda)\)
• Sample \(N^2\) times from \(\text{Exp}(1)\)
• Sum these \(N^2\) values

Repeating this several times and averaging the sums across all runs will give us an estimate of the expectation. Invoking the law of large numbers, we expect this estimate to be close to the true expectation. Here is a simulation of this setup using Python and NumPy:

import numpy as np
rng = np.random.default_rng(seed = 1033)

lamb = 5
L = [ ]
for _ in range(1_000_000):
    N = rng.poisson(lamb)
    L.append(
        sum([rng.exponential() 
            for _ in range(N ** 2)]
        )
    )

print('Estimate:', np.mean(L))
print('True expectation:', lamb + lamb ** 2)

Estimate: 30.008593300093473
True expectation: 30