Structural Properties of EDMs

Stat 203 Lecture 20

Dr. Janssen

Generating Functions in General

Moment Generating Function (MGF)

The moment generating function, denoted $M(t)$, for some variable $y$ with probability function $\mathcal{P}(y)$ is

\[ M(t) = E[e^{ty}] = \begin{cases} \displaystyle\int_S \mathcal{P}(y) e^{ty} dy & \text{ for $y$ continuous}\\ \sum_{y\in S} \mathcal{P}(y) e^{ty} & \text{ for $y$ discrete},\end{cases} \]

for all values of $t$ for which the expectation exists.

Cumulant Generating Function (CGF)

The cumulant generating function is defined to be

\[ K(t) = \log M(t) = \log E [e^{ty}], \]

for all values of $t$ for which the expectation exists.

The Cumulants

In general:

\[ \kappa_r = \frac{d^r K(t)}{dt^r}\biggr\vert_{t=0} \]

Exercises:

\[ E[y] = \kappa_1 = \frac{dK(t)}{dt}\biggr\vert_{t=0} \qquad \text{ and } \qquad \text{var}[y] = \kappa_2 = \frac{d^2 K(t)}{dt^2}\biggr\vert_{t=0}. \]

Generating Functions for EDMs

MGF for EDM

Theorem 1 The MGF and CGF for an EDM are

\[ \begin{align*} M(t) &= \exp \left(\frac{\kappa(\theta + t\phi) - \kappa(\theta)}{\phi}\right); \\ K(t) &= \frac{\kappa(\theta + t\phi) - \kappa(\theta)}{\phi}. \end{align*} \qquad(1)\]

Using Equation 1, the $r$th cumulant for an EDM is

\[ \kappa_r = \phi^{r-1} \frac{d^r \kappa(\theta)}{d \theta^r}. \]

Examples: Normal and Poisson

Example 1 The CGF for the normal distribution is

\[ K(t) = \frac{(\mu + t\sigma^2)^2}{2\sigma^2} - \frac{\mu^2}{2\sigma^2} = \mu t + \frac{\sigma^2 t^2}{2}. \]

Example 2 For the Poisson distribution, we obtain

\[ K(t) = \mu(\exp t - 1). \]

Mean and Variance of an EDM

Mean and Variance

Note:

\[ E[y] = \mu = \frac{d\kappa(\theta)}{d\theta} \qquad \text{ and } \qquad \text{var}[y] = \phi \frac{d^2\kappa(\theta)}{d\theta^2}. \]

Observe:

\[ \frac{d^2\kappa(\theta)}{d\theta^2} = \frac{d}{d\theta} \left(\frac{d\kappa(\theta)}{d\theta}\right) = \frac{d\mu}{d\theta}. \]

Example: Normal distribution

For the normal distribution, $\kappa(\theta) = \theta^2/2$, so $E[y] = d\kappa/d\theta = \theta = \mu$.

For the variance, we see $V(\mu) = d^2\kappa /d\theta^2 = 1$, and since $\phi = \sigma^2$, $\text{var}[y] = \phi V(\mu) = \sigma^2$.

Example: Poisson

We have $\kappa(\theta) = \mu$ and $\theta = \log\mu$. The mean is thus

\[ E[y] = \frac{d\kappa}{d\theta} = \frac{d\kappa}{d\mu} \frac{d\mu}{d\theta} = \mu. \]

For the variance, $V(\mu) = d\mu/d\theta = \mu$. Since $\phi = 1$ for the Poisson distribution, $\text{var}[y] = \mu$.

Variance Function

Example

Consider an EDM with $V(\mu) = \mu^2$.

Then

\[ \mathcal{P}(y) = a(y,\phi) \exp \left( \frac{y(-1/\mu) - \log\mu}{\phi} \right), \]

for an appropriate function $a(y, \phi)$.

Recognize this?

Since $V(\mu) = d\mu/d\theta$, solve $d\theta/d\mu = \mu^{-2}$ for $\theta$ to obtain $\theta = -1/\mu$ (i.e., integrate with constant of integration equal to 0). Then, since $\mu = d\kappa/d\theta$ and $\theta = -1/\mu$, $\kappa(\theta) = -\log(-\theta) = \log\mu$.

With these forms for $\theta$ and $\kappa(\theta)$, the EDM uniquely corresponding to $V(\mu) = \mu^2$ has the probability function on the screen.

The constants of integration are not functions of $\mu$, so can be absorbed into $a$ if not set to zero.

This is the probability function of the gamma distribution.

Thus, the variance function $V(\mu) = \mu^2$ uniquely refers to a gamma distribution within the EDM family.

As a consequence, if the mean-variance relationship can be established for a given dataset, and quantified using the variance, the corresponding EDM is uniquely identified.

In general, $\text{var}[y] = \phi V(\mu)$ means that the variance of an EDM depends on the mean. The normal distribution is unique in the EDM family since its variance does not depend on the mean as $V(\mu)= 1$. For other EDMs, the variance is a nonconstant function of the mean, and the role of the variance function $V$ is to specify precisely that function.