EDMs in Dispersion Model Form and Unit Deviance

Stat 203 Lecture 21

Dr. Janssen

Unit Deviance and Dispersion Model Form

Relating \(\mu\) and \(\theta\)

Recall:

\(\theta\) is the canonical parameter
\(\mu\) is the mean of the response
Each can be written as a one-to-one function of the other
The canonical parametrization of the probability function is given in terms of \(\theta\)
But we must be able to rewrite it in terms of \(\mu\)

Unit Deviance

Definition 1 We define the unit deviance to be the quantity

\[ d(y,\mu) = 2 \left( t(y,y) - t(y,\mu)\right). \qquad(1)\]

Dispersion model form

Definition 2 In terms of the unit deviance, the probability function for an EDM is

\[ \mathcal{P}(y; \mu, \phi) = b(y,\phi) \exp \left[-\frac{1}{2\phi} d(y,\mu)\right]. \]

where:

\(b(y,\phi) = a(y,\phi) \exp \left[ t(y,y)/\phi \right]\), which cannot always be written in closed form
all other definitions are the same as the canonical parameter form that we saw a few days ago.

Normal distribution in dispersion model form

\[ \mathcal{P}(y;\mu,\sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(- \frac{(y-\mu)^2}{2\sigma^2} \right) \]

A technical note

Definition 1 for the unit deviance assumes that we are allowed to set \(\mu\) equal to \(y\), but this need not be possible.

Our workaround:

\[ d(y,\mu) = 2 \left( \lim\limits_{\epsilon \to 0} t(y+\epsilon, y+\epsilon) - t(y,\mu) \right). \]

Unit deviance for Poisson

Recall that

\[ \mathcal{P}(y; \mu) = \frac{\exp(-\mu) \mu^y}{y!} = \frac{\exp(y\log \mu - \mu)}{y!} \]

A technical note that may interest only me

The distributions in our class have the property that \(\Omega = S\), but this need not be true in general.

In certain cases, we can generalize the unit deviance to

\[ d(y,\mu) = 2 \left( \sup\limits_{\mu\in \Omega} t(y,\mu) - t(y,\mu) \right). \]

That is, the domain for \(\mu\) is the same as the support for \(y\), at least in a limiting sense; technically, that \(S \subseteq \overline{\Omega}\).

EDMs exist for which \(\mu\) are far more restricted than those for \(y\), such as the Tweedie models with power variance functions \(V(\mu) = \mu^\xi\). When \(\xi < 0\), the resulting distributions can take all values \(y\) on the real line, where as the mean must be positive. So we can generalize the unit deviance again to

\[ d(y,\mu) = 2 \left( \sup\limits_{\mu\in \Omega} t(y,\mu) - t(y,\mu) \right) \]

However such distributions do not have any useful applications for modelling real data, as least not yet, so we can ignore this technicality in practice.

The limiting definition is sufficient.

So, we have this new way of writing the probability function for the EDM family that relies on the unit deviance. What are its properties? How, for instance, are unit deviances distributed? to answer that question, we need a new tool, the saddlepoint approximation.

The Saddlepoint Approximation

Definition

Definition 3 The saddlepoint approximation to the EDM density function \(\mathcal{P}(y; \mu, \phi)\) is defined by

\[ \tilde{\mathcal{P}}(y; \mu,\phi) = \frac{1}{\sqrt{2\pi\phi V(y)}} \exp \left( - \frac{d(y,\mu)}{2\phi} \right). \]

Compared to the dispersion model form, \(b(y,\phi) \approx 1/\sqrt{2\pi \phi V(y)}\).

The saddlepoint approximation is often remarkably accurate, even in the extreme tails of the distribution. As well as being computationally useful in some cases, the approximation aids our theoretical understanding of the properties of edms.

In mathematics, the method of steepest descent or saddle-point method is an extension of Laplace’s method for approximating an integral, where one deforms a contour integral in the complex plane to pass near a stationary point (saddle point), in roughly the direction of steepest descent or stationary phase. The saddle-point approximation is used with integrals in the complex plane, whereas Laplace’s method is used with real integrals.

In practical terms, we tend to modify the term \(V(y)\) so that it can never take the value zero. For example, in the saddlepoint approximation to the Poisson distribution, we replace \(V(y)\) with \(V(y+\epsilon)\), where \(\epsilon = 1/6\).

This has the benefits of improved accuracy and being defined at \(y = 0\). This is called the modified saddlepoint approximation.

When \(b(y,\phi)\) is not available in any closed form, we can approximate it by a simple(ish) analytic function.

If this approximation looks familiar, it’s basically using the normal distribution’s form to approximate other EDMs. For the normal distribution, we have \(V(\mu) = 1 = V(y)\) and \(\phi = \sigma^2\), so the saddlepoint approximation is exact.

Saddlepoint approximation of Poisson

For the Poisson distribution, \(V(\mu) = \mu\), so that \(V(y) = y\).

Thus:

\[ {\tilde{\mathcal{P}}} = \frac{1}{\sqrt{2\pi y}} \exp (-y\log(y/\mu) + (y-\mu)). \]

An important consequence

If the saddlepoint approximation to the probability function of an EDM is accurate, then \(d(y,\mu)/\phi \sim \chi_1^2\).