Two Components of a GLM

Stat 203 Lecture 19

Dr. Janssen

Two Components

Two Questions to Answer

Question

What probability distribution is appropriate?

Question

How are the explanatory variables related to the mean of the response \(\mu\)?

Random Component: EDM

A different kind of EDM

Continuous examples:

Normal distribution
Gamma distribution

Discrete examples:

Poisson
Binomial
Negative binomial

Definition (for real)

Definition 1 (EDM) Distributions in the EDM family have a probability function of the form

\[ \mathcal{P}(y; \theta, \phi) = a(y,\phi) \exp \left( \frac{y\theta - \kappa(\theta)}{\phi} \right) \qquad(1)\]

where:

\(\theta\) is called the canonical parameter.
\(\kappa(\theta)\) is a known function called the cumulant function.
\(\phi > 0\) is the dispersion parameter.
\(a(y,\phi)\) is a normalizing function ensuring that Equation 1 is a probability function, i.e., that \(\int \mathcal{P}(y; \theta, \phi) dy = 1\) or that \(\sum_y \mathcal{P}(y; \theta, \phi) dy = 1\).

Other notation

The support of \(y\) is denoted by \(S\) (where \(S\) does not depend on \(\phi\) or \(\theta\))
Domain of \(\theta\) is \(\Theta\), an open interval of values containing 0 and satisfying \(\kappa(\theta) < \infty\).
The corresponding domain of \(\mu\) is denoted \(\Omega\)

All of our favorite distributions are EDMs

The normal distribution

The pdf for the normal distribution with mean \(\mu\) and variance \(\sigma^2\) is

\[ \mathcal{P}(y;\mu,\sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left(- \frac{(y-\mu)^2}{2\sigma^2} \right) \]

The Poission distribution

For \(\mu > 0\) and \(y = 0, 1 ,2 ,\ldots\), the Poisson probability function is

\[ \mathcal{P}(y; \mu) = \frac{\exp(-\mu) \mu^y}{y!} \]

The binomial distribution

The binomial probability function is

\[ \mathcal{P}(y; \mu, m) = \binom{m}{my} \mu^y (1-\mu)^{m(1-y)}, \] where \(y = 0, 1/m, 2/m, \ldots, 1\) and \(0 < \mu < 1\).

Exponential Exploration

The exponential distribution has probability function

\[ \mathcal{P}(y; \mu) = \mu \exp(-\mu y). \]

Show that the exponential distribution is in the family of EDMs.

Weibull

The Weibull distribution has the probability function

\[ \mathcal{P}(y; \alpha, \gamma) = \frac{\alpha}{\gamma} \left(\frac{y}{\gamma}\right)^{\alpha - 1} \exp \left[ - \left(\frac{y}{\gamma}\right)^{\alpha}\right] \]

for \(y > 0\) with \(\alpha,\gamma > 0\).

Weibull distribution is a continuous probability distribution. It models a broad range of random variables, largely in the nature of a time to failure or time between events. Examples are maximum one-day rainfalls and the time a user spends on a web page.

We can rewrite as

\[ \mathcal{P}(y; \alpha, \gamma) = \exp \left[ -\left(\frac{y}{\gamma}\right)^\alpha + \log \left( \frac{\alpha}{\gamma}\right) + (\alpha - 1)\log \frac{y}{\gamma} \right]. \]

Inside the exponential, a term of the form \(y\theta\) cannot be extracted unless \(\alpha = 1\). Hence the Weibull distribution is not an EDM in general. When \(\alpha = 1\),

\[ \mathcal{P}(y; \gamma) = \exp(-y/\gamma)/\gamma = \exp(-(y/\gamma) - \log \gamma), \]

which is the exponential distribution with mean \(\gamma\). In this form, we have an EDM where \(\theta = -1/\gamma\) is the canonical parameter, \(\kappa(\theta) = \log\gamma\) and \(\phi = 1\).