Quadratic Forms and the Principal Axis Theorem

Math 303: Section 32

Dr. Janssen

Intro

\[ \def\R{{\mathbb R}} \def\P{{\mathbb P}} \def\B{{\mathcal B}} \def\C{{\mathcal C}} \def\S{{\mathcal S}} \def\b{{\mathbf{b}}} \def\a{{\mathbf{a}}} \def\c{{\mathbf{c}}} \def\x{{\mathbf{x}}} \def\y{{\mathbf{y}}} \def\u{{\mathbf{u}}} \def\v{{\mathbf{v}}} \def\w{{\mathbf{w}}} \def\z{{\mathbf{z}}} \def\e{{\mathbf{e}}} \def\r{{\mathbf{r}}} \def\M{{\mathcal{M}}} \DeclareMathOperator{\null}{Nul} \DeclareMathOperator{\span}{Span} \DeclareMathOperator{\dim}{dim} \DeclareMathOperator{\proj}{proj} \DeclareMathOperator{\row}{Row} \DeclareMathOperator{\col}{Col} \DeclareMathOperator{\trace}{trace} \newcommand{\set}[1]{\left\{ {#1} \right\}} \newcommand{\setof}[2]{{\left\{#1\,\colon\,#2\right\}}} \newcommand{\norm}[1]{{\left|\! \left| #1 \right| \! \right|}} \newcommand{\ip}[1]{{\left\langle #1 \right\rangle}} \newcommand{\tr}[1]{{{#1}^\textsf{T}}} \]

What’s a quadratic in variables \(x_1, x_2, \ldots, x_n\)?

Preview Activity 32.1

Definition 1 Let \(A\) be an \(n\times n\) symmetric matrix. A quadratic form on \(\R^n\) is a function \(Q\) defined by

\[ Q(\x) = \tr{\x} A \x. \]

For a given quadratic form, the matrix \(A\) is unique, so will be referred to as the matrix of the quadratic form.

Note that we can assume \(A\) is symmetric by replacing \(a_{ij}\) and \(a_{ji}\) with their average since \(x_i x_j\) and \(x_j x_i\) will both show up in the expansion.

PA solutions.

A question

Given an equation of the form \(Q(\x) = d\), where \(Q\) is a quadratic form on \(\R^2\) and \(d\) is a constant, how can we eliminate the cross terms?

That is, given a quadratic form \(Q\) in variables \(x_1, x_2, \ldots, x_n\), we want to find variables \(y_1, y_2, \ldots, y_n\) in terms of the \(x\)’s so that when written in terms of \(y_1, y_2, \ldots, y_n\), \(Q\) contains no cross terms, i.e.:

\[ Q(\x) = \tr{\y} D \y, \]

where \(D\) is diagonal.

by using a diagonal matrix!

Details. Let \(Q\) be the qf defined by \(Q(\x) = \tr{\x} A \x\), where \(A\) is an \(n\times n\) symmetric matrix. Since \(A\) is symmetric, we can find an orthogonal matrix \(P = [\mathbf{p}_1 \ \cdots \ \mathbf{p}_n]\) whose columns are orthonormal eigenvectors of \(A\) corresponding to eigenvalues \(\lambda_1, \lambda_2, \ldots, \lambda_n\), respectively. Letting \(\y = \tr{P} \x\) gives us \(\x = P\y\) and

\[ \tr{\x} A \x = \tr{(P\y)} A (P\y) = \tr{\y} (\tr{P} A P) \y = \tr{\y} D \y, \]

where \(D\) is the diagonal matrix whose \(i\)th diagonal entry is \(\lambda_i\).

Moreover, \(\B = \set{\mathbf{p}_1, \ldots, \mathbf{p}_n}\) is an orthonormal basis for \(\R^n\) and so defines a coordinate system for \(\R^n\). note that if \(\y = \tr{[y_1 \ \cdots \ y_n]}\), then

\[ \x = P \y = \sum y_i \mathbf{p}_i, \]

so \([\x]_\B = \y\).

Summary: Theorem 32.2

Theorem 1 (Principal Axis Theorem) Let \(A\) be an \(n\times n\) symmetric matrix. There is an orthogonal change of variables \(\x = P \y\) so that the quadratic form \(Q\) defined by \(Q(\x) = \tr{\x} A \x\) is transformed into the quadratic form \(\tr{\y} D \y\), where \(D\) is a diagonal matrix.

The columns of the orthogonal matrix \(P\) form an orthogonal basis for \(\R^n\) and are called the principal axes for the quadratic form \(Q\). The coordinate vector wrt this basis for \(\x\) is \(\y\).

Activity 32.1

Let \(Q\) be the quadratic form defined by \(Q(\x) = 2x^2 + 4xy + 5y^2 = \tr{\x} A\x\), where \(\x = \left[\begin{matrix} x \\ y \end{matrix}\right]\) and \(A = \left[\begin{matrix} 2 & 2 \\ 2 & 5 \end{matrix}\right]\).

• The eigenvalues of \(A\) are \(\lambda_1 = 6\) and \(\lambda_2 = 1\) with corresponding eigenvectors \(\v_1 = \tr{[1 \ 2]}\) and \(\v_2 = \tr{[-2 \ 1]}\), respectively. Find an orthogonal matrix \(P\) with determinant 1 that diagonalizes \(A\). Is \(P\) unique? Is there a matrix without determinant 1 that orthogonally diagonalizes \(A\)? Explain.
• Use the matrix \(P\) to write the quadratic form without the cross-product.

Classifying Quadratic Forms

Definition

A symmetric matrix \(A\) (and its associated quadratic form \(Q\)) is

• positive definite if \(\tr{\x} A \x > 0\) for all \(\x\ne \mathbf{0}\),
• positive semidefinite if \(\tr{\x} A \x \ge 0\) for all \(\x\),
• negative definite if \(\tr{\x} A \x < 0\) for all \(\x\ne \mathbf{0}\),
• negative semidefinite if \(\tr{\x} A \x \le 0\) for all \(\x\), and
• indefinite if \(\tr{\x} A \x\) takes on both positive and negative values.

Connecting to eigenvalues

A quadratic form \(Q(\x) = \tr{\x} A \x\) is

• positive definite if \(A\) has all positive eigenvalues,
• negative definite if \(A\) has all negative eigenvalues,
• positive semidefinite if \(A\) has all nonnegative eigenvalues,
• negative semidefinite if \(A\) has all nonpositive eigenvalues, and
• indefinite if it has some of both.

Making an inner product

Let \(A\) be a symmetric \(n\times n\) matrix, and define \(\ip{,} : \R^n\times \R^n \to \R\) by

\[ \ip{\u,\v} = \tr{\u} A\v. \]

• Explain why it is necessary for \(A\) to be positive definite in order for this to define an inner product on \(\R^n\).
• Show that in this case, the function does define an inner product.
• Let \(\ip{,} : \R^2 \times \R^2 \to \R\) be defined by

\[ \ip{\left[\begin{matrix} x_1 \\ x_2 \end{matrix}\right], \left[\begin{matrix} y_1 \\ y_2 \end{matrix}\right]} = 2 x_1 y_1 - x_1 y_2 - x_2 y_1 + x_2 y_2. \]

Find a matrix \(A\) so that \(\ip{\x,\y} = \tr{\x} A \y\) and explain why \(\ip{,}\) defines an inner product.