Math 303: Section 29

Dr. Janssen

\[ \def\R{{\mathbb R}} \def\P{{\mathbb P}} \def\B{{\mathcal B}} \def\C{{\mathcal C}} \def\S{{\mathcal S}} \def\b{{\mathbf{b}}} \def\c{{\mathbf{c}}} \def\x{{\mathbf{x}}} \def\y{{\mathbf{y}}} \def\u{{\mathbf{u}}} \def\v{{\mathbf{v}}} \def\w{{\mathbf{w}}} \def\z{{\mathbf{z}}} \def\e{{\mathbf{e}}} \def\r{{\mathbf{r}}} \def\M{{\mathcal{M}}} \DeclareMathOperator{\null}{Nul} \DeclareMathOperator{\span}{Span} \DeclareMathOperator{\dim}{dim} \DeclareMathOperator{\proj}{proj} \DeclareMathOperator{\row}{Row} \DeclareMathOperator{\col}{Col} \DeclareMathOperator{\trace}{trace} \newcommand{\set}[1]{\left\{ {#1} \right\}} \newcommand{\setof}[2]{{\left\{#1\,\colon\,#2\right\}}} \newcommand{\norm}[1]{{\left|\! \left| #1 \right| \! \right|}} \newcommand{\ip}[1]{{\left\langle #1 \right\rangle}} \]

An **inner product** \(\ip{\ , \ }\) on a vector space \(V\) is a mapping \(V\times V\to \R\) satisfying:

- \(\ip{\u,\v} = \ip{\v,\u}\) for all \(\u,v\in V\)
- \(\ip{\u + \v, \w} = \ip{\u,\w} + \ip{\v,\w}\) for all \(\u,\v,\w\in V\)
- \(\ip{c\u, \v} = c\ip{\u,\v}\) for all \(\u,\v\in V\) and scalars \(c\)
- \(\ip{\u, \u} \ge 0\) for all \(\u\) in \(V\) and \(\ip{\u,\u} = 0\) if and only if \(\u = \mathbf{0}\)

A vector space \(V\) with an inner product \(\ip{\ , \ }\) defined on \(V\) is called an **inner product space**.

- If \(a_1, \ldots, a_n\) are positive scalars, then \(\ip{[u_1 \ u_2 \cdots \ u_n]^\textsf{T}, [v_1 \ v_2 \cdots \ v_n]^\textsf{T}} = a_1 u_1 v_1 + a_2 u_2 v_2 + \cdots + a_n u_n v_n\) defines an inner product on \(\R^n\)
- Every invertible \(n\times n\) matrix \(A\) defines an inner product on \(\R^n\) by \(\ip{\u, \v} = (A\u) \cdot (A\v)\).
- The definite integral defines an inner product: \(\ip{f,g} = \displaystyle\int_a^b f(x) g(x)\, dx\), for \(f,g\in C[a,b]\)
- The trace of an \(n\times n\) matrix defines an inner product on \(\M_{n\times n}\): \(\ip{A,B} = \trace\left(AB^\textsf{T}\right)\). This is called the
*Frobenius*inner product.

Let \(\u\) be a vector in an inner product space \(V\).

- Why is \(\ip{\u, \mathbf{0}} = \ip{\u,\mathbf{0}} + \ip{\u, \mathbf{0}}\)?
- How does the equation in the previous part show that \(\ip{\u,\mathbf{0}} = 0\)?

Let \(\ip{\ , \ }\) be an inner product on a vector space \(V\) and let \(\u,\v, \w\) be vectors in \(V\) and \(c\) a scalar. Then

- \(\ip{\mathbf{0},\v} = \ip{\v,\mathbf{0}} = 0\)
- \(\ip{\u,c\v} = c\ip{\u,\v}\)
- \(\ip{\u,\v+\w} = \ip{\u,\v} + \ip{\u,\w}\)
- \(\ip{\u-\v,\w} = \ip{\u,\w} - \ip{\v,\w}\)

Let \(\v\) be a vector in an inner product space \(V\). The **length**^{1} of \(\v\) is the real number

\[ \norm{\v} = \sqrt{\ip{\v,\v}}. \]

A vector \(\v\) in an inner product space is a **unit vector** if \(\norm{\v} = 1\).

If \(\u,\v\) are vectors in an inner product space \(V\), the **distance between \(\u\) and \(\v\)** is the length of the difference \(\u - \v\), \(\norm{\u - \v}\).

Find the length of the vectors \(\u = [1 \ 3]^\textsf{T}\) and \(\v = [3 \ 1]^\textsf{T}\) using the inner product

\[ \ip{[u_1 \ u_2]^\textsf{T}, [v_1 \ v_2]^\textsf{T}} = 2u_1 v_1 + 3 u_2 v_2 \]

in \(\R^2\).

Then, find the distance between the polynomials \(p(t) = t+1\) and \(q(t) = t^2 - 1\) in \(C[0,1]\) using the inner product \(\ip{f,g} = \int_0^1 f(x) g(x)\, dx\).

Let \(\u,\v\) be nonzero vectors in an inner product space \(V\). The **angle \(\theta\) between \(\u\) and \(\v\)** is such that

\[ \cos(\theta) = \frac{\ip{\u,\v}}{\norm{\u}\ \norm{\v}} \]

and \(0\le \theta \le \pi\).

We say \(\u\) and \(\v\) are **orthogonal** if \(\ip{\u,\v} = 0\).

- Find a nonzero vector in \(\R^2\) orthogonal to the vector \(\u = [3 \ 1]^\textsf{T}\) using the inner product \(\ip{[u_1 \ u_2]^\textsf{T}, [v_1\ v_2]^\textsf{T}} = 2 u_1 v_1 + 3 u_2 v_2\).
- Determine if the vector \(\v = \left[\begin{matrix} 0 \\ 3 \\ -2 \end{matrix}\right]\) is orthogonal to the vector \(\w = \left[\begin{matrix} -1 \\ 0 \\ 1 \end{matrix}\right]\) using the inner product \(\ip{\u, \v} = (A\u)\cdot (A\v)\) on \(\R^3\), where \(A = \left[\begin{matrix} 0 & 1 & 1 \\ 1 & 1 & 0 \\ 0 & 1 & 0 \end{matrix}\right]\).
- Find the angle between the two polynomials \(p(t) = 1\) and \(q(t) =t\) in \(\P_1\), with inner product \(\ip{r(t), s(t)} = \int_0^1 r(t) s(t)\, dt\).

A subset \(S\) of an inner product space \(V\) for which \(\ip{\u,\v} = 0\) for all \(\u\ne \v\) in \(S\) is called an **orthogonal set**.

**Theorem 1** Let \(\set{\v_1, \v_2, \ldots, \v_m}\) be a set of nonzero orthogonal vectors in an inner product space \(V\). Then the vectors \(\v_1, \v_2, \ldots, \v_m\) are linearly independent.

An **orthogonal basis** \(\mathcal{B}\) for a subspace \(W\) of an inner product space \(V\) is a basis of \(W\) that is also an orthogonal set.

**Theorem 2** Let \(\mathcal{B} = \set{\v_1, \v_2, \ldots, \v_m}\) be an orthogonal basis for a subspace of an inner product space \(V\). Let \(\x\) be a vector in \(W\). Then

\[ \x = \frac{\ip{\x, \v_1}}{\ip{\v_1, \v_1}} \v_1 + \frac{\ip{\x, \v_2}}{\ip{\v_2, \v_2}} \v_2 + \cdots + \frac{\ip{\x, \v_m}}{\ip{\v_m, \v_m}} \v_m. \]

Let \(p_1(t) = 1- t\), \(p_2(t) = -2 + 4t + 4t^2\), and \(p_3(t) = 7 - 41t +40t^2\) be vectors in the inner product space \(\P_2\) with inner product defined by \(\ip{p(t), q(t)} = \int_0^1 p(t) q(t)\, dt\). Let \(\B = \set{p_1(t), p_2(t), p_3(t)}\). You may assume that \(\B\) is an orthogonal basis for \(\P_2\). Let \(z(t) = 4 - 2t^2\). Find the weight \(x_3\) so that \(z(t) = x_1 p_1(t) + x_2 p_2(t) + x_3 p_3(t)\). Use technology as appropriate to evaluate any integrals.

An **orthonormal basis** \(\B = \set{\v_1, \v_2, \ldots, \v_m}\) for a subspace \(W\) of an inner product space \(V\) is an orthogonal basis such that \(\norm{\v_k} = 1\) for \(1\le k\le m\).

**Activity 29.5**. Construct an orthonormal basis for the subspace

\[ W = \span \set{\left[\begin{matrix} 1 \\ 1 \\ 1 \\ 0 \end{matrix}\right], \left[\begin{matrix} -1 \\ 1 \\ -1 \\ 2 \end{matrix}\right], \left[\begin{matrix} 8 \\ 5 \\ -31 \\ -3 \end{matrix}\right]} \]

of the inner product space \(\R^4\) with inner product

\[ \ip{[u_1 \ u_2 \ u_3 \ u_4]^\textsf{T}, [v_1 \ v_2 \ v_3 \ v_4]^\textsf{T}} = 2 u_1 v_1 + 3 u_2 v_2 + u_3 v_3 + 5 u_4 v_4. \]

(Note that you need to check for orthogonality!)

Let \(W\) be a subspace of an inner product space \(V\) and let \(\B = \set{\w_1, \w_2, \ldots, \w_m}\) be an orthogonal basis for \(W\). For a vector \(\v\) in \(V\), the **orthogonal projection of \(\v\) onto \(W\)** is the vector

\[ \proj_W \v = \frac{\ip{\v,\w_1}}{\ip{\w_1,\w_1}}\w_1 + \frac{\ip{\v,\w_2}}{\ip{\w_2,\w_2}}\w_2 + \cdots + \frac{\ip{\v,\w_m}}{\ip{\w_m,\w_m}}\w_m. \]

The **projection of \(\v\) orthogonal to \(W\)** is the vector

\[ \proj_{W^\perp} \v = \v - \proj_W \v. \]

Let \(W = \span \set{\left[\begin{matrix} 1 \\ 1 \\ 1 \\ 0 \end{matrix}\right], \left[\begin{matrix} -1 \\ 1 \\ -1 \\ 2 \end{matrix}\right], \left[\begin{matrix} 8 \\ 5 \\ -31 \\ -3 \end{matrix}\right]}\) in \(\R^4\). Find the projection of the vector \(\v = [2 \ 0 \ 0 \ 1]^\textsf{T}\) onto \(W\) using the inner product

\[ \ip{[u_1 \ u_2 \ u_3 \ u_4]^\textsf{T}, [v_1 \ v_2 \ v_3 \ v_4]^\textsf{T}} = 2 u_1 v_1 + 3 u_2 v_2 + u_3 v_3 + 5 u_4 v_4. \]

Show directly that \(\proj_{W^\perp} \v\) is orthogonal to the basis vectors for \(W\).

**Theorem 3 (Generalized Pythagorean Theorem)** Let \(\u\) and \(\v\) be orthogonal vectors in an inner product space \(V\). Then

\[ \norm{\u - \v}^2 = \norm{\u}^2 + \norm{\v}^2. \]

Let \(W\) be a subspace of an inner product space \(V\) and let \(\u\) be a vector in \(V\). Then

\[ \norm{\u - \proj_W \u} < \norm{\u - \x} \]

for every vector \(\x\) in \(W\) different from \(\proj_W \u\).

In \(\R^n\) with the dot product, if \(\v = [v_1 \ v_2 \ \cdots \ v_n]^\textsf{T}\) and \(\proj_W \v = [w_1 \ w_2 \ \cdots \ w_n]^\textsf{T}\), then the square of the error in approximating \(\v\) by \(\proj_W \v\) is given by

\[ \norm{\v - \proj_W \v}^2 = \sum\limits_{i=1}^n (v_i - w_i)^2. \]

Thus \(\proj_W \v\) minimizes this sum of squares over all vectors in \(W\), and so we call it the *least squares approximation* to \(\v\).

The set \(\B = \set{1, t - \frac{1}{2}, t^3 - \frac{9}{10}t + \frac{1}{5}}\) is an orthogonal basis for a subspace \(W\) of the inner product space \(\P_3\) using the inner product \(\ip{p(t), q(t)} = \int_0^1 p(t) q(t)\, dt\).

Find the polynomial in \(W\) that is closest to the polynomial \(r(t) = t^2\) and give a numeric estimate of how good this approximation is.

The **orthogonal complement** of a subset \(S\) of an inner product space \(V\) is the set

\[ S^\perp = \setof{\v\in V}{\ip{\v,\u} = 0 \text{ for all } \u\in S}. \]

As in \(\R^n\):

**Theorem 4** Let \(\B = \set{\w_1, \ldots, \w_m}\) be a basis for a subspace \(W\) of an inner product space \(V\). A vector \(\v\) in \(V\) is orthogonal to every vector in \(W\) if and only if \(\v\) is orthogonal to every vector in \(\B\).

Consider \(\P_2\) with the inner product \(\ip{p(t), q(t)} = \int_0^1 p(t) q(t)\, dt\).

- Find \(\ip{p(t), 1-t}\) where \(p(t) = a + bt + ct^2\) is in \(\P_2\).
- Describe as best you can the orthogonal complement of \(\span \set{1-t}\) in \(\P_2\). Is \(p(t) = 1 - 2t - 2t^2\) in this orthogonal complement? Is \(p(t) = 1 + t - t^2\)?

Let \(V\) be an inner product space of dimension \(n\) and let \(W\) be a subspace of \(V\). Let \(\x\) be any vector in \(V\). We will demonstrate that \(\x\) can be written uniquely as a sum of a vector in \(W\) and a vector in \(W^\perp\).

- Explain why \(\proj_W \x\) is in \(W\) and \(\proj_{W^\perp} \x\) is in \(W^\perp\).
- Explain why \(\x\) can be written as a sum of vectors, one in \(W\) and one in \(W^\perp\).
- Suppose \(\x = \w + \w_1\) and \(\x = \u + \u_1\), where \(\w,\u\in W\) and \(\w_1, \u_1 \in W^\perp\). Show that \(\w = \u\) and \(\w_1 = \u_1\), so that the representation of \(\x\) as a sum of a vector in \(W\) and a vector in \(W^\perp\) is unique. [Hint: what is \(W\cap W^\perp\)?]

Let \(V\) be a finite-dimensional inner product space and let \(W\) be a subspace of \(V\). Any vector in \(V\) can be written in a unique way as a sum of a vector in \(W\) and a vector in \(W^\perp\).