The Dot Product in Euclidean Space

Math 303: Section 27

Dr. Janssen

Intro

\[ \def\R{{\mathbb R}} \def\P{{\mathbb P}} \def\B{{\mathcal B}} \def\C{{\mathcal C}} \def\S{{\mathcal S}} \def\b{{\mathbf{b}}} \def\c{{\mathbf{c}}} \def\x{{\mathbf{x}}} \def\y{{\mathbf{y}}} \def\u{{\mathbf{u}}} \def\v{{\mathbf{v}}} \def\w{{\mathbf{w}}} \def\z{{\mathbf{z}}} \def\e{{\mathbf{e}}} \def\r{{\mathbf{r}}} \def\M{{\mathcal{M}}} \DeclareMathOperator{\null}{Nul} \DeclareMathOperator{\span}{Span} \DeclareMathOperator{\dim}{dim} \DeclareMathOperator{\proj}{proj} \DeclareMathOperator{\row}{Row} \DeclareMathOperator{\col}{Col} \newcommand{\set}[1]{\left\{ {#1} \right\}} \newcommand{\setof}[2]{{\left\{#1\,\colon\,#2\right\}}} \newcommand{\norm}[1]{{\left|\! \left| #1 \right| \! \right|}} \]

Recall

The length/magnitude/norm of the vector \(\v = \left[\begin{matrix} v_1 \\ v_2 \end{matrix}\right]\) in \(\R^2\) is

\[ \norm{\v} = \sqrt{v_1^2 + v_2^2}. \]

The dot product of vectors \(\u = [ u_1 \ u_2 \ \cdots \ u_n]^\textsf{T}\) and \(\v = [ v_1 \ v_2 \ \cdots \ v_n]^\textsf{T}\) in \(\R^n\) is

\[ \u\cdot \v = u_1 v_1 + u_2 v_2 + \cdots + u_n v_n. \]

The norm \(\norm{\v}\) of \(\v\in\R^n\) is

\[ \norm{\v} = \sqrt{\v\cdot\v}. \]

Alternatively, \(\u\cdot\v = \u^\textsf{T}\v\).

Preview Activity 27.1 Discussion

Theorem 1 Let \(\u,\v,\w\) be vectors in \(\R^n\) and let \(c\) be a scalar. Then

• \(\u\cdot \v = \v\cdot \u\)
• \((\u + \v)\cdot \w = (\u\cdot \w) + (\v\cdot \w)\)
• \((c\u) \cdot \v = \u\cdot (c\v) = c(\u\cdot \v)\)
• \(\u\cdot \u \ge 0\) with equality if and only if \(\u = \mathbf{0}\)
• \(\norm{c\u} = |c| \norm{\u}\)

Distance between vectors

Definition 1 Let \(\u,\v\in\R^n\). The distance between \(\u\) and \(\v\) is the length of the difference \(\u - \v\), or

\[ \norm{\u - \v}. \]

draw from same point. Looking at the length of the vector between \(Q\) and \(P\).

Angle between two vectors

Thus,

\[ \cos\theta = \frac{\u\cdot\v}{\norm{\u} \norm{\v}}. \]

Determining a “best” solution to a problem often involves finding a solution that minimizes a dis- tance. We generally accomplish a minimization through orthogonality – which depends on the angle between vectors. Given two vectors u and v in Rn, we position the vectors so that they emanate from the same initial point. If the vectors are nonzero, then they determine a plane in Rn. In that plane there are two angles that these vectors create. We will define the angle between the vectors to be the smaller of these two angles.

Derivation on pp. 481–482.

Activity 27.1

• The vectors \(\e_1 = [1 \ 0]^\textsf{T}\) and \(\e_2 = [0 \ 1]^\textsf{T}\) are perpendicular in \(\R^2\). What is \(\e_1 \cdot \e_2\)?
• Suppose \(\u,\v\) are any vectors in \(\R^n\). If their angle is \(90^\circ\), what does the previous equation tell us about \(\u\cdot \v\)? What if \(\u\cdot\v = 0\)–what is the angle between the two vectors?
• Explain why the following definition makes sense: two vectors \(\u\), \(\v\) in \(\R^n\) are orthogonal if \(\u\cdot\v = 0\).
• According to the previous definition, to which vectors is \(\mathbf{0}\) orthogonal? Does this make intuitive sense? Explain.

Activity 27.2

• Find the angle between the two vectors \(\u = [1 \ 3 \ -2 \ 5]^\textsf{T}\) and \(\v = [5 \ 2 \ 3 \ -1]^\textsf{T}\).
• Find, if possible, two non-parallel vectors orthogonal to \(\u = [ 0 \ 3 \ -2 \ 1]^\textsf{T}\).

Orthogonal projections

When running a sprint, the racers may be aided or slowed by the wind. The wind assistance is a measure of the wind speed that is helping push the runners down the track. It is much easier to run a very fast race if the wind is blowing hard in the direction of the race. So that world records aren’t dependent on the weather conditions, times are only recorded as record times if the wind aiding the runners is less than or equal to 2 meters per second. Wind speed for a race is recorded by a wind gauge that is set up close to the track. It is important to note, however, that weather is not always as cooperative as we might like. The wind does not always blow exactly in the direction of the track, so the gauge must account for the angle the wind makes with the track. If the wind is blowing in the direction of the vector u in Figure 27.6 and the track is in the direction of the vector v in Figure 27.6, then only part of the total wind vector is actually working to help the runners. This part is called the orthogonal projection of the vector u onto the vector v.

Draw the picture

Activity 27.3: Calculating Projection

Since the orthogonal project \(\proj_\v \u\) is in the direction of \(\v\), there is a constant \(c\) such that \(\proj_\v \u = c\v\). So let’s solve for \(c\).

• The component of \(\u\) acting perpendicular to \(\v\) is called the projection of \(\u\) orthogonal to \(\v\) and is denoted \(\proj_{\perp \v} \u\). Write an equation involving \(\proj_{\perp \v} \u\), \(\proj_\v \u\), and \(\u\). Solve for \(\proj_{\perp \v} \u\).
• Given that \(\v\) and \(\proj_{\perp \v} \u\) are orthogonal, what does that tell us about \(\v \cdot \proj_{\perp \v} \u\)? Combine this with the previous part and the observation that \(\proj_\v \u = c\v\) to obtain an equation involving \(\v\), \(\u\), and \(c\).
• Solve for \(c\).
• Use the value of \(c\) to identify \(\proj_\v \u\).

Definition

Let \(\u, \v\) be vectors in \(\R^n\) with \(\v\ne \mathbf{0}\).

The orthogonal projection of \(\u\) onto \(\v\) is the vector

\[ \proj_\v \u = \frac{\u\cdot\v}{\norm{\v}^2} \v. \]

The projection of \(\u\) orthogonal to \(\v\) is the vector

\[ \proj_{\perp \v} \u = \u - \proj_\v \u. \]

Activity 27.4

Let \(\u = \left[\begin{matrix} 5 \\ 8 \end{matrix}\right]\) and \(\v = \left[\begin{matrix} 6 \\ - 10 \end{matrix}\right]\). Find \(\proj_\v \u\) and \(\proj_{\perp \v} \u\) and draw a picture to illustrate.

Orthogonal Complements

The definition

Let \(W\) be a subspace of \(\R^n\) for some \(n\ge 1\). The orthogonal complement of \(W\) is the set

\[ W^\perp = \setof{\x \in \R^n}{\x \cdot \w = 0 \text{ for all } \w\in W}. \]

PA 27.2

Let \(W = \span\set{[1 \ -1]^\textsf{T}}\) in \(\R^2\). Completely describe all vectors in \(W^\perp\) both algebraically and geometrically.

observation: orthogonal projection onto a vector is really a projection on to the span of that vector, i.e., the whole line.

The vector \(\proj_\v \u\) is the best approximation to \(\u\) of all the vectors in \(\span{\v}\) in the sense that it’s the closes to \(\u\) among all vectors in \(\span \set{\v}\).

we’ve seen this before in the definition of a plane in \(\R^3\) as the set of all vectors normal to a given vector \(\mathbf{n}\).

Activity 27.5

We have seen another example of orthogonal complements. Let \(A\) be an \(m\times n\) matrix with rows \(\r_1, \r_2, \ldots, \r_m\) in order. Consider the three spaces \(\null A\), \(\row A\), and \(\col A\) related to \(A\), where \(\row A = \span\set{\r_1, \r_2, \ldots, \r_m}\). Let \(\x\) be a vector in \(\row A\).

• What does it mean for \(\x\) to be in \(\row A\)?
• Now let \(\y\) be a vector in \(\null A\). Use the result of the previous part and the fact that \(A\y = \mathbf{0}\) to explain why \(\x\cdot y = \mathbf{0}\). Explain how this verifies that \((\row A)^\perp = \null A\).
• Use \(A^\textsf{T}\) in place of \(A\) in the previous part to show that \((\col A)^\perp = \null A^\textsf{T}\).

Summary Theorem 27.9

Let \(A\) be \(m\times n\). Then

\[ (\row A)^\perp = \null A \text{ and } (\col A)^\perp = \null A^\textsf{T}. \]

It’s enough to check a basis

Theorem 2 Let \(\mathcal{B} = \set{\w_1, \w_2, \ldots, \w_m}\) be a basis for a subspace \(W\) of \(\R^n\). A vector \(\v\) in \(\R^n\) is orthogonal to every vector in \(W\) if and only if \(\v\) is orthogonal to every vector in \(\mathcal{B}\).

Prove the reverse direction? Forward is obvious.

Activity 27.6

Let \(W = \span \set{\left[\begin{matrix} 1 \\ 1 \\ 0 \end{matrix}\right], \left[\begin{matrix} 0 \\ 0 \\ 1 \end{matrix}\right]}\). Find all vectors in \(W^\perp\).