in 
is said to be an orthogonal set if
Orthogonal Sets
Summary
orthogonal set: A set of vectors in
is said to be an orthogonal set if
In many cases we like our bases to be orthogonal (that is, the vectors to be mutually perpendicular). Even better are orthonormal bases, in which the orthogonal vectors are of unit length.
Theorem 4: If 
is an orthogonal set of
nonzero vectors in , then S is linearly independent and hence is a basis
for the subspace spanned by S.
orthogonal basis: an orthogonal basis for a subspace W of
is a basis for W that is also an orthogonal set.
Theorem 5: Let 
be an orthogonal basis
for a subspace W of . For each 
in W, the weights in the linear
combination
are given by
Whoops! We have a notational collision: the author wants to use the ``hat''
symbol to indicate the orthogonal projection of y onto another vector. I
don't like this, because the vector 
is different for different
vectors. Mathworld (maintainers of Mathematica), many other mathematicians, and
I like to reserve the ``hat'' for unit vectors
To make your lives easier, I'll give up my notation, albeit unhappily. I'll
indicate unit vectors by the notation 
: hence,
So
I prefer this right-most form of the projection, as it makes clear what's going
on: we form a unit vector in the direction of u, cast a
shadow along this unit vector using the inner product, and then weight the
normal vector by this coefficient. This corresponds to the
``shadow'' cast by the vector onto the of
vector u. Then we can write
where z is orthogonal to u. We can rewrite equation 1 as
which just says that we break vector y into its components along the
orthogonal direction to represent it. This is what we do with our ordinary
basis of vectors 
, 
, and 
.
The fact of the matter is that orthonormal bases are used more often than orthogonal bases, so we generally are working with normalized vectors.
Theorem 6: An 
matrix U has orthonormal columns if and only
if 
.
Theorem 7: Let U be an matrix with orthonormal columns, and
let x and y be in . Then
if and only if 
.
Exercise #25, p. 393
Orthogonal matrix: a square matrix such that 
, having
orthonormal columns. It's ironic that the name is ``orthogonal'', rather than
``orthonormal''. Feel free to call such a matrix an orthonormal matrix.
Curiously enough, orthonormal columns in an orthogonal matrix imply that the rows are also orthonormal:
Example: #28, p. 393
