TABLE OF CONTENTS
Preface 1
1 Preliminaries 5
1.1 Banach spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Hilbert spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.3 Self-adjoint operators . . . . . . . . . . . . . . . . . . . . . . . 26
1.4 Spectral decomposition . . . . . . . . . . . . . . . . . . . . . . . 30
2 Bilinear Maps and Forms 34
2.1 Bilinear maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.2 Bilinear Forms and Spaces . . . . . . . . . . . . . . . . . . . . . 37
2.3 Notions of Orthogonality. Orthogonal bases . . . . . . . . . . . 42
2.4 Isometries and Similarities of a nondegenerate bilinear form . . . 45
2.5 Matrix representation and Diagonization Theorem . . . . . . . . 45
2.6 Representation of bounded bilinear forms on real Hilbert spaces 50
3 Quadratic forms 52
3.1 Generalities on Quadratic Forms and Spaces . . . . . . . . . . . 52
3.2 N-ary quadratic forms (Quadratic forms on Kn) . . . . . . . . . . 55
3.3 Reduction of Quadratic forms on nite dimensional real vector
spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.4 Quadratic forms on Hilbert spaces . . . . . . . . . . . . . . . . . 60
– Quadratic forms dened with compact self-adjoint operators
on separable spaces . . . . . . . . . . . . . . . . . . . . . 60
– Gelfand triples. Rayleigh quotients. Eigenvalue Problems. . . . 62
4 Applications 63
4.1 Quadratic forms and Unconstrained Optimization . . . . . . . . 63
– Quadratic Optimization . . . . . . . . . . . . . . . . . . . . . . 64
4.2 Optimization of Convex Functions . . . . . . . . . . . . . . . . . 65
– Optimization of convex functions of class C2. . . . . . . . . . . 65
– Linear Regression in Statistics . . . . . . . . . . . . . . . . . . 67
– Mean-Square Approximation . . . . . . . . . . . . . . . . . . . 67
– Lax-Milgram Theorem . . . . . . . . . . . . . . . . . . . . . . 68
ii
CHAPTER ONE
Preliminaries
Throughout the work, K will hold either for the eld of real numbers R or for
the eld of complex numbers C.
This part is devoted to a review of some basic notions from Functional
Analysis and to the introduction of bilinear forms.. The ideal spaces on which
we shall work are Hilbert spaces (e.g. euclidean spaces) that are particular
(and even pratical) Banach spaces.
1.1 Banach spaces
Denition 1.1.1
Let X be a linear space over K. A norm on X is any nonnegative real-valued
function jj jj on X satisfying the following conditions :
i) 8 x 2 X, jjxjj = 0 () x = 0: (nondegeneracy)
ii) jjxjj = jj jjxjj ; 8 x 2 X and 8 2 K: (homogeneity)
iii) jjx + yjj jjxjj + jjyjj, 8 x; y 2 X : (subadditivity)
A linear space X endowed with a norm jj jj is called a normed linear space
and is denoted by (X; jj jj).
A normed linear space (X; jj jj) of which norm is not ambiguous will be
simply denoted by X.
Denition 1.1.2
Let X be a K-linear space. Two norms jj jj1 and jj jj2 on X are said to be
equivalent if there exist positive constants and such that
jjxjj1 jjxjj2 jjxjj1 8 x 2 X :
Denition 1.1.3
5
Let (X; jj jj) be a normed linear space. A sequence (xn)n of elements of X
converges in (X; jj jj), if there exists an element a 2 X such that
lim
n!+1
jjxn ajj = 0 in R:
In this case a is unique (due to the triangular inequality property of the
norm) and we also say that (xn)n converges to a with respect to the norm jj jj
and then write
lim
n!+1
xn = a :
Denition 1.1.4
Let (X; jj jj) be a normed linear space.
1. A subset F of X is said to be closed if every sequence (an)n of elements
of A which converges to some element x 2 X, has its limit x in A. That
is:
(an)n A converges in X =) lim
n!+1
an 2 A:
2. A subset U of X is said to be open if its complement; Uc = X n U, is
closed.
3. The collection = of all open sets of the normed linear space (X; jj jj)
denes a topology on X. In this topological space (X; =), = is called the
norm topology.
4. The closure of a subset A of X is the smallest closed set (of X) that
contains A.
The closure of A is in fact the intersection of all closed sets of X containing
A.
It is denoted by A or cl(A).
A subset D X of which closure is equal to X, is said to be dense in
X.
5. The interior of a subset A of X is the largest open set (of X) which is
contained in A.
The interior of A is in fact the union of all open sets contained in A.
It is denoted by A or int(A).
6. A subset A X is said to be bounded if there exists a positive constant
M such that
x 2 A =) jjxjj < M :
This means that A is contained in some open ball B(0; M).
Now we dene the most important topological concept for computational
purposes, namely the concept of separability.
6
Denition 1.1.5
A normed linear space (X; jj jj) is said to be separable if it contains a dense
subset D which is at most countable.
Proposition 1.1.6
Let X be a K-linear space and assume that jj jj1 and jj jj2 on X are two
equivalent norms on X. Then a sequence of elements of X converges with
respect to jj jj1 if and only if it does with respect to jj jj2 :
Therefore two equivalent norms dene the same topology on X:
Vice-versa if two norms dene the same topology, then they are equivalent.
However two metrics may dene the same topology without being strongly equiv-
alent!
Denition 1.1.7
Let (X; jj jj) be a normed linear space. A sequence (xn)n of elements of X is
said to be a Cauchy sequence if
lim
m;n!+1
jjxn xmjj = 0 ; that is,
8″ > 0; 9N 2 N : jjxn xmjj < ” for all n N and all m N :
Proposition 1.1.8.
In a normed linear space,
1) every convergent sequence is a Cauchy sequence,
2) and every Cauchy sequence is bounded.
Denition 1.1.9 (Banach space)
A normed linear space (X; jj jj) in which every Cauchy sequence is convergent
is called a Banach space.
Denition 1.1.10
Given a normed linear space (X; jj jjX), any Banach space (E; jj jjE) such
that there exists an isometry j : X ! E with dense range in E; i.e.,
(i) jjj(x)jjE = jjxjjX for all x 2 X, and
(ii) j(X) = E,
7
is called a completion of X.
This means that E is a completion of X if E is a Banach space which contains
a dense subset isometric to X.
For instance :
i) The completion of C([0; 1]) equipped with the norm jj jj2 dened by
jjfjj2 =
hR 1
0 jf(t)j2dt
i1
2 , is (isometric to) L2(]0; 1[).
ii) The completion of C1([0; 1]) equipped with the norm jj jjW1;2 dened by
jjfjjW1;2 =
hR 1
0 jf(t)j2dt +
R 1
0 jf0(t)j2dt
i1
2 , is (isometric to) H1(]0; 1[).
Theorem 1.1.11 (Hausdor)
Every normed linear space has a completion.
Denition 1.1.12
Let (X; jj jjX) and (Y; jj jjY ) be two arbitrary normed linear spaces. A map
f : X ! Y is said to be continuous at a point a 2 X if for every sequence
(xn)n of X converging to a with respect to jj jjX, the sequence
f(xn)
n converges
to f(a) in Y with respect to jj jjY .
f is said to be continuous (on X) if it is continuous at every point of X.
Equivalently, f is continuous if and only if the pre-image of every open set in
Y is an open set in X.
Recall that given two K-linear spaces X and Y ,
a map or operator
T : X ! Y
is said to be linear if for all x1; x2 2 X and for all 1; 2 2 K we
have
T(1×1 + 2×2) = 1T(x1) + 2T(x2) :
Equivalently, T : X ! Y is linear if for all x1; x2 2 X and for all
2 K, we have
T(x1 + x2) = T(x1) + T(x2) :
a map or operator
T : X ! Y
8
is said to be antilinear if for all x1; x2 2 X and for all 1; 2 2 K
we have
T(1×1 + 2×2) = 1T(x1) + 2T(x2) :
Equivalently, T : X ! Y is linear if for all x1; x2 2 X and for all
2 K, we have
T(x1 + x2) = T(x1) + T(x2) :
Theorem 1.1.13
Let (X; jj jjX) and (Y; jj jjY ) be normed linear spaces. Then a linear map
T : X ! Y is continuous if and only if T is a bounded linear map in the sense
that there exists a constant real number 0 such that
jjT(x)jjY jjxjjX 8x 2 X:
Notations 1.1.14
Let X and Y be two given arbitrary normed linear spaces.
The set of all bounded linear maps (i.e. continuous linear maps) from X into
Y is a linear space that will be denoted by B(X; Y ). When X = Y , we
simply write B(X) instead of B(X;X).
Given a bounded linear map T : X ! Y , we shall set
jjTjjB(X;Y ) = inf
n
k : jjT(x)jjY kjjxjjX 8 x 2 X
o
that will be simply written as jjTjj when there is no ambiguity.
We denote by X := B(X;K) the topological dual of X; that is, the set of all
continuous linear functionals of X.
Elements of X are also called continuous linear forms or bounded linear
forms.
Proposition 1.1.15
Let (X; jj jjX) be a nontrivial normed linear space and (Y; jj jjY ) be an
arbitrary normed linear space. Then for every T 2 B(X; Y ), we have
jjT(x)jjY jjTjj jjxjjX 8 x 2 X ;
and
jjTjj = sup
jjxjjX1
jjT(x)jjY = sup
jjxjjX=1
jjT(x)jjY = sup
jjxjjX6=0
jjT(x)jjY
jjxjjX
:
9
Theorem 1.1.16
Let (X; jj jjX) and (Y; jj jjY ) be normed linear spaces. Then
1.
B(X; Y ) ; jj jjB(X;Y )
is a normed linear space.
2. If moreover (Y; jj jjY ) is a Banach space, then
B(X; Y ); jj jjB(X;Y )
is
a Banach space.
Corollary 1.1.17
The dual X of any normed linear space X is (always) a Banach space.
Remark 1.1.18
Given a normed linear space (X; jj jjX), the dual X being a normed linear
space (in fact a Banach space) has also a dual X called the bidual of X.
Moreover there exists a canonical injection J : X ,! X dened by
J : X ! X
x 7! J(x) ;
where J(x) is the continuous form on X dened by
hJ(x); fi := hf; xi := f(x) ; 8 f 2 X :
Denition 1.1.19 (Re exive space)
A normed linear space (X; jj jjX) is re exive if it is a Banach space such that
the canonical injection J : X ,! X is surjective.
Theorem 1.1.20 (Hahn-Banach)
Let (X; jj : jj) be a normed linear space and E X be a linear subspace of X:
If f : E ! R is a continuous linear functional, then there exists F 2 X that
extends f such that
jjFjjX = jjfjjE :
Corollary 1.1.21
For every x0 2 X, there exists f0 2 X such that
jjf0jj = jjx0jj and hf0; x0i = jjx0jj2:
10
Theorem 1.1.22 (Uniform Boundedness Principle)
Let (X; jj jjX) and (Y; jj jjY ) be two Banach spaces and let fTigi2I be a
family (not necessarily countable) of continuous linear operators from X into
Y . Assume that
sup
i2I
jjTi(x)jj < 1 8x 2 X:
Then
sup
i2I
jjTijjB(X;Y ) < 1:
In other words there exists a constant c > 0 such that
jjTi(x)jj cjjxjj 8x 2 X; 8i 2 I:
Theorem 1.1.23 (Open Mapping Theorem)
Let (X; jj jjX) and (Y; jj jjY ) be two Banach spaces and let T be a continuous
linear operator from X into Y that is surjective. Then there exists a constant
c > 0 such that
BY (0; c) T(BX(0; 1)):
Corollary 1.1.24 (Banach Isomorphism Theorem)
Let (X; jj jjX) and (Y; jj jjY ) be two Banach spaces and let T be a continuous
linear operator from X into Y that is bijective. Then the inverse map T1 is
also continuous ( from Y into X).
Theorem 1.1.25 (Closed Graph Theorem)
Let (X; jj jjX) and (Y; jj jjY ) be two Banach spaces and let T be a linear
operator from X into Y: Assume that the graph of T, G(T), is closed in EF:
Then T is continuous.
Remark 1.1.26
The converse is obviously true, since the graph of any continuous map (linear
or not) is closed.
Denition 1.1.27 (Compactness)
A subset A of a Banach space X is said to be compact if every cover of A by
open sets of X, has a nite subcover.
Proposition 1.1.28
Every compact set of a Banach space is closed and bounded.
The converse is not true in general.
11
Theorem 1.1.29 (Bolzano-Weierstrass)
A subset A of a Banach space X is said to be compact if and only if any
sequence of A has a subsequence that converges to some point of A.
Theorem 1.1.30 (Heine-Borel)
Given a natural number n, a subset of the euclidean space Rn is compact if
and only if it is closed and bounded.
Heine-Borel Theorem fails in innite dimensional Banach spaces. In fact we
have the following characterization of nite dimensional normed spaces over
R.
Theorem 1.1.31 (Riesz)[19]
Let E be a Banach space over the eld K. Then the closed unit ball of E is
compact if and only if the dimension of E is nite.
Thus Riesz Theorem characterizes the compactness of the closed unit ball
of a Banach space E by the niteness of the dimension of E.
Therefore, we need other types of topologies on innite dimensional spaces
Denition 1.1.32
Let E be a real Banach space. To each f in E, we assign the map
f : E ! R
x 7! f (x) = hf; xi:
We denote the family of all such maps from E into R by ffgf2E .
The weak topology on E (denoted by !) is the smallest topology on E which
makes the maps f continuous.
Proposition 1.1.33
Let (E; !) be a real Banach space endowed with the weak topology ! . Then
! is Hausdor; that is, for any two dierent points x1 and x2 taken in (E; !);
there exists two respective disjoint weakly open neighbourhoods U1 and U2;
that is U1, U2 belongs to ! and U1 \ U2 = ;:
Consequently if a sequence fxngn2N in (E; !) converges weakly to some x
(i.e xn * x) in (E; !), then x is unique.
12
Proposition 1.1.34
A sequence fxngn2N in a real Banach space E converges weakly to some x in
E if and only if f(xn) ! f(x) for each f 2 E:
Proposition 1.1.35
Given a nite dimensional normed space E, the norm-topology (strong topology)
and the weak topology coincide on E.
Theorem 1.1.36 (Eberlein-Smulyan)
A real Banach space E is re exive if and only if every (norm) bounded sequence
in E has a subsequence which converges weakly to an element of E.
On the dual of a normed space, we can dene a weaker topology as follows.
Denition 1.1.37
Let E be a real Banach space. Dene for every x 2 E, the map ‘x dened on
E by
‘x(f) = f(x) ; 8 f 2 E :
Then the weak topology on E is dened as the smallest topology on E for
which the maps ‘x; x 2 E, are continuous.
The weak topology of E (denoted by !) is the
Proposition 1.1.38
Let E be a Banach space. A sequence of bounded linear forms ffngn2N of
elements of E converges weakly to some f 2 E if and only if fn(x) ! f(x)
for each x 2 E:
Proposition 1.1.39
Given a nite dimensional normed space E, the norm-topology (strong topology),
the weak topology and the weak topology coincide on East.
13
Theorem 1.1.40 (Banach-Alaoglu)[19]
For every Banach space E, the closed unit ball is weakly compact.
Now we consider the spectral properties of bounded linear operators.
Denitions 1.1.41 (Spectra and resolvents)
Let E be a Banach space over K and T be a bounded linear operator of E;
i.e., T 2 B(E). The spectrum (T) of T is dened by
(T) = f 2 K : I T is not invertible in B(E)g :
The resolvent set (T) of T is the complementary of (T) in K; that is,
(T) = K n (T):
The elements of (T) are called the regular values of T.
If 2 (T), then the operator R(T) = (I T)1 is called the resolvent
operator of T at .
The spectrum is decomposed into the disjoint union of the following three
sets:
a) The point spectrum of T :
p(T) =
n
2 K : Ker(I T) 6= f0g
o
:
b) The continuous spectrum of T :
c(T) =
n
2 K : Ker(IT) = f0g; R(I T) = E but R(IT) 6= E
o
:
c) The residual spectrum of T :
r(T) =
n
2 K : Ker(I T) = f0g; R(I T) 6= E
o
:
Remark/Denition 1.1.42 [24],[19]
1. Given T 2 B(E), an element of p(T) is called an eigenvalue of T and a
non-zero vector v such that Tv = v is called an eigenvector of T associated
to the eigenvalue . Eigenvalues and eigenvectors are sometimes
called characteristic values and characteristic vectors respectively.
More generally if 2 p(T), n 2 N and v is a nonzero element of E such
that (I T)nv = 0, then v is called a principal vector associated with
the eigenvalue .
14
2. Given an eigenvalue of an operator T, the geometric multiplicity of is
by denition the dimension of Ker(IT). And the algebraic multiplicity
of is dened as follows:
when E is nite-dimensional and equipped with a basis, it is the multiplicity
of as a root of the characteristic polynomial of the matrix
associated to T,
more generally for an arbitrary nontrivivial Banach space E, it is the
dimension of the vector subspace
1[
k=1
Ker(I T)k :
Thus the algebraic multiplicity of an eigenvalue is always greater
than or equal to its geometric multiplicity.
The notion of spectrum can be dened for unbounded operators dened
in a normed space.
Example 1.1.43 [24],[30]
1. Let n be a natural number and A : Cn ! Cn be a linear operator with
corresponding matrix A. then it is well-known that T has n eigenvalues
1; : : : ; n (counted with their multiplicities) and that the eigenvalues
are the roots of the characteristic polynomial P() = det(A In) for
A. Since A In) is invertible when is not an eigenvalue, it follows
that the spectrum (A) of A is a pure point spectrum, namely
(A) =
k : 1 k n
= p(A) ;
and that the resolvent set of A is the complex plane except nitely many
points, namely
(A) = C n
k : 1 k n
:
2. Let E = C([0; 1]) be equipped with the supremum norm and consider the
operator T : E ! E; f 7! Tf dened by
[Tf](x) =
Z x
0
f(s) ds ; 8x 2 [0; 1] :
Then it is not hard to check that T has no eigenvalue, T is injective
but not surjective and its range is not dense in E, and moreover TI is
invertible in B(E). Therefore
(T) = f0g = r(T) :
15
Theorem 1.1.44 [20]
Let E be a Banach space over C and T 2 B(E). Then the following holds for
the spectrum of T.
i) (T) is a closed subset of C
ii) (T) B(0; jjTjjB(X))
iii) (T) is a compact subset of C
Denition 1.1.45 (Spectral radius) [20]
Let E be a Banach space over C and T 2 B(E). Then the spectral radius of T
is dened by
r(T) := sup
n
jj : 2 (T)
o
= max
n
jj : 2 (T)
o
:
Futhermore, we have the Gelfang formula
r(T) = lim
n!1
kTnk
1
n = inf
n2N
kTnk
1
n :
Denition 1.1.46 (Compact linear maps)
Let E and F be two Banach Spaces over K. A linear map, T : E ! F is said
to be compact if the image of the closed unit ball BE(0; 1) by T is a relatively
compact subset of F.
In other words, T is compact if T
BE(0; 1)
is compact.
This denition is equivalent to each of the following properties.
i) For each bounded subset B E, the image set T(B) is relatively compact
in F.
ii) For every bounded sequence fxngn2N E, the sequence fTxngn2N has a
convergent subsequence in F:
Proposition 1.1.47 (Properties of compact linear maps) [20]
1. Every compact linear map is bounded.
2. For all Banach spaces E and F over the eld K, the set K(E; F) of
compact linear maps from E into F is a linear subspace of the space
B(E; F) of bounded linear maps from E into F.
3. Let E, F and G be Banach spaces, and let T : E ! F, S1 : F ! G
and S2 : G ! E be bounded linear operators. Then
16
i) If the range of T; R(T), is nite dimensional, then T is compact.
Therefore, every linear map dened on a nite dimensional normed
space is not only bounded, but also compact.
ii) If T is compact, then S1 T and T S2 are compact.
Therefore the set K(E) of compact linear operators (endomorphisms)
of E is a two-sided ideal of the algebra B(E) of bounded linear operators
(endomorphisms) of E.
iii) The limit of a convergent sequence of compact linear operators with
respect to the operator norm, is compact. Thus K(E) in closed in
B(E).
Next, we state the Riesz-Schauder spectral theory of compact linear operators.
Theorem 1.1.48 (Riesz-Schauder spectral theory)[19],[22]
Let E be a complex Banach space and T be a compact linear operator of E.
Then:
1. The spectrum of T consists of an at most countable set of points of the
complex plane which has no point of accumulation except possibly = 0.
2. Every nonzero number of the spectrum of T is an eigenvalue of T of nite
multiplicity.
3. The dual operator of T denoted by T 2 B(E) and dened by
T(f) = f T for every f 2 E ;
is also compact and a nonzero number is an eigenvalue of T if and only
if it is an eigenvalue of T.
Remark 1.1.49 [22]
The notion of dual operator is an extension of the notion of transposed matrix.
In fact, if E is nite dimensional and equipped with a given basis, then the
matrix of the dual of a linear operator of E is the transposed matrix of the
matrix of T. And More generally, if E is a normed space and T 2 B(E), then
using the duality pairing of E E, we have
hf; Txi = hT f; xi 8 x 2 E :
The above Theorem 1.1.48 can be rephrased as follows.
17
Theorem 1.1.50
Let E be a complex Banach space and T be a compact linear operator of E.
1. If 2 (T) and 6= 0, then is an eigenvalue of T of nite multiplicity.
2. (T) is either nite or countably innite.
3. If (T) is innite, then
(T) = f0g [
n : n = 1; 2; : : :
;
where fngn2N is a sequence of complex numbers converging to 0.
As a corollary we have
Theorem 1.1.51 [20]
Let E be an innite dimensional complex Banach space and T 2 B(E) be
compact. Then The following holds.
1. 0 2 (T)
2. (T) n f0g consists of eigenvalues of nite multiplicity.
3. (T) n f0g is either empty, nite or a sequence of complex numbers
converging to 0. That is (T) n f0g is a discrete set with no limit point
other than 0.
1.2 Hilbert spaces
Denition 1.2.1 (Inner product)
An inner product on a K-linear space E is any functional h ; i dened on EE
which is a positive hermitian and nondegenerate form; that is,
h ; i : E E ! K
(x; y) 7! hx; yi
and satises the following conditions :
1. hx; xi 0 8 x 2 E, and hx; xi = 0 if and only if x = 0.
2. hy; xi = hx; yi 8 x; y 2 E.
3. h1x1 + 2×2; yi = 1hx1; yi + 2hx2; yi 8 x1; x2; y 2 E
and 8 x1; 2 2 K.
18
Remark 1.2.2
1. A form : E E ! K which is linear with respect to its rst argument
and antilinear with respect to its second argument is said to be
sesquilinear.
If : E E ! K is sesquilinear and satisfy
(x; y) = (y; x) 8 x; y 2 E ;
then it is also called a hermitian sesquilinear form.
2. When K = R; that is, E is a real vector space, a hermitian (sesquilinear)
form on E is just called a symmetric bilinear form on E.
Denition 1.2.3
A linear space endowed with an inner product is called an inner product space
or a prehilbertian space.
A nite dimensional real prehilbertian space is also called a euclidean space
Theorem 1.2.4 (Cauchy-Schwarz-Bunyakovsky Inequal-
ity)
Let (E; h ; i) be an inner product space. Then
jhx; yij2 hx; xi hy; yi 8 x; y 2 E:
The equality of this inequality holds if and only if x and y are linearly dependent.
Theorem 1.2.5 (Norm induced by an inner product)
Let (E; h ; i) be an inner product space. Then the function
jj x jjE : E ! R
x 7!
p
hx; xi
denes a norm on E:
Denition 1.2.6 (Hilbert space)
An inner product space E is called a Hilbert space (usually denoted by H), if
(E; jj : jjE) is a Banach space.
19
Remark 1.2.7 (Hilbert space)
Finite dimensional real Hilbert spaces are also called Euclidean spaces. And
since all nite dimensional normed spaces are Banach spaces, there is no dierence
between nite dimensional real prehilbertian spaces and nite dimensional
real Hilbert spaces.
Theorem 1.2.8 (Parallelogram law and Polarization identity)
Let (E; h ; i) be an inner product space. Then we have
1. the parallelogram law :
jjx + yjj2 + jjx yjj2 = 2(jjxjj2 + jjyjj2) 8 x; y 2 E ;
2. and the Polarization identity :
hx; yi =
1
4
jjx+yjj2jjxyjj2+ijjx+iyjj2ijjxiyjj2
8 x; y 2 E :
Note that the parallelogram law characterizes the norms that are induced
by inner products according to a theorem by Von Neumann.
Denitions 1.2.9 (Orthogonality)
Let (E; h ; i) be an inner product space.
Two vectors x and y in E are said to be orthogonal (written x?y and read x
‘perp’ y) if hx; yi = 0:
If M is a non empty subset of E, we write x?M (and read x orthogonal
to M) if x is orthogonal to every element of M.
Given a non-empty subset M of E, we denote by M?, the set of all elements
of E which are orthogonal to M: That is,
M? =
n
x 2 E ; hx; yi = 0 8 y 2 M
o
:
The set M? is then called the orthogonal of M, it is a closed vector subspace
of E.
Theorem 1.2.10 (Projection Theorem)
Let H be a Hilbert space and M a closed subspace of H. For arbitrary vector
x in H, there exists a unique vector y? 2 M such that,
jjx y?jjH = inf
y2M
jjx yjjH:
Furthermore, x 2 M is such a vector if and only if (x x)?M.
The Projection Theorem yields the following denition end theorems.
20
Denition 1.2.11 (Direct Sum of vectors spaces)
Let X and Y be two subspaces of a vector space E. Then E is said to be the
direct sum of X and Y if
E = X + Y and X \ Y = f0g .
This means that every vector u 2 E has a unique decomposition of the form
u = x + y with x 2 X and y 2 Y:
in this case we write E = X Y .
Theorem 1.2.12 (Direct Sum Decomposition)
Let F be a closed subspace of a Hilbert space H. Then,
H = F F? :
Therefore we say that F? is the orthogonal complement of F
Theorem 1.2.13 (Riesz Representation)
Let H be a Hilbert space and let f be a bounded linear functional on H. Then,
(i) There exists a unique vector yo in H such that
f(x) = hx; yoi; for every x 2 H:
(ii) Moreover, jjfjjH = jjyojjH:
Corollary 1.2.14
If H be a Hilbert space, then H ‘ H via the canonical map
‘ : H ! H
a 7! ‘a ;
where ‘a is dened by
‘a = hx; ai ; 8 x 2 H :
This canonical map is bijective, isometric and antilinear.
Corollary 1.2.15
Real Euclidean and Hilbert spaces are re exive.
21
Denitions 1.2.16 (Hilbertian basis)
Let H be a Hilbert space.
1. A unit vector of H is a vector of H is equal to 1.
2. A family
u
2 of nonzero vectors of H is said to be orthogonal, if the
vectors of the family are pairwise orthogonal; that is,
hu ; ui = 0 ; for all 6= in :
3. A family
u
2 of (nonzero) vectors of H is said to be orthonormal
if these vectors are all unit vectors and pairwise orthogonal; that is,
8<
:
jjujj =
p
hu ; ui = 1 ; for all 2 :
hu ; ui = 0 ; for all 6= in :
4. A family A of vectors of H is said to be total or complete if the vector
subspace spanned by A is dense is H; that is,
Span(A ) = H :
5. A family of vectors of H that is both orthonormal and complete is called
a hilbertian basis.
A hilbertian basis can be nite, countable or uncountable.
The crucial dierence between a hilbertian basis and a (nite) orthonor-
mal basis is that in the rst case the expansion of a vector may not be a
linear combination of some of the elements of the hilbertian basis, but a
series of vectors!
Examples 1.2.17
In the real space
`2 =
(
u = (un)n1 R;
X1
n=1
u2
n < 1
)
endowed with the inner product dened by
hu; vi =
X1
n=1
unvn ;
the following vectors orthonormal and complete.
e1 = (1; 0; 0; 0; : : 🙂
…
ek = (k;n)n where k;n = 1 si n = k and k;n = 0 otherwise
…
22
Proposition 1.2.18
Let H be a Hilbert space. Then
1. For every vector subspace F of H, we have
F??
= F :
In particular if F is a closed subspace, then
F?
?
= F :
2. For every nonempty subset A of H, A? is a closed vector subspace of H
(and so a Hilbert subspace) and
A??
= Span(A) :
3. A nonempty set S of H is complete (or total) if and only if
S? = f0g :
Theorem 1.2.19 [17]
Let (H; h ; i) be a Hilbert space. Then
1. H has a hilbertian basis (i.e., an orthonormal basis) that can be nite
countable or uncountable. Furthermore, every orthonormal set in H is
contained in some hermitian basis.
2. A hilbertian basis of H is at most countable if and only if H is separable;
that is, H contains a dense subset that is at most countable.
3. For every orthonormal sequence of vectors fekgk2N of elements of H, we
have Bessel Inequality:
8 x 2 H;
X+1
k=1
jhx; ekij2 jjxjj2 :
4. If H is separable and fekgk2N is a complete and orthonormal sequence
of vectors of H, then we have Parseval Identity:
8 x 2 H;
X+1
k=1
jhx; ekij2 = jjxjj2 :
23
Proposition 1.2.20
Let (H; h ; i) be an innite dimensional Hilbert space and fekgk2N be an
orthonormal set of H. Then
1. For every x 2 H, we have
lim
k!+1
hx; eki = 0 :
2. Every bounded sequence of scalars fkgk2N gives a bounded linear operator
T on H dened by
Tx =
X+1
k=1
khx; ekiek ; 8x 2 H ;
of which norm is jjTjj = supk1 jkj.
Proof
1. Follows from the convergence of the numerical series
P+1
k=1 jhx; ekij2
according to Bessel inequality.
2. Follows from the convergence in H of the series
P+1
k=1 khx; ekij2 and
the bounds
jjTxjj sup
k1
jkj jjxjj ; 8x 2 H
and
sup
k1
jjTekjj = sup
k1
jkj :
Theorem 1.2.21 (Construction of compact linear operators) [24]
Let (H; h ; i) be an innite dimensional, separable complex Hilbert space with
orthonormal basis (ek)k2N and let (k)k2N be an arbitrary sequence of
complex numbers. For every x 2 H, consider the series
Tx =
X+1
k=1
khx; ekiek :
Then
1. The series is convergent and the sum denes a linear operator T on H if
the sequence (k)k2N is bounded.
2. T exists and is bounded if and only if the sequence (k)k2N is bounded.
When k 2 f0; 1g for all k, T is an orthogonal projection.
3. T exists and is a compact linear operator if and only if the sequence
(k)k2N converges to 0.
When only nitely many of the terms k are nonzero, T is a nite rank
operator.
24
Denition 1.2.22 (Numerical range)[19]
Let H be a complex Hilbert space and T be a bounded linear operator on H.
The numerical range of T is dened by
W(T) =
n
hTx; xi : x 2 H; jjxjj = 1
o
:
Proposition 1.2.23 [19]
Let H be a complex Hilbert space and T be a bounded linear operator on H.
Then
(T) W(T) ;
where
W(T) =
n
hTx; xi : x 2 H; jjxjj = 1
o
: :
More precisely, if 62 W(T), then 2 (T) with
(T I)1
1
dist(; W(T)
:
Denition 1.2.24 [6],[27]
Let H be a complex Hilbert space and T be a bounded linear operator on H.
The numerical radius of T is dened by
w(T) = sup
jhTx; xij : x 2 H; jjxjj = 1
o
:
It satises the following inequalities
r(T) w(T) and
jjTjj
2
w(T) jjTjj :
As a result of the Riesz representation Theorem, we have the following characterization
of weak convergence in a Hilbert space.
Proposition 1.2.25
Let H be a Hilbert space.
A sequence fxngn2N of elements of H converges weakly to some a 2 H if and
only if
lim
n!+1
hxn; yi = ha; yi ; 8 y 2 H :
25
Theorem 1.2.26 [24],[6]
Let H be a Hilbert space and (xn)n2N be a weakly convergent sequence with
limit point x. then for every compact linear operator T on H, the image
sequence
Txn
n2N converges strongly (in norm) to Tx.
That is, for any compact linear operator T 2 B(H), we have
xn * x for n ! +1 =) Txn ! Tx for n ! +1:
Corollary 1.2.27
Let (en)n2N be an orthonormal sequence of a Hilbert space H. Then for every
compact linear operator T of H, we have
lim
n!+1
Ten = 0 with respect to the norm topology of H :
Proof. By Bessel inequality, we have that (en)n2N converges weakly to 0 in H.
Therefore the corollary follows from the above Theorem [ ].
1.3 Self-adjoint operators
Let n be a natural number and T : Cn ! Cn be a linear operator. Then T
can be represented by a complex square matrix of order n. Suppose moreover
that T is self-adjoint. Then the matrix associated to T is conjugate symmetric
and it is well-known from basic Linear Algebra that all the eigenvalues of this
matrix are all real and that there exists an orthonormal basis (e1; : : : ; en) for
Cn in which the matrix of T is diagonal, meaning also that the linear operator
T can be expressed as follows :
Tx =
Xn
k=1
khx; ekiek ; 8 x 2 Cn ;
where hx; eki coincide with kth coordinate of x in the orthonormal basis (e1; : : : ; en)
of eigenvectors for T corresponding respectively to the eigenvalues
k
1kn.
For innite dimensional Hilbert spaces, the situation is much more tremendous,
but for self-adjoint compact linear operators, a corresponding theory can
be well developped. It will be culminated in the famous spectral theorem.
Denition 1.3.1 (Symmetric or self-adjoint operators) [19],[18],[20],[24]
Let H be a Hilbert space over K and A : H ! H be a bounded linear operator.
A is said to be symmetric or self-adjoint if
hAx; yi = hx; Ayi ; 8 x; y 2 H :
26
Remark 1.3.2 [24],[22]
1. Given an arbitrary bounded linear operator T on a Hilbert space H, the
adjoint operator of T is the unique bounded linear operator of H denoted
by T and satisfying
hTx; yi = hx; Tyi ; 8 x; y 2 H :
The existence and properties of T are based on the Riesz representation
theorem and can be proved following the idea of the proof of [Representation
theorem of a real bounded bilinear form].
Therefore a bounded linear operator T on a Hilbert space is symmetric
or self-adjoint if T = T.
2. Let H be a Hilbert space. Then any map T dened from H into H that
satises
hTx; yi = hx; Tyi ; 8 x; y 2 H ;
is necessarily linear and bounded. In fact :
a) the linearity holds since for all x; y 2 H and for all ; 2 K, we
have
T(x + y) Tx Ty
2 H? = f0g ;
b) and the boundedness follows from the Closed graph Theorem […]
3. Given a Hilbert space H and a linear operator T dened from a dense
domain D(T) H into H, the adjoint operator of T is dened as the
unique operator T with domain D(T) and such that
hTx; yi = hx; Tyi ; 8 x 2 D(T) and 8y 2 D(T) :
In this case, a linear unbounded operator T dened on a dense domain
D(T) of a Hilbert space H is said to be symmetric if T is an extension
of T, this amounts to:
hTx; yi = hx; Tyi ; 8 x; y 2 D(T) :
And again, a linear unbounded operator T dened on a dense domain
D(T) of a Hilbert space H is said to be self-adjoint if T = T. This
means that not only T is symmetric, but also D(T) = D(T).
Proposition 1.3.3 [24],[19]
The numerical range of any bounded, self-adjoint linear operator on a complex
Hilbert space is a subset of R.
In particular for any bounded, self-adjoint linear operator T on a Hilbert
space over K, hTx; xi is a real number for all x 2 H.
27
Proof. For all x 2 H, we have
hTx; xi = hx; Txi = hTx; xi :
Proposition 1.3.4 [24]
Let T be a bounded self-adjoint operator on a complex Hilbert space H. Then
all its eigenvalues are real numbers. Furthermore, any pair of eigenvectors
corresponding to dierent eigenvalues are orthogonal.
Proof. This is a well-known result.
If Tv = v for 2 C and v 2 H n f0g, we get
hv; vi = hv; vi = hTv; vi = hv; Tvi = hv; vi = hv; vi :
And since hv; vi = jjvjj2 6= 0; we get = meaning that is real.
Note that roughly prouving, is real because hTv; vi is real and hv; vi
is a nonzero real number.
If 1 6= 2 are two dierent eigenvalues corresponding respectively to two
eigenvectors v1 and v2, then 1 and 2 are real numbers and we have
1hv1; v2i = h1v1; v2i = hTv1; v2i
= hv1; Tv2i
= hv1; 2v2i = 2hv1; v2i = 2hv1; v2i :
Therefore
(1 2)hv1; v2i = 0
and so hv1; v2i = 0 since 1 2 6= 0.
We have the following alternative formular for the norm of a bounded se –
adjoint operator.
Proposition 1.3.5 [24],[19]
Let T be a bounded self-adjoint operator on a Hilbert space H (over K). Then
jjTjj = sup
x2H; jjxjj=1
jhTx; xij :
Proof.(Sketch)
Set = supjjxjj=1 jhTx; xij:
28
– First of all we have clearly jjTjj.
– Let x; y 2 H. By expending hT(x + y); x + yi and hT(x y); x yi
we get
hTx; yi + hTy; xi = 1
2
hT(x + y); x + yi hT(x y); x yi
2 (jjx + yjj2 + jjx yjj2)
(jjxjj2 + jjyjj2) (using parallelogram law):
And so
hTx; yi + hTy; xi
jjxjj2 + jjyjj2
:
– If x 2 H is such that Tx 6= 0, then by setting y = jjxjj
jjTxjjTx and by
applying the inequality of the previous step, we get
jjTxjj jjxjj :
The latter inequality is also (directly) satised even tough Tx = 0.
Therefore,
jjTxjj jjxjj ; 8x 2 H ;
yielding jjTjj .
Proposition 1.3.6 [18]
Let T be a bounded self-adjoint operator on a real Hilbert space H. Dene
the lower and upper bounds
m = inf
x2H; jjxjj=1
hTx; xi and M = sup
x2H; jjxjj=1
hTx; xi :
Then
1. (T) [m; M].
2. m; M 2 (T).
3. jjTjj = maxfm; Mg.
Corollary 1.3.7 [18]
1. For every bounded linear self-adjoint operator T on a real Hilbert space,
either jjTjj or jjTjj is an approximate eigenvalue (i.e., a limit of a sequence
of eigenvalues of T).
2. Consequently, every nonzero compact linear self-adjoint operator T on a
real Hilbert space, has either jjTjj or jjTjj as an eigenvalue; that is,
jjTjj; jjTjj
\ p(T) 6= ; :
29
1.4 Spectral decomposition
This section deals with the spectral decomposition of a compact linear selfadjoint
operator on a separable Hilbert space.
Theorem 1.4.1 (Hilbert-Schmidt) [24], [18], [28]
Let T be a compact self-adjoint operator on a separable Hilbert space H of
nite or innite dimension. Then H admits an at most countable orthonormal
basis consisting of eigenvectors for T. More precisely
1. In the nite dimensional case, the numbering of the nite sequence of
basis vectors (e1; :::; en) can be chosen such that the corresponding nite
sequence of eigenvalues (1; :::; n) decreases numerically (in absolute
value) :
j1j j2j : : : jnj :
And in the basis of eigenvectors, the operator T is described by :
Tx =
Xn
k=1
khx; ekiek for all x 2 H = Span
ek ; k = 1; :::; n
:
2. In the separable, innite dimensional case :
a) If T = 0, then any orthonormal basis of the separable Hilbert space
H is a countable orthonormal basis consisting of eigenvectors of T.
b) If T 6= 0, is of nite rank n 1, then
H = ker(T)R(T) with [ker(T)]? = R(T) and dim[R(T)] = n :
In this case R(T) is nite dimensional and invariant under T.
Therefore R(T) has a nite orthonomal basis (e1; :::; en) consisting
of eigenvectors of the restriction of T to its range R(T) such that
their corresponding eigenvalues satisfy
j1j j2j : : : jnj (all nonzero):
Again T is described by
Tx =
Xn
k=1
khx; ekiek for all x 2 H :
By adding to (e1; :::; en) an orthonormal basis of ker(T) (also separable),
we shall obtain a countable orthonormal basis for H consisting
of eigenvectors of T.
30
c) Otherwise
R(T) is innite dimensional
, we can nd an innite sequence
of orthonormal eigenvectors
ek
k2N ; in fact an orthonormal
basis of R(T) consisting of eigenvectors, with a corresponding sequence
of nonzero eigenvalues
k
k2N that decreases numerically
and tends to 0;
j1j j2j : : : jkj : : : with lim
k!+1
k = 0 ;
and for which T can be described as
Tx =
X+1
k=1
khx; ekiek for all x 2 H :
Note in this case that
p(T) f0g [
k : k = 1; 2; : : :
:
Proof (Sketch).
If the range of T is nite dimensional, then we can proceed by induction.
Now we only consider the case in which the range of T is innite dimensional.
– Set 0 = 0 and let
1 ; 2 ; : : :
be the countable set of all the nonzero eigenvalues of T. Consider the
subspace
H0 = ker(T) (that may be null) ;
and the eigenspaces
Hk = ker(T kI) ; k = 1; 2; : : : :
For k 1, dim(Hk) 1 since Hk is an eigenspace, and dim(Hk) <
+1 since T is compact (cf. Theorem 1.1.48, by which every nonzero
eigenvalue k must have a nite multiplicity). Thus
1 dim(Hk) < +1; 8 k 1 :
1. For any m 6= n in
0; 1; 2 : : : g, Hm and Hn are orthogonal. (See the
proof of theorem …)
2. We have H = k0Hk where (recall)
k0Hk =
vk0 + : : : + vkn : n 0; 0 j n; kj 0; vkj 2 Hkj
:
To see this, set
V = k=0Hk
31
and suppose by contradiction that V 6= H. Therefore V ? 6= f0g and
V ? \ ker(T) V ? \ H0 V ? \ V = f0g :
Moreover, it is not hard to show that T maps V ? into V ?. Thus the
restriction of T to V ? would be a nonzero, compact self-adjoint operator
of V ? and would have at least one nonzero eigenvalue. This would imply
by Corrolary 1.3.7 that T has an eigenvector in V ? with a nonzero
eigenvalue contradicting the fact that
V ? \ [k1Hk V ? \ V = f0g :
3. For each k 1, the nite dimensional subspace Hk possesses a nite
orthonormal basis
Bk =
n
ek; 1 ; ek; 2 ; : : : ; ek; nk
o
:
Besides the closed subspace H0 = ker(T) of the separable space H, is
either null, in which case we set B0 = ;, or admits an at most countable
orthonormal basis B0.
It follows that
B = [+1
k=0Bk
is a countable orthonormal basis of eigenvectors of T. Furthermore
Tx =
X+1
k=1
Xnk
j=1
khx; ek; jiek; j for all x 2 H :
Constructive proof (Sketchy)
We shall prove again this Theorem 1.4.1 by successive applications of Corollary
1.3.7.
Let H1 = H assumed to be nontrivial and set T1 = T.
By the second part of Corollary 1.3.7,
there exist an eigenvalue 1 of T1 and a corresponding eigenvector ‘1 such
that jj’1jj = 1 and j1j = jjT1jj. Set H2 := f’1g?. Thus H2 is a closed
subspace of H1 and T(H2) H2 (i.e H2 is T-invariant).
Now let T2 be the restriction of T to H2. Then T2 is compact self adjoint
operator in B(H2)
If T2 6= 0,then there exists an eigenvalue 2 of T2 and corresponding eigenvector
‘2 such that jj’2jj = 1 and j2j = jjT2jj jjT1jj = j1j
f’1; ‘2g is orthonormal.
H3 = f’1; ‘2g?
H3 is a closed subspace of H and TH3 H3
32
Letting T3 be the restriction of T to H3, we have that T3 is a compact selfadjoint
operator in B(H3). Continuing in this manner, the process stops when
Tn = 0 or else we get a sequence fng of eigenvalues of T and corresponding
orthonormal set f’1; ‘2; ‘3:::g of eigenvectors such that
jn+1j = jjTn+1jj jjTnjj = jnj n = 1; 2; 3::: (1.4.1)
Claim : If fng is an innite sequence, then n ! 0; n ! 1.
Proof of Claim. Suppose by contradiction , there exist > 0 such that jnj
for all n 2 N
Hence for n 6= m, we have that,
jjT’n T’mjj2 = jj’n ‘mjj2 = 2
n + 2
m > (1.4.2)
But this is impossible, since fT’ng has a convergent subsequence due to the
compactness of T. We therefore conclude that n ! 0; n ! 1.
Now, we prove the representation of T as asserted in the theorem.
Case I. Tn = 0 for some n
xn := x
Xn
k=1
< x; ‘k > ‘k
It is evident that xn is orthogonal to ‘i for 1 i n
Therefore, xn 2 Hn
0 = Tnxn = Tx T(
Xn
k=1
hx; ‘ki’k)
) Tx =
Xn
k=1
khx; ‘ki’k
Case II. Tn 6= 0 for all n 2 N
jjTx
Xn
k=1
khx; ‘ki’kjj = jjTnxnjj jjTnjj jjxnjj
= jnj jjxnjj
jnjjjxjj ! 0
) jjTx
Xn
k=1
khx; ‘ki’kjj as ! 0; n ! 1.
Hence, Tx =
X1
k=1
khx; ‘ki’k :
33
Do you need help? Talk to us right now: (+234) 08060082010, 08107932631 (Call/WhatsApp). Email: [email protected].
IF YOU CAN'T FIND YOUR TOPIC, CLICK HERE TO HIRE A WRITER»
