## TABLE OF CONTENTS

Preface 1

1 Preliminaries 5

1.1 Banach spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.2 Hilbert spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.3 Self-adjoint operators . . . . . . . . . . . . . . . . . . . . . . . 26

1.4 Spectral decomposition . . . . . . . . . . . . . . . . . . . . . . . 30

2 Bilinear Maps and Forms 34

2.1 Bilinear maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.2 Bilinear Forms and Spaces . . . . . . . . . . . . . . . . . . . . . 37

2.3 Notions of Orthogonality. Orthogonal bases . . . . . . . . . . . 42

2.4 Isometries and Similarities of a nondegenerate bilinear form . . . 45

2.5 Matrix representation and Diagonization Theorem . . . . . . . . 45

2.6 Representation of bounded bilinear forms on real Hilbert spaces 50

3 Quadratic forms 52

3.1 Generalities on Quadratic Forms and Spaces . . . . . . . . . . . 52

3.2 N-ary quadratic forms (Quadratic forms on Kn) . . . . . . . . . . 55

3.3 Reduction of Quadratic forms on nite dimensional real vector

spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

3.4 Quadratic forms on Hilbert spaces . . . . . . . . . . . . . . . . . 60

– Quadratic forms dened with compact self-adjoint operators

on separable spaces . . . . . . . . . . . . . . . . . . . . . 60

– Gelfand triples. Rayleigh quotients. Eigenvalue Problems. . . . 62

4 Applications 63

4.1 Quadratic forms and Unconstrained Optimization . . . . . . . . 63

– Quadratic Optimization . . . . . . . . . . . . . . . . . . . . . . 64

4.2 Optimization of Convex Functions . . . . . . . . . . . . . . . . . 65

– Optimization of convex functions of class C2. . . . . . . . . . . 65

– Linear Regression in Statistics . . . . . . . . . . . . . . . . . . 67

– Mean-Square Approximation . . . . . . . . . . . . . . . . . . . 67

– Lax-Milgram Theorem . . . . . . . . . . . . . . . . . . . . . . 68

ii

## CHAPTER ONE

Preliminaries

Throughout the work, K will hold either for the eld of real numbers R or for

the eld of complex numbers C.

This part is devoted to a review of some basic notions from Functional

Analysis and to the introduction of bilinear forms.. The ideal spaces on which

we shall work are Hilbert spaces (e.g. euclidean spaces) that are particular

(and even pratical) Banach spaces.

1.1 Banach spaces

Denition 1.1.1

Let X be a linear space over K. A norm on X is any nonnegative real-valued

function jj jj on X satisfying the following conditions :

i) 8 x 2 X, jjxjj = 0 () x = 0: (nondegeneracy)

ii) jjxjj = jj jjxjj ; 8 x 2 X and 8 2 K: (homogeneity)

iii) jjx + yjj jjxjj + jjyjj, 8 x; y 2 X : (subadditivity)

A linear space X endowed with a norm jj jj is called a normed linear space

and is denoted by (X; jj jj).

A normed linear space (X; jj jj) of which norm is not ambiguous will be

simply denoted by X.

Denition 1.1.2

Let X be a K-linear space. Two norms jj jj1 and jj jj2 on X are said to be

equivalent if there exist positive constants and such that

jjxjj1 jjxjj2 jjxjj1 8 x 2 X :

Denition 1.1.3

5

Let (X; jj jj) be a normed linear space. A sequence (xn)n of elements of X

converges in (X; jj jj), if there exists an element a 2 X such that

lim

n!+1

jjxn ajj = 0 in R:

In this case a is unique (due to the triangular inequality property of the

norm) and we also say that (xn)n converges to a with respect to the norm jj jj

and then write

lim

n!+1

xn = a :

Denition 1.1.4

Let (X; jj jj) be a normed linear space.

1. A subset F of X is said to be closed if every sequence (an)n of elements

of A which converges to some element x 2 X, has its limit x in A. That

is:

(an)n A converges in X =) lim

n!+1

an 2 A:

2. A subset U of X is said to be open if its complement; Uc = X n U, is

closed.

3. The collection = of all open sets of the normed linear space (X; jj jj)

denes a topology on X. In this topological space (X; =), = is called the

norm topology.

4. The closure of a subset A of X is the smallest closed set (of X) that

contains A.

The closure of A is in fact the intersection of all closed sets of X containing

A.

It is denoted by A or cl(A).

A subset D X of which closure is equal to X, is said to be dense in

X.

5. The interior of a subset A of X is the largest open set (of X) which is

contained in A.

The interior of A is in fact the union of all open sets contained in A.

It is denoted by A or int(A).

6. A subset A X is said to be bounded if there exists a positive constant

M such that

x 2 A =) jjxjj < M :

This means that A is contained in some open ball B(0; M).

Now we dene the most important topological concept for computational

purposes, namely the concept of separability.

6

Denition 1.1.5

A normed linear space (X; jj jj) is said to be separable if it contains a dense

subset D which is at most countable.

Proposition 1.1.6

Let X be a K-linear space and assume that jj jj1 and jj jj2 on X are two

equivalent norms on X. Then a sequence of elements of X converges with

respect to jj jj1 if and only if it does with respect to jj jj2 :

Therefore two equivalent norms dene the same topology on X:

Vice-versa if two norms dene the same topology, then they are equivalent.

However two metrics may dene the same topology without being strongly equiv-

alent!

Denition 1.1.7

Let (X; jj jj) be a normed linear space. A sequence (xn)n of elements of X is

said to be a Cauchy sequence if

lim

m;n!+1

jjxn xmjj = 0 ; that is,

8″ > 0; 9N 2 N : jjxn xmjj < ” for all n N and all m N :

Proposition 1.1.8.

In a normed linear space,

1) every convergent sequence is a Cauchy sequence,

2) and every Cauchy sequence is bounded.

Denition 1.1.9 (Banach space)

A normed linear space (X; jj jj) in which every Cauchy sequence is convergent

is called a Banach space.

Denition 1.1.10

Given a normed linear space (X; jj jjX), any Banach space (E; jj jjE) such

that there exists an isometry j : X ! E with dense range in E; i.e.,

(i) jjj(x)jjE = jjxjjX for all x 2 X, and

(ii) j(X) = E,

7

is called a completion of X.

This means that E is a completion of X if E is a Banach space which contains

a dense subset isometric to X.

For instance :

i) The completion of C([0; 1]) equipped with the norm jj jj2 dened by

jjfjj2 =

hR 1

0 jf(t)j2dt

i1

2 , is (isometric to) L2(]0; 1[).

ii) The completion of C1([0; 1]) equipped with the norm jj jjW1;2 dened by

jjfjjW1;2 =

hR 1

0 jf(t)j2dt +

R 1

0 jf0(t)j2dt

i1

2 , is (isometric to) H1(]0; 1[).

Theorem 1.1.11 (Hausdor)

Every normed linear space has a completion.

Denition 1.1.12

Let (X; jj jjX) and (Y; jj jjY ) be two arbitrary normed linear spaces. A map

f : X ! Y is said to be continuous at a point a 2 X if for every sequence

(xn)n of X converging to a with respect to jj jjX, the sequence

f(xn)

n converges

to f(a) in Y with respect to jj jjY .

f is said to be continuous (on X) if it is continuous at every point of X.

Equivalently, f is continuous if and only if the pre-image of every open set in

Y is an open set in X.

Recall that given two K-linear spaces X and Y ,

a map or operator

T : X ! Y

is said to be linear if for all x1; x2 2 X and for all 1; 2 2 K we

have

T(1×1 + 2×2) = 1T(x1) + 2T(x2) :

Equivalently, T : X ! Y is linear if for all x1; x2 2 X and for all

2 K, we have

T(x1 + x2) = T(x1) + T(x2) :

a map or operator

T : X ! Y

8

is said to be antilinear if for all x1; x2 2 X and for all 1; 2 2 K

we have

T(1×1 + 2×2) = 1T(x1) + 2T(x2) :

Equivalently, T : X ! Y is linear if for all x1; x2 2 X and for all

2 K, we have

T(x1 + x2) = T(x1) + T(x2) :

Theorem 1.1.13

Let (X; jj jjX) and (Y; jj jjY ) be normed linear spaces. Then a linear map

T : X ! Y is continuous if and only if T is a bounded linear map in the sense

that there exists a constant real number 0 such that

jjT(x)jjY jjxjjX 8x 2 X:

Notations 1.1.14

Let X and Y be two given arbitrary normed linear spaces.

The set of all bounded linear maps (i.e. continuous linear maps) from X into

Y is a linear space that will be denoted by B(X; Y ). When X = Y , we

simply write B(X) instead of B(X;X).

Given a bounded linear map T : X ! Y , we shall set

jjTjjB(X;Y ) = inf

n

k : jjT(x)jjY kjjxjjX 8 x 2 X

o

that will be simply written as jjTjj when there is no ambiguity.

We denote by X := B(X;K) the topological dual of X; that is, the set of all

continuous linear functionals of X.

Elements of X are also called continuous linear forms or bounded linear

forms.

Proposition 1.1.15

Let (X; jj jjX) be a nontrivial normed linear space and (Y; jj jjY ) be an

arbitrary normed linear space. Then for every T 2 B(X; Y ), we have

jjT(x)jjY jjTjj jjxjjX 8 x 2 X ;

and

jjTjj = sup

jjxjjX1

jjT(x)jjY = sup

jjxjjX=1

jjT(x)jjY = sup

jjxjjX6=0

jjT(x)jjY

jjxjjX

:

9

Theorem 1.1.16

Let (X; jj jjX) and (Y; jj jjY ) be normed linear spaces. Then

1.

B(X; Y ) ; jj jjB(X;Y )

is a normed linear space.

2. If moreover (Y; jj jjY ) is a Banach space, then

B(X; Y ); jj jjB(X;Y )

is

a Banach space.

Corollary 1.1.17

The dual X of any normed linear space X is (always) a Banach space.

Remark 1.1.18

Given a normed linear space (X; jj jjX), the dual X being a normed linear

space (in fact a Banach space) has also a dual X called the bidual of X.

Moreover there exists a canonical injection J : X ,! X dened by

J : X ! X

x 7! J(x) ;

where J(x) is the continuous form on X dened by

hJ(x); fi := hf; xi := f(x) ; 8 f 2 X :

Denition 1.1.19 (Re exive space)

A normed linear space (X; jj jjX) is re exive if it is a Banach space such that

the canonical injection J : X ,! X is surjective.

Theorem 1.1.20 (Hahn-Banach)

Let (X; jj : jj) be a normed linear space and E X be a linear subspace of X:

If f : E ! R is a continuous linear functional, then there exists F 2 X that

extends f such that

jjFjjX = jjfjjE :

Corollary 1.1.21

For every x0 2 X, there exists f0 2 X such that

jjf0jj = jjx0jj and hf0; x0i = jjx0jj2:

10

Theorem 1.1.22 (Uniform Boundedness Principle)

Let (X; jj jjX) and (Y; jj jjY ) be two Banach spaces and let fTigi2I be a

family (not necessarily countable) of continuous linear operators from X into

Y . Assume that

sup

i2I

jjTi(x)jj < 1 8x 2 X:

Then

sup

i2I

jjTijjB(X;Y ) < 1:

In other words there exists a constant c > 0 such that

jjTi(x)jj cjjxjj 8x 2 X; 8i 2 I:

Theorem 1.1.23 (Open Mapping Theorem)

Let (X; jj jjX) and (Y; jj jjY ) be two Banach spaces and let T be a continuous

linear operator from X into Y that is surjective. Then there exists a constant

c > 0 such that

BY (0; c) T(BX(0; 1)):

Corollary 1.1.24 (Banach Isomorphism Theorem)

Let (X; jj jjX) and (Y; jj jjY ) be two Banach spaces and let T be a continuous

linear operator from X into Y that is bijective. Then the inverse map T1 is

also continuous ( from Y into X).

Theorem 1.1.25 (Closed Graph Theorem)

Let (X; jj jjX) and (Y; jj jjY ) be two Banach spaces and let T be a linear

operator from X into Y: Assume that the graph of T, G(T), is closed in EF:

Then T is continuous.

Remark 1.1.26

The converse is obviously true, since the graph of any continuous map (linear

or not) is closed.

Denition 1.1.27 (Compactness)

A subset A of a Banach space X is said to be compact if every cover of A by

open sets of X, has a nite subcover.

Proposition 1.1.28

Every compact set of a Banach space is closed and bounded.

The converse is not true in general.

11

Theorem 1.1.29 (Bolzano-Weierstrass)

A subset A of a Banach space X is said to be compact if and only if any

sequence of A has a subsequence that converges to some point of A.

Theorem 1.1.30 (Heine-Borel)

Given a natural number n, a subset of the euclidean space Rn is compact if

and only if it is closed and bounded.

Heine-Borel Theorem fails in innite dimensional Banach spaces. In fact we

have the following characterization of nite dimensional normed spaces over

R.

Theorem 1.1.31 (Riesz)[19]

Let E be a Banach space over the eld K. Then the closed unit ball of E is

compact if and only if the dimension of E is nite.

Thus Riesz Theorem characterizes the compactness of the closed unit ball

of a Banach space E by the niteness of the dimension of E.

Therefore, we need other types of topologies on innite dimensional spaces

Denition 1.1.32

Let E be a real Banach space. To each f in E, we assign the map

f : E ! R

x 7! f (x) = hf; xi:

We denote the family of all such maps from E into R by ffgf2E .

The weak topology on E (denoted by !) is the smallest topology on E which

makes the maps f continuous.

Proposition 1.1.33

Let (E; !) be a real Banach space endowed with the weak topology ! . Then

! is Hausdor; that is, for any two dierent points x1 and x2 taken in (E; !);

there exists two respective disjoint weakly open neighbourhoods U1 and U2;

that is U1, U2 belongs to ! and U1 \ U2 = ;:

Consequently if a sequence fxngn2N in (E; !) converges weakly to some x

(i.e xn * x) in (E; !), then x is unique.

12

Proposition 1.1.34

A sequence fxngn2N in a real Banach space E converges weakly to some x in

E if and only if f(xn) ! f(x) for each f 2 E:

Proposition 1.1.35

Given a nite dimensional normed space E, the norm-topology (strong topology)

and the weak topology coincide on E.

Theorem 1.1.36 (Eberlein-Smulyan)

A real Banach space E is re exive if and only if every (norm) bounded sequence

in E has a subsequence which converges weakly to an element of E.

On the dual of a normed space, we can dene a weaker topology as follows.

Denition 1.1.37

Let E be a real Banach space. Dene for every x 2 E, the map ‘x dened on

E by

‘x(f) = f(x) ; 8 f 2 E :

Then the weak topology on E is dened as the smallest topology on E for

which the maps ‘x; x 2 E, are continuous.

The weak topology of E (denoted by !) is the

Proposition 1.1.38

Let E be a Banach space. A sequence of bounded linear forms ffngn2N of

elements of E converges weakly to some f 2 E if and only if fn(x) ! f(x)

for each x 2 E:

Proposition 1.1.39

Given a nite dimensional normed space E, the norm-topology (strong topology),

the weak topology and the weak topology coincide on East.

13

Theorem 1.1.40 (Banach-Alaoglu)[19]

For every Banach space E, the closed unit ball is weakly compact.

Now we consider the spectral properties of bounded linear operators.

Denitions 1.1.41 (Spectra and resolvents)

Let E be a Banach space over K and T be a bounded linear operator of E;

i.e., T 2 B(E). The spectrum (T) of T is dened by

(T) = f 2 K : I T is not invertible in B(E)g :

The resolvent set (T) of T is the complementary of (T) in K; that is,

(T) = K n (T):

The elements of (T) are called the regular values of T.

If 2 (T), then the operator R(T) = (I T)1 is called the resolvent

operator of T at .

The spectrum is decomposed into the disjoint union of the following three

sets:

a) The point spectrum of T :

p(T) =

n

2 K : Ker(I T) 6= f0g

o

:

b) The continuous spectrum of T :

c(T) =

n

2 K : Ker(IT) = f0g; R(I T) = E but R(IT) 6= E

o

:

c) The residual spectrum of T :

r(T) =

n

2 K : Ker(I T) = f0g; R(I T) 6= E

o

:

Remark/Denition 1.1.42 [24],[19]

1. Given T 2 B(E), an element of p(T) is called an eigenvalue of T and a

non-zero vector v such that Tv = v is called an eigenvector of T associated

to the eigenvalue . Eigenvalues and eigenvectors are sometimes

called characteristic values and characteristic vectors respectively.

More generally if 2 p(T), n 2 N and v is a nonzero element of E such

that (I T)nv = 0, then v is called a principal vector associated with

the eigenvalue .

14

2. Given an eigenvalue of an operator T, the geometric multiplicity of is

by denition the dimension of Ker(IT). And the algebraic multiplicity

of is dened as follows:

when E is nite-dimensional and equipped with a basis, it is the multiplicity

of as a root of the characteristic polynomial of the matrix

associated to T,

more generally for an arbitrary nontrivivial Banach space E, it is the

dimension of the vector subspace

1[

k=1

Ker(I T)k :

Thus the algebraic multiplicity of an eigenvalue is always greater

than or equal to its geometric multiplicity.

The notion of spectrum can be dened for unbounded operators dened

in a normed space.

Example 1.1.43 [24],[30]

1. Let n be a natural number and A : Cn ! Cn be a linear operator with

corresponding matrix A. then it is well-known that T has n eigenvalues

1; : : : ; n (counted with their multiplicities) and that the eigenvalues

are the roots of the characteristic polynomial P() = det(A In) for

A. Since A In) is invertible when is not an eigenvalue, it follows

that the spectrum (A) of A is a pure point spectrum, namely

(A) =

k : 1 k n

= p(A) ;

and that the resolvent set of A is the complex plane except nitely many

points, namely

(A) = C n

k : 1 k n

:

2. Let E = C([0; 1]) be equipped with the supremum norm and consider the

operator T : E ! E; f 7! Tf dened by

[Tf](x) =

Z x

0

f(s) ds ; 8x 2 [0; 1] :

Then it is not hard to check that T has no eigenvalue, T is injective

but not surjective and its range is not dense in E, and moreover TI is

invertible in B(E). Therefore

(T) = f0g = r(T) :

15

Theorem 1.1.44 [20]

Let E be a Banach space over C and T 2 B(E). Then the following holds for

the spectrum of T.

i) (T) is a closed subset of C

ii) (T) B(0; jjTjjB(X))

iii) (T) is a compact subset of C

Denition 1.1.45 (Spectral radius) [20]

Let E be a Banach space over C and T 2 B(E). Then the spectral radius of T

is dened by

r(T) := sup

n

jj : 2 (T)

o

= max

n

jj : 2 (T)

o

:

Futhermore, we have the Gelfang formula

r(T) = lim

n!1

kTnk

1

n = inf

n2N

kTnk

1

n :

Denition 1.1.46 (Compact linear maps)

Let E and F be two Banach Spaces over K. A linear map, T : E ! F is said

to be compact if the image of the closed unit ball BE(0; 1) by T is a relatively

compact subset of F.

In other words, T is compact if T

BE(0; 1)

is compact.

This denition is equivalent to each of the following properties.

i) For each bounded subset B E, the image set T(B) is relatively compact

in F.

ii) For every bounded sequence fxngn2N E, the sequence fTxngn2N has a

convergent subsequence in F:

Proposition 1.1.47 (Properties of compact linear maps) [20]

1. Every compact linear map is bounded.

2. For all Banach spaces E and F over the eld K, the set K(E; F) of

compact linear maps from E into F is a linear subspace of the space

B(E; F) of bounded linear maps from E into F.

3. Let E, F and G be Banach spaces, and let T : E ! F, S1 : F ! G

and S2 : G ! E be bounded linear operators. Then

16

i) If the range of T; R(T), is nite dimensional, then T is compact.

Therefore, every linear map dened on a nite dimensional normed

space is not only bounded, but also compact.

ii) If T is compact, then S1 T and T S2 are compact.

Therefore the set K(E) of compact linear operators (endomorphisms)

of E is a two-sided ideal of the algebra B(E) of bounded linear operators

(endomorphisms) of E.

iii) The limit of a convergent sequence of compact linear operators with

respect to the operator norm, is compact. Thus K(E) in closed in

B(E).

Next, we state the Riesz-Schauder spectral theory of compact linear operators.

Theorem 1.1.48 (Riesz-Schauder spectral theory)[19],[22]

Let E be a complex Banach space and T be a compact linear operator of E.

Then:

1. The spectrum of T consists of an at most countable set of points of the

complex plane which has no point of accumulation except possibly = 0.

2. Every nonzero number of the spectrum of T is an eigenvalue of T of nite

multiplicity.

3. The dual operator of T denoted by T 2 B(E) and dened by

T(f) = f T for every f 2 E ;

is also compact and a nonzero number is an eigenvalue of T if and only

if it is an eigenvalue of T.

Remark 1.1.49 [22]

The notion of dual operator is an extension of the notion of transposed matrix.

In fact, if E is nite dimensional and equipped with a given basis, then the

matrix of the dual of a linear operator of E is the transposed matrix of the

matrix of T. And More generally, if E is a normed space and T 2 B(E), then

using the duality pairing of E E, we have

hf; Txi = hT f; xi 8 x 2 E :

The above Theorem 1.1.48 can be rephrased as follows.

17

Theorem 1.1.50

Let E be a complex Banach space and T be a compact linear operator of E.

1. If 2 (T) and 6= 0, then is an eigenvalue of T of nite multiplicity.

2. (T) is either nite or countably innite.

3. If (T) is innite, then

(T) = f0g [

n : n = 1; 2; : : :

;

where fngn2N is a sequence of complex numbers converging to 0.

As a corollary we have

Theorem 1.1.51 [20]

Let E be an innite dimensional complex Banach space and T 2 B(E) be

compact. Then The following holds.

1. 0 2 (T)

2. (T) n f0g consists of eigenvalues of nite multiplicity.

3. (T) n f0g is either empty, nite or a sequence of complex numbers

converging to 0. That is (T) n f0g is a discrete set with no limit point

other than 0.

1.2 Hilbert spaces

Denition 1.2.1 (Inner product)

An inner product on a K-linear space E is any functional h ; i dened on EE

which is a positive hermitian and nondegenerate form; that is,

h ; i : E E ! K

(x; y) 7! hx; yi

and satises the following conditions :

1. hx; xi 0 8 x 2 E, and hx; xi = 0 if and only if x = 0.

2. hy; xi = hx; yi 8 x; y 2 E.

3. h1x1 + 2×2; yi = 1hx1; yi + 2hx2; yi 8 x1; x2; y 2 E

and 8 x1; 2 2 K.

18

Remark 1.2.2

1. A form : E E ! K which is linear with respect to its rst argument

and antilinear with respect to its second argument is said to be

sesquilinear.

If : E E ! K is sesquilinear and satisfy

(x; y) = (y; x) 8 x; y 2 E ;

then it is also called a hermitian sesquilinear form.

2. When K = R; that is, E is a real vector space, a hermitian (sesquilinear)

form on E is just called a symmetric bilinear form on E.

Denition 1.2.3

A linear space endowed with an inner product is called an inner product space

or a prehilbertian space.

A nite dimensional real prehilbertian space is also called a euclidean space

Theorem 1.2.4 (Cauchy-Schwarz-Bunyakovsky Inequal-

ity)

Let (E; h ; i) be an inner product space. Then

jhx; yij2 hx; xi hy; yi 8 x; y 2 E:

The equality of this inequality holds if and only if x and y are linearly dependent.

Theorem 1.2.5 (Norm induced by an inner product)

Let (E; h ; i) be an inner product space. Then the function

jj x jjE : E ! R

x 7!

p

hx; xi

denes a norm on E:

Denition 1.2.6 (Hilbert space)

An inner product space E is called a Hilbert space (usually denoted by H), if

(E; jj : jjE) is a Banach space.

19

Remark 1.2.7 (Hilbert space)

Finite dimensional real Hilbert spaces are also called Euclidean spaces. And

since all nite dimensional normed spaces are Banach spaces, there is no dierence

between nite dimensional real prehilbertian spaces and nite dimensional

real Hilbert spaces.

Theorem 1.2.8 (Parallelogram law and Polarization identity)

Let (E; h ; i) be an inner product space. Then we have

1. the parallelogram law :

jjx + yjj2 + jjx yjj2 = 2(jjxjj2 + jjyjj2) 8 x; y 2 E ;

2. and the Polarization identity :

hx; yi =

1

4

jjx+yjj2jjxyjj2+ijjx+iyjj2ijjxiyjj2

8 x; y 2 E :

Note that the parallelogram law characterizes the norms that are induced

by inner products according to a theorem by Von Neumann.

Denitions 1.2.9 (Orthogonality)

Let (E; h ; i) be an inner product space.

Two vectors x and y in E are said to be orthogonal (written x?y and read x

‘perp’ y) if hx; yi = 0:

If M is a non empty subset of E, we write x?M (and read x orthogonal

to M) if x is orthogonal to every element of M.

Given a non-empty subset M of E, we denote by M?, the set of all elements

of E which are orthogonal to M: That is,

M? =

n

x 2 E ; hx; yi = 0 8 y 2 M

o

:

The set M? is then called the orthogonal of M, it is a closed vector subspace

of E.

Theorem 1.2.10 (Projection Theorem)

Let H be a Hilbert space and M a closed subspace of H. For arbitrary vector

x in H, there exists a unique vector y? 2 M such that,

jjx y?jjH = inf

y2M

jjx yjjH:

Furthermore, x 2 M is such a vector if and only if (x x)?M.

The Projection Theorem yields the following denition end theorems.

20

Denition 1.2.11 (Direct Sum of vectors spaces)

Let X and Y be two subspaces of a vector space E. Then E is said to be the

direct sum of X and Y if

E = X + Y and X \ Y = f0g .

This means that every vector u 2 E has a unique decomposition of the form

u = x + y with x 2 X and y 2 Y:

in this case we write E = X Y .

Theorem 1.2.12 (Direct Sum Decomposition)

Let F be a closed subspace of a Hilbert space H. Then,

H = F F? :

Therefore we say that F? is the orthogonal complement of F

Theorem 1.2.13 (Riesz Representation)

Let H be a Hilbert space and let f be a bounded linear functional on H. Then,

(i) There exists a unique vector yo in H such that

f(x) = hx; yoi; for every x 2 H:

(ii) Moreover, jjfjjH = jjyojjH:

Corollary 1.2.14

If H be a Hilbert space, then H ‘ H via the canonical map

‘ : H ! H

a 7! ‘a ;

where ‘a is dened by

‘a = hx; ai ; 8 x 2 H :

This canonical map is bijective, isometric and antilinear.

Corollary 1.2.15

Real Euclidean and Hilbert spaces are re exive.

21

Denitions 1.2.16 (Hilbertian basis)

Let H be a Hilbert space.

1. A unit vector of H is a vector of H is equal to 1.

2. A family

u

2 of nonzero vectors of H is said to be orthogonal, if the

vectors of the family are pairwise orthogonal; that is,

hu ; ui = 0 ; for all 6= in :

3. A family

u

2 of (nonzero) vectors of H is said to be orthonormal

if these vectors are all unit vectors and pairwise orthogonal; that is,

8<

:

jjujj =

p

hu ; ui = 1 ; for all 2 :

hu ; ui = 0 ; for all 6= in :

4. A family A of vectors of H is said to be total or complete if the vector

subspace spanned by A is dense is H; that is,

Span(A ) = H :

5. A family of vectors of H that is both orthonormal and complete is called

a hilbertian basis.

A hilbertian basis can be nite, countable or uncountable.

The crucial dierence between a hilbertian basis and a (nite) orthonor-

mal basis is that in the rst case the expansion of a vector may not be a

linear combination of some of the elements of the hilbertian basis, but a

series of vectors!

Examples 1.2.17

In the real space

`2 =

(

u = (un)n1 R;

X1

n=1

u2

n < 1

)

endowed with the inner product dened by

hu; vi =

X1

n=1

unvn ;

the following vectors orthonormal and complete.

e1 = (1; 0; 0; 0; : : 🙂

…

ek = (k;n)n where k;n = 1 si n = k and k;n = 0 otherwise

…

22

Proposition 1.2.18

Let H be a Hilbert space. Then

1. For every vector subspace F of H, we have

F??

= F :

In particular if F is a closed subspace, then

F?

?

= F :

2. For every nonempty subset A of H, A? is a closed vector subspace of H

(and so a Hilbert subspace) and

A??

= Span(A) :

3. A nonempty set S of H is complete (or total) if and only if

S? = f0g :

Theorem 1.2.19 [17]

Let (H; h ; i) be a Hilbert space. Then

1. H has a hilbertian basis (i.e., an orthonormal basis) that can be nite

countable or uncountable. Furthermore, every orthonormal set in H is

contained in some hermitian basis.

2. A hilbertian basis of H is at most countable if and only if H is separable;

that is, H contains a dense subset that is at most countable.

3. For every orthonormal sequence of vectors fekgk2N of elements of H, we

have Bessel Inequality:

8 x 2 H;

X+1

k=1

jhx; ekij2 jjxjj2 :

4. If H is separable and fekgk2N is a complete and orthonormal sequence

of vectors of H, then we have Parseval Identity:

8 x 2 H;

X+1

k=1

jhx; ekij2 = jjxjj2 :

23

Proposition 1.2.20

Let (H; h ; i) be an innite dimensional Hilbert space and fekgk2N be an

orthonormal set of H. Then

1. For every x 2 H, we have

lim

k!+1

hx; eki = 0 :

2. Every bounded sequence of scalars fkgk2N gives a bounded linear operator

T on H dened by

Tx =

X+1

k=1

khx; ekiek ; 8x 2 H ;

of which norm is jjTjj = supk1 jkj.

Proof

1. Follows from the convergence of the numerical series

P+1

k=1 jhx; ekij2

according to Bessel inequality.

2. Follows from the convergence in H of the series

P+1

k=1 khx; ekij2 and

the bounds

jjTxjj sup

k1

jkj jjxjj ; 8x 2 H

and

sup

k1

jjTekjj = sup

k1

jkj :

Theorem 1.2.21 (Construction of compact linear operators) [24]

Let (H; h ; i) be an innite dimensional, separable complex Hilbert space with

orthonormal basis (ek)k2N and let (k)k2N be an arbitrary sequence of

complex numbers. For every x 2 H, consider the series

Tx =

X+1

k=1

khx; ekiek :

Then

1. The series is convergent and the sum denes a linear operator T on H if

the sequence (k)k2N is bounded.

2. T exists and is bounded if and only if the sequence (k)k2N is bounded.

When k 2 f0; 1g for all k, T is an orthogonal projection.

3. T exists and is a compact linear operator if and only if the sequence

(k)k2N converges to 0.

When only nitely many of the terms k are nonzero, T is a nite rank

operator.

24

Denition 1.2.22 (Numerical range)[19]

Let H be a complex Hilbert space and T be a bounded linear operator on H.

The numerical range of T is dened by

W(T) =

n

hTx; xi : x 2 H; jjxjj = 1

o

:

Proposition 1.2.23 [19]

Let H be a complex Hilbert space and T be a bounded linear operator on H.

Then

(T) W(T) ;

where

W(T) =

n

hTx; xi : x 2 H; jjxjj = 1

o

: :

More precisely, if 62 W(T), then 2 (T) with

(T I)1

1

dist(; W(T)

:

Denition 1.2.24 [6],[27]

Let H be a complex Hilbert space and T be a bounded linear operator on H.

The numerical radius of T is dened by

w(T) = sup

jhTx; xij : x 2 H; jjxjj = 1

o

:

It satises the following inequalities

r(T) w(T) and

jjTjj

2

w(T) jjTjj :

As a result of the Riesz representation Theorem, we have the following characterization

of weak convergence in a Hilbert space.

Proposition 1.2.25

Let H be a Hilbert space.

A sequence fxngn2N of elements of H converges weakly to some a 2 H if and

only if

lim

n!+1

hxn; yi = ha; yi ; 8 y 2 H :

25

Theorem 1.2.26 [24],[6]

Let H be a Hilbert space and (xn)n2N be a weakly convergent sequence with

limit point x. then for every compact linear operator T on H, the image

sequence

Txn

n2N converges strongly (in norm) to Tx.

That is, for any compact linear operator T 2 B(H), we have

xn * x for n ! +1 =) Txn ! Tx for n ! +1:

Corollary 1.2.27

Let (en)n2N be an orthonormal sequence of a Hilbert space H. Then for every

compact linear operator T of H, we have

lim

n!+1

Ten = 0 with respect to the norm topology of H :

Proof. By Bessel inequality, we have that (en)n2N converges weakly to 0 in H.

Therefore the corollary follows from the above Theorem [ ].

1.3 Self-adjoint operators

Let n be a natural number and T : Cn ! Cn be a linear operator. Then T

can be represented by a complex square matrix of order n. Suppose moreover

that T is self-adjoint. Then the matrix associated to T is conjugate symmetric

and it is well-known from basic Linear Algebra that all the eigenvalues of this

matrix are all real and that there exists an orthonormal basis (e1; : : : ; en) for

Cn in which the matrix of T is diagonal, meaning also that the linear operator

T can be expressed as follows :

Tx =

Xn

k=1

khx; ekiek ; 8 x 2 Cn ;

where hx; eki coincide with kth coordinate of x in the orthonormal basis (e1; : : : ; en)

of eigenvectors for T corresponding respectively to the eigenvalues

k

1kn.

For innite dimensional Hilbert spaces, the situation is much more tremendous,

but for self-adjoint compact linear operators, a corresponding theory can

be well developped. It will be culminated in the famous spectral theorem.

Denition 1.3.1 (Symmetric or self-adjoint operators) [19],[18],[20],[24]

Let H be a Hilbert space over K and A : H ! H be a bounded linear operator.

A is said to be symmetric or self-adjoint if

hAx; yi = hx; Ayi ; 8 x; y 2 H :

26

Remark 1.3.2 [24],[22]

1. Given an arbitrary bounded linear operator T on a Hilbert space H, the

adjoint operator of T is the unique bounded linear operator of H denoted

by T and satisfying

hTx; yi = hx; Tyi ; 8 x; y 2 H :

The existence and properties of T are based on the Riesz representation

theorem and can be proved following the idea of the proof of [Representation

theorem of a real bounded bilinear form].

Therefore a bounded linear operator T on a Hilbert space is symmetric

or self-adjoint if T = T.

2. Let H be a Hilbert space. Then any map T dened from H into H that

satises

hTx; yi = hx; Tyi ; 8 x; y 2 H ;

is necessarily linear and bounded. In fact :

a) the linearity holds since for all x; y 2 H and for all ; 2 K, we

have

T(x + y) Tx Ty

2 H? = f0g ;

b) and the boundedness follows from the Closed graph Theorem […]

3. Given a Hilbert space H and a linear operator T dened from a dense

domain D(T) H into H, the adjoint operator of T is dened as the

unique operator T with domain D(T) and such that

hTx; yi = hx; Tyi ; 8 x 2 D(T) and 8y 2 D(T) :

In this case, a linear unbounded operator T dened on a dense domain

D(T) of a Hilbert space H is said to be symmetric if T is an extension

of T, this amounts to:

hTx; yi = hx; Tyi ; 8 x; y 2 D(T) :

And again, a linear unbounded operator T dened on a dense domain

D(T) of a Hilbert space H is said to be self-adjoint if T = T. This

means that not only T is symmetric, but also D(T) = D(T).

Proposition 1.3.3 [24],[19]

The numerical range of any bounded, self-adjoint linear operator on a complex

Hilbert space is a subset of R.

In particular for any bounded, self-adjoint linear operator T on a Hilbert

space over K, hTx; xi is a real number for all x 2 H.

27

Proof. For all x 2 H, we have

hTx; xi = hx; Txi = hTx; xi :

Proposition 1.3.4 [24]

Let T be a bounded self-adjoint operator on a complex Hilbert space H. Then

all its eigenvalues are real numbers. Furthermore, any pair of eigenvectors

corresponding to dierent eigenvalues are orthogonal.

Proof. This is a well-known result.

If Tv = v for 2 C and v 2 H n f0g, we get

hv; vi = hv; vi = hTv; vi = hv; Tvi = hv; vi = hv; vi :

And since hv; vi = jjvjj2 6= 0; we get = meaning that is real.

Note that roughly prouving, is real because hTv; vi is real and hv; vi

is a nonzero real number.

If 1 6= 2 are two dierent eigenvalues corresponding respectively to two

eigenvectors v1 and v2, then 1 and 2 are real numbers and we have

1hv1; v2i = h1v1; v2i = hTv1; v2i

= hv1; Tv2i

= hv1; 2v2i = 2hv1; v2i = 2hv1; v2i :

Therefore

(1 2)hv1; v2i = 0

and so hv1; v2i = 0 since 1 2 6= 0.

We have the following alternative formular for the norm of a bounded se –

adjoint operator.

Proposition 1.3.5 [24],[19]

Let T be a bounded self-adjoint operator on a Hilbert space H (over K). Then

jjTjj = sup

x2H; jjxjj=1

jhTx; xij :

Proof.(Sketch)

Set = supjjxjj=1 jhTx; xij:

28

– First of all we have clearly jjTjj.

– Let x; y 2 H. By expending hT(x + y); x + yi and hT(x y); x yi

we get

hTx; yi + hTy; xi = 1

2

hT(x + y); x + yi hT(x y); x yi

2 (jjx + yjj2 + jjx yjj2)

(jjxjj2 + jjyjj2) (using parallelogram law):

And so

hTx; yi + hTy; xi

jjxjj2 + jjyjj2

:

– If x 2 H is such that Tx 6= 0, then by setting y = jjxjj

jjTxjjTx and by

applying the inequality of the previous step, we get

jjTxjj jjxjj :

The latter inequality is also (directly) satised even tough Tx = 0.

Therefore,

jjTxjj jjxjj ; 8x 2 H ;

yielding jjTjj .

Proposition 1.3.6 [18]

Let T be a bounded self-adjoint operator on a real Hilbert space H. Dene

the lower and upper bounds

m = inf

x2H; jjxjj=1

hTx; xi and M = sup

x2H; jjxjj=1

hTx; xi :

Then

1. (T) [m; M].

2. m; M 2 (T).

3. jjTjj = maxfm; Mg.

Corollary 1.3.7 [18]

1. For every bounded linear self-adjoint operator T on a real Hilbert space,

either jjTjj or jjTjj is an approximate eigenvalue (i.e., a limit of a sequence

of eigenvalues of T).

2. Consequently, every nonzero compact linear self-adjoint operator T on a

real Hilbert space, has either jjTjj or jjTjj as an eigenvalue; that is,

jjTjj; jjTjj

\ p(T) 6= ; :

29

1.4 Spectral decomposition

This section deals with the spectral decomposition of a compact linear selfadjoint

operator on a separable Hilbert space.

Theorem 1.4.1 (Hilbert-Schmidt) [24], [18], [28]

Let T be a compact self-adjoint operator on a separable Hilbert space H of

nite or innite dimension. Then H admits an at most countable orthonormal

basis consisting of eigenvectors for T. More precisely

1. In the nite dimensional case, the numbering of the nite sequence of

basis vectors (e1; :::; en) can be chosen such that the corresponding nite

sequence of eigenvalues (1; :::; n) decreases numerically (in absolute

value) :

j1j j2j : : : jnj :

And in the basis of eigenvectors, the operator T is described by :

Tx =

Xn

k=1

khx; ekiek for all x 2 H = Span

ek ; k = 1; :::; n

:

2. In the separable, innite dimensional case :

a) If T = 0, then any orthonormal basis of the separable Hilbert space

H is a countable orthonormal basis consisting of eigenvectors of T.

b) If T 6= 0, is of nite rank n 1, then

H = ker(T)R(T) with [ker(T)]? = R(T) and dim[R(T)] = n :

In this case R(T) is nite dimensional and invariant under T.

Therefore R(T) has a nite orthonomal basis (e1; :::; en) consisting

of eigenvectors of the restriction of T to its range R(T) such that

their corresponding eigenvalues satisfy

j1j j2j : : : jnj (all nonzero):

Again T is described by

Tx =

Xn

k=1

khx; ekiek for all x 2 H :

By adding to (e1; :::; en) an orthonormal basis of ker(T) (also separable),

we shall obtain a countable orthonormal basis for H consisting

of eigenvectors of T.

30

c) Otherwise

R(T) is innite dimensional

, we can nd an innite sequence

of orthonormal eigenvectors

ek

k2N ; in fact an orthonormal

basis of R(T) consisting of eigenvectors, with a corresponding sequence

of nonzero eigenvalues

k

k2N that decreases numerically

and tends to 0;

j1j j2j : : : jkj : : : with lim

k!+1

k = 0 ;

and for which T can be described as

Tx =

X+1

k=1

khx; ekiek for all x 2 H :

Note in this case that

p(T) f0g [

k : k = 1; 2; : : :

:

Proof (Sketch).

If the range of T is nite dimensional, then we can proceed by induction.

Now we only consider the case in which the range of T is innite dimensional.

– Set 0 = 0 and let

1 ; 2 ; : : :

be the countable set of all the nonzero eigenvalues of T. Consider the

subspace

H0 = ker(T) (that may be null) ;

and the eigenspaces

Hk = ker(T kI) ; k = 1; 2; : : : :

For k 1, dim(Hk) 1 since Hk is an eigenspace, and dim(Hk) <

+1 since T is compact (cf. Theorem 1.1.48, by which every nonzero

eigenvalue k must have a nite multiplicity). Thus

1 dim(Hk) < +1; 8 k 1 :

1. For any m 6= n in

0; 1; 2 : : : g, Hm and Hn are orthogonal. (See the

proof of theorem …)

2. We have H = k0Hk where (recall)

k0Hk =

vk0 + : : : + vkn : n 0; 0 j n; kj 0; vkj 2 Hkj

:

To see this, set

V = k=0Hk

31

and suppose by contradiction that V 6= H. Therefore V ? 6= f0g and

V ? \ ker(T) V ? \ H0 V ? \ V = f0g :

Moreover, it is not hard to show that T maps V ? into V ?. Thus the

restriction of T to V ? would be a nonzero, compact self-adjoint operator

of V ? and would have at least one nonzero eigenvalue. This would imply

by Corrolary 1.3.7 that T has an eigenvector in V ? with a nonzero

eigenvalue contradicting the fact that

V ? \ [k1Hk V ? \ V = f0g :

3. For each k 1, the nite dimensional subspace Hk possesses a nite

orthonormal basis

Bk =

n

ek; 1 ; ek; 2 ; : : : ; ek; nk

o

:

Besides the closed subspace H0 = ker(T) of the separable space H, is

either null, in which case we set B0 = ;, or admits an at most countable

orthonormal basis B0.

It follows that

B = [+1

k=0Bk

is a countable orthonormal basis of eigenvectors of T. Furthermore

Tx =

X+1

k=1

Xnk

j=1

khx; ek; jiek; j for all x 2 H :

Constructive proof (Sketchy)

We shall prove again this Theorem 1.4.1 by successive applications of Corollary

1.3.7.

Let H1 = H assumed to be nontrivial and set T1 = T.

By the second part of Corollary 1.3.7,

there exist an eigenvalue 1 of T1 and a corresponding eigenvector ‘1 such

that jj’1jj = 1 and j1j = jjT1jj. Set H2 := f’1g?. Thus H2 is a closed

subspace of H1 and T(H2) H2 (i.e H2 is T-invariant).

Now let T2 be the restriction of T to H2. Then T2 is compact self adjoint

operator in B(H2)

If T2 6= 0,then there exists an eigenvalue 2 of T2 and corresponding eigenvector

‘2 such that jj’2jj = 1 and j2j = jjT2jj jjT1jj = j1j

f’1; ‘2g is orthonormal.

H3 = f’1; ‘2g?

H3 is a closed subspace of H and TH3 H3

32

Letting T3 be the restriction of T to H3, we have that T3 is a compact selfadjoint

operator in B(H3). Continuing in this manner, the process stops when

Tn = 0 or else we get a sequence fng of eigenvalues of T and corresponding

orthonormal set f’1; ‘2; ‘3:::g of eigenvectors such that

jn+1j = jjTn+1jj jjTnjj = jnj n = 1; 2; 3::: (1.4.1)

Claim : If fng is an innite sequence, then n ! 0; n ! 1.

Proof of Claim. Suppose by contradiction , there exist > 0 such that jnj

for all n 2 N

Hence for n 6= m, we have that,

jjT’n T’mjj2 = jj’n ‘mjj2 = 2

n + 2

m > (1.4.2)

But this is impossible, since fT’ng has a convergent subsequence due to the

compactness of T. We therefore conclude that n ! 0; n ! 1.

Now, we prove the representation of T as asserted in the theorem.

Case I. Tn = 0 for some n

xn := x

Xn

k=1

< x; ‘k > ‘k

It is evident that xn is orthogonal to ‘i for 1 i n

Therefore, xn 2 Hn

0 = Tnxn = Tx T(

Xn

k=1

hx; ‘ki’k)

) Tx =

Xn

k=1

khx; ‘ki’k

Case II. Tn 6= 0 for all n 2 N

jjTx

Xn

k=1

khx; ‘ki’kjj = jjTnxnjj jjTnjj jjxnjj

= jnj jjxnjj

jnjjjxjj ! 0

) jjTx

Xn

k=1

khx; ‘ki’kjj as ! 0; n ! 1.

Hence, Tx =

X1

k=1

khx; ‘ki’k :

33

Do you need help? Talk to us right now: (+234) 08060082010, 08107932631 (Call/WhatsApp). Email: [email protected].

**IF YOU CAN'T FIND YOUR TOPIC, CLICK HERE TO HIRE A WRITER»**