View source for Operator norm

{{Short description|Measure of the "size" of linear operators}}
In [[mathematics]], the '''operator norm''' measures the "size" of certain [[linear operator]]s by assigning each a [[real number]] called its {{em|operator norm}}. Formally, it is a [[Norm (mathematics)|norm]] defined on the space of [[bounded linear operator]]s between two given [[normed vector space]]s. Informally, the operator norm <math>\|T\|</math> of a linear map <math>T : X \to Y</math> is the maximum factor by which it "lengthens" vectors.

== Introduction and definition ==

Given two normed vector spaces <math>V</math> and <math>W</math> (over the same base [[Field (mathematics)|field]], either the [[real number]]s <math>\R</math> or the [[complex number]]s <math>\Complex</math>), a [[linear map]] <math>A : V \to W</math> is continuous [[if and only if]] there exists a real number <math>c</math> such that<ref>{{Citation|last1=Kreyszig|first1=Erwin|title=Introductory functional analysis with applications|publisher=John Wiley & Sons|year=1978|isbn=9971-51-381-1|page=97}}</ref>
<math display="block">\|Av\| \leq c \|v\| \quad \text{ for all } v\in V.</math>

The norm on the left is the one in <math>W</math> and the norm on the right is the one in <math>V</math>. 
Intuitively, the continuous operator <math>A</math> never increases the length of any vector by more than a factor of <math>c.</math> Thus the [[Image (mathematics)|image]] of a bounded set under a continuous operator is also bounded. Because of this property, the continuous linear operators are also known as [[bounded operator]]s. 
In order to "measure the size" of <math>A,</math> one can take the [[infimum]] of the numbers <math>c</math> such that the above inequality holds for all <math>v \in V.</math> 
This number represents the maximum scalar factor by which <math>A</math> "lengthens" vectors. 
In other words, the "size" of <math>A</math> is measured by how much it "lengthens" vectors in the "biggest" case. So we define the operator norm of <math>A</math> as
<math display="block">\|A\|_\text{op} = \inf\{ c \geq 0 : \|Av\| \leq c \|v\| \text{ for all } v \in V \}.</math>

The infimum is attained as the set of all such <math>c</math> is [[Closed set|closed]], [[Empty set|nonempty]], and [[Bounded set|bounded]] from below.<ref>See e.g. Lemma 6.2 of {{harvtxt|Aliprantis|Border|2007}}.</ref>

It is important to bear in mind that this operator norm depends on the choice of norms for the normed vector spaces <math>V</math> and <math>W</math>.

== Examples ==

Every real <math>m</math>-by-<math>n</math> [[matrix (mathematics)|matrix]] corresponds to a linear map from <math>\R^n</math> to <math>\R^m.</math> Each pair of the plethora of (vector) [[norm (mathematics)|norms]] applicable to real vector spaces induces an operator norm for all <math>m</math>-by-<math>n</math> matrices of real numbers; these induced norms form a subset of [[matrix norm]]s.

If we specifically choose the [[Euclidean norm]] on both <math>\R^n</math> and <math>\R^m,</math> then the matrix norm given to a matrix <math>A</math> is the [[square root]] of the largest [[eigenvalue]] of the matrix <math>A^{*} A</math> (where <math>A^{*}</math> denotes the [[conjugate transpose]] of <math>A</math>).<ref>{{Cite web|url=https://mathworld.wolfram.com/OperatorNorm.html|title=Operator Norm|last=Weisstein|first=Eric W.|authorlink = Eric W. Weisstein|website=mathworld.wolfram.com|language=en|access-date=2020-03-14}}</ref> 
This is equivalent to assigning the largest [[singular value]] of <math>A.</math>

Passing to a typical infinite-dimensional example, consider the [[sequence space]] <math>\ell^2,</math> which is an [[Lp space|L<sup>''p''</sup> space]], defined by
<math display="block">\ell^2 = \left\{ (a_n)_{n \geq 1} : \; a_n \in \Complex, \; \sum_n |a_n|^2 < \infty \right\}.</math>

This can be viewed as an infinite-dimensional analogue of the [[Euclidean space]] <math>\Complex^n.</math> 
Now consider a bounded sequence <math>s_{\bull} = \left(s_n\right)_{n=1}^\infty.</math> The sequence <math>s_{\bull}</math> is an element of the space <math>\ell^\infty,</math> with a norm given by
<math display="block">\left\|s_{\bull}\right\|_\infty = \sup _n \left|s_n\right|.</math>

Define an operator <math>T_s</math> by pointwise multiplication:
<math display="block">\left(a_n\right)_{n=1}^{\infty} \;\stackrel{T_s}{\mapsto}\;\ \left(s_n \cdot a_n\right)_{n=1}^{\infty}.</math>

The operator <math>T_s</math> is bounded with operator norm
<math display="block">\left\|T_s\right\|_\text{op} = \left\|s_{\bull}\right\|_\infty.</math>

This discussion extends directly to the case where <math>\ell^2</math> is replaced by a general <math>L^p</math> space with <math>p > 1</math> and <math>\ell^\infty</math> replaced by <math>L^\infty.</math>

==Equivalent definitions==

Let <math>A : V \to W</math> be a linear operator between normed spaces. The first four definitions are always equivalent, and if in addition <math>V \neq \{0\}</math> then they are all equivalent:
:<math>
\begin{alignat}{4}
\|A\|_\text{op} &= \inf &&\{ c \geq 0 ~&&:~ \| A v \| \leq c \| v \| ~&&~ \text{ for all } ~&&v \in V \} \\
&= \sup &&\{ \| Av \| ~&&:~ \| v \| \leq 1 ~&&~\mbox{ and } ~&&v \in V \} \\
&= \sup &&\{ \| Av \| ~&&:~ \| v \| < 1 ~&&~\mbox{ and } ~&&v \in V \} \\
&= \sup &&\{ \| Av \| ~&&:~ \| v \| \in \{0,1\}  ~&&~\mbox{ and } ~&&v \in V \} \\
&= \sup &&\{ \| Av \| ~&&:~ \| v \| = 1 ~&&~\mbox{ and } ~&&v \in V \}  \;\;\;\text{ this equality holds if and only if } V \neq \{ 0 \} \\
&= \sup &&\bigg\{ \frac{\| Av \|}{\| v \|} ~&&:~ v \ne 0 ~&&~\mbox{ and } ~&&v \in V \bigg\}  \;\;\;\text{ this equality holds if and only if } V \neq \{ 0 \}. \\
\end{alignat}
</math>
If <math>V = \{0\}</math> then the sets in the last two rows will be empty, and consequently their [[supremum]]s over the set <math>[-\infty, \infty]</math> will equal <math>-\infty</math> instead of the correct value of <math>0.</math> If the supremum is taken over the set <math>[0, \infty]</math> instead, then the supremum of the empty set is <math>0</math> and the formulas hold for any <math>V.</math> 

Importantly, a linear operator <math>A : V \to W</math> is not, in general, guaranteed to achieve its norm <math>\|A\|_\text{op} = \sup \{\|A v\| : \|v\| \leq 1, v \in V\}</math> on the closed unit ball <math>\{v \in V : \|v\| \leq 1\},</math> meaning that there might not exist any vector <math>u \in V</math> of norm <math>\|u\| \leq 1</math> such that <math>\|A\|_\text{op} = \|A u\|</math> (if such a vector does exist and if <math>A \neq 0,</math> then <math>u</math> would necessarily have unit norm <math>\|u\| = 1</math>). R.C. James proved [[James's theorem]] in 1964, which states that a [[Banach space]] <math>V</math> is [[reflexive space|reflexive]] if and only if every [[bounded linear functional]] <math>f \in V^*</math> achieves its [[Dual norm|norm]] on the closed unit ball.{{sfn|Diestel|1984|p=6}} 
It follows, in particular, that every non-reflexive Banach space has some bounded linear functional (a type of bounded linear operator) that does not achieve its norm on the closed unit ball.

If <math>A : V \to W</math> is bounded then{{sfn|Rudin|1991|pp=92-115}} 
<math display="block">\|A\|_\text{op} = \sup \left\{\left|w^*(A v)\right| : \|v\| \leq 1, \left\|w^*\right\| \leq 1 \text{ where } v \in V, w^* \in W^*\right\}</math>
and{{sfn|Rudin|1991|pp=92-115}}  
<math display="block">\|A\|_\text{op} = \left\|{}^tA\right\|_\text{op}</math>
where <math>{}^t A : W^* \to V^*</math> is the [[Transpose of a linear map|transpose]] of <math>A : V \to W,</math> which is the linear operator defined by <math>w^* \,\mapsto\, w^* \circ A.</math>

== Properties ==

The operator norm is indeed a norm on the space of all [[bounded operator]]s between <math>V</math> and <math>W</math>. This means
<math display="block">\|A\|_\text{op} \geq 0 \mbox{ and } \|A\|_\text{op} = 0 \mbox{ if and only if } A = 0,</math>
<math display="block">\|aA\|_\text{op} = |a| \|A\|_\text{op} \mbox{ for every scalar } a ,</math>
<math display="block">\|A + B\|_\text{op} \leq \|A\|_\text{op} + \|B\|_\text{op}.</math>

The following inequality is an immediate consequence of the definition:
<math display="block">\|Av\| \leq \|A\|_\text{op} \|v\| \ \mbox{ for every }\ v \in V.</math>

The operator norm is also compatible with the composition, or multiplication, of operators: if <math>V</math>, <math>W</math> and <math>X</math> are three normed spaces over the same base field, and <math>A : V \to W</math> and <math>B : W \to X</math> are two bounded operators, then it is a [[sub-multiplicative norm]], that is: 
<math display="block">\|BA\|_\text{op} \leq \|B\|_\text{op} \|A\|_\text{op}.</math>

For bounded operators on <math>V</math>, this implies that operator multiplication is jointly continuous.

It follows from the definition that if a sequence of operators converges in operator norm, it [[converges uniformly]] on bounded sets.

== Table of common operator norms ==

By choosing different norms for the codomain, used in computing <math>\|Av\|</math>, and the domain, used in computing <math>\|v\|</math>, we obtain different values for the operator norm. Some common operator norms are easy to calculate, and others are [[NP-hard]]. 
Except for the NP-hard norms, all these norms can be calculated in <math>N^2</math> operations (for an <math>N \times N</math> matrix), with the exception of the <math>\ell_2 - \ell_2</math> norm (which requires <math>N^3</math> operations for the exact answer, or fewer if you approximate it with the [[Power iteration|power method]] or [[Lanczos algorithm|Lanczos iterations]]).

{| class="wikitable" style="text-align: center; width: 500px; height: 200px;"
|+ Computability of Operator Norms<ref>section 4.3.1, [[Joel Tropp]]'s PhD thesis, [http://users.cms.caltech.edu/~jtropp/papers/Tro04-Topics-Sparse.pdf]</ref>
|-
! scope="col" colspan = "2" rowspan = "2" |
! scope="col" colspan = "3" | Co-domain
|-
! scope="col" | <math>\ell_1</math>
! scope="col" | <math>\ell_2</math>
! scope="col" | <math>\ell_\infty</math>
|-
! scope = "row" rowspan="3" | Domain
! scope="row" | <math>\ell_1</math>
| Maximum <math>\ell_1</math> norm of a column || Maximum <math>\ell_2</math> norm of a column || Maximum <math>\ell_{\infty}</math> norm of a column
|-
! scope="row" | <math>\ell_2</math>
| NP-hard || Maximum singular value || Maximum <math>\ell_2</math> norm of a row
|-
! scope="row" | <math>\ell_\infty</math>
| NP-hard || NP-hard || Maximum <math>\ell_1</math> norm of a row
|}

The norm of the [[Conjugate transpose|adjoint]] or transpose can be computed as follows. 
We have that for any <math>p, q,</math> then <math>\|A\|_{p\rightarrow q} = \|A^*\|_{q'\rightarrow p'}</math> where <math>p', q'</math> are [[Hölder's inequality|Hölder conjugate]] to <math>p, q,</math> that is, <math>1/p + 1/p' = 1</math> and <math>1/q + 1/q' = 1.</math>

== Operators on a Hilbert space ==

Suppose <math>H</math> is a real or complex [[Hilbert space]]. If <math>A : H \to H</math> is a bounded linear operator, then we have
<math display="block">\|A\|_\text{op} = \left\|A^*\right\|_\text{op}</math>
and
<math display="block">\left\|A^* A\right\|_\text{op} = \|A\|_\text{op}^2,</math>
where <math>A^{*}</math> denotes the [[adjoint operator]] of <math>A</math> (which in [[Euclidean space]]s with the standard [[inner product]] corresponds to the [[conjugate transpose]] of the matrix <math>A</math>).

In general, the [[spectral radius]] of <math>A</math> is bounded above by the operator norm of <math>A</math>:
<math display="block">\rho(A) \leq \|A\|_\text{op}.</math>

To see why equality may not always hold, consider the [[Jordan canonical form]] of a matrix in the finite-dimensional case. Because there are non-zero entries on the superdiagonal, equality may be violated. The [[quasinilpotent operator]]s is one class of such examples. A nonzero quasinilpotent operator <math>A</math> has spectrum <math>\{0\}.</math> So <math>\rho(A) = 0</math> while <math>\|A\|_\text{op} > 0.</math>

However, when a matrix <math>N</math> is [[Normal matrix|normal]], its [[Jordan canonical form]] is diagonal (up to unitary equivalence); this is the [[spectral theorem]]. In that case it is easy to see that
<math display="block">\rho(N) = \|N\|_\text{op}.</math>

This formula can sometimes be used to compute the operator norm of a given bounded operator <math>A</math>: define the [[Hermitian operator]] <math>B = A^{*} A,</math> determine its spectral radius, and take the [[square root of a matrix|square root]] to obtain the operator norm of <math>A.</math>

The space of bounded operators on <math>H,</math> with the [[Topological space|topology]] induced by operator norm, is not [[Separable space|separable]]. 
For example, consider the [[Lp space]] <math>L^2[0, 1],</math> which is a Hilbert space. 
For <math>0 < t \leq 1,</math> let <math>\Omega_t</math> be the [[Indicator function|characteristic function]] of <math>[0, t],</math> and <math>P_t</math> be the [[multiplication operator]] given by <math>\Omega_t,</math> that is,
<math display="block">P_t (f) = f \cdot \Omega_t.</math>

Then each <math>P_t</math> is a bounded operator with operator norm 1 and
<math display="block">\left\|P_t - P_s\right\|_\text{op} = 1 \quad \mbox{ for all } \quad t \neq s.</math>

But <math>\{P_t : 0 < t \leq 1\}</math> is an [[uncountable set]]. 
This implies the space of bounded operators on <math>L^2([0, 1])</math> is not separable, in operator norm. 
One can compare this with the fact that the sequence space <math>\ell^{\infty}</math> is not separable.

The [[associative algebra]] of all bounded operators on a Hilbert space, together with the operator norm and the adjoint operation, yields a [[C*-algebra]].

==See also==

* {{annotated link|Banach–Mazur compactum}}
* {{annotated link|Continuous linear operator}}
* {{annotated link|Contraction (operator theory)}}
* {{annotated link|Discontinuous linear map}}
* {{annotated link|Dual norm}}
* {{annotated link|Matrix norm}}
* {{annotated link|Norm (mathematics)}}
* {{annotated link|Normed space}}
* {{annotated link|Operator algebra}}
* {{annotated link|Operator theory}}
* {{annotated link|Topologies on the set of operators on a Hilbert space}}
* {{annotated link|Unbounded operator}}

==Notes==

{{reflist}}

==References==

* {{citation|last1=Aliprantis|first1=Charalambos D.|last2=Border|first2=Kim C.|title=Infinite Dimensional Analysis: A Hitchhiker's Guide|year=2007|publisher=Springer|isbn=9783540326960|page=229|url=https://books.google.com/books?id=4hIq6ExH7NoC&pg=PA229}}.
* {{Citation|last=Conway|first=John B.|authorlink=John B. Conway|year=1990|contribution=III.2 Linear Operators on Normed Spaces|title=A Course in Functional Analysis|pages=67–69|isbn=0-387-97245-5|publisher=Springer-Verlag|location=New York|url=https://books.google.com/books?id=ix4P1e6AkeIC&pg=PA67}}
* {{cite book|last=Diestel|first=Joe|title=Sequences and series in Banach spaces|publisher=Springer-Verlag|publication-place=New York|date=1984|isbn=0-387-90859-5|oclc=9556781}} <!--{{sfn|Diestel|1984|p=}}-->
* {{Rudin Walter Functional Analysis|edition=2}} <!--{{sfn|Rudin|1991|p=}}-->

{{Banach spaces}}
{{Hilbert space}}
{{Functional analysis}}
{{Duality and spaces of linear maps}}

[[Category:Functional analysis]]
[[Category:Norms (mathematics)]]
[[Category:Operator theory]]