View source for Second moment method

{{Use American English|date = March 2019}}
{{Short description|Method in probability theory}}

In mathematics, the '''second moment method''' is a technique used in [[probability theory]] and [[analysis]] to show that a [[random variable]] has positive probability of being positive. More generally, the "moment method" consists of bounding the probability that a random variable fluctuates far from its mean, by using its moments.<ref>{{cite web |url=http://terrytao.wordpress.com/2008/06/18/the-strong-law-of-large-numbers/ |title=The strong law of large numbers | author = Terence Tao | author-link = Terence Tao | accessdate=2009-02-10 |work=What’s new?|date=2008-06-18 }}</ref>

The method is often quantitative, in that one can often deduce a lower bound on the probability that the random variable is larger than some constant times its expectation. The method involves comparing the second [[moment (mathematics)|moment]] of random variables to the square of the first moment.

==First moment method==
The first moment method is a simple application of [[Markov's inequality]] for integer-valued variables.  For a '''non-negative''', '''integer-valued''' random variable {{math|''X''}}, we may want to prove that {{math|1=''X'' = 0}} with high probability. To obtain an upper bound for {{math|Pr(''X'' > 0)}}, and thus a lower bound for {{math|1=Pr(''X'' = 0)}}, we first note that since {{math|''X''}} takes only integer values, {{math|1=Pr(''X'' > 0) = Pr(''X'' ≥ 1)}}.  Since {{math|''X''}} is non-negative we can now apply [[Markov's inequality]] to obtain {{math|Pr(''X'' ≥ 1) ≤ E[''X'']}}. Combining these we have {{math|Pr(''X'' > 0) ≤ E[''X'']}}; the first moment method is simply the use of this inequality.

==Second moment method==
In the other direction, {{math|E[''X'']}} being "large" does not directly imply that {{math|1=Pr(''X'' = 0)}} is small. However, we can often use the second moment to derive such a conclusion, using [[Cauchy–Schwarz inequality]].

{{math theorem | math_statement = If {{math|''X'' ≥ 0}} is a [[random variable]] with
finite variance, then
<math display="block">
\Pr( X > 0 )
\ge \frac{(\operatorname{E}[X])^2}{\operatorname{E}[X^2]}.
</math>}}

{{math proof | proof = Using the [[Cauchy–Schwarz inequality]], we have
<math display="block">
\operatorname{E}[X] = \operatorname{E}[ X \, \mathbf{1}_{\{ X > 0 \}} ] \le \operatorname{E}[X^2]^{1/2} \Pr( X > 0)^{1/2}.
</math>
Solving for <math>\Pr( X > 0)</math>, the desired inequality then follows. [[Q.E.D.]]}}

The method can also be used on distributional limits of random variables. Furthermore, the estimate of the previous theorem can be refined by means of the so-called  [[Paley–Zygmund inequality]]. Suppose that {{math|''X<sub>n</sub>''}} is a sequence of non-negative real-valued random variables which [[converge in law]] to a random variable {{math|''X''}}. If there are finite positive constants {{math|''c''<sub>1</sub>}}, {{math|''c''<sub>2</sub>}} such that
<math display="block">\begin{align}
\operatorname{E} \left [X_n^2 \right ] &\le c_1 \operatorname{E}[X_n]^2 \\
\operatorname{E} \left [X_n \right ] &\ge c_2
\end{align}</math>

hold for every {{mvar|n}}, then it follows from the [[Paley–Zygmund inequality]] that for every {{mvar|n}} and {{mvar|θ}} in {{open-open|0, 1}}
<math display="block"> \Pr (X_n \geq c_2 \theta) \geq \frac{(1-\theta)^2}{c_1}.</math>

Consequently, the same inequality is satisfied by {{mvar|X}}.

==Example application of method==

===Setup of problem===
The [[Bernoulli bond percolation]] [[Glossary of graph theory#Subgraphs|subgraph]] of a graph {{mvar|G}} at parameter {{mvar|p}} is a random subgraph obtained from {{mvar|G}} by deleting every edge of {{mvar|G}} with probability {{math|1−''p''}}, independently. The [[binary tree|infinite complete binary tree]] {{mvar|T}} is an infinite [[tree (graph theory)|tree]] where one vertex (called the root) has two neighbors and every other vertex has three neighbors. The second moment method can be used to show that at every parameter {{math|''p'' ∈ {{open-closed|1/2, 1}}}} with positive probability the connected component of the root in the percolation subgraph of {{mvar|T}} is infinite.

===Application of method===
Let {{mvar|K}} be the percolation component of the root, and let {{math|''T<sub>n</sub>''}} be the set of vertices of {{mvar|T}} that are at distance {{mvar|n}} from the root. Let {{math|''X<sub>n</sub>''}} be the number of vertices in {{math|''T<sub>n</sub>'' ∩ ''K''}}.

To prove that {{mvar|K}} is infinite with positive probability, it is enough to show that <math>\Pr(X_n>0\ \ \forall n)>0</math>. Since the events <math>\{X_n>0\}</math> form a decreasing sequence, by continuity of probability measures this is equivalent to showing that <math>\inf_{n}\Pr(X_n>0)>0</math>.

The [[Cauchy–Schwarz inequality]] gives
<math display="block">\operatorname{E}[X_n]^2\le \operatorname{E}[X_n^2] \, \operatorname{E}\left [(1_{X_n>0})^2\right ] = \operatorname{E}[X_n^2]\,\Pr(X_n>0).</math>
Therefore, it is sufficient to show that
<math display="block"> \inf_n \frac{\operatorname{E} \left[ X_n \right ]^2}{\operatorname{E} \left[ X_n^2 \right ]}>0\,,</math>
that is, that the second moment is bounded from above by a constant times the first moment squared (and both are nonzero). In many applications of the second moment method, one is not able to calculate the moments precisely, but can nevertheless establish this inequality.

In this particular application, these moments can be calculated. For every specific {{mvar|v}} in {{math|''T<sub>n</sub>''}},
<math display="block">\Pr(v\in K) = p^n. </math>

Since <math>|T_n| = 2^n</math>, it follows that
<math display="block">\operatorname{E}[X_n] = 2^n\,p^n</math>
which is the first moment. Now comes the second moment calculation.

<math display="block">\operatorname{E}\!\left[X_n^2 \right ] = \operatorname{E}\!\left[\sum_{v\in T_n} \sum_{u\in T_n}1_{v\in K}\,1_{u\in K}\right] = \sum_{v\in T_n} \sum_{u\in T_n} \Pr(v,u\in K).</math>

For each pair {{mvar|v}}, {{mvar|u}} in {{math|''T<sub>n</sub>''}} let {{math|''w''(''v'', ''u'')}} denote the vertex in {{math|''T''}} that is farthest away from the root and lies on the simple path in {{math|''T''}} to each of the two vertices {{mvar|v}} and {{mvar|u}}, and let {{math|''k''(''v'', ''u'')}} denote the distance from {{math|''w''}} to the root. In order for {{mvar|v}}, {{mvar|u}} to both be in {{math|''K''}}, it is necessary and sufficient for the three simple paths from {{math|''w''(''v'', ''u'')}} to {{mvar|v}}, {{mvar|u}} and the root to be in {{math|''K''}}. Since the number of edges contained in the union of these three paths is {{math|2''n'' − ''k''(''v'', ''u'')}}, we obtain
<math display="block">\Pr(v,u\in K) = p^{2n-k(v,u)}.</math>

The number of pairs {{math|(''v'', ''u'')}} such that {{math|1=''k''(''v'', ''u'') = ''s''}} is equal to <math>2^s\,2^{n-s}\,2^{n-s-1} = 2^{2n-s-1}</math>, for {{math|1=''s'' = 0, 1, ..., ''n''}}. Hence,
<math display="block">\operatorname{E}\!\left[X_n^2 \right ] = \sum_{s=0}^n 2^{2n-s-1} p^{2n-s} = \frac 1 2\,(2p)^n \sum_{s=0}^n (2p)^s = \frac 1 2 \,(2p)^n \, \frac{(2p)^{n+1}-1}{2p-1} \le \frac p{2p-1} \,\operatorname{E}[X_n]^2,</math>
which completes the proof.

===Discussion===
*The choice of the random variables {{math|''X''<sub>''n''</sub>}} was rather natural in this setup. In some more difficult applications of the method, some ingenuity might be required in order to choose the random variables {{math|''X''<sub>''n''</sub>}} for which the argument can be carried through.
*The [[Paley–Zygmund inequality]] is sometimes used instead of the [[Cauchy–Schwarz inequality]] and may occasionally give more refined results.
*Under the (incorrect) assumption that the events {{mvar|v}}, {{mvar|u}} in {{mvar|K}} are always independent, one has <math>\Pr(v,u\in K) = \Pr(v\in K) \, \Pr(u\in K)</math>, and the second moment is equal to the first moment squared. The second moment method typically works in situations in which the corresponding events or random variables are “nearly independent".
*In this application, the random variables {{math|''X''<sub>''n''</sub>}} are given as sums <math display="block">X_n = \sum_{v \in T_n} 1_{v\in K}.</math> In other applications, the corresponding useful random variables are [[integral]]s <math display="block">X_n = \int f_n(t)\,d\mu(t),</math> where the functions {{math|''f''<sub>''n''</sub>}} are random. In such a situation, one considers the product measure {{math|''μ'' × ''μ''}} and calculates <math display="block"> \begin{align}
\operatorname{E} \left[X_n^2 \right ] & = \operatorname{E}\left[\iint f_n(x)\,f_n(y)\,d\mu(x)\,d\mu(y)\right ] \\
& = \operatorname{E}\left[ \iint \operatorname{E}\left[f_n(x)\,f_n(y)\right]\,d\mu(x)\,d\mu(y)\right ],
\end{align}</math> where the last step is typically justified using [[Fubini's theorem]].

==References==
<references/>
{{refbegin}}
*{{Citation | last1=Burdzy | first1=Krzysztof | last2=Adelman | first2=Omer | last3=Pemantle | first3=Robin | title=Sets avoided by Brownian motion | hdl=1773/2194  | year=1998 | journal=Annals of Probability | volume=26 | issue=2 | pages=429–464 | doi=10.1214/aop/1022855639| arxiv=math/9701225 | s2cid=7338064 }}
*{{Citation | last1=Lyons | first1=Russell | title=Random walk, capacity, and percolation on trees | year=1992 | journal=Annals of Probability | volume=20 | issue=4 | pages=2043–2088 | doi=10.1214/aop/1176989540| doi-access=free }}
*{{Citation | last1=Lyons | first1=Russell | last2=Peres | first2=Yuval | title=Probability on trees and networks | url=http://mypage.iu.edu/~rdlyons/prbtree/prbtree.html}}
{{refend}}

[[Category:Probabilistic inequalities]]
[[Category:Articles containing proofs]]
[[Category:Moment (mathematics)]]