Isserlis' theorem

In probability theory, Isserlis' theorem or Wick's probability theorem is a formula that allows one to compute higher-order moments of the multivariate normal distribution in terms of its covariance matrix. It is named after Leon Isserlis.

This theorem is also particularly important in particle physics, where it is known as Wick's theorem after the work of Wick (1950).^[1] Other applications include the analysis of portfolio returns,^[2] quantum field theory^[3] and generation of colored noise.^[4]

Statement

If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (X_1,\dots, X_{n})} is a zero-mean multivariate normal random vector, thenFailed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \operatorname{E} [\,X_1 X_2\cdots X_{n}\,] = \sum_{p\in P_n^2}\prod_{\{i,j\}\in p} \operatorname{E}[\,X_i X_j\,] = \sum_{p\in P_n^2}\prod_{\{i,j\}\in p} \operatorname{Cov}(\,X_i, X_j\,), } where the sum is over all the pairings of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \{1,\ldots,n\}} , i.e. all distinct ways of partitioning Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \{1,\ldots,n\}} into pairs Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \{i,j\}} , and the product is over the pairs contained in Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p} .^[5]^[6]

More generally, if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (Z_1,\dots, Z_{n})} is a zero-mean complex-valued multivariate normal random vector, then the formula still holds.

The expression on the right-hand side is also known as the hafnian of the covariance matrix of $(X_{1},\dots ,X_{n})$ .

Odd case

If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n=2m+1} is odd, there does not exist any pairing of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \{1,\ldots,2m+1\}} . Under this hypothesis, Isserlis' theorem implies thatFailed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \operatorname{E}[\,X_1 X_2\cdots X_{2m+1}\,] = 0.} This also follows from the fact that $-X=(-X_{1},\dots ,-X_{n})$ has the same distribution as Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X} , which implies that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \operatorname{E}[\,X_1 \cdots X_{2m+1}\,]=\operatorname{E}[\,(-X_1) \cdots (-X_{2m+1})\,]=-\operatorname{E}[\,X_1 \cdots X_{2m+1}\,] = 0} .

Even case

In his original paper,^[7] Leon Isserlis proves this theorem by mathematical induction, generalizing the formula for the Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 4^{\text{th}}} order moments,^[8] which takes the appearance

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \operatorname{E}[\,X_1 X_2 X_3 X_4\,] = \operatorname{E}[X_1X_2]\,\operatorname{E}[X_3X_4] + \operatorname{E}[X_1X_3]\,\operatorname{E}[X_2X_4] + \operatorname{E}[X_1X_4]\,\operatorname{E}[X_2X_3]. }

If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n=2m} is even, there exist Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (2m)!/(2^{m}m!) = (2m-1)!!} (see double factorial) pair partitions of $\{1,\ldots ,2m\}$ : this yields Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (2m)!/(2^{m}m!) = (2m-1)!!} terms in the sum. For example, for Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 4^{\text{th}}} order moments (i.e. Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 4} random variables) there are three terms. For Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 6^{\text{th}}} -order moments there are Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 3\times 5=15} terms, and for Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 8^{\text{th}}} -order moments there are Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 3\times5\times7 = 105} terms.

Example

We can evaluate the characteristic function of gaussians by the Isserlis theorem:

E[e^{-iX}]=\sum _{k}{\frac {(-i)^{k}}{k!}}E[X^{k}]=\sum _{k}{\frac {(-i)^{2k}}{(2k)!}}E[X^{2k}]=\sum _{k}{\frac {(-i)^{2k}}{(2k)!}}{\frac {(2k)!}{k!2^{k}}}E[X^{2}]^{k}=e^{-{\frac {1}{2}}E[X^{2}]}

Proof

Since both sides of the formula are multilinear in Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X_1, ..., X_n} , if we can prove the real case, we get the complex case for free.

Let Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Sigma_{ij} = \operatorname{Cov}(X_i, X_j)} be the covariance matrix, so that we have the zero-mean multivariate normal random vector Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (X_1, ..., X_n) \sim N(0, \Sigma)} . Since both sides of the formula are continuous with respect to Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Sigma} , it suffices to prove the case when Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Sigma} is invertible.

Using quadratic factorization Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle -x^T\Sigma^{-1}x/2 + v^Tx - v^T\Sigma v/2 = -(x-\Sigma v)^T\Sigma^{-1}(x-\Sigma v)/2} , we get

{\frac {1}{\sqrt {(2\pi )^{n}\det \Sigma }}}\int e^{-x^{T}\Sigma ^{-1}x/2+v^{T}x}dx=e^{v^{T}\Sigma v/2}

Differentiate under the integral sign with Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \partial_{v_1, ..., v_n}|_{v_1, ..., v_n=0}} to obtain

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle E[X_1\cdots X_n] = \partial_{v_1, ..., v_n}|_{v_1, ..., v_n=0}e^{v^T\Sigma v/2}} .

That is, we need only find the coefficient of term Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle v_1\cdots v_n} in the Taylor expansion of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle e^{v^T\Sigma v/2}} .

If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n} is odd, this is zero. So let Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n = 2m} , then we need only find the coefficient of term Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle v_1\cdots v_n} in the polynomial ${\frac {1}{m!}}(v^{T}\Sigma v/2)^{m}$ .

Expand the polynomial and count, we obtain the formula. Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \square}

Generalizations

Gaussian integration by parts

An equivalent formulation of the Wick's probability formula is the Gaussian integration by parts. If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (X_1,\dots X_{n})} is a zero-mean multivariate normal random vector, then

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \operatorname{E}(X_1 f(X_1,\ldots,X_n))=\sum_{i=1}^{n} \operatorname{Cov}(X_1, X_i)\operatorname{E}(\partial_{X_i}f(X_1,\ldots,X_n)).} This is a generalization of Stein's lemma.

The Wick's probability formula can be recovered by induction, considering the function Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f:\mathbb{R}^n\to\mathbb{R}} defined by Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle f(x_1,\ldots,x_n)=x_2\ldots x_n} . Among other things, this formulation is important in Liouville conformal field theory to obtain conformal Ward identities, BPZ equations^[9] and to prove the Fyodorov-Bouchaud formula.^[10]

Non-Gaussian random variables

For non-Gaussian random variables, the moment-cumulants formula^[11] replaces the Wick's probability formula. If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (X_1,\dots X_{n})} is a vector of random variables, then Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \operatorname{E}(X_1 \ldots X_n)=\sum_{p\in P_n} \prod_{b\in p} \kappa\big((X_i)_{i\in b}\big),} where the sum is over all the partitions of $\{1,\ldots ,n\}$ , the product is over the blocks of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p} and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \kappa\big((X_i)_{i\in b}\big)} is the joint cumulant of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (X_i)_{i\in b}} .

References

^ Wick, G.C. (1950). "The evaluation of the collision matrix". Physical Review. 80 (2): 268–272. Bibcode:1950PhRv...80..268W. doi:10.1103/PhysRev.80.268.
^ Repetowicz, Przemysław; Richmond, Peter (2005). "Statistical inference of multivariate distribution parameters for non-Gaussian distributed time series" (PDF). Acta Physica Polonica B. 36 (9): 2785–2796. Bibcode:2005AcPPB..36.2785R.
^ Perez-Martin, S.; Robledo, L.M. (2007). "Generalized Wick's theorem for multiquasiparticle overlaps as a limit of Gaudin's theorem". Physical Review C. 76 (6): 064314. arXiv:0707.3365. Bibcode:2007PhRvC..76f4314P. doi:10.1103/PhysRevC.76.064314. S2CID 119627477.
^ Bartosch, L. (2001). "Generation of colored noise". International Journal of Modern Physics C. 12 (6): 851–855. Bibcode:2001IJMPC..12..851B. doi:10.1142/S0129183101002012. S2CID 54500670.
^ Janson, Svante (June 1997). Gaussian Hilbert Spaces. Cambridge Core. doi:10.1017/CBO9780511526169. ISBN 9780521561280. Retrieved 2019-11-30.
^ Michalowicz, J.V.; Nichols, J.M.; Bucholtz, F.; Olson, C.C. (2009). "An Isserlis' theorem for mixed Gaussian variables: application to the auto-bispectral density". Journal of Statistical Physics. 136 (1): 89–102. Bibcode:2009JSP...136...89M. doi:10.1007/s10955-009-9768-3. S2CID 119702133.
^ Isserlis, L. (1918). "On a formula for the product-moment coefficient of any order of a normal frequency distribution in any number of variables". Biometrika. 12 (1–2): 134–139. doi:10.1093/biomet/12.1-2.134. JSTOR 2331932.
^ Isserlis, L. (1916). "On Certain Probable Errors and Correlation Coefficients of Multiple Frequency Distributions with Skew Regression". Biometrika. 11 (3): 185–190. doi:10.1093/biomet/11.3.185. JSTOR 2331846.
^ Kupiainen, Antti; Rhodes, Rémi; Vargas, Vincent (2019-11-01). "Local Conformal Structure of Liouville Quantum Gravity". Communications in Mathematical Physics. 371 (3): 1005–1069. arXiv:1512.01802. Bibcode:2019CMaPh.371.1005K. doi:10.1007/s00220-018-3260-3. ISSN 1432-0916. S2CID 55282482.
^ Remy, Guillaume (2020). "The Fyodorov–Bouchaud formula and Liouville conformal field theory". Duke Mathematical Journal. 169. arXiv:1710.06897. doi:10.1215/00127094-2019-0045. S2CID 54777103.
^ Leonov, V. P.; Shiryaev, A. N. (January 1959). "On a Method of Calculation of Semi-Invariants". Theory of Probability & Its Applications. 4 (3): 319–329. doi:10.1137/1104031.