10. The Cramér Representation

10. The Cramér Representation#

The spectral density accomplishes an orthogonal decomposition by frequency of the variance of a covariance stationary stochastic process. The Cramér representation exhibits this decomposition in a general way.

Let \(x(t)\) be a covariance stationary, zero mean stochastic process with autocorrelation function \(R(\tau)\). Let the spectral density of \(x\) be denoted \(S(\omega)\), where

\[ S(\omega) = \int^\infty_{-\infty} R(\tau) e^{-i\omega\tau} d\tau. \]

Associated with \(x(t)\), there exists a complex valued random process \(Z(\omega)\) defined by

(i) \(Z(0) = 0\)

(ii) \(Z(-\omega) = \overline{Z(\omega)}\), where the bar denotes complex conjugation.

(iii) \(Z(\omega)\) is a process with orthogonal increments, and in particular

\[ E Z'(\omega) \overline{Z'(\nu)} = S(\nu) \delta(\nu - \omega), \]

where the prime denotes the (possibly generalized) derivative of \(Z(\omega)\) with respect to \(\omega\). The process \(Z(\omega)\) is called the “random spectral measure” of the \(x(t)\) process. This orthogonal-increments random measure reappears as the foundational object \(W\) of the Hansen–Sargent prediction calculus in 19. Prediction Formulas for Continuous Time Linear Rational Expectations Models. In terms of this random process, the \(x(t)\) process has the Cramér representation

\[ x(t) = \frac{1}{\sqrt{2\pi}}\, \int^\infty_{-\infty} e^{i\omega t} dZ(\omega) \]

or

(50)#\[x(t) = \frac{1}{\sqrt{2\pi}} \int^\infty_{-\infty} e^{i\omega t}\, Z'(\omega) d\omega.\]

To help motivate this representation, we use (50) to calculate \(R(\tau) = Ex(t) x(t-\tau)\),

\[\begin{split} \begin{aligned} Ex(t) x(t-\tau) &= E\, \frac{1}{2\pi} \int^\infty_{-\infty} e^{i\omega t}\, Z' d\omega \int^\infty_{-\infty} e^{-i\nu (t-\tau)}\, \overline{Z'(\nu)} d\nu \\ &= \frac{1}{2\pi} \int^\infty_{-\infty} e^{-i\nu (t - \tau)} \int^\infty_{-\infty} e^{i\omega t}\, EZ'(\omega)\, \overline{Z'(\nu)}\, d\omega d\nu \\ &= \frac{1}{2\pi} \int^\infty_{-\infty} e^{-i\nu (t - \tau)} \int^\infty_{-\infty} e^{i\omega t}\, S(\nu) \delta(\nu - \omega) d\omega d\nu \\ &= \frac{1}{2\pi} \int^\infty_{-\infty} e^{-i\nu (t - \tau)}\, e^{i\nu t}\, S(\nu) d\nu \end{aligned} \end{split}\]

or

\[ R(\tau) = \frac{1}{2\pi} \int^\infty_{-\infty} e^{i\nu \tau} S(\nu) d\nu. \]

This is the inversion formula (39) for recovering the autocorrelation function from the spectral density.

There is a sense in which the random spectral measure can be defined formally as the Fourier transform of \(x(t)\),

(51)#\[ Z'(\omega,\, w) = \frac{1}{\sqrt{2\pi}}\, \int^\infty_{-\infty} e^{-i\omega t}\, x(t,\, w) dt, \]

provided that the integral is interpreted delicately. In (51), we have added the argument \(w \in \Omega\) explicitly to emphasize that both \(x(t,\, w)\) and \(Z'(\omega,\, w)\) are random processes defined on the same underlying probability space \((\Omega,\, \mathcal{F},\, P)\).

Differentiating (50) formally with respect to time, we have that the mean square derivative of \(x(t)\), if it exists, has Cramér representation

\[ x'(t) = \frac{1}{\sqrt{2\pi}} \int^\infty_{-\infty} i\, \omega\, e^{i\omega t}\, Z'(\omega) d\omega. \]

The derivative of the random spectral measure associated with \(x'(t)\) is thus \(\omega Z'(\omega)\), which being proportional to \(Z'(\omega)\) is itself the derivative of a process with orthogonal increments and obeys

\[ E \omega Z'(\omega)\ \overline{\nu Z'(\nu)} = \nu^2 S(\nu) \delta(\nu - \omega). \]

Thus the spectral density of \(Dx(t)\) is \(\nu^2 S(\nu)\). More generally, consider the distributed lag

\[ y(t) = \int^\infty_{-\infty} b(\tau) x(t - \tau) d\tau \]

where \(b(\tau) \in L_2\, (-\infty,\, \infty)\). Then using (50), we have

(52)#\[\begin{split}\begin{aligned} y(t) &= \frac{1}{\sqrt{2\pi}} \int^\infty_{-\infty} b(\tau) \int^\infty_{-\infty} e^{i\omega (t-\tau)}\, Z'(\omega) d\omega \\ &= \frac{1}{\sqrt{2\pi}} \int^\infty_{-\infty} \int^\infty_{-\infty} b(\tau)\, e^{-i\omega \tau}\, d\tau\, e^{i\omega t}\, Z'(\omega) d\omega \\ y(t) &= \frac{1}{\sqrt{2\pi}} \int^\infty_{-\infty} e^{i\omega t}\, B(\omega)\, Z'(\omega) d\omega \end{aligned}\end{split}\]

where \(B(\omega)\) is the Fourier transform of \(b(t)\), namely

\[ B(\omega) = \int^\infty_{-\infty} b(\tau) e^{-i\omega \tau}\, d\tau. \]

From (52), it follows that the random spectral measure of \(y(t)\) has derivative \(B(\omega) Z'(\omega)\), and that \(y(t)\) has spectral density

\[ B(\omega)\ \overline{B(\omega)}\ S(\omega) \]

or

(53)#\[S_y(\omega) = |B(\omega)|^2 S_x(\omega)\]

where \(S_x(\omega) \equiv S(\omega)\) is the spectral density of \(x(t)\).

Equation (53) can be used to view from another angle the orthogonal decomposition across frequencies induced by \(S_x(\omega)\). For \(0 < c < d\), we create the “window” \(B_{cd}(\omega)\) according to

\[\begin{split} B_{cd}(\omega) = \begin{cases} 1 & \omega \in [c,\, d]\ \text{ or }\ [-d,\,-c] \\ 0 & \text{otherwise} \end{cases} \end{split}\]
\[ \text{where }\ d > c > 0. \]

The band-pass filter \(B_{cd}(\omega)\) is illustrated in Fig. 4.

../_images/fig-10-1_bandpass_window.png

Fig. 4 Figure 1. The band-pass filter (“window”) \(B_{cd}(\omega)\), defined on the frequency axis \(\omega \in (-\infty,\, \infty)\). It equals \(1\) on the two symmetric frequency bands \([c,\, d]\) and \([-d,\, -c]\) and equals \(0\) everywhere else, with \(d > c > 0\).#

Define the time function

\[\begin{split} \begin{aligned} b_{cd}(t) &= \frac{1}{2\pi} \int^\infty_{-\infty} B_{cd}(\omega) e^{+i\omega t}\, d\omega \\ &= \frac{1}{\pi}\ \left[ \frac{\sin\ dt}{t}\ - \ \frac{\sin\ ct}{t}\right] \end{aligned} \end{split}\]

We have that \(y_{cd}(t)\), defined by

\[ y_{cd}(t) \equiv \int^\infty_{-\infty} b_{cd}(\tau) x(t-\tau) d\tau, \]

has the spectral density \(S_{cd}\) given by

\[\begin{split} S_{cd}(\omega) = \begin{cases} S_x(\omega) & \omega \in [c,\, d]\ \text{ or }\ [-d,\, -c] \\ 0 & \text{otherwise} \end{cases} \end{split}\]

If we choose an interval parameterized by \(0 < e < f\) such that \([e,\,f] \cap [c,\, d] = 0\), it follows from the orthogonal increments property of \(Z(\omega)\) that \(y_{ef}(t)\) is orthogonal to \(y_{cd}(t-\tau)\) for all \(t\). For using the Cramér representation, we have

\[\begin{split} \begin{aligned} Ey_{cd}(t) y_{ef}(t-\tau) &= \frac{1}{2\pi} \int^\infty_{-\infty} e^{i\omega t}\, B_{cd}(\omega) Z'(\omega) \\ &\quad \int^\infty_{-\infty} e^{-i\nu (t-\tau)}\, \overline{B_{ef}(\nu)}\, Z'(\nu) d\nu d\omega \\ &= \frac{1}{2\pi} \int^\infty_{-\infty} e^{-i\nu (t - \tau)}\, \overline{B_{ef}(\nu)} \\ &\quad \int^\infty_{-\infty} B_{cd}(\omega) S(\nu) \delta(\nu - \omega) d\omega d\nu \\ &= \frac{1}{2\pi} \int^\infty_{-\infty} e^{-i\nu (t - \tau)}\, \overline{B_{ef}(\nu)}\ B_{cd}(\nu) S(\nu) d\nu \\ &= 0 \end{aligned} \end{split}\]

since \(\overline{B_{ef}(\nu)}\ B_{cd}(\nu) = 0\) for all \(\nu\).

We also have that \(y_{ef}(t)\) has spectral density

\[\begin{split} S_{ef}(\omega) = \begin{cases} S_x(\omega) & \omega \in [e,\, f]\ \text{ or }\ [-f,\, -e] \\ 0 & \text{otherwise} \end{cases} \end{split}\]

By filtering the process \(x(t)\) with filters formed by taking disjoint frequency intervals, \([c,\,d]\), \([e,\,f]\), we create processes that are orthogonal at all leads and lags, but whose respective spectral densities equal that of \(x(t)\) over the intervals \([c,\,d]\) and \([e,\,f]\).

To motivate from another angle the frequency decomposition induced by the spectral density, it is useful to note that the Cramér representation

\[ x(t) = \frac{1}{\sqrt{2\pi}} \int^\infty_{-\infty} e^{i\omega t}\, Z'(\omega) d\omega \]

can be also represented in a couple of alternative ways. Let the complex valued random process \(Z'(\omega)\) be represented in the alternative forms

\[\begin{split} \begin{aligned} z'(\omega) &= a(\omega) + b(\omega)i \\ z'(\omega) &= r(\omega)e^{i\theta(\omega)}. \end{aligned} \end{split}\]

Then it is straightforward to derive the Cramér representation in the forms

(54)#\[x(t) = \sqrt{\frac{2}{\pi}} \int^\infty_0\ a(\omega)\ \cos\ \omega t\, d\omega \ - \ \sqrt{\frac{2}{\pi}} \int^\infty_0 b(\omega)\ \sin\ \omega t\, d\omega\]

where

\[ E (a(\omega) + ib(\omega))\ (a(\nu) - ib(\nu)) = S(\nu) \delta(\nu - \omega); \]

and

(55)#\[x(t) = \sqrt{\frac{2}{\pi}} \int^\infty_0 r(\omega)\ \cos\ (\omega t + \theta(\omega)) d\omega\]

where

\[ Er(\omega) e^{i\theta(\omega)}\, r(\nu) e^{-i\theta(\nu)} = S(\nu) \delta(\nu - \omega). \]

The factor \(\sqrt{2/\pi}\) arises because the conjugate symmetry (ii) makes the negative frequencies duplicate the positive ones: \(x(t) = (2/\sqrt{2\pi})\,\operatorname{Re} \int_0^\infty e^{i\omega t} Z'(\omega)\, d\omega\), and \(2/\sqrt{2\pi} = \sqrt{2/\pi}\).

Representation (54) expresses \(x(t)\) as a weighted sum of cosine and sine waves, with the weights being random processes \(a(\omega)\) and \(b(\omega)\). Equation (55) represents \(x(t)\) as a sum of cosine waves with amplitude \(r(\omega)\) and phase \(\theta(\omega)\) being governed by random processes.

Using representation (54), it is possible to show that

\[ Ex(t)^2 = \frac{1}{2\pi} \int^\infty_{-\infty} S(\omega)\, d\omega = \frac{1}{\pi} \int^\infty_0 S(\omega)\, d\omega , \]

the last equality using \(S(\omega) = S(-\omega)\). Formally \(E\,[a(\omega)^2 + b(\omega)^2] = E\,|Z'(\omega)|^2 = S(\omega)\,\delta(0)\), so the “random amplitudes” \(a(\omega),\ b(\omega)\) are, like \(Z'\) itself, generalized processes: the statement that has ordinary meaning is the one about increments, namely that the variance contributed by the frequency band \([c,\, d]\) is \(\frac{1}{\pi}\int_c^d S(\omega)\, d\omega\), which is what the band-pass construction above computed.

The time average as a filter: mean square ergodicity#

The machinery just assembled settles a question raised in 1. Covariance Stationary Stochastic Processes and left open there: when does averaging a single realization over time reveal the ensemble mean? The answer costs nothing extra, because the time average is simply one more filter of the kind (53) describes.

Let \(x(t)\) be covariance stationary with mean \(\mu\), and form the time average over a record of length \(2T\),

\[ \bar x_T = \frac{1}{2T}\int^{T}_{-T} x(t)\, dt . \]

Call \(x(t)\) mean square ergodic if \(E(\bar x_T - \mu)^2 \to 0\) as \(T \to \infty\), so that \(\bar x_T\) converges in mean square to \(\mu\).

Now observe that \(\bar x_T\) is the output of a filter applied to \(x\), with weighting function \(b(\tau) = 1/2T\) on \([-T,T]\) and zero elsewhere. Its transfer function is

\[ B_T(\omega) = \frac{1}{2T}\int^{T}_{-T} e^{-i\omega t}\, dt = \frac{\sin \omega T}{\omega T}, \]

so by (52) the centered time average has the Cramér representation \(\bar x_T - \mu = (2\pi)^{-1/2}\int B_T(\omega) Z'(\omega)\, d\omega\), and by (53) its variance is

(56)#\[E(\bar x_T - \mu)^2 = \frac{1}{2\pi}\int^\infty_{-\infty} \left(\frac{\sin \omega T}{\omega T}\right)^{2} S(\omega)\, d\omega .\]

Equation (56) answers the question by inspection. The kernel \((\sin \omega T/\omega T)^2\) never exceeds \(1\), equals \(1\) at \(\omega = 0\), and tends to \(0\) at every \(\omega \neq 0\) as \(T\) grows. So as the record lengthens, the filter annihilates every frequency except the origin: time averaging is a band-pass filter whose band collapses onto zero frequency. What survives in the limit is whatever variance the process carries exactly at \(\omega = 0\).

For processes with an ordinary spectral density that limit is zero, and \(x\) is mean square ergodic. But the linearly deterministic component of Wold’s theorem (8. Spectral Densities) carries \(\delta\)-functions, \(S_d(\omega) = \sum_j a_j\pi[\delta(\omega - \omega_j) + \delta(\omega + \omega_j)]\), and if one of the \(\omega_j\) is zero then (56) retains a term that never vanishes. Hence

Criterion 3 (Mean square ergodicity)

A covariance stationary process is mean square ergodic if and only if its spectral density carries no \(\delta\)-function at frequency zero, equivalently iff its linearly deterministic component contains no zero-frequency term.

Failure has exactly one form: a random constant. If \(x(t) \equiv A\) with \(EA = 0\) and \(EA^2 = \sigma^2\), then \(R(\tau) = \sigma^2\) for every \(\tau\), the spectral density is \(2\pi\sigma^2\delta(\omega)\), and \(\bar x_T = A\) for every \(T\). The time average never approaches the ensemble mean, because there is nothing to average. Every purely linearly indeterministic process is therefore mean square ergodic, since its spectral density \(|\tilde P(i\omega)|^2\) is an ordinary function with no atoms. The book’s standing assumption already delivers the property.

Two consequences follow. First, when \(\int |R(\tau)|\,d\tau < \infty\), letting \(T \to \infty\) in the time-domain counterpart of (56),

\[ E(\bar x_T - \mu)^2 = \frac{1}{2T}\int^{2T}_{-2T}\Big(1 - \frac{|\tau|}{2T}\Big) R(\tau)\, d\tau, \]

gives

(57)#\[2T \cdot E(\bar x_T - \mu)^2 \;\longrightarrow\; \int^\infty_{-\infty} R(\tau)\, d\tau = S(0) .\]

The variance of the sample mean is asymptotically \(S(0)/2T\): the spectral density at the origin, divided by the length of the record. The “long-run variance” of time series econometrics is nothing but the spectrum at zero frequency.

Second, and less comfortably: this settles the sample mean only. Whether the sample autocovariances converge to \(R(\tau)\) is a question about fourth moments. That is the property on which the estimation in 21. Inferring a Continuous-Time System from Discrete-Time Data: An Appreciation of A. W. Phillips (1959) and 22. The Dimensionality of the Aliasing Problem in Models with Rational Spectral Densities relies, which the second-moment theory of this book does not settle. A1. Ergodicity and the Consistent Estimation of Second Moments takes that up.

The band decomposition and sampling#

This chapter exhibits a process as a superposition of mutually orthogonal frequency bands, each carrying a definite share of the variance. Keep that picture in mind when reading 17. Discrete Sampling: The Folding Formula. Sampling at interval \(T\) does not destroy the bands; it wraps them. Every band is translated by a multiple of \(2\pi/T\) onto the observable interval \([-\pi/T,\, \pi/T]\) and there added to whatever was already present, which is precisely what the folding formula asserts. Because the bands were orthogonal to begin with, their variances simply add, and the discrete spectrum at a given frequency is the total variance of all the continuous bands that fold onto it. Nothing tells the observer how that total was divided among them.

That last sentence is the aliasing problem, and 22. The Dimensionality of the Aliasing Problem in Models with Rational Spectral Densities turns it into a construction: given any continuous spectral density, one can build a different one, supported entirely on high-frequency bands, that folds onto the same discrete spectrum. The alternative process is manufactured with exactly the band-pass window \(B_{cd}(\omega)\) of this chapter, applied at frequencies above the Nyquist rate. Two processes that could hardly look less alike in continuous time, one concentrated at low frequencies and the other carrying all of its power above \(\pi\), are then indistinguishable in the sampled data.

Exercises#

import numpy as np
from scipy.integrate import quad

Exercise 11

Variance by frequency band, and what sampling does to it. Take the Ornstein–Uhlenbeck process of 7. Stochastic Differential Equations Driven by a Wiener Process with \(a = 1\), \(b = 0.7\), whose spectral density is \(S(\omega) = b^2/(a^2+\omega^2)\).

(a) Verify that the band variances add up: partition \([0,\infty)\) into bands and check that \(\sum_{\text{bands}} \frac{1}{\pi}\int_c^d S(\omega)\, d\omega = R(0) = b^2/(2a)\), which is property (iii) of 8. Spectral Densities.

(b) Report the share of the variance lying above the Nyquist frequency \(\pi/T\) for \(T = 0.5\), \(1\) and \(3\). This is the power that sampling at interval \(T\) folds back into the observable band rather than discarding. That quantity governs how badly the formula of 17. Discrete Sampling: The Folding Formula distorts the spectrum.

(c) Confirm the orthogonality claim of the text numerically in the frequency domain: for disjoint bands, \(\int B_{cd}(\omega)\,B_{ef}(\omega)\,S(\omega)\,d\omega = 0\), so the band-pass filtered series are uncorrelated at all leads and lags.

Exercise 12

Mean square ergodicity, and how it fails. Continue with the Ornstein–Uhlenbeck process, \(R(\tau) = \frac{b^2}{2a}e^{-a|\tau|}\), \(S(\omega) = b^2/(a^2+\omega^2)\), \(a=1\), \(b=0.7\).

(a) Evaluate \(E(\bar x_T-\mu)^2\) from the time-domain formula \(\frac{1}{2T}\int_{-2T}^{2T}(1-|\tau|/2T)R(\tau)d\tau\) for \(T = 5, 20, 100\), and confirm it approaches \(S(0)/2T\) of (57).

(b) Confirm the same numbers from the frequency-domain formula (56), integrating \((\sin\omega T/\omega T)^2 S(\omega)\). The two routes are the same computation seen from the two sides of the Cramér representation.

(c) Now break ergodicity. Add a random constant: \(y(t) = x(t) + A\) with \(A\) independent of \(x\), mean zero, variance \(\sigma^2 = 0.5\). Show that \(E(\bar y_T - \mu)^2 \to \sigma^2\) rather than to zero, however long the record. The spectral density of \(y\) carries a \(\delta\)-function at \(\omega = 0\).