14. Examples of Nonstationary Processes

We briefly consider an example of a nonstationary process, for which many of the results above can still be used. We consider a process \(x(t)\) that is described by

(74)\[ x(t) = \int^\infty_0 p(\tau)\, w(t-\tau)\, d\tau \]

where \(w(t)\) is a nonstationary white noise with

\[\begin{split} E w(t)\, w(t-\tau) = \begin{cases} \delta(\tau) & \text{for } t \geq -T \\ 0 & t < -T,\ T > 0. \end{cases} \end{split}\]

This specification states that \(w(t) \equiv 0\) for \(t < -T\), so that the system can be imagined to start up at \(t = -T\). We also assume that

\[ \int^\infty_0 p(\tau)^2\, d\tau = +\infty. \]

It follows from (74) that \(x(t)\) can also be expressed as

\[ x(t) = \int^t_{-T} p(t-s)\, w(s)\, ds. \]

It then follows that for \(\tau > 0\)

(75)\[ E x(t)\, x(t-\tau) = \int^{t-\tau}_{-T} p(t-s)\, p(t-\tau-s)\, ds. \]

It can be verified from (75), using the mean square differentiability criterion of 2. Mean Square Continuity and Differentiability of a Stochastic Process, the same \(p(0) = 0\) condition developed in 9. Characterizations of Mean Square Differentiability and Mean Square Continuity and 13. Locally Unpredictable Stochastic Processes, that \(x(t)\) is mean square differentiable if (a) \(p(0) = 0\), and (b) \(p(s)\) is twice differentiable.[1]

As an example, we take a nonstationary process governed by

(76)\[ x(t) = \frac{\beta + D}{D^2}\, w(t) \qquad \beta > 0 \]

or

(77)\[ x(t) = \int^{t+T}_0 (1 + \beta s)\, w(t-s)\, ds \]

where

\[ 1 + \beta t \leftrightarrow \frac{\beta + i\omega}{(i\omega)^2}\,. \]

The operator \(\dfrac{\beta + D}{D^2}\) has the rational form \(\psi(D)/\theta(D)\) of the linear stochastic differential equations of 11. Linear Stochastic Differential Equations, but with the denominator \(\theta(D) = D^2\) placing its double root at the origin rather than in the open left half plane. It is exactly this departure from the stability condition \(\operatorname{re}(\lambda_j) < 0\) assumed there, with the characteristic roots lying on the imaginary axis, that makes \(x(t)\) nonstationary, with the integrated kernel \(p(\tau) = 1 + \beta\tau\) and \(\int_0^\infty p(\tau)^2\, d\tau = \infty\). The operator calculus and prediction formulas of 11. Linear Stochastic Differential Equations and 12. Linear Least Squares Prediction nonetheless continue to apply, as the next steps show.

Applying the Wiener–Kolmogorov prediction formula of 12. Linear Least Squares Prediction to (77), here extended to this nonstationary process, we have that \(\hat E_t\, x(t+v)\) is given by

(78)\[ \hat E_t x(t+v) = \int^{t+T}_0 \left(1 + \beta (s+v)\right) w(t-s)\, ds, \qquad v > 0 \]

Since for \(E_t x(t+v)\), the “kernel” \(p(s) = 1 + \beta (s+v)\) does not satisfy \(p(0) = 0\), the process is not mean square differentiable (with respect to \(t\)); like the processes of 13. Locally Unpredictable Stochastic Processes, it is locally unpredictable, and its increments behave locally like those of a martingale. However, \(d/dv\, E_t x(t+v)\) does exist as a mean square derivative. In particular, note that

\[\begin{split} \begin{aligned} \lim_{\epsilon \to 0}\ E\ &\left[ \frac{\int^\infty_0 (1 + \beta (\tau + v + \epsilon)) w(t-\tau)\, d\tau - \int^\infty_0 (1 + \beta (\tau + v)) w(t-\tau)\, d\tau}{\epsilon} \right. \\ &\left. - \beta \int^\infty_0 w(t-\tau)\, d\tau \right]^2 = 0. \end{aligned} \end{split}\]

This shows that

(79)\[\frac{d}{dv}\ E_t\, x(t+v) = \beta \int^{t+T}_0 w(t-\tau)\, d\tau,\]

which is the expression for \(d/dv\, E_t x(t+v)\) that is obtained by differentiating the right side of (78) formally.

Equation (79) can be expressed as

(80)\[ \frac{d}{dv}\, E_t\, x(t+v) = \frac{\beta}{D}\, w(t). \]

Solving (76) formally for \(w(t)\), and using the result in (80) gives

\[\begin{split} \begin{aligned} \frac{d}{dv}\ E_t\, x(t+v) &=\ \frac{\beta}{D}\ \frac{D^2}{\beta + D}\ x(t) \\ &= \frac{\beta}{\beta + D}\ Dx(t) \end{aligned} \end{split}\]

(81)\[\frac{d}{dv}\ E_t\, x(t+v) = \beta \int^\infty_0 e^{-\beta s}\, Dx(t-s)\, ds\]

or

\[ \frac{d}{dv}\ E_t\, x(t+v) = \beta \int^{t+T}_0 e^{-\beta s}\, Dx(t-s)\, ds \]

which formally expresses the mean square derivative \(d/dv\ E_t x(t+v)\) as a geometric distributed lag of the “derivative” of \(x(t)\), it being understood that the derivative of \(x(t)\) exists only as a generalized stochastic process. Equation (81) is a version of Cagan’s adaptive expectations scheme in continuous time. Notice that the expected rate of change \(d/dv\ E_t x(t+v)\) is independent of the value of \(v\) at which it is evaluated. This is the counterpart in continuous time of the adaptive expectations scheme that was noted by Muth (1960) in discrete time, namely, that the optimal forecast is independent of horizon. This is a special property of the stochastic process (76) for which adaptive expectations are optimal or rational. 20. Aggregation Over Time and the Inverse Optimal Predictor Problem for Adaptive Expectations in Continuous Time develops exactly this example into a full bivariate money-creation/inflation model, solving the “inverse optimal predictor” problem for Cagan’s scheme, and asks what sampling and aggregation over time do to it.

Notes

[1]

Write \(x(t) = \int_{-T}^{t} p(t-s)\, w(s)\, ds\) and split the increment over \([t, t+\epsilon]\) into the revision of the existing kernel (over \(s \in [-T, t]\)) and the fresh innovations (over \(s \in [t, t+\epsilon]\)). Since \(E\, w(s) w(s') = \delta(s-s')\) and the two pieces occupy disjoint intervals of \(s\), they are uncorrelated, so $\(E\big[(x(t+\epsilon) - x(t))^2\big] = \int_{0}^{t+T} \big[p(\tau+\epsilon) - p(\tau)\big]^2\, d\tau + \int_{0}^{\epsilon} p(\tau)^2\, d\tau .\)\( Dividing by \)\epsilon^2\(, the first term tends to \)\int_0^{t+T} p’(\tau)^2, d\tau\( when \)p\( is twice continuously differentiable, while the second behaves like \)p(0)^2/\epsilon\(, which diverges unless \)p(0) = 0\(. Hence the difference quotient \)[x(t+\epsilon) - x(t)]/\epsilon\( converges in mean square if and only if \)p(0) = 0\(, making \)x\( mean square differentiable with \)Dx(t) = \int_{-T}^{t} p’(t-s), w(s), ds\(. The smoothness of \)p\( supplying convergence of the first term. This is the nonstationary counterpart of the boundary term \)p(0), w(t)$ in (49) of 9. Characterizations of Mean Square Differentiability and Mean Square Continuity.