---
jupytext:
  text_representation:
    extension: .md
    format_name: myst
    format_version: 0.13
    jupytext_version: 1.11.1
kernelspec:
  display_name: Python 3
  language: python
  name: python3
---

# 21. Inferring a Continuous-Time System from Discrete-Time Data: An Appreciation of A. W. Phillips (1959)

```{eval-rst}
.. index::
   single: Phillips, A. W.
   single: aliasing problem
   single: identification
   single: transfer function
   single: estimation; of continuous time systems
   pair: ergodicity; estimation
```

```{note}
This chapter presents A. W. Phillips's 1959 *Biometrika* paper, "The Estimation of Parameters
in Systems of Stochastic Differential Equations." Rather than reproduce the paper, we paraphrase
Phillips's argument step by step and read it as an early application of machinery built up in
earlier chapters: covariance-stationary processes and their spectral densities, white noise and
physical realizability, rational transfer functions, the sum-of-exponentials structure of
rational autocovariances, and the folding formula for sampled processes. The framing and the
closing survey draw on an appreciation of Phillips written by Lars Peter Hansen and Thomas J.
Sargent.[^fn21-appreciation]
```

## Why Phillips's 1959 paper matters

One way to honour A. W. Phillips is to trace the continuing influence of one of his most
enduring contributions to economic dynamics: his 1959 *Biometrika* paper about how
*discrete-time* observations can be used to restrict a *continuous-time* linear model. That
paper described, with great precision, what later came to be called the problem of
**aggregation over time**, set out a framework for studying it, and obtained useful
characterizations of it. Phillips's paper shared something of the destiny of John F. Muth's
1960 and 1961 papers on rational expectations: it took the profession years to recognize how
much could be done with the ideas. In 1960, both Phillips and Muth were well ahead of most
economists in their command of the technicalities of time series analysis and in their
appreciation of its potential for economic dynamics. Economists did not really take up the
inquiry where Phillips left it until the early 1970s, when A. R. Bergstrom, Christopher Sims,
P. C. B. Phillips, and others returned to aggregation over time.

Phillips's setting is one we have been preparing for throughout these notes. He supposes that
a vector of data is generated by a continuous-time stochastic process with a **rational
spectral density matrix**[^fn21-rational], the class of processes that Chapter 8
singled out through the spectral factorization theorem, and that Chapter 11 generated by
driving constant-coefficient linear differential equations with white noise. As Phillips
recognized, processes with rational spectral densities are the natural environment for
studying time aggregation, for a chain of interrelated reasons. A rational continuous-time
spectral density implies all of the following:

(a) the discrete-time process obtained by *skip-sampling* (point-in-time sampling) the
continuous process also has a rational spectral density;

(b) the continuous-time autocovariance function is a positive semidefinite *function* on the
real line that is a weighted sum of exponentials $e^{\lambda_i \tau}$, with $\operatorname{Re}\lambda_i < 0$,
the structure that Chapter 11 derived for solutions of linear stochastic
differential equations;

(c) the discrete-time autocovariance *sequence* is a positive semidefinite sequence on the
integers that is a weighted sum of geometric sequences $\mu_i^{|\tau|}$, with decay factors
$\mu_i = e^{\lambda_i}$, with *the same* weights and *the same* exponents as in (b), mapped
through the exponential;

(d) the continuous-time data solve a system of linear differential equations driven by a
continuous-time vector white noise, itself physically *unrealizable* in the sense of Chapter 4
but well defined once convolved with an $L^2$ kernel;

(e) the discrete-time data solve a system of linear difference equations driven by a
discrete-time white noise;

(f) the continuous differential equation can be solved to express the data as a convolution
of an $L^2$ kernel with the white noise, the continuous-time Wold representation of Chapter 8;

(g) the discrete difference equation can be solved to express the sampled data as a
convolution of an $\ell^2$ sequence with the discrete white noise.

The bridge between (b) and (c) carries the same weights and relates the exponents by
$\mu_i = e^{\lambda_i}$. It is the foundation of Phillips's approach and the source of the
**aliasing problem**. Because the complex logarithm is multivalued, for any integer $k$ the decay factor
$\mu_i = \exp(\lambda_i \pm 2\pi i k)$ is consistent with the same observed discrete data; many
continuous-time models therefore share one discrete-time autocovariance sequence. P. C. B.
Phillips (1972, 1973, 1974) and Hansen and Sargent (1983, 1991) pursued this thread and, in a
sense, completed it; the precise count of observationally equivalent models is the subject of
{doc}`22_dimensionality_aliasing_problem`. What follows reconstructs the steps by which
Phillips reached the representations (a)–(g) and turned them into a procedure for inference.

```{eval-rst}
.. index::
   single: differential equation system; linear
   single: structural parameters; recovery of
```

## The continuous-time system

Phillips begins with a system of linear stochastic *lagged dependences* among $n$ variables
$x_i(t)$ of continuous time, written as a system of convolution integrals,

```{math}
:label: eq-21-sys
x_i(t) = \sum_{j \neq i} \int_0^\infty w_{ij}(h)\, x_j(t-h)\, dh + \xi_i(t), \qquad i = 1, \ldots, n.
```

Each weighting function $w_{ij}(h)$ records how the value of $x_i$ at time $t$ depends on the
history of $x_j$, and the $\xi_i(t)$ are stochastic disturbances. This is the multivariate,
two-sided cousin of the one-sided distributed lags that appeared as far back as Chapter 3 (the
shot-noise integral $\int_0^\infty L(\tau) z(t-\tau)\, d\tau$) and that organized the Wold
representation of Chapter 8.

Phillips's pivotal modelling choice is to require every lagged dependence to be **rational**:
the Laplace transform of each weighting function, which Phillips calls its *transfer function*,
is a ratio of finite-order polynomials,

```{math}
:label: eq-21-transfer
\int_0^\infty e^{-sh}\, w_{ij}(h)\, dh = \frac{\beta_{ij}(s)}{\alpha_{ij}(s)}.
```

This is exactly the rational-transfer-function language of Chapters 8 and 11. Recalling that
the Laplace transform of the derivative operator $D = d/dt$ is multiplication by $s$, the
identity that ran through the operational calculus of Chapters 8, 11, and 12, Phillips
rewrites the convolution system {eq}`eq-21-sys` as a system of *differential equations* and
then, collecting terms, as the single operator equation

```{math}
:label: eq-21-Q
Q(D)\, x(t) = \xi(t),
```

where $Q(D)$ is the $n \times n$ matrix of operators with diagonal entries $1$ and
off-diagonal entries $-\beta_{ij}(D)/\alpha_{ij}(D)$.

At this point Phillips imposes a restriction whose payoff appears only later, and which we are
in a position to read structurally. He insists that each numerator $\beta_{ij}(s)$ be of
*strictly lower degree* than its denominator $\alpha_{ij}(s)$. His own justification is that
behavioural units cannot respond to change *instantaneously*: an instantaneous response would
require a Dirac $\delta$-impulse in the weighting function $w_{ij}$, and a $\delta$-impulse is
exactly what a numerator of equal or higher degree would produce in the inverse transform.[^fn21-delta]
In the language of Chapter 4, Phillips is asking that the lagged dependences be *physically
realizable*, built from ordinary rather than generalized kernels. By the criteria of Chapters 2
and 9, a strict numerator–denominator degree gap is precisely what buys mean-square continuity,
indeed a definite *order of mean-square differentiability*, for the response. Phillips will
later show that this same restriction is what makes the separate behavioural equations
**identifiable**. A modelling assumption motivated by economics turns out to coincide with the
smoothness condition the earlier chapters identified, and to carry the weight of
identification as well.

```{eval-rst}
.. index::
   single: white noise; and physical realizability
   single: spectral factorization theorem; in Phillips's argument
   single: unrealizable disturbance
```

## The disturbances, white noise, and spectral factorization

Phillips next specifies the disturbance vector $\xi(t)$. He takes the system to be stable and
the $\xi_i(t)$ to be stationary with rational spectral densities, and he generates them by
passing a more primitive vector $\zeta(t)$ through a rational filter,

```{math}
:label: eq-21-R
\xi(t) = R(D)\, \zeta(t),
```

with $R(D)$ again a matrix of proper rational operators. Writing $\vartheta$, $\psi$, $\lambda$
for the (cross-)covariance functions of $\xi$, of $\zeta$, and between them, and $\theta$,
$\Psi$, $\Lambda$ for their two-sided Laplace transforms, a short convolution argument gives
$\theta(s) = R(s)\,\Psi(s)\,R'(-s)$. It is the continuous-time analogue of the filtering
calculation in Chapter 8.

Phillips now makes the primitive process $\zeta(t)$ a **white noise**: he sets $\Psi(s) = I$.
This is the boldest and most modern step in the paper, and it is the white noise of Chapters 3,
4, 6 and 7. Replacing $s$ by $i\omega$ turns Laplace transforms into spectral densities, and
$\Psi = I$ says that $\zeta$ has a flat spectral density at every frequency and zero cross-spectra.
Its covariance matrix is $\delta(\tau) I$, a $\delta$-impulse at the origin. As Chapter 4
stressed, such a process has infinite instantaneous variance and is *not physically
realizable*; it exists only as a generalized process, exactly like $dN/dt$ in Chapter 3 or the
formal derivative $dW/dt$ of the Wiener process in Chapter 6. It is nonetheless a perfectly
legitimate *input*, because convolving it with the proper rational kernel $R$ returns an
ordinary, physically realizable $\xi(t)$, which is the point of Chapter 4. With
$\Psi = I$,

```{math}
:label: eq-21-theta
\theta(s) = R(s)\, R'(-s).
```

Equation {eq}`eq-21-theta` is the **spectral factorization theorem** of Chapter 8, written for a
vector disturbance. Phillips has factored
the rational spectral density $\theta$ of the observable disturbance into a stable rational
operator $R$ times its conjugate $R'(-s)$. That is the factorization
$S(\omega) = \tilde P(i\omega)\tilde P(-i\omega)$ that identified the fundamental white-noise
representation in Chapter 8. The condition that the roots of the denominators of $R$ have negative real parts is
the fundamentalness (minimum-phase) condition from that chapter.

Phillips *performs* the factorization one way; {doc}`15_kalman_filter_spectral_factorization`
performs it another. Phillips works
in the frequency domain and by hand: he forms the generating function of the sampled
autocovariances, factors its numerator polynomial by finding roots, and keeps the factor whose
zeros lie outside the unit circle. That is a discrete-time spectral factorization carried out
explicitly. The Kalman–Bucy filter reaches the same object by a different route.
It solves an algebraic Riccati equation in the time domain, whose stabilizing solution delivers
the minimum-phase factor $T(s) = I + C(sI-A)^{-1}K$ automatically, with no root-finding at all.
Phillips in 1959 was doing by hand, and for the sampled process, what the Riccati equation does
in one step; the two are the frequency-domain and time-domain faces of the same theorem, and
recognizing that is what later made large systems computationally tractable.

```{eval-rst}
.. index::
   single: autocovariance function; matrix sum of exponentials
   single: rational autocovariance
   single: poles and residues
```

## The system autocovariance is a matrix sum of exponentials

Combining {eq}`eq-21-Q` and {eq}`eq-21-R`, the whole system reads $Q(D)x(t) = R(D)\zeta(t)$.
Clearing denominators by a diagonal operator $P(D)$ puts it in polynomial form,

```{math}
:label: eq-21-reduced
F(D)\, x(t) = G(D)\, \zeta(t), \qquad F = PQ,\ G = PR,
```

with $F(D)$ and $G(D)$ now matrices of *polynomials*. Because of the strict
numerator–denominator degree gap imposed earlier, the diagonal entry in each row of $F$ has
strictly higher degree than the off-diagonal entries and than the entries of $G$ in that row;
hence $\det F(D)$ does not vanish identically and the system can be solved,
$x(t) = F^{-1}(D) G(D)\, \zeta(t)$. This is the multivariate version of the operator solution
$x = \tilde P(D) w$ of Chapter 11. Reading $F^{-1}G$ as a one-sided convolution kernel makes it
the continuous-time Wold representation (f) listed above.

Repeating the filtering calculation that produced {eq}`eq-21-theta`, the spectral density
matrix of the observable vector $x$ is

```{math}
:label: eq-21-Phi
\Phi(s) = F^{-1}(s)\, G(s)\, G'(-s)\, F'^{-1}(-s).
```

Phillips now extracts the autocovariance. Writing $p(s) = |F(s)| = \prod_{r=1}^m (s - \lambda_r)$
for the characteristic polynomial and assuming the roots $\lambda_r$ (the *poles* of the system)
are distinct with negative real parts, he expands {eq}`eq-21-Phi` in partial fractions and
inverts. The unique inverse compatible with stationarity is the one that decays as
$\tau \to \pm\infty$:

```{math}
:label: eq-21-phi-cont
\phi(\tau) = \sum_{r=1}^m K_r\, e^{\lambda_r \tau}, \qquad \tau \geq 0, \qquad \phi(-\tau) = \phi(\tau)'.
```

Each coefficient $K_r$ is a *rank-one* matrix, $K_r = k_r l_r$, built from the left and right
null vectors of $F(\lambda_r)$. It is the residue of the pole $\lambda_r$. Equation
{eq}`eq-21-phi-cont` is the centrepiece, and it is precisely the result (b) promised at the
outset: **a rational spectral density produces an autocovariance that is a weighted sum of
exponentials**, with decay rates equal to the poles. In the scalar case it is exactly the
$p(\tau) = \sum_j g_j e^{\lambda_j \tau}$ of Chapter 11 and the sum-of-exponentials entries in
the table of Chapter 8. Phillips's contribution is the matrix bookkeeping, the rank-one residues
$K_r$, that carries the scalar fact into a multivariate system.

```{eval-rst}
.. index::
   single: aliasing; in Phillips's system
   single: folding formula; applied to a structural system
   single: skip sampling; and identification
```

## From continuous time to discrete time: sampling and aliasing

Here Phillips builds the bridge to the data. Sample the continuous autocovariance at unit
intervals, $\phi_\tau \equiv \phi(\tau)$ for integer $\tau$, and form the one-sided generating
function $M(z) = \sum_{\tau \ge 0} \phi_\tau z^\tau$. Substituting {eq}`eq-21-phi-cont` and
summing the geometric series gives

```{math}
:label: eq-21-M
M(z) = \sum_{r=1}^m \frac{K_r}{1 - \mu_r z}, \qquad \mu_r = e^{\lambda_r}.
```

This is result (c), and it is the exact statement that *sampling preserves the
sum-of-exponentials structure*: the same residues $K_r$ reappear, and each continuous decay
rate $\lambda_r$ is mapped to a discrete decay factor $\mu_r = e^{\lambda_r}$. The
discrete-time autocovariance sequence is a sum of geometric sequences $K_r \mu_r^{|\tau|}$, the
autocovariances of a vector ARMA process. Sampling a rational continuous-time model lands us
inside the rational *discrete-time* models studied by Bartlett, Quenouille, Whittle, and Wold,
which is exactly why Phillips can hope to estimate the continuous system from discrete data.

The map from the discrete data back to the continuous parameters runs through

```{math}
:label: eq-21-log
\lambda_r = \log \mu_r.
```

Equation {eq}`eq-21-log` is where the **aliasing problem** is born, and it is the same
phenomenon as the *folding formula* of Chapter 17. The complex logarithm is multivalued: if
$\mu_r$ is complex, then $\lambda_r$, $\lambda_r + 2\pi i$, $\lambda_r - 2\pi i$, $\ldots$ all
exponentiate to the same $\mu_r$, so infinitely many continuous-time decay rates produce the
same sampled autocovariance. Sampling at unit intervals can resolve frequencies only up to the
Nyquist frequency $\pi$; higher continuous-time frequencies are *folded* (aliased) back into
the observable band, exactly as Chapter 17's $S^d(\omega) = \sum_n S(\omega - 2\pi n)$ folds the
continuous spectrum. The data always recover the decay rate $\operatorname{Re}\lambda_r$, since
$\operatorname{Re}\lambda_r = \log|\mu_r|$. The data leave the oscillation frequency
$\operatorname{Im}\lambda_r$ undetermined. Counting precisely how many continuous models survive is
the project of {doc}`22_dimensionality_aliasing_problem`, where the requirement that the implied
continuous spectral density be positive semidefinite is shown to cut the countable infinity down
to a finite set.

Phillips also gives the constructive direction. From a finite stretch of sampled
autocovariances $\phi_\tau$ one can fit a matrix generating function $M(z) = U^{-1}(z) V(z)$
with polynomial $U, V$; the coefficients of $U$ satisfy a linear difference equation in the
$\phi_\tau$, after which a partial-fraction expansion returns the residues $K_r$ and decay
factors $\mu_r$, and {eq}`eq-21-log` returns the continuous poles $\lambda_r$. The whole
continuous spectral density $\Phi(s)$ can then be reassembled from the $K_r$ and $\lambda_r$ via
{eq}`eq-21-phi-cont`.

```{eval-rst}
.. index::
   single: identification; of behavioural equations
   single: observational equivalence; in Phillips's system
```

## Recovering and identifying the behavioural equations

Knowing $\Phi(s)$ is not the same as knowing the *structural* operators $Q$ and $\theta$, the
behavioural equations and the disturbance covariances one wants. Phillips shows how to
recover them, and exactly here the degree restriction from §"The continuous-time system" earns
its keep.

When every equation of the system has the same order $u$, the strict degree gap forces the
diagonal of $F(s)$ to be a degree-$u$ polynomial while the off-diagonal entries have degree at
most $u-1$. Counting coefficients against the residue conditions $F(\lambda_r) K_r = 0$, Phillips
finds exactly enough independent equations to pin down every coefficient of $F$; the structural
$Q(s) = P^{-1}(s) F(s)$ and the disturbance spectrum $\theta(s)$ then follow. The behavioural
equations are *identified*. When the equations are of different orders, identification can fail,
but Phillips shows it is restored if one knows that the structural disturbances are mutually
*uncorrelated*: requiring the off-diagonal entries of $G(s)G'(-s)$ to vanish forces the
admissible change-of-basis matrix to be diagonal, so the system is again identified. This is the
same kind of side information, exclusion and orthogonality restrictions, that
{doc}`22_dimensionality_aliasing_problem` and the rational-expectations literature later used to
overcome aliasing.

```{eval-rst}
.. index::
   single: ARMA process; as the discrete shadow of a continuous model
   single: impulse response; continuous versus discrete
```

## The discrete-time system implied by sampling

Phillips closes by writing the sampled data's own dynamics. Forming the two-sided generating
function and factoring it, he shows that the sampled vector $x_t$ obeys a stochastic *difference*
equation

```{math}
:label: eq-21-discrete
U(E^{-1})\, x_t = W(E^{-1})\, \epsilon_t,
```

driven by a discrete white noise $\epsilon_t$, where $E^{-1}$ is the lag operator $E^{-h} y_t =
y_{t-h}$. This is representation (e)–(g): the continuous system $F(D)x = G(D)\zeta$ has a
discrete-time ARMA shadow, the discrete Wold representation of the sampled data. Phillips's
point is that because the sampled continuous process and this discrete
ARMA process have *identical* second-moment properties, the sampling behaviour of estimators of
the continuous system can be studied through the well-developed theory of estimators for
discrete ARMA models.

Every step of Phillips's procedure runs on *sample* autocovariances $\hat\phi_\tau$ computed
from a single realization, and the argument presupposes that these converge to the
population $\phi_\tau$. That is the property called *covariance ergodicity*, and it is not
implied by covariance stationarity: it is a restriction on fourth moments, not second. For the
processes Phillips works with it does hold. A Gaussian process with a rational spectral density
has an exponentially decaying autocovariance, which suffices.
{doc}`/appendices/ergodicity` gives the conditions, and notes the one circumstance, exhibited in
{doc}`17_discrete_sampling_folding`, in which sampling itself destroys the property. It is the same identification of continuous-time content with a
discrete-time representation that organizes the companion chapters on time aggregation
({doc}`18_time_aggregation_var`, {doc}`20_aggregation_inverse_optimal_predictor`).

```{eval-rst}
.. index::
   single: continuous time econometrics; history of
   single: covariance ergodicity; and consistent estimation
```

## Phillips's legacy

Phillips set out the framework; the profession spent decades unpacking it. Two threads run
directly out of the 1959 paper. The first is the **aliasing / identification** thread of
{eq}`eq-21-log`: P. C. B. Phillips (1972, 1973, 1974) and Hansen and Sargent (1983, 1991)
characterized exactly when, and how nearly, a continuous-time model can be recovered from
skip-sampled data. {doc}`22_dimensionality_aliasing_problem` gives that analysis. The second,
pursued by Christopher Sims (1971) and John Geweke (1978), asks how closely discrete-time
distributed lags resemble continuous-time ones. {doc}`18_time_aggregation_var` takes up that
question, asking when the discrete moving-average coefficients $C_k$ resemble a sampled
continuous kernel $p(\tau)$; Sims's formula expresses the discrete lag as a
convolution of the continuous lag with a function of the regressors' autocovariance, a
time-series version of the omitted-variable theorem in which the "omitted variables" are the
continuum of unobserved sampling instants.

A related strand compares continuous and discrete *impulse responses*, Phillips's
representations (d)–(e) and (f)–(g). Hansen and Sargent (1991), staying within Phillips's
rational world, traced when discrete impulse responses resemble the underlying continuous ones
to the mean-square continuity and differentiability of the process. Chapters 2 and 9 showed that
the numerator and denominator degrees of the rational spectral density fix those properties.
That is the degree gap Phillips insisted on. Albert Marcet (1991) broke with the
rational assumption, keeping the kernel in $L^2$ but allowing discontinuities, and showed how
much harder aggregation over time becomes when the smoothness Phillips assumed is absent
({doc}`23_temporal_aggregation_streamlined`).

Finally, **rational expectations** threw new light on the aliasing problem. The cross-equation
restrictions that rational expectations imposes often supply just enough identifying information
to pin down a unique continuous-time model, sometimes without using the rational
spectral-density restriction. Christiano (1984), Christiano and Eichenbaum (1987), Heaton (1993),
and Hansen and Sargent (1991) exploited this route. Christiano offered an image of how it
works.[^fn21-wheels] More recently, Hansen and Scheinkman (1995) extended the questions Phillips
raised under (b) and (c) to *nonlinear* continuous-time Markov processes, using moment conditions
for test functions to identify them from discrete data; the operator counterparts of Phillips's
$\lambda_r \mapsto \mu_r = e^{\lambda_r}$ reappear there as the relation between the generator of
the continuous process and the transition operator of its sampled version.

In ten pages Phillips assembles rational transfer functions, the spectral factorization theorem,
white noise as a generalized input, the sum-of-exponentials autocovariance, the sampling map
$\mu = e^{\lambda}$, and the discrete ARMA shadow of a continuous system. He turns them into a
program for inference. Almost every structure that this book builds appears in his paper.

## Exercises

These exercises make Phillips's continuous-to-discrete bridge concrete. They use only `numpy`.

```{code-cell} ipython3
import numpy as np
```

```{exercise-start}
:label: ex-21-1
```

**Aliasing and the multivalued logarithm.** Phillips's map {eq}`eq-21-log`,
$\lambda_r = \log \mu_r$, is the root of the aliasing problem because the complex logarithm is
multivalued. Take a complex continuous-time pole $\lambda = -0.3 + i$.

(a) Show that $\lambda$ and the aliases $\lambda_k = -0.3 + i(1 + 2\pi k)$, for
$k = -3, \ldots, 3$, all map to the *same* discrete decay factor $\mu = e^{\lambda}$.

(b) Confirm that the decay rate $\operatorname{Re}\lambda = \log|\mu|$ is recovered from $\mu$,
whereas the oscillation frequency $\operatorname{Im}\lambda$ is determined only up to a multiple
of $2\pi$, that is, only up to the Nyquist frequency $\pi$ of unit-interval sampling
(Chapter 17).

```{exercise-end}
```

```{solution-start} ex-21-1
:class: dropdown
```

```{code-cell} ipython3
lam = -0.3 + 1j
mu = np.exp(lam)
print(f"discrete decay factor mu = {mu:.6f},  |mu| = {abs(mu):.6f}\n")

print(" k        lambda_k            exp(lambda_k)        |exp(lambda_k) - mu|")
for k in range(-3, 4):
    lam_k = -0.3 + 1j*(1 + 2*np.pi*k)
    print(f"{k:2d}   {lam_k.real:+.3f}{lam_k.imag:+.3f}i      "
          f"{np.exp(lam_k).real:+.4f}{np.exp(lam_k).imag:+.4f}i     {abs(np.exp(lam_k) - mu):.2e}")

print(f"\nrecovered decay rate  Re(lambda) = log|mu| = {np.log(abs(mu)):+.4f}  (true {lam.real:+.4f})")
print(f"frequencies consistent with the data: Im = 1 + 2*pi*k for any integer k")
```

Every alias exponentiates to the same $\mu$ to machine precision, so the discrete data cannot
distinguish them. The data pin down $\operatorname{Re}\lambda = \log|\mu| = -0.3$ exactly, but
leave $\operatorname{Im}\lambda$ ambiguous by multiples of $2\pi$. That is the folding and
aliasing of Chapter 17, quantified in {doc}`22_dimensionality_aliasing_problem`.

```{solution-end}
```

```{exercise-start}
:label: ex-21-2
```

**Phillips's procedure end-to-end on the Ornstein–Uhlenbeck process.** Let $x$ be the
scalar Ornstein–Uhlenbeck process of Chapters 7–8, $dx = -a\, x\, dt + b\, dW$, with $a = 0.7$,
$b = 1$. Its continuous-time autocovariance is $\phi(\tau) = \tfrac{b^2}{2a} e^{-a|\tau|}$
(a one-term version of {eq}`eq-21-phi-cont` with pole $\lambda = -a$).

(a) Sample $\phi$ at the integers and form Phillips's generating function $M(z) = \sum_{\tau \ge 0}
\phi_\tau z^\tau$. Verify it equals $\phi_0/(1 - \mu z)$ with $\mu = e^{-a}$, as {eq}`eq-21-M`
predicts.

(b) Recover the continuous-time parameters from the sampled autocovariances alone:
$\mu = \phi_1/\phi_0$, then $a = -\log\mu$ from {eq}`eq-21-log`, and $b^2 = 2a\,\phi_0$.

(c) Exhibit the discrete-time shadow {eq}`eq-21-discrete`: the sampled process is the AR(1)
$x_t = \mu\, x_{t-1} + \epsilon_t$. Report the innovation variance
$\sigma_\epsilon^2 = \phi_0(1 - \mu^2)$ and check that the AR(1) autocovariances $\phi_0\mu^{|\tau|}$
reproduce the sampled $\phi_\tau$.

```{exercise-end}
```

```{solution-start} ex-21-2
:class: dropdown
```

```{code-cell} ipython3
a, b = 0.7, 1.0
phi = lambda tau: (b**2/(2*a))*np.exp(-a*abs(tau))

# (a) generating function M(z)
phi_tau = np.array([phi(t) for t in range(0, 400)])
z = 0.5
M_sum = np.sum(phi_tau * z**np.arange(400))
mu_true = np.exp(-a)
M_closed = phi(0)/(1 - mu_true*z)
print(f"(a) M(z) summed   = {M_sum:.8f}")
print(f"    phi_0/(1-mu z) = {M_closed:.8f}   (mu = e^-a = {mu_true:.6f})")

# (b) recover continuous-time parameters from sampled autocovariances
mu_hat = phi(1)/phi(0)
a_hat  = -np.log(mu_hat)
b2_hat = 2*a_hat*phi(0)
print(f"\n(b) recovered: a = {a_hat:.6f} (true {a}),  b^2 = {b2_hat:.6f} (true {b**2})")

# (c) discrete-time AR(1) shadow
sig2 = phi(0)*(1 - mu_hat**2)
print(f"\n(c) AR(1):  x_t = {mu_hat:.4f} x_(t-1) + eps,  var(eps) = {sig2:.6f}")
ar1 = np.array([phi(0)*mu_hat**t for t in range(6)])
print("    AR(1) autocov vs sampled phi_tau:")
for t in range(6):
    print(f"      tau={t}:  {ar1[t]:.6f}   {phi(t):.6f}")
```

The generating function matches Phillips's closed form; the continuous decay $a$ and intensity
$b^2$ are recovered exactly from the sampled autocovariances; and the sampled process is exactly
the AR(1) whose autocovariances are $\phi_0\mu^{|\tau|}$, representation
{eq}`eq-21-discrete` in one dimension. (Because the pole $\lambda = -a$ is *real*, there is no
aliasing here: $a = -\log\mu$ is unambiguous.)

```{solution-end}
```

```{exercise-start}
:label: ex-21-3
```

**Sampling preserves the sum-of-exponentials structure.** Consider the second-order scalar
system $(D + a_1)(D + a_2)\, x = w$ driven by unit white noise, with $a_1 = 0.5$, $a_2 = 1.3$.
Its spectral density is $S(\omega) = 1/[(a_1^2 + \omega^2)(a_2^2 + \omega^2)]$, and (Chapter 8,
table; Chapter 11) its autocovariance is the two-term sum of exponentials
$\phi(\tau) = K_1 e^{-a_1|\tau|} + K_2 e^{-a_2|\tau|}$ with
$K_1 = \tfrac{1}{2a_1(a_2^2 - a_1^2)}$, $K_2 = -\tfrac{1}{2a_2(a_2^2 - a_1^2)}$.

(a) Sample at the integers and verify Phillips's identity {eq}`eq-21-M`: the generating function
$\sum_{\tau \ge 0}\phi_\tau z^\tau$ equals $\dfrac{K_1}{1 - \mu_1 z} + \dfrac{K_2}{1 - \mu_2 z}$
with $\mu_r = e^{-a_r}$, the *same residues* $K_r$, and exponents mapped by
$\mu = e^{\lambda}$.

(b) Recover the continuous poles from the discrete decay factors via {eq}`eq-21-log`,
$\lambda_r = \log\mu_r$, and confirm $\lambda_r = -a_r$.

```{exercise-end}
```

```{solution-start} ex-21-3
:class: dropdown
```

```{code-cell} ipython3
a1, a2 = 0.5, 1.3
K1 =  1/(2*a1*(a2**2 - a1**2))
K2 = -1/(2*a2*(a2**2 - a1**2))
phi = lambda tau: K1*np.exp(-a1*abs(tau)) + K2*np.exp(-a2*abs(tau))
mu1, mu2 = np.exp(-a1), np.exp(-a2)

# (a) generating-function identity
z = 0.4
M_sum = np.sum([phi(t)*z**t for t in range(4000)])
M_closed = K1/(1 - mu1*z) + K2/(1 - mu2*z)
print(f"(a) sum_tau phi_tau z^tau = {M_sum:.10f}")
print(f"    K1/(1-mu1 z)+K2/(1-mu2 z) = {M_closed:.10f}   diff {abs(M_sum-M_closed):.1e}")

# (b) recover continuous poles
print(f"\n(b) log(mu1) = {np.log(mu1):+.4f}  (true lambda_1 = {-a1})")
print(f"    log(mu2) = {np.log(mu2):+.4f}  (true lambda_2 = {-a2})")
```

The discrete generating function reproduces Phillips's partial-fraction form {eq}`eq-21-M`
exactly: sampling carries the continuous sum-of-exponentials autocovariance (b) into the discrete
sum-of-geometrics autocovariance (c) with the *same* weights and exponents related by
$\mu_r = e^{\lambda_r}$. Because both poles are real, $\lambda_r = \log\mu_r$ recovers them
without ambiguity. A real-pole system is free of the aliasing that complex poles bring.

```{solution-end}
```

## References

Aït-Sahalia, Y. (1996). Nonparametric Pricing of Interest Rate Derivative Securities.
*Econometrica*, **64**, 527–560.

Bartlett, M. S. (1946). On the theoretical specification and sampling properties of
autocorrelated time series. *Journal of the Royal Statistical Society B*, **8**, 27–41.

Bergstrom, A. R. (1983). Gaussian Estimation of Structural Parameters in Higher Order
Continuous Time Dynamic Models. *Econometrica*, **51**, 117–152.

Christiano, L. J. (1984). The Effects of Aggregation over Time on Tests of the Representative
Agent Model of Consumption. Mimeo, Federal Reserve Bank of Minneapolis.

Christiano, L. J., and M. Eichenbaum (1987). Temporal Aggregation and Structural Inference in
Macroeconomics. *Carnegie–Rochester Conference Series on Public Policy*, **26**, 63–130.

Frazer, R. A., Duncan, W. J., and Collar, A. R. (1950). *Elementary Matrices*. Cambridge
University Press.

Geweke, J. (1978). Temporal Aggregation in the Multiple Regression Model. *Econometrica*,
**46**, 643–661.

Hansen, L. P., and T. J. Sargent (1983). The Dimensionality of the Aliasing Problem in Models
with Rational Spectral Densities. *Econometrica*, **51**, 377–387.

Hansen, L. P., and T. J. Sargent (1991). Identification of Continuous Time Rational
Expectations Models from Discrete Time Data. In L. P. Hansen and T. J. Sargent, *Rational
Expectations Econometrics*. Boulder: Westview Press.

Hansen, L. P., and J. A. Scheinkman (1995). Back to the Future: Generating Moment Implications
for Continuous-Time Markov Processes. *Econometrica*, **63**, 767–804.

Heaton, J. (1993). The Interaction Between Time-Nonseparable Preferences and Time Aggregation.
*Econometrica*, **61**, 353–385.

Kareken, J. H., Muench, T., and Wallace, N. (1973). Optimal Open Market Strategy: The Use of
Information Variables. *American Economic Review*, **63**, 156–172.

Marcet, A. (1991). Temporal Aggregation of Economic Time Series. In L. P. Hansen and T. J.
Sargent, *Rational Expectations Econometrics*. Boulder: Westview Press.

Muth, J. F. (1960). Optimal Properties of Exponentially Weighted Forecasts. *Journal of the
American Statistical Association*, **55**, 299–306.

Muth, J. F. (1961). Rational Expectations and the Theory of Price Movements. *Econometrica*,
**29**, 315–335.

Phillips, A. W. (1959). The Estimation of Parameters in Systems of Stochastic Differential
Equations. *Biometrika*, **46**, 67–76.

Phillips, P. C. B. (1972). The Structural Estimation of a Stochastic Differential Equation
System. *Econometrica*, **40**, 1021–1041.

Phillips, P. C. B. (1973). The Problem of Identification in Finite Parameter Continuous Time
Models. *Journal of Econometrics*, **1**, 351–362.

Phillips, P. C. B. (1974). The Estimation of Some Continuous Time Models. *Econometrica*,
**42**, 803–823.

Quenouille, M. H. (1957). *The Analysis of Multiple Time Series*. London: Charles Griffin.

Sims, C. A. (1971). Discrete Approximations to Continuous Time Distributed Lags in
Econometrics. *Econometrica*, **39**, 545–563.

Whittle, P. (1953). The Analysis of Multiple Stationary Time Series. *Journal of the Royal
Statistical Society B*, **15**, 125–139.

Wold, H. (1949). A Large-Sample Test for Moving Averages. *Journal of the Royal Statistical
Society B*, **11**, 297–305.

## Notes

[^fn21-appreciation]: The framing of this chapter, the list of consequences (a)–(g) of a
    rational spectral density, and the closing survey follow L. P. Hansen and T. J. Sargent,
    "An Appreciation of A. W. Phillips" (November 1995), an essay surveying Phillips's 1959 work
    and the literature on aggregation over time that grew out of it.

[^fn21-rational]: For a continuous-time process, a rational spectral density is one expressible
    as a (matrix) ratio of finite-order polynomials in the angular frequency $\omega$; for a
    matrix, "division" means multiplication by the inverse. See Chapter 8.

[^fn21-delta]: If a numerator $\beta_{ij}(s)$ were of degree equal to or greater than its
    denominator $\alpha_{ij}(s)$, the partial-fraction expansion of $\beta_{ij}/\alpha_{ij}$
    would contain a constant (or polynomial) term, whose inverse Laplace transform is a Dirac
    $\delta$-impulse and its derivatives, the generalized, physically unrealizable objects of
    Chapter 4.

[^fn21-wheels]: A manifestation of aliasing familiar to movie-goers is the illusion that a
    wheel appears to spin backwards, or that as a vehicle accelerates its wheels first speed up,
    then appear to reverse. The film camera skip-samples a continuous rotation, leading the eye
    to solve $\mu = \exp(\lambda + 2\pi i k)$ for a negative apparent frequency when the true one
    is positive. Watching the wheel alone, one cannot tell; watching the vehicle move against the
    fixed background of trees and road resolves the ambiguity. That background does what the
    cross-equation restrictions of rational expectations do for an econometrician who reads
    people's behaviour as forecasts of an underlying continuous-time process.