Linear Time Series Analysis

Linear Time Series Analysis#

Thomas J. Sargent

This book is an updated and extended version of Chapters IX, X, XI, and XIV of

Thomas J. Sargent, Macroeconomic Theory, 2nd edition (1987), Academic Press.

Those four chapters describe basic tools that are used throughout modern macroeconomics and applied dynamics. Chapter IX develops linear difference equations and the algebra of lag operators. Chapter X develops the linear least squares projection. Chapter XI combines them into a theory of covariance stationary processes and of linear prediction. Chapter XIV uses all three to construct and interpret a rational expectations equilibrium.

This book builds on those foundations in several ways. Graphs are redrawn with data that have arrived since 1987. Python code illustrates many of the concepts, and each figure links to the script that draws it. Answers now accompany the exercises of the 1987 book, and many new exercises appear with answers of their own. A dozen new sections extend the theory or connect it to work published since.

A reader will find that this book mostly uses what are now called sequence space methods rather than the recursive formulations of the macroeconomics textbooks that I wrote after the 1987 Academic Press “black book”. It is handy to know both.

Section numbering

Sections 1–38 of the time series part follow the development, and in most places the text, of Chapter XI of Macroeconomic Theory. Added sections carry letter suffixes — 5a, 7a, 18a, 33a, 34a, 36a–36e — and sit just after the section whose argument they extend. Sections 39–41 are postscripts that leave the linear, stationary theory behind. Readers of Macroeconomic Theory can navigate by the original section numbers.

What each chapter does#

  • Chapter IX — Difference Equations and Lag Operators develops the lag operator \(L\) and the calculus of polynomials in \(L\), and uses that calculus to solve linear difference equations: to factor a characteristic polynomial into stable and unstable roots, to invert an operator by partial fractions, and to solve the linear-quadratic (Euler equation) optimization problems of dynamic economics by the rule solve stable roots backward and unstable roots forward.

  • Chapter X — Linear Least Squares Projections (Regressions) develops the linear least squares projection — the regression viewed through the orthogonality principle — together with recursive projection (the Kalman filter), the law of iterated projections, and the static signal extraction problem.

  • Linear Time Series, the bulk of the book, combines the two. It studies covariance stationary stochastic processes built from white noise by linear difference equations, and it asks two questions about them: what is a process’s structure — its spectrum, its Wold representation — and how does one predict it, by the Wiener–Kolmogorov theory of linear prediction, filtering, and signal extraction?

  • Chapter XIV — Investment Under Uncertainty deploys all three to construct, compute, and interpret a rational expectations equilibrium.

How the time series theory builds on Chapters IX and X#

Chapters IX and X are not prerequisites to be gotten out of the way. Their ideas are the working parts of the time series theory, reused at every turn.

1. Slutsky’s reinterpretation: a difference equation driven by chance. Chapter IX solved \((1 - a_1 L - \cdots - a_n L^n)\,y_t = x_t\) for a known forcing sequence \(\{x_t\}\). The time series theory opens (Section 1) by making \(\{x_t\}\) a sequence of random shocks — a linear stochastic difference equation. Slutsky’s (1937) insight was that even a low-order difference equation, if driven by erratic shocks, produces realizations that look like observed business cycles. Everything Chapter IX taught about roots, stability, and oscillation now describes the second moments — the covariogram and spectrum — of a random process.

2. Wold’s theorem: every stationary process is such a difference equation. The pivot of the book is Representation Theory. There, a sequence of the linear least squares projections of Chapter X — regressing \(x_t\) on ever-longer stretches of its own past — produces the process’s innovations \(\epsilon_t = x_t - P[x_t\mid x_{t-1},x_{t-2},\ldots]\), and Wold’s decomposition theorem shows that any covariance-stationary process is a one-sided moving average \(x_t = d(L)\epsilon_t\) of that white noise. This is exactly the fusion of the two earlier chapters: the object is a stochastic difference equation (Chapter IX), and the white noise driving it is manufactured by projection (Chapter X). Whittle’s spectral factorization, added to that section, makes the construction computable from the spectrum in a few Fourier transforms.

3. Prediction is projection organized by lag-operator algebra. The Wiener–Kolmogorov prediction formula — the central result of the book — projects the future of a process onto its past. The projection is Chapter X; the bookkeeping is Chapter IX. Its annihilation operator \([\,\cdot\,]_+\) (“discard negative powers of \(L\)”) is computed pole by pole in the very partial-fraction / first-order \(1/(1-\lambda L)\) basis of Chapter IX — a connection made explicit in The Residue Theorem Behind Partial Fractions. Selecting the fundamental representation in Sections 16–18 is Chapter IX’s root-sorting (roots inside vs. outside the unit circle, in reciprocal pairs) applied to a covariance generating function.

4. “Stable roots backward, unstable roots forward” returns as forecasting the future. The device that solved Chapter IX’s Euler equations reappears throughout the middle part whenever agents must act on forecasts. The Hansen–Sargent formula for a geometric distributed lead \(P_t\sum_j \lambda^j x_{t+j}\) is the stochastic counterpart of the forward-looking operator of Chapter IX; it is applied to rational expectations models (Cagan’s hyperinflation, supply and demand with inventories), where factoring \((1-\lambda^{-1}B)(1-\lambda B)\) and inverting the unstable root forward inside a projection operator delivers the equilibrium and its cross-equation restrictions. The chain rule of forecasting (pure law of iterated projections) and the compact state-space predictor \(P[x_{t+\tau}\mid x_t]=A^\tau x_t\) are the same idea in recursive and matrix form.

5. Causality, filtering, and exogeneity are statements about one-sided projections. Sims’s theorem that Wiener–Granger causality equals econometric exogeneity says a projection is one-sided — again Chapter X’s projection meeting Chapter IX’s factorization. This single idea organizes the applications: Sims’s money–income test, the invariance of two-sided (but not one-sided) projections under common filtering, the forward-versus-backward filtering that preserves orthogonality conditions, and the way errors in variables manufacture spurious causality by turning a one-sided projection two-sided.

The arc of the middle part#

Read in sequence, the middle part moves through five stages:

Chapter XIV: a rational expectations equilibrium in action#

Chapter XIV — Investment Under Uncertainty is where the whole apparatus is put to work. It extends the linear-quadratic Euler-equation problems of Chapter IX to stochastic forcing processes — the certainty-equivalence principle lets forecasting and optimization separate — solves them with the Wiener–Kolmogorov / Hansen–Sargent geometric-lead formula (282) of §20, and assembles the pieces into Lucas and Prescott’s (1971) model of a competitive industry. Its centerpiece is a precise, constructive account of a rational expectations equilibrium and two complementary ways to compute and interpret one:

  • As a fixed point. Each firm forecasts the endogenous output price by projecting on the very price process that all firms’ investment decisions jointly generate. The equilibrium is a fixed point of the mapping from the price law of motion firms perceive to the one their behavior actually produces (Chapter XIV, §§6, 9). This is the operational content of “rational expectations”: the agents in the model forecast prices as well as the economist who models them.

  • As a social planning problem. Because the competitive industry equilibrium implicitly maximizes a welfare criterion — discounted consumer surplus minus producer surplus — one can replace the awkward fixed-point calculation with a straightforward maximization (Chapter XIV, §5). Lucas and Prescott’s device turns “find the equilibrium” into “solve a planning problem,” a trick that pervades modern macroeconomics.

The link back to the dynamic supply and demand curves of §36a. From the equilibrium Chapter XIV (§7) reads off a dynamic supply curve: current output depends on lagged output and on current and expected future prices, so — because forecasting future prices requires the parameters of the demand process — the demand curve’s parameters appear inside the supply curve, subverting the exclusion restrictions that ordinarily identify a supply schedule. The identifying information that remains lives entirely in the cross-equation restrictions that a rational expectations equilibrium stamps onto the data. This is the very same object studied from the opposite side in A Difficulty in Interpreting Vector Autoregressions (the Hansen–Sargent section added to the middle part): there the equilibrium of a market with forward-looking supply and demand is a covariance-stationary vector process — a vector autoregression — and the lesson is that its Wold innovations are generally not the structural supply and demand shocks that hit agents, so the innovation accounting of a fitted VAR misattributes its shocks. The two sections are two views of one equilibrium: Chapter XIV constructs the dynamic supply and demand curves and shows why their parameters are entangled; §36a takes such an equilibrium as given and shows why a vector autoregression cannot recover the agents’ surprises from it. Both derive the forward-looking decision rules by the same stable-roots-backward / unstable-roots-forward factorization of Chapter IX, and both turn on the cross-equation restrictions of rational expectations — the thread that also runs through exact linear rational expectations models. Chapter XIV closes with a precise statement of Lucas’s critique: because decision rules like the dynamic supply curve inherit the parameters of the processes agents forecast, they are not invariant to policy interventions in those processes — the reason estimation must reach for the “deep” parameters of preferences and technology.

What is new in this edition#

Beyond modernizing the 1987 text, this edition adds several sections that extend the theory or connect it to recent work:

  • The uncertainty principle for Fourier transform pairs — the time–frequency trade-off that limits what any filter or spectral window can resolve.

  • Estimating spectra, cross spectra, and bispectra with the FFT — following Hinich and Clay (1968), the practical estimation counterpart to the spectrum and cross-spectrum theory: the periodogram’s inconsistency, the resolution–variance trade-off, and the bispectrum, whose non-vanishing is a fingerprint of nonlinearity a flat spectrum cannot see.

  • The residue theorem behind partial fractions — the complex-analysis foundation of the partial-fraction calculus used throughout the prediction and signal-extraction chapters, showing why the annihilation operator acts one pole at a time.

  • Whittle’s spectral factorization (in Representation Theory) — a constructive, FFT-based way to recover the Wold moving-average kernel and the innovation variance from a spectral density; the computational complement to the existence theorems of Sections 16–18.

  • Seasonality and approximation errors — following Hansen and Sargent (1993), a frequency-domain criterion for when a misspecified model is better estimated with seasonally adjusted data.

  • Sims’s formula, derived in the time domain — a self-contained pedagogical companion to the aggregation-over-time section, tying Sims’s discrete/continuous-time result to Theil’s specification theorem.

  • A difficulty in interpreting vector autoregressions and exact linear rational expectations models — following Hansen and Sargent (1991), examples in which the innovations a vector autoregression recovers are not the shocks that hit agents, with dynamic supply and demand curves derived by the stable-roots-backward / unstable-roots-forward method of Chapter IX (the counterpart, from the econometrician’s side, to the equilibrium constructed in Chapter XIV).

  • Money demand in hyperinflations — a capstone application, following Sargent (1977), that reads Cagan’s money-demand regression as a misspecified distributed lag under rational expectations. It shows that the notorious inconsistency of Cagan’s estimator is an instance of Sims’s frequency-domain approximation-error formula (Exercise 33, and §33a), and it ties together the rational-expectations Cagan model, Granger causality versus mere leading (§08), and the regime-dependence at the heart of Lucas’s critique.

  • Decomposing an explosive autoregression — a short note that applies the chapter’s “unstable roots forward” device to a scalar explosive AR(1), splitting it into an explosive trend with a random amplitude plus a stationary anticipative component. It is the explosive-root counterpart of the Beveridge–Nelson permanent/transitory decomposition and sets up Exercise 36.

  • Lucas’s two illustrations of the quantity theory and Whiteman’s critique — Lucas (1980) read the slope of a scatter of low-pass-filtered series as evidence for the quantity theory; Whiteman (1984) showed, via Sims’s approximation-error formula once more, that this slope estimates the sum of coefficients in a two-sided distributed lag — the ratio of cross spectrum to spectrum at frequency zero — and that inside Lucas’s own (1975) model the statistic is not a test of the Mundell–Tobin effect. Sargent and Surico (2011) make the point quantitative: the slope is not policy invariant. This is the third application of the approximation formula in the book, after §33a and §36c.

  • Two postscripts that move beyond the linear, stationary theory: nonlinear (Volterra / Wiener–Itô) moving-average representations, where a nonzero bispectrum is the fingerprint of nonlinearity a flat spectrum cannot see; and complex demodulation, a tool for estimating a moving spectrum and cross spectrum, applied to the changing seasonal in U.S. interest rates.

What this book adds throughout#

Relative to the 1987 original, this version:

  • Corrects typographical and mathematical errors present in the LaTeX source.

  • Adds Python code that generates modern versions of every figure, using current U.S. and international data; each figure links to the script that produced it.

  • Adds worked solutions to the exercises of Chapters IX, X, XI, and XIV as collapsible dropdowns.

  • Extends several sections with updated empirical examples and commentary, and cross-references QuantEcon lectures where related code already exists (see ARMA and Spectral Estimation).

How to read this book#

A reader new to the material can proceed linearly: Chapters IX and X first, then the time series part, then the Chapter XIV capstone. A reader already comfortable with lag operators and regressions can begin directly at Introduction and refer back to Chapter IX and Chapter X as needed; Chapter XIV can then be read as a self-contained application once the prediction theory of Sections 14 and 20 is in hand. Three mathematical or estimation-oriented digressions — Fourier and z-transforms, the uncertainty principle, and FFT estimation — are marked optional on first reading and may be skipped without loss of continuity. The Index at the end collects the book’s concepts, named results, and people with links to the sections where they appear.

Notation#

Throughout, \(L\) denotes the lag operator, \(Lx_t = x_{t-1}\), introduced in Section 1 of Chapter IX. \(E\) denotes the mathematical expectation operator, and \(\hat E\) or \(P[\,\cdot \mid \cdot\,]\) the linear least squares projection operator of Chapter X. The operator \([\,\cdot\,]_+\) is the annihilation operator (discard negative powers of \(L\)) of the Wiener–Kolmogorov formula. A polynomial \(d(z)\) is fundamental when its zeros lie outside the unit circle, so that \(d(L)^{-1}\) is one-sided in nonnegative powers of \(L\). All stochastic processes are discrete time unless otherwise stated.

References#

[IAnd71]

Theodore W. Anderson. The Statistical Analysis of Time Series. John Wiley & Sons, New York, 1971.

[IHS91]

Lars Peter Hansen and Thomas J. Sargent. Rational Expectations Econometrics. Westview Press, Boulder, CO, 1991.

[IHS93]

Lars Peter Hansen and Thomas J. Sargent. Seasonality and approximation errors in rational expectations models. Journal of Econometrics, 55(1-2):21–55, 1993.

[IHC68]

Melvin J. Hinich and Clarence S. Clay. The application of the discrete fourier transform in the estimation of power spectra, coherence, and bispectra of geophysical data. Reviews of Geophysics, 6(3):347–363, 1968.

[ILP71]

Jr. Lucas, Robert E. and Edward C. Prescott. Investment under uncertainty. Econometrica, 39(5):659–681, 1971.

[ISar87]

Thomas J. Sargent. Macroeconomic Theory. Academic Press, New York, 2 edition, 1987.

[ISlu37]

Eugen Slutsky. The summation of random causes as the source of cyclic processes. Econometrica, 5(2):105–146, 1937.

[IWhi83]

Peter Whittle. Prediction and Regulation by Linear Least-Square Methods. University of Minnesota Press, Minneapolis, 2 edition, 1983.

[IWol38]

Herman O. A. Wold. A Study in the Analysis of Stationary Time Series. Almqvist and Wiksell, Uppsala, 1938.