# Linear Time Series Analysis **Thomas J. Sargent** This book is an updated and extended version of Chapters IX, X, XI, and XIV of > Thomas J. Sargent, *Macroeconomic Theory*, 2nd edition (1987), Academic Press. Those four chapters describe basic tools that are used throughout modern macroeconomics and applied dynamics. Chapter IX develops linear difference equations and the algebra of lag operators. Chapter X develops the linear least squares projection. Chapter XI combines them into a theory of covariance stationary processes and of linear prediction. Chapter XIV uses all three to construct and interpret a rational expectations equilibrium. This book builds on those foundations in several ways. Graphs are redrawn with data that have arrived since 1987. Python code illustrates many of the concepts, and each figure links to the script that draws it. Answers now accompany the exercises of the 1987 book, and many new exercises appear with answers of their own. A dozen new sections extend the theory or connect it to work published since. A reader will find that this book mostly uses what are now called **sequence space methods** rather than the recursive formulations of the macroeconomics textbooks that I wrote after the 1987 Academic Press "black book". It is handy to know both. ```{admonition} Section numbering :class: note Sections 1–38 of the time series part follow the development, and in most places the text, of Chapter XI of *Macroeconomic Theory*. Added sections carry letter suffixes — 5a, 7a, 18a, 33a, 34a, 36a–36e — and sit just after the section whose argument they extend. Sections 39–41 are postscripts that leave the linear, stationary theory behind. Readers of *Macroeconomic Theory* can navigate by the original section numbers. ``` ## What each chapter does - {doc}`Chapter IX — Difference Equations and Lag Operators ` develops the lag operator $L$ and the calculus of polynomials in $L$, and uses that calculus to solve linear difference equations: to factor a characteristic polynomial into stable and unstable roots, to invert an operator by partial fractions, and to solve the linear-quadratic (Euler equation) optimization problems of dynamic economics by the rule *solve stable roots backward and unstable roots forward*. - {doc}`Chapter X — Linear Least Squares Projections (Regressions) ` develops the linear least squares projection — the regression viewed through the orthogonality principle — together with recursive projection (the Kalman filter), the law of iterated projections, and the static signal extraction problem. - **{doc}`Linear Time Series `**, the bulk of the book, combines the two. It studies covariance stationary stochastic processes built from white noise by linear difference equations, and it asks two questions about them: what is a process's structure — its {doc}`spectrum `, its {doc}`Wold representation ` — and how does one predict it, by the {doc}`Wiener–Kolmogorov theory ` of linear prediction, {doc}`filtering `, and {doc}`signal extraction `? - {doc}`Chapter XIV — Investment Under Uncertainty ` deploys all three to construct, compute, and interpret a rational expectations equilibrium. ## How the time series theory builds on Chapters IX and X Chapters IX and X are not prerequisites to be gotten out of the way. Their ideas are the working parts of the time series theory, reused at every turn. **1. Slutsky's reinterpretation: a difference equation driven by chance.** Chapter IX solved $(1 - a_1 L - \cdots - a_n L^n)\,y_t = x_t$ for a *known* forcing sequence $\{x_t\}$. The time series theory opens ({doc}`Section 1 `) by making $\{x_t\}$ a sequence of *random* shocks — a **linear stochastic difference equation**. Slutsky's (1937) insight was that even a low-order difference equation, if driven by erratic shocks, produces realizations that look like observed business cycles. Everything Chapter IX taught about roots, stability, and oscillation now describes the second moments — the {doc}`covariogram ` and {doc}`spectrum ` — of a random process. **2. Wold's theorem: every stationary process *is* such a difference equation.** The pivot of the book is {doc}`Representation Theory `. There, a sequence of the linear least squares projections of Chapter X — regressing $x_t$ on ever-longer stretches of its own past — produces the process's **innovations** $\epsilon_t = x_t - P[x_t\mid x_{t-1},x_{t-2},\ldots]$, and Wold's decomposition theorem shows that *any* covariance-stationary process is a one-sided moving average $x_t = d(L)\epsilon_t$ of that white noise. This is exactly the fusion of the two earlier chapters: the object is a stochastic difference equation (Chapter IX), and the white noise driving it is manufactured by projection (Chapter X). Whittle's spectral factorization, added to that section, makes the construction *computable* from the spectrum in a few Fourier transforms. **3. Prediction is projection organized by lag-operator algebra.** The {doc}`Wiener–Kolmogorov prediction formula ` — the central result of the book — projects the future of a process onto its past. The projection is Chapter X; the bookkeeping is Chapter IX. Its **annihilation operator** $[\,\cdot\,]_+$ ("discard negative powers of $L$") is computed pole by pole in the very partial-fraction / first-order $1/(1-\lambda L)$ basis of Chapter IX — a connection made explicit in {doc}`The Residue Theorem Behind Partial Fractions `. Selecting the *fundamental* representation in {doc}`Sections 16–18 ` is Chapter IX's root-sorting (roots inside vs. outside the unit circle, in reciprocal pairs) applied to a covariance generating function. **4. "Stable roots backward, unstable roots forward" returns as forecasting the future.** The device that solved Chapter IX's Euler equations reappears throughout the middle part whenever agents must act on forecasts. The {doc}`Hansen–Sargent formula ` for a geometric distributed *lead* $P_t\sum_j \lambda^j x_{t+j}$ is the stochastic counterpart of the forward-looking operator of Chapter IX; it is applied to {doc}`rational expectations models ` (Cagan's hyperinflation, supply and demand with inventories), where factoring $(1-\lambda^{-1}B)(1-\lambda B)$ and inverting the unstable root forward *inside a projection operator* delivers the equilibrium and its **cross-equation restrictions**. The {doc}`chain rule of forecasting ` (pure law of iterated projections) and the {doc}`compact state-space predictor ` $P[x_{t+\tau}\mid x_t]=A^\tau x_t$ are the same idea in recursive and matrix form. **5. Causality, filtering, and exogeneity are statements about one-sided projections.** Sims's theorem that {doc}`Wiener–Granger causality equals econometric exogeneity ` says a projection is *one-sided* — again Chapter X's projection meeting Chapter IX's factorization. This single idea organizes the applications: Sims's {doc}`money–income test `, the invariance of two-sided (but not one-sided) projections under {doc}`common filtering `, the {doc}`forward-versus-backward filtering ` that preserves orthogonality conditions, and the way {doc}`errors in variables ` manufacture spurious causality by turning a one-sided projection two-sided. ## The arc of the middle part Read in sequence, the middle part moves through five stages: - **Foundations and the frequency domain** ({doc}`2 `–{doc}`7 `): covariance stationarity, the covariogram and its cross-series analogue, the {doc}`Fourier / z-transform ` machinery (with the {doc}`inverse-transform residue calculus `), and the {doc}`spectrum ` and {doc}`cross spectrum ` — gain, phase, and coherence — together with their {doc}`FFT-based estimation `. - **Filters and the business cycle** ({doc}`8 `–{doc}`12 `): what filters do to a spectrum (the {doc}`Slutsky and Kuznets ` spurious cycles), competing {doc}`definitions of the business cycle `, and the {doc}`index (dynamic-factor) model ` of comovement. - **Representation and prediction** ({doc}`13 `–{doc}`19 `): Wold's theorem, the Wiener–Kolmogorov formulas, how to build a {doc}`Wold representation ` for MA, AR, and ARMA processes, and dynamic {doc}`signal extraction `. - **Optimization, expectations, and the vector case** ({doc}`20 `–{doc}`26 `): geometric leads, the chain rule, rational expectations, vector stochastic difference equations and their {doc}`compact notation `, and optimal prediction and filtering. - **Causality and its econometric pitfalls** ({doc}`27 `–{doc}`36 `): Granger causality, filtering and projections, one-sided-projection theories, seasonal adjustment and {doc}`temporal aggregation `, errors in variables, {doc}`rational bubbles `, and — the section that Chapter XIV speaks to directly — {doc}`the difficulty of interpreting vector autoregressions `, where the equilibrium of a market with forward-looking supply and demand is read as a vector autoregression. ## Chapter XIV: a rational expectations equilibrium in action {doc}`Chapter XIV — Investment Under Uncertainty ` is where the whole apparatus is put to work. It extends the linear-quadratic Euler-equation problems of {doc}`Chapter IX ` to *stochastic* forcing processes — the certainty-equivalence principle lets forecasting and optimization separate — solves them with the Wiener–Kolmogorov / Hansen–Sargent geometric-lead formula {eq}`eq-90` of {doc}`§20 `, and assembles the pieces into Lucas and Prescott's (1971) model of a competitive industry. Its centerpiece is a precise, constructive account of a **rational expectations equilibrium** and *two* complementary ways to compute and interpret one: - **As a fixed point.** Each firm forecasts the endogenous output price by projecting on the very price process that all firms' investment decisions jointly generate. The equilibrium is a *fixed point* of the mapping from the price law of motion firms **perceive** to the one their behavior **actually** produces (Chapter XIV, §§6, 9). This is the operational content of "rational expectations": the agents in the model forecast prices as well as the economist who models them. - **As a social planning problem.** Because the competitive industry equilibrium implicitly maximizes a welfare criterion — discounted consumer surplus minus producer surplus — one can replace the awkward fixed-point calculation with a straightforward maximization (Chapter XIV, §5). Lucas and Prescott's device turns "find the equilibrium" into "solve a planning problem," a trick that pervades modern macroeconomics. **The link back to the dynamic supply and demand curves of {doc}`§36a `.** From the equilibrium Chapter XIV (§7) reads off a *dynamic supply curve*: current output depends on lagged output and on current and **expected future** prices, so — because forecasting future prices requires the parameters of the demand process — *the demand curve's parameters appear inside the supply curve*, subverting the exclusion restrictions that ordinarily identify a supply schedule. The identifying information that remains lives entirely in the **cross-equation restrictions** that a rational expectations equilibrium stamps onto the data. This is the very same object studied from the opposite side in {doc}`A Difficulty in Interpreting Vector Autoregressions ` (the Hansen–Sargent section added to the middle part): there the equilibrium of a market with **forward-looking supply and demand** is a covariance-stationary vector process — a vector autoregression — and the lesson is that its *Wold innovations are generally not the structural supply and demand shocks* that hit agents, so the innovation accounting of a fitted VAR misattributes its shocks. The two sections are two views of one equilibrium: Chapter XIV **constructs** the dynamic supply and demand curves and shows why their parameters are entangled; {doc}`§36a ` takes such an equilibrium as given and shows why a vector autoregression cannot recover the agents' surprises from it. Both derive the forward-looking decision rules by the same *stable-roots-backward / unstable-roots-forward* factorization of {doc}`Chapter IX `, and both turn on the cross-equation restrictions of {doc}`rational expectations ` — the thread that also runs through {doc}`exact linear rational expectations models `. Chapter XIV closes with a precise statement of **Lucas's critique**: because decision rules like the dynamic supply curve inherit the parameters of the processes agents forecast, they are not invariant to policy interventions in those processes — the reason estimation must reach for the "deep" parameters of preferences and technology. ## What is new in this edition Beyond modernizing the 1987 text, this edition adds several sections that extend the theory or connect it to recent work: - {doc}`The uncertainty principle for Fourier transform pairs ` — the time–frequency trade-off that limits what any filter or spectral window can resolve. - {doc}`Estimating spectra, cross spectra, and bispectra with the FFT ` — following Hinich and Clay (1968), the practical estimation counterpart to the spectrum and cross-spectrum theory: the periodogram's inconsistency, the resolution–variance trade-off, and the **bispectrum**, whose non-vanishing is a fingerprint of nonlinearity a flat spectrum cannot see. - {doc}`The residue theorem behind partial fractions ` — the complex-analysis foundation of the partial-fraction calculus used throughout the prediction and signal-extraction chapters, showing why the annihilation operator acts one pole at a time. - **Whittle's spectral factorization** (in {doc}`Representation Theory `) — a constructive, FFT-based way to recover the Wold moving-average kernel and the innovation variance from a spectral density; the computational complement to the existence theorems of {doc}`Sections 16–18 `. - {doc}`Seasonality and approximation errors ` — following Hansen and Sargent (1993), a frequency-domain criterion for when a misspecified model is better estimated with seasonally adjusted data. - {doc}`Sims's formula, derived in the time domain ` — a self-contained pedagogical companion to the {doc}`aggregation-over-time ` section, tying Sims's discrete/continuous-time result to Theil's specification theorem. - {doc}`A difficulty in interpreting vector autoregressions ` and {doc}`exact linear rational expectations models ` — following Hansen and Sargent (1991), examples in which the innovations a vector autoregression recovers are *not* the shocks that hit agents, with dynamic supply and demand curves derived by the stable-roots-backward / unstable-roots-forward method of {doc}`Chapter IX ` (the counterpart, from the econometrician's side, to the equilibrium **constructed** in Chapter XIV). - {doc}`Money demand in hyperinflations ` — a capstone application, following Sargent (1977), that reads Cagan's money-demand regression as a *misspecified distributed lag* under rational expectations. It shows that the notorious inconsistency of Cagan's estimator is an instance of **Sims's frequency-domain approximation-error formula** ({numref}`ex-1`, and {doc}`§33a `), and it ties together the {doc}`rational-expectations Cagan model `, {doc}`Granger causality ` versus mere *leading* ({doc}`§08 `), and the regime-dependence at the heart of Lucas's critique. - {doc}`Decomposing an explosive autoregression ` — a short note that applies the chapter's "unstable roots forward" device to a scalar explosive AR(1), splitting it into an explosive trend with a random amplitude plus a *stationary anticipative* component. It is the explosive-root counterpart of the Beveridge–Nelson permanent/transitory decomposition and sets up {numref}`ex-4`. - {doc}`Lucas's two illustrations of the quantity theory and Whiteman's critique ` — Lucas (1980) read the slope of a scatter of low-pass-filtered series as evidence for the quantity theory; Whiteman (1984) showed, via **Sims's approximation-error formula** once more, that this slope estimates the *sum of coefficients* in a two-sided distributed lag — the ratio of cross spectrum to spectrum at frequency zero — and that inside Lucas's own (1975) model the statistic is *not* a test of the Mundell–Tobin effect. Sargent and Surico (2011) make the point quantitative: the slope is not policy invariant. This is the third application of the approximation formula in the book, after {doc}`§33a ` and {doc}`§36c `. - Two postscripts that move beyond the linear, stationary theory: {doc}`nonlinear (Volterra / Wiener–Itô) moving-average representations `, where a nonzero bispectrum is the fingerprint of nonlinearity a flat spectrum cannot see; and {doc}`complex demodulation `, a tool for estimating a *moving* spectrum and cross spectrum, applied to the changing seasonal in U.S. interest rates. ## What this book adds throughout Relative to the 1987 original, this version: - **Corrects typographical and mathematical errors** present in the LaTeX source. - **Adds Python code** that generates modern versions of every figure, using current U.S. and international data; each figure links to the script that produced it. - **Adds worked solutions** to the exercises of Chapters IX, X, XI, and XIV as collapsible dropdowns. - **Extends several sections** with updated empirical examples and commentary, and **cross-references QuantEcon lectures** where related code already exists (see [ARMA](https://python-advanced.quantecon.org/arma.html) and [Spectral Estimation](https://python-advanced.quantecon.org/estspec.html)). ## How to read this book A reader new to the material can proceed linearly: Chapters IX and X first, then the time series part, then the Chapter XIV capstone. A reader already comfortable with lag operators and regressions can begin directly at {doc}`chapters/01_introduction` and refer back to {doc}`Chapter IX ` and {doc}`Chapter X ` as needed; {doc}`Chapter XIV ` can then be read as a self-contained application once the prediction theory of {doc}`Sections 14 ` and {doc}`20 ` is in hand. Three mathematical or estimation-oriented digressions — {doc}`Fourier and z-transforms `, {doc}`the uncertainty principle `, and {doc}`FFT estimation ` — are marked *optional on first reading* and may be skipped without loss of continuity. The {doc}`Index ` at the end collects the book's concepts, named results, and people with links to the sections where they appear. ## Notation Throughout, $L$ denotes the **lag operator**, $Lx_t = x_{t-1}$, introduced in {doc}`Section 1 of Chapter IX `. $E$ denotes the mathematical expectation operator, and $\hat E$ or $P[\,\cdot \mid \cdot\,]$ the **linear least squares projection** operator of {doc}`Chapter X `. The operator $[\,\cdot\,]_+$ is the **annihilation operator** (discard negative powers of $L$) of the Wiener–Kolmogorov formula. A polynomial $d(z)$ is **fundamental** when its zeros lie outside the unit circle, so that $d(L)^{-1}$ is one-sided in nonnegative powers of $L$. All stochastic processes are discrete time unless otherwise stated. ## References ```{bibliography} :labelprefix: I :filter: key in {"sargent1987macro", "lucasprescott1971investment", "hansensargent1991book", "hansensargent1993seasonality", "hinichclay1968application", "whittle1983prediction", "wold1938study", "anderson1971statistical", "slutsky1937summation"} ```