Terminology#

Each field that solves inverse problems has its own words and letters for the same things. This page defines the words fips uses, once each, with the symbol, the name in the code, and the other names you may know them by. The rest of the documentation uses these words and no others.

The linear model#

fips solves the linear model

\[z = H x + c + \varepsilon\]

for the state \(x\), given the observations \(z\), the forward operator \(H\), the offset \(c\), a prior \(x_0\) with its prior covariance \(S_0\), and the observation covariance \(S_z\) (the covariance of the error \(\varepsilon\)). With \(n\) state elements and \(m\) observations, the answer is the posterior

\[\hat{x} = x_0 + K (z - H x_0 - c), \qquad \hat{S} = S_0 - K H S_0, \qquad K = S_0 H^T \left(H S_0 H^T + S_z / \gamma\right)^{-1}.\]

The letters follow one convention: lowercase for vectors, uppercase for matrices, a subscript \(0\) for the prior, a subscript \(z\) for the observations, and a hat for the posterior.

The words#

Word	Symbol	In fips	What it is
state	\(x\)	the prior’s labels (`problem.prior.space`)	The unknowns the problem solves for (fluxes, densities, slownesses), \(n\) of them. Also called the control vector or the parameters.
observation	\(z\)	`obs`	The measured data, \(m\) of them. Also called the data or the measurement vector (Rodgers writes \(y\)).
forward operator	\(H\)	`forward`	The matrix that maps a state to the observations it would produce, \(m \times n\). Also called the Jacobian (of a nonlinear model, linearized), the observation operator, the sensitivity matrix or, in atmospheric transport, the footprints. `Problem.predict` gives \(H x + c\) for any state.
offset	\(c\)	`offset`	The part of the observations the state does not explain, added to \(H x\): a background concentration, a known bias. Optional. An offset the data should decide is a block of the state instead (`offset_block()`).
prior	\(x_0\)	`prior`	What is known of the state before the observations: an inventory, a climatology, a first guess. Also called the a priori or the background state.
prior covariance	\(S_0\)	`prior_cov`	The covariance of the prior’s error, \(n \times n\). Also called the prior error covariance (\(B\) in data assimilation, \(S_a\) in Rodgers, \(C_M\) in Tarantola).
observation covariance	\(S_z\)	`obs_cov`	The covariance of the error \(\varepsilon\) between the observations and what the forward operator models, \(m \times m\): measurement error plus the errors of the model and of representing the state. Also called the model-data mismatch or the observation error covariance (\(R\) in data assimilation, \(S_\epsilon\) in Rodgers, \(C_D\) in Tarantola).
innovation	\(d\)	`post.innovation_cov`	The observations minus what the prior models, \(d = z - H x_0 - c\). Its covariance, \(H S_0 H^T + S_z / \gamma\), is \(m \times m\). The solve factors it, whitened by the model-data mismatch, unless there are fewer state elements than observations, the mismatch is dense, diagonal, sparse, block diagonal or Kronecker, and the prior error covariance has a square root (a positive-definite one does): it then factors an \(n \times n\) matrix instead.
posterior	\(\hat{x}\), \(\hat{S}\)	`fips.solve(problem)`, a `Posterior`	The answer: the state estimate \(\hat{x}\) (`post.estimate`) and its posterior covariance \(\hat{S}\) (`post.cov()`, whose diagonal is `post.variances`). Also called the a posteriori, the analysis or the retrieval. `post.modeled_obs` is \(H \hat{x} + c\).
gain	\(K\)	`post.kalman_gain`	The \(n \times m\) matrix that weighs how much each observation moves the estimate (the Kalman gain, the contribution function).
averaging kernel	\(A\)	`post.averaging_kernel()`	\(A = K H\), how the estimate responds to the true state: a diagonal near 1 is a state element the observations constrain, near 0 one left at its prior.
DOFS	\(\operatorname{tr}(A)\)	`post.dofs`	Degrees of freedom for signal: the number of independent pieces of information the observations give about the state. `post.dofs_per_element` is the diagonal of \(A\), each state element’s share, which sums to DOFS.
information content	\(-\tfrac{1}{2} \ln \lvert I - A \rvert\)	`post.information_content`	The Shannon information content, in nats (divide by \(\ln 2\) for bits): how much the observations shrink the volume of the state’s uncertainty, \(\tfrac{1}{2} \ln (\lvert S_0 \rvert / \lvert \hat{S} \rvert)\), on a log scale.
gamma	\(\gamma\)	`gamma`, `post.with_gamma(g)`	The regularization factor: the solve uses \(S_z / \gamma\), so a gamma above 1 trusts the observations more and below 1 the prior more. 1 by default.

The posterior also has these statistics:

Statistic	In fips	What it is
chi-squared	`post.chi2`	The misfit of the estimate to the observations and to the prior, each weighed by its covariance; `post.reduced_chi2` divides it by \(m\), and is near 1 when the covariances are right. `post.chi2_tests()` gives its degrees of freedom and p-value, and the same test of three more vectors (Rodgers’ four tests): the residual \(z - H \hat{x} - c\), the increment \(\hat{x} - x_0\), and the modeled increment \(H (\hat{x} - x_0)\), the increment as the observations see it. In a linear problem the residual’s test gives the innovation’s numbers and the modeled increment’s the increment’s.
\(R^2\), RMSE	`post.r2`, `post.rmse`	The squared correlation of the observations less the offset, \(z - c\), with \(H \hat{x}\), and the root mean square of \(z - H \hat{x} - c\). With no offset, or a constant one, the squared correlation is that of \(z\) and \(H \hat{x} + c\).
uncertainty reduction	`post.uncertainty_reduction`	\(1 - \sqrt{\operatorname{tr}(\hat{S})} / \sqrt{\operatorname{tr}(S_0)}\).
Desroziers diagnostic	`post.desroziers(by=...)`	The observation error variances the residuals \(z - H \hat{x} - c\) and the innovation diagnose, beside those the solve used.

fips’s own words#

label: What names one element of a vector or one row or column of a matrix: a time, a cell, a site and a time. A label is a row of a pandas index.
block: A named part of a state or of the observations whose elements share their kind of label: the flux cells and the bias sites of a state, the station and satellite blocks of the observations.
space (Space): The labels of a state, of the observations, or of one axis of a matrix: an ordered set of blocks, each with its own pandas index.
vector (Vector): Values on a space: a prior, the observations, an estimate.
operator (Operator): A matrix with a space on each axis: the forward operator, an aggregation, a gain. A covariance (Covariance) is an operator from a space to itself.
covariance model: A covariance written down before its labels exist: a correlation model (Exponential, Kronecker, …) with standard deviations, which a Problem puts on the prior’s or the observations’ space.
aggregation (\(W\)): An operator from a space to groups of its labels (sums or means): hourly observations to daily means, the state to monthly totals (fips.aggregate).
method: How fips.solve() solves a problem. The one method is "analytic", the closed-form Gaussian posterior.