Solving and Posterior

solve() solves a Problem with a method and returns a Posterior. The problem is the question and holds what needs no solve and no \(m \times m\) matrix (Problem.predict()); the posterior is the answer and holds what depends on the solve or on the \(m \times m\) innovation covariance (Posterior.innovation_cov). Each method declares its options once (for "analytic", AnalyticOptions), and an unknown method or option raises with the names it knows. The posterior labels the answer and the method keeps its own machinery: the posterior asks what the method kept from the solve (a Solution) for unlabeled arrays and labels them with the problem’s spaces. The analytic method keeps the factorization of its whitened system (\(m \times m\)), or of its information system (\(n \times n\)) when there are fewer state elements than observations, the model-data mismatch is dense, diagonal, sparse, block diagonal or Kronecker, and the prior error covariance has a square root (a positive-definite one does). In the whitened form nothing \(n \times n\) is formed unless the full covariance or averaging kernel is asked for. The solve factors by Cholesky at its gamma; the first Posterior.with_gamma() eigendecomposes once, and later ones reuse it. Posterior.with_data() reuses the factorization in either form.

See also

See Terminology for the notation (\(\hat{x}\), \(\hat{S}\), \(K\), \(A\), DOFS).

Solving

solve(problem[, method])	Solve an inverse problem and return its posterior.
available_methods()	Return the names of the methods solve() can use.

AnalyticOptions([gamma])

The options of the "analytic" method.

Constructor

Posterior(problem, solution, options)

The solution of an inverse problem, labeled.

Inputs

Posterior.problem	The problem that was solved.
Posterior.options	The options the method solved with (gamma).

State estimate

Posterior.estimate	The posterior state estimate, \(\hat{x}\), on the prior's space.
Posterior.mean	The posterior mean, estimate itself.
Posterior.variances	The posterior error variances, the diagonal of \(\hat{S}\).
Posterior.cov([W])	Return the posterior error covariance, of the state or of aggregates of it.
Posterior.uncertainty([W])	Return the prior and posterior uncertainty of each aggregate, and how much the observations reduce it.

Observation space

Posterior.modeled_obs	The observations the estimate models, \(H \hat{x} + c\).
Posterior.innovation_cov	The covariance of the innovation, \(H S_0 H^T + S_z / \gamma\), dense.
Posterior.desroziers([by])	Compare the observation error variances Desroziers et al. (2005) diagnose with those the solve used.
Posterior.leverage(x)	Return the leverage matrix of the state x: which observations weigh most on it.

Operators

Posterior.kalman_gain	The gain \(K\), state by observations, dense.
Posterior.averaging_kernel([W])	Return the averaging kernel, of the state or of aggregates of it.

Diagnostics

Posterior.dofs	Degrees of freedom for signal, the trace of averaging_kernel().
Posterior.dofs_per_element	the diagonal of averaging_kernel(), on the prior's space.
Posterior.information_content	The Shannon information content of the observations about the state, in nats.
Posterior.chi2	the sum of the data and the state misfits.
Posterior.reduced_chi2	chi2 divided by the number of observations; near 1 when the covariances are right.
Posterior.chi2_tests()	Test whether the innovation, the residual and the increment are as large as the covariances say: Rodgers' four chi-squared tests.
Posterior.r2	The R² of the enhancement, \(\operatorname{corr}(z - c, H\hat{x})^2\).
Posterior.rmse	The root mean square of the observations minus modeled_obs.
Posterior.uncertainty_reduction	One minus the ratio of the posterior to the prior total error.
Posterior.summary()	Return the diagnostics of the solve as a table of one row.

Reusing the factorization

Posterior.with_gamma(gamma)	Return the posterior of the same problem with another regularization factor.
Posterior.with_data(obs)	Return the posterior of the same problem with other observations.

Recovering a known truth

The exact mean and spread of the estimate’s error when the data come from a world (the problem itself, a fixed truth, or another Problem on the same labels): the answer of a Monte Carlo loop over Problem.simulate() and Posterior.with_data(), without sampling.

Posterior.recovery([world, truth, W, level])

Return how well the estimate recovers the truth when the data come from world: exactly, without sampling.

What groups of observations add

For each group of observations (a site, a platform, a month), in the problem or a candidate not yet in it, the posterior uncertainty of aggregates of the state without the group and with it, and the group’s share of DOFS: a rank update of the posterior covariance per group, without solving again and without an \(n \times n\) matrix.

Posterior.data_worth(by, W)	Return what each group of observations adds to the certainty of each aggregate, and its share of DOFS.
Posterior.candidate_worth(forward, obs_cov, ...)	Return what each group of candidate observations, not in the problem, would add to the certainty of each aggregate.

Serialization

A posterior pickles with the standard library (pickle.dump(post, file)): it keeps the problem, the options and the method’s solution (for the analytic method, the factorization), and rebuilds every output when it is read.

Writing a method

What a Posterior asks of a method: its solution answers the questions of Solution, and its options have the regularization factor (Options). How a method is added is in the docstring of fips.methods.

Solution(*args, **kwargs)	What a Posterior asks of the object a method keeps from its solve.
Solution.estimate()	Return the point estimate of the state, \(\hat{x}\) (n).
Solution.variances()	Return the posterior error variances, the diagonal of \(\hat{S}\) (n).
Solution.cov([W])	Return the posterior error covariance of the state, or of aggregates of it.
Solution.gain([W])	Return the gain \(K\), or \(W K\) for an aggregation.
Solution.dofs()	Return the degrees of freedom for signal, \(\operatorname{tr}(K H)\).
Solution.dofs_per_element()	Return the diagonal of the averaging kernel, \(\operatorname{diag}(K H)\) (n), whose sum is dofs().
Solution.information()	Return the Shannon information content, \(-\tfrac{1}{2} \ln \lvert I - K H \rvert\), in nats.
Solution.chi2()	Return the chi-squared of the solution: the sum of the data and the state misfits.
Solution.chi2_tests()	Return Rodgers' four chi-squared tests, each its chi-squared and degrees of freedom.
Options(*args, **kwargs)	What a Posterior reads from the options of a method.
Options.gamma	the solve divided the model-data mismatch by it.