fips.Posterior.information_content#

property Posterior.information_content: float#

The Shannon information content of the observations about the state, in nats.

\[H = -\tfrac{1}{2} \ln \lvert I - A \rvert = \tfrac{1}{2} \ln \frac{\lvert S_0 \rvert}{\lvert \hat{S} \rvert} = \tfrac{1}{2} \sum_k \ln(1 + s_k^2)\]

with \(A\) the averaging kernel and \(s_k^2\) the eigenvalues of \(\gamma R_0^T H^T S_z^{-1} H R_0\) (\(S_0 = R_0 R_0^T\)), the observations’ information on the whitened state; dofs is \(\sum_k s_k^2 / (1 + s_k^2)\). It is how much the observations reduce the entropy of the state, from the prior to the posterior: the log of how many times smaller the volume of uncertainty is. Divide by \(\ln 2\) for bits. A direction the observations constrain counts at most 1 in dofs, but its information keeps growing with the precision of the observations.

The analytic method reads the log-determinant from the factor it solved with (the Cholesky factor at the gamma of fips.solve(), or the eigenvalues after with_gamma()), so it costs nothing more than the solve.

References

Rodgers, C. D. (2000). Inverse Methods for Atmospheric Sounding: Theory and Practice. World Scientific.

See also

dofs: The degrees of freedom for signal.
dofs_per_element: DOFS, element by element.

Examples

>>> import numpy as np
>>> import pandas as pd
>>> import fips
>>> cells = pd.Index(["a", "b"], name="cell")
>>> obs_id = pd.Index(["o1", "o2", "o3"], name="obs_id")
>>> prior = fips.Vector.from_series(pd.Series(0.0, index=cells), block="flux")
>>> obs = fips.Vector.from_series(
...     pd.Series([1.0, 2.0, 3.0], index=obs_id), block="co2"
... )
>>> problem = fips.Problem(
...     prior=prior,
...     prior_cov=fips.Independent().with_std(1.0),
...     obs=obs,
...     obs_cov=fips.Independent().with_std(np.sqrt(0.1)),
...     forward=fips.Operator(
...         np.array([[1.0, 0.0], [0.0, 1.0], [1.0, 1.0]]),
...         obs.space,
...         prior.space,
...     ),
... )
>>> post = fips.solve(problem)
>>> round(post.information_content, 3)
2.916
>>> round(float(post.information_content / np.log(2)), 3)  # in bits
4.207