fips.Posterior.candidate_worth#

Posterior.candidate_worth(forward, obs_cov, by, W)[source]#

Return what each group of candidate observations, not in the problem, would add to the certainty of each aggregate.

Candidate observations are rows of a forward operator that the problem does not have (a new site, another platform, a season not yet measured) and the covariance of their errors; their values are not needed. For each group of candidates, the posterior standard deviation of each aggregate without the group (this posterior) and with it, from a rank \(m_c\) update of \(\hat{S}\), without solving again:

\[\hat{S}_{+c} = \hat{S} - \hat{S} H_c^T F^{-1} H_c \hat{S}, \qquad F = H_c \hat{S} H_c^T + R_c / \gamma,\]

with \(H_c\) the group’s rows of forward and \(R_c\) its block of obs_cov. The errors of the candidates are taken to be independent of the problem’s observations, and are divided by the regularization factor \(\gamma\) as the problem’s are, so the result is the posterior of the problem solved again with the group’s observations added. For \(W\), the three blocks \(W \hat{S} W^T\), \(W \hat{S} H_c^T\) and \(H_c \hat{S} H_c^T\) are one cov() of \(W\) and \(H_c\) stacked: nothing n x n is formed. Each group is added on its own, to the problem as it is.

Parameters:

  • forward (Operator) – The candidates’ forward operator, from the state (its columns, the state’s labels in any order) to the candidate observations (its rows, labeled as observations are, on the levels by groups).

  • obs_cov (Covariance | CovarianceModel) – The covariance of the candidates’ errors, on the rows of forward; a model (fips.Independent().with_std(5.0)) is built there. It must be positive-definite.

  • by (Any) – The levels of the candidates’ labels to group by, as data_worth() takes them ("site").

  • W (Operator) – An aggregation, as cov() takes it: one row per aggregate.

Returns:

As data_worth(): one row per group and aggregate, with n_obs, dofs (the group’s share of DOFS in the problem with it added, \(m_c - \operatorname{tr}(R_c F^{-1}) / \gamma\)), std_without (this posterior’s), std_with (with the group added) and reduction, \(1 - \sigma_{with} / \sigma_{without}\). A variance below zero by more than roundoff, which only a covariance that is not positive semidefinite can give, makes its standard deviation and the reduction NaN, with a logged warning (as in uncertainty()).

Return type:

DataFrame

Raises:

  • TypeError – If the method’s estimate is not linear in the observations (its solution does not set estimate_is_linear), forward or W is not an Operator, or obs_cov is not a covariance.

  • ValueError – If the columns of forward or W do not have exactly the state’s labels, obs_cov is not on the rows of forward, or by puts no candidate in a group.

See also

data_worth

What the groups of observations in the problem add.

uncertainty

The prior and posterior uncertainty of each aggregate.

Notes

A candidate group is always added on top of the problem’s observations, so there are no std_alone and reduction_alone columns, which data_worth() has for the groups in the problem.

Examples

One flux observed four times at site A; would a candidate site B, three observations with errors of the same size, help more than one more observation at A?

>>> import numpy as np
>>> import pandas as pd
>>> import fips
>>> state = fips.Space({"flux": pd.Index(["a"], name="cell")})
>>> observed = fips.Space(
...     {
...         "co2": pd.MultiIndex.from_product(
...             [["A"], range(4)], names=["site", "hour"]
...         )
...     }
... )
>>> problem = fips.Problem(
...     prior=fips.Vector(np.zeros(1), state),
...     prior_cov=fips.Covariance.diagonal(1.0, state),
...     obs=fips.Vector(np.ones(4), observed),
...     obs_cov=fips.Covariance.diagonal(1.0, observed),
...     forward=fips.Operator(np.ones((4, 1)), observed, state),
... )
>>> post = fips.solve(problem)
>>> candidates = fips.Space(
...     {
...         "co2": pd.MultiIndex.from_tuples(
...             [("A", 4), ("B", 0), ("B", 1), ("B", 2)], names=["site", "hour"]
...         )
...     }
... )
>>> H_new = fips.Operator(np.ones((4, 1)), candidates, state)
>>> W = fips.Operator(np.eye(1), state, state)
>>> table = post.candidate_worth(
...     H_new, fips.Independent().with_std(1.0), by="site", W=W
... )
>>> table.droplevel("block").round(3).reset_index()
  site cell  n_obs   dofs  std_without  std_with  reduction
0    A    a      1  0.167        0.447     0.408      0.087
1    B    a      3  0.375        0.447     0.354      0.209