Description of all functions and classes

Description of all functions and classes#

Top-Level#

`version`()	Version of `spey` package
`about`()	Prints the information regarding spey installation
`check_updates`()	Check Spey Updates.
`ExpectationType`(value)	Expectation type has been used to determine the nature of the statistical model through out the package.
`AvailableBackends`()	Returns a list of available backends.
`get_backend`(name)	Statistical model backend retreiver.
`get_backend_metadata`(name)	Retreive metadata about the backend.
`get_backend_bibtex`(name)	Retreive BibTex entry for backend plug-in if available.
`cite`()	Retreive BibTex information for Spey
`reset_backend_entries`()	Scan the system for backends and reset the entries
`statistical_model_wrapper`(func)	Backend wrapper for `StatisticalModel`.
`helper_functions.correlation_to_covariance`(...)	Convert correlation matrix into covariance matrix
`helper_functions.covariance_to_correlation`(...)	Convert covariance matrix into correlation matrix.
`optimizer.core.fit`(func, model_configuration)	Operating the fitting of the given function.
`set_log_level`(level)	Set log level for spey

Main Classes#

class spey.StatisticalModel(backend: BackendBase, analysis: str, xsection: float = nan, ntoys: int = 1000)[source]#

Bases: HypothesisTestingBase

Statistical model base. This class wraps around the various statistical model backends available through spey’s plugin system. Each backend has to inherit BackendBase which sets certain requirements on the available functionality to be used for hypothesis testing. These requirements are such as accessibility to log-likelihood, \(\log\mathcal{L}\), it’s derivative with respect to \(\mu\) and nuisance parameters, \(\partial_\theta\log\mathcal{L}\), its Hessian and Assimov data generation. Depending on availablility StatisticalModel will take propriate action to perform requested computation. The goal of this class is to collect all different backends under same roof in order to perform combination of different likelihood recipies.

Parameters:

backend (BackendBase) – Statistical model backend
analysis (Text) – Unique identifier of the statistical model. This attribue will be used for book keeping purposes.
xsection (float, default np.nan) – cross section, unit is determined by the user. Cross section value is only used for computing upper limit on excluded cross-section value.
ntoys (int, default 1000) – Number of toy samples for hypothesis testing. (Only used for toy-based hypothesis testing)

Raises:

AssertionError – If the given backend does not inherit BackendBase

Returns:

General statistical model object that wraps around different likelihood prescriptions.

Return type:

StatisticalModel

analysis: str#: Unique identifier as analysis name

asimov_likelihood(poi_test: float = 1.0, expected: ExpectationType = ExpectationType.observed, return_nll: bool = True, test_statistics: str = 'qtilde', init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → float[source]#

Compute likelihood of the statistical model generated with the Asimov data.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
test_statistics (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
The choice of test_statistics will effect the generation of the Asimov data where the fit is performed via \(\mu=1\) if test_statistics="q0" and \(\mu=0\) for others. Note that this \(\mu\) does not correspond to the poi_test input of this function but it determines how Asimov data is generated.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

likelihood computed for asimov data

Return type:

float

property available_calculators: List[str]#: Retruns available calculator names i.e. 'toy', 'asymptotic' and 'chi_square'.

property backend: BackendBase#: Accessor to the backend

property backend_type: str#: Return type of the backend

chi2(poi_test: float = 1.0, poi_test_denominator: float | None = None, expected: ExpectationType = ExpectationType.observed, allow_negative_signal: bool = False, **kwargs) → float#

If poi_test_denominator=None computes

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\hat\mu,\hat\theta)}\right)\]

else

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\mu_{\rm denom},\theta_{\mu_{\rm denom}})}\right)\]

where \(\mu_{\rm denom}\) is poi_test_denominator which is typically zero to compare signal model with the background only model.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
poi_test_denominator (float, default None) – parameter of interest for the denominator, \(\mu\). If None maximum likelihood will be computed.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative. Only valid when poi_test_denominator=None.
kwargs – keyword arguments for the optimiser.

Returns:

value of the \(\chi^2\).

Return type:

float

combine(other, **kwargs)[source]#

Combination routine between two statistical models.

Note

This function’s availability is backend dependent.

Parameters:

other (StatisticalModel) – Statistical model to be combined with this model
kwargs – backend specific arguments.

Raises:

CombinerNotAvailable – If this statistical model does not have a combination routine implementation.
AssertionError – If the combination routine in the backend does not return a BackendBase object.

Returns:

Returns a new combined statistical model.

Return type:

StatisticalModel

excluded_cross_section(expected: ExpectationType = ExpectationType.observed) → float[source]#

Compute excluded cross section value at 95% CL

Parameters:

expected (ExpectationType) –

Sets which values the fitting algorithm should focus and p-values to be computed.

observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.

Raises:

UnknownCrossSection – If the cross-section is nan.

Returns:

Returns the upper limit at 95% CL on cross section value where the unit is defined by the user.

Return type:

float

exclusion_confidence_level(poi_test: float = 1.0, expected: ExpectationType = ExpectationType.observed, allow_negative_signal: bool = False, calculator: Literal['asymptotic', 'toy', 'chi_square'] = 'asymptotic', poi_test_denominator: float | None = None, **kwargs) → List[float]#

Compute exclusion confidence level (\(1-CL_s\)) at a given POI, \(\mu\).

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed. If expected is set to "all" it will return both p-values and expected p-values where the likelihood will be fitted to observations.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
  
  Note
  
  In case of observed, function will return one value which has been fit to the observed data.
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
  
  Note
  
  In case of aposteriori, function will return five value for expected p-values which has been fit to the observed data. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The order of the output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  In case of apriori, function will return five value for expected p-values which has been fit to the SM background. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
allow_negative_signal (bool, default False) – If True \(\hat\mu\) value will be allowed to be negative.
calculator (Literal["asymptotic", "toy", "chi_square"], default "asymptotic") –
Chooses the computation basis for hypothesis testing
- "asymptotic": Uses asymptotic hypothesis testing to compute p-values.
- "toy": Uses generated toy samples to compute p-values.
- "chi_square": Computes p-values via chi-square; \(\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}\).
poi_test_denominator (float, default None) – Set the POI value for the null hypothesis. if None, signal hypothesis will be compared against maximum likelihood otherwise with respect to the hypothesis determined with the POI value provided with this input. Only used when calculator="chi-square".
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Raises:

CalculatorNotAvailable – If calculator is not available.

Returns:

Exclusion confidence level i.e. \(1-CL_s\).

Return type:

List[float]

fixed_poi_sampler(poi_test: float, size: int | None = None, expected: ExpectationType = ExpectationType.observed, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → ndarray | Callable[[int], ndarray][source]#

Sample data from the statistical model with fixed parameter of interest.

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
size (int, default None) – sample size. If None a callable function will be returned which takes sample size as input.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have sampler implementation.

Returns:

Sampled data with shape of (size, number of bins) or callable function to sample from directly.

Return type:

Union[np.ndarray, Callable[[int], np.ndarray]]

generate_asimov_data(expected: ExpectationType = ExpectationType.observed, test_statistic: str = 'qtilde', init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → List[float][source]#

Generate Asimov data for the statistical model. This function generates a set of parameters (nuisance and poi i.e. \(\theta\) and \(\mu\)) with respect to test_statistic input which determines the value of \(\mu\) i.e. if test_statistic="q0" \(\mu=1\) and 0 for anything else. The objective function is used to optimize the statistical model to find the fit parameters for fixed poi optimisation. Then fit parameters are used to retreive the expected data through expected_data() function.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistic (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

Asimov data

Return type:

List[float]

property is_alive: bool#: Returns True if at least one bin has non-zero signal yield.

property is_asymptotic_calculator_available: bool#: Check if Asymptotic calculator is available for the backend

property is_chi_square_calculator_available: bool#: Check if chi-square calculator is available for the backend

property is_toy_calculator_available: bool#: Check if Toy calculator is available for the backend

likelihood(poi_test: float = 1.0, expected: ExpectationType = ExpectationType.observed, return_nll: bool = True, data: List[float] | ndarray | None = None, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → float[source]#

Compute the likelihood of the statistical model at a fixed parameter of interest

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

Likelihood of the statistical model at a fixed signal strength.

Return type:

float

maximize_asimov_likelihood(return_nll: bool = True, expected: ExpectationType = ExpectationType.observed, test_statistics: str = 'qtilde', init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → Tuple[float, float][source]#

Find the maximum of the likelihood which computed with respect to Asimov data.

Parameters:

return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default "qtilde") –
test statistic.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

Find the maximum of the likelihood.

Parameters:

return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

ntoys#: Number of toy samples for sample generator during hypothesis testing

poi_upper_limit(expected: ExpectationType = ExpectationType.observed, confidence_level: float = 0.95, low_init: float | None = 1.0, hig_init: float | None = 1.0, expected_pvalue: Literal['nominal', '1sigma', '2sigma'] = 'nominal', maxiter: int = 10000, optimiser_arguments: Dict[str, Any] | None = None) → float | List[float]#

Compute the upper limit for the parameter of interest i.e. \(\mu\).

Note

This function uses "qtilde" test statistic which means signal values are always assumed to be positive i.e. \(\hat\mu>0\).

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
confidence_level (float, default 0.95) – Determines the confidence level of the upper limit i.e. the value of \(1-CL_s\). It needs to be between [0,1].
low_init (Optional[float], default 1.0) –
Lower limit for the search algorithm to start If None it the lower limit will be determined by \(\hat\mu + 1.5\sigma_{\hat\mu}\).

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
hig_init (Optional[float], default 1.0) –
Upper limit for the search algorithm to start If None it the upper limit will be determined by \(\hat\mu + 2.5\sigma_{\hat\mu}\).

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
expected_pvalue (Literal["nominal", "1sigma", "2sigma"], default "nominal") –
In case of aposteriori and apriori expectation, gives the choice to find excluded upper limit for statistical deviations as well.
- "nominal": only find the upper limit for the central p-value. Returns a single value.
- "1sigma": find the upper limit for central p-value and \(1\sigma\) fluctuation from background. Returns 3 values.
- "2sigma": find the upper limit for central p-value and \(1\sigma\) and \(2\sigma\) fluctuation from background. Returns 5 values.
Note

For expected=spey.ExpectationType.observed, expected_pvalue argument will be overwritten to "nominal".
maxiter (int, default 10000) – Maximum iteration limit for the optimiser.
optimiser_arguments (Dict, default None) – Arguments for optimiser that is used to compute likelihood and its maximum.

Returns:

In case of nominal values it returns a single value for the upper limit. In case of expected_pvalue="1sigma" or expected_pvalue="2sigma" it will return a list of multiple upper limit values for fluctuations as well as the central value. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.

Return type:

Union[float, List[float]]

prepare_for_fit(data: List[float] | ndarray | None = None, expected: ExpectationType = ExpectationType.observed, allow_negative_signal: bool | None = True, **kwargs) → Dict[source]#

Prepare backend for the optimiser.

Parameters:

data (Union[List[float], np.ndarray], default None) – input data that to fit
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.

Returns:

Dictionary of necessary toolset for the fit. objective function, "func", use gradient boolean, "do_grad" and function to compute negative log-likelihood with given fit parameters, "nll".

Return type:

Dict

property s95exp: float#

Compute excluded cross section value at 95% CL with apriori expectation. See excluded_cross_section() for reference.

Raises:: UnknownCrossSection – If the cross-section is nan.

property s95obs: float#

Compute excluded cross section value at 95% CL with observed expectation. See excluded_cross_section() for reference.

Raises:: UnknownCrossSection – If the cross-section is nan.

sigma_mu(poi_test: float, expected: ExpectationType = ExpectationType.observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float#

If available, \(\sigma_\mu\) will be computed through Hessian of negative log-likelihood see spey.StatisticalModel.sigma_mu_from_hessian() for details. However, if not available it will be estimated via \(q_{\mu,A}\)

\[\sigma^2_A = \frac{(\mu - \mu^\prime)^2}{q_{\mu,A}}\quad , \quad q_{\mu,A} = -2\ln\lambda_A(\mu)\]

see eq. (31) in [arXiv:1007.1727]

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

value of the variance on \(\mu\).

Return type:

float

sigma_mu_from_hessian(poi_test: float, expected: ExpectationType = ExpectationType.observed, init_pars: List[float] | None = None, par_bounds: List[Tuple[float, float]] | None = None, **kwargs) → float[source]#

Compute variance of \(\mu\) from inverse Hessian. See eq. (27-28) in [arXiv:1007.1727].

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have Hessian implementation.

Returns:

variance on parameter of interest.

Return type:

float

significance(expected: ExpectationType = ExpectationType.observed, **kwargs) → Tuple[float, float, List[float], List[float]]#

Compute the discovery of a positive signal. See [arXiv:1007.1727] eq. (53). and sec. 5.1.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  Since aposteriori and observed are both represent post-fit computation the result will be the same. The only difference can be seen via prefit, apriori, computation.
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

(\(\sqrt{q_{0,A}}\), \(\sqrt{q_0}\), p-values and expected p-values)

Return type:

Tuple[float, float, List[float], List[float]]

xsection: float#: Value of the cross section, unit is defined by the user.

Base Classes#

class spey.BackendBase[source]#

An abstract class construction to enforce certain behaviour on statistical model backend. In order to perform certain computations, spey needs to have access to specific function constructions such as precsription to form likelihood. Hence, each backend is required to inherit BackendBase.

asimov_negative_loglikelihood(poi_test: float = 1.0, expected: ExpectationType = ExpectationType.observed, test_statistics: str = 'qtilde', **kwargs) → Tuple[float, ndarray][source]#

Compute negative log-likelihood at fixed \(\mu\) for Asimov data.

Note

Interface first calls backend specific methods to compute likelihood. If they are not implemented, it optimizes objective function through spey interface. Either prescription to optimizing the likelihood or objective function must be available for a backend to be sucessfully integrated to the spey interface.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood at POI of interest and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

combine(other, **kwargs)[source]#

A routine to combine to statistical models.

Note

This function is only available if the backend has a specific routine for combination between same or other backends.

Parameters:: other (BackendBase) – Statistical model object to be combined.
Raises:: NotImplementedError – If the backend does not have a combination scheme.
Returns:: Create a new statistical model from combination of this and other one.
Return type:: BackendBase

abstract config(allow_negative_signal: bool = True, poi_upper_bound: float = 10.0) → ModelConfig[source]#

Model configuration.

Parameters:

allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
poi_upper_bound (float, default 10.0) – upper bound for parameter of interest, \(\mu\).

Returns:

Model configuration. Information regarding the position of POI in parameter list, suggested input and bounds.

Return type:

ModelConfig

expected_data(pars: List[float]) → List[float][source]#

Compute the expected value of the statistical model. This function is mainly used to generate Asimov data within the package, see generate_asimov_data().

Parameters:: pars (List[float]) – nuisance, \(\theta\) and parameter of interest, \(\mu\).
Returns:: Expected data of the statistical model
Return type:: List[float]

get_hessian_logpdf_func(expected: ExpectationType = ExpectationType.observed, data: List[float] | ndarray | None = None) → Callable[[ndarray], float][source]#

Currently Hessian of \(\log\mathcal{L}(\mu, \theta)\) is only used to compute variance on \(\mu\). This method returns a callable function which takes fit parameters (\(\mu\) and \(\theta\)) and returns Hessian.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
data (Union[List[float], np.ndarray], default None) – input data that to fit

Raises:

NotImplementedError – If the Hessian of the backend has not been implemented.

Returns:

Function that takes fit parameters (\(\mu\) and \(\theta\)) and returns Hessian of \(\log\mathcal{L}(\mu, \theta)\).

Return type:

Callable[[np.ndarray], float]

abstract get_logpdf_func(expected: ExpectationType = ExpectationType.observed, data: List[float] | ndarray | None = None) → Callable[[ndarray], float][source]#

Generate function to compute \(\log\mathcal{L}(\mu, \theta)\) where \(\mu\) is the parameter of interest and \(\theta\) are nuisance parameters.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
data (Union[List[float], np.ndarray], default None) – input data that to fit

Returns:

Function that takes fit parameters (\(\mu\) and \(\theta\)) and computes \(\log\mathcal{L}(\mu, \theta)\).

Return type:

Callable[[np.ndarray], float]

get_objective_function(expected: ExpectationType = ExpectationType.observed, data: List[float] | ndarray | None = None, do_grad: bool = True) → Callable[[ndarray], float | Tuple[float, ndarray]][source]#

Objective function is the function to perform the optimisation on. This function is expected to be negative log-likelihood, \(-\log\mathcal{L}(\mu, \theta)\). Additionally, if available it canbe bundled with the gradient of negative log-likelihood.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
data (Union[List[float], np.ndarray], default None) – input data that to fit
do_grad (bool, default True) – If True return objective and its gradient as tuple (subject to availablility) if False only returns objective function.

Returns:

Function which takes fit parameters (\(\mu\) and \(\theta\)) and returns either objective or objective and its gradient.

Return type:

Callable[[np.ndarray], Union[float, Tuple[float, np.ndarray]]]

get_sampler(pars: ndarray) → Callable[[int], ndarray][source]#

Retreives the function to sample from.

Parameters:: pars (np.ndarray) – fit parameters (\(\mu\) and \(\theta\))
Raises:: NotImplementedError – If the sampler for the backend has not been implemented.
Returns:: Function that takes number_of_samples as input and draws as many samples from the statistical model.
Return type:: Callable[[int], np.ndarray]

property is_alive: bool#: Returns True if at least one bin has non-zero signal yield.

minimize_asimov_negative_loglikelihood(expected: ExpectationType = ExpectationType.observed, test_statistics: str = 'qtilde', **kwargs) → Tuple[float, ndarray][source]#

A backend specific method to minimize negative log-likelihood for Asimov data.

Note

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

minimize_negative_loglikelihood(expected: ExpectationType = ExpectationType.observed, allow_negative_signal: bool = True, **kwargs) → Tuple[float, ndarray][source]#

A backend specific method to minimize negative log-likelihood.

Note

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

negative_loglikelihood(poi_test: float = 1.0, expected: ExpectationType = ExpectationType.observed, **kwargs) → Tuple[float, ndarray][source]#

Backend specific method to compute negative log-likelihood for a parameter of interest \(\mu\).

Note

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
kwargs – keyword arguments for the optimiser.

Raises:

NotImplementedError – If the method is not available for the backend.

Returns:

value of negative log-likelihood at POI of interest and fit parameters (\(\mu\) and \(\theta\)).

Return type:

Tuple[float, np.ndarray]

class spey.base.hypotest_base.HypothesisTestingBase(ntoys: int = 1000)[source]#

Abstract class that ensures classes that are performing hypothesis teststing includes certain set of function to perform necessary computations. This class gives the ability to compute exclusion limits and upper limits for the class inherits it.

Parameters:: ntoys (int, default 1000) – Number of toy samples for hypothesis testing. (Only used for toy-based hypothesis testing)

abstract asimov_likelihood(poi_test: float = 1.0, expected: ExpectationType = ExpectationType.observed, return_nll: bool = True, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float[source]#

Compute likelihood at fixed \(\mu\) for Asimov data

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs – keyword arguments for the optimiser.

Returns:

value of the likelihood.

Return type:

float

chi2(poi_test: float = 1.0, poi_test_denominator: float | None = None, expected: ExpectationType = ExpectationType.observed, allow_negative_signal: bool = False, **kwargs) → float[source]#

If poi_test_denominator=None computes

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\hat\mu,\hat\theta)}\right)\]

else

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\mu_{\rm denom},\theta_{\mu_{\rm denom}})}\right)\]

where \(\mu_{\rm denom}\) is poi_test_denominator which is typically zero to compare signal model with the background only model.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
poi_test_denominator (float, default None) – parameter of interest for the denominator, \(\mu\). If None maximum likelihood will be computed.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative. Only valid when poi_test_denominator=None.
kwargs – keyword arguments for the optimiser.

Returns:

value of the \(\chi^2\).

Return type:

float

Compute exclusion confidence level (\(1-CL_s\)) at a given POI, \(\mu\).

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed. If expected is set to "all" it will return both p-values and expected p-values where the likelihood will be fitted to observations.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
  
  Note
  
  In case of observed, function will return one value which has been fit to the observed data.
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
  
  Note
  
  In case of aposteriori, function will return five value for expected p-values which has been fit to the observed data. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The order of the output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  In case of apriori, function will return five value for expected p-values which has been fit to the SM background. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
allow_negative_signal (bool, default False) – If True \(\hat\mu\) value will be allowed to be negative.
calculator (Literal["asymptotic", "toy", "chi_square"], default "asymptotic") –
Chooses the computation basis for hypothesis testing
- "asymptotic": Uses asymptotic hypothesis testing to compute p-values.
- "toy": Uses generated toy samples to compute p-values.
- "chi_square": Computes p-values via chi-square; \(\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}\).
poi_test_denominator (float, default None) – Set the POI value for the null hypothesis. if None, signal hypothesis will be compared against maximum likelihood otherwise with respect to the hypothesis determined with the POI value provided with this input. Only used when calculator="chi-square".
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Raises:

CalculatorNotAvailable – If calculator is not available.

Returns:

Exclusion confidence level i.e. \(1-CL_s\).

Return type:

List[float]

Sample data from the statistical model with fixed parameter of interest.

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
size (int, default None) – sample size. If None a callable function will be returned which takes sample size as input.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have sampler implementation.

Returns:

Sampled data with shape of (size, number of bins) or callable function to sample from directly.

Return type:

Union[np.ndarray, Callable[[int], np.ndarray]]

abstract property is_alive: bool#: Returns True if at least one bin has non-zero signal yield.

abstract property is_asymptotic_calculator_available: bool#: Check if Asymptotic calculator is available for the backend

abstract property is_chi_square_calculator_available: bool#: Check if chi-square calculator is available for the backend

abstract property is_toy_calculator_available: bool#: Check if Toy calculator is available for the backend

abstract likelihood(poi_test: float = 1.0, expected: ExpectationType = ExpectationType.observed, return_nll: bool = True, data: List[float] | ndarray | None = None, **kwargs) → float[source]#

Compute likelihood of the statistical model

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.
data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.
kwargs – keyword arguments for the optimiser.

Returns:

value of likelihood at fixed \(\mu\).

Return type:

float

abstract maximize_asimov_likelihood(return_nll: bool = True, expected: ExpectationType = ExpectationType.observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → Tuple[float, float][source]#

Compute maximum of the likelihood for Asimov data.

Parameters:

return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs – keyword arguments for the optimiser.

Returns:

value of \(\hat\mu\) and maximum likelihood.

Return type:

Tuple[float, float]

abstract maximize_likelihood(return_nll: bool = True, expected: ExpectationType = ExpectationType.observed, allow_negative_signal: bool = True, data: List[float] | ndarray | None = None, **kwargs) → Tuple[float, float][source]#

Compute maximum of the likelihood.

Parameters:

return_nll (bool, default True) – If True returns negative log-likelihood, else likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
data (Union[List[float], np.ndarray], default None) – input data that to fit. If None data will be set according to expected input.
kwargs – keyword arguments for the optimiser.

Returns:

value of \(\hat\mu\) and maximum likelihood.

Return type:

Tuple[float, float]

ntoys#: Number of toy samples for sample generator during hypothesis testing

Compute the upper limit for the parameter of interest i.e. \(\mu\).

Note

This function uses "qtilde" test statistic which means signal values are always assumed to be positive i.e. \(\hat\mu>0\).

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
confidence_level (float, default 0.95) – Determines the confidence level of the upper limit i.e. the value of \(1-CL_s\). It needs to be between [0,1].
low_init (Optional[float], default 1.0) –
Lower limit for the search algorithm to start If None it the lower limit will be determined by \(\hat\mu + 1.5\sigma_{\hat\mu}\).

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
hig_init (Optional[float], default 1.0) –
Upper limit for the search algorithm to start If None it the upper limit will be determined by \(\hat\mu + 2.5\sigma_{\hat\mu}\).

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
expected_pvalue (Literal["nominal", "1sigma", "2sigma"], default "nominal") –
In case of aposteriori and apriori expectation, gives the choice to find excluded upper limit for statistical deviations as well.
- "nominal": only find the upper limit for the central p-value. Returns a single value.
- "1sigma": find the upper limit for central p-value and \(1\sigma\) fluctuation from background. Returns 3 values.
- "2sigma": find the upper limit for central p-value and \(1\sigma\) and \(2\sigma\) fluctuation from background. Returns 5 values.
Note

For expected=spey.ExpectationType.observed, expected_pvalue argument will be overwritten to "nominal".
maxiter (int, default 10000) – Maximum iteration limit for the optimiser.
optimiser_arguments (Dict, default None) – Arguments for optimiser that is used to compute likelihood and its maximum.

Returns:

Return type:

Union[float, List[float]]

sigma_mu(poi_test: float, expected: ExpectationType = ExpectationType.observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float[source]#

\[\sigma^2_A = \frac{(\mu - \mu^\prime)^2}{q_{\mu,A}}\quad , \quad q_{\mu,A} = -2\ln\lambda_A(\mu)\]

see eq. (31) in [arXiv:1007.1727]

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

value of the variance on \(\mu\).

Return type:

float

significance(expected: ExpectationType = ExpectationType.observed, **kwargs) → Tuple[float, float, List[float], List[float]][source]#

Compute the discovery of a positive signal. See [arXiv:1007.1727] eq. (53). and sec. 5.1.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  Since aposteriori and observed are both represent post-fit computation the result will be the same. The only difference can be seen via prefit, apriori, computation.
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

(\(\sqrt{q_{0,A}}\), \(\sqrt{q_0}\), p-values and expected p-values)

Return type:

Tuple[float, float, List[float], List[float]]

class spey.base.model_config.ModelConfig(poi_index: int, minimum_poi: float, suggested_init: List[float], suggested_bounds: List[Tuple[float, float]], parameter_names: List[str] | None = None, suggested_fixed: List[bool] | None = None)[source]#

Container to hold certain properties of the backend and statistical model. This will ensure the consistency of the computation through out the package.

Parameters:

poi_index (int) – index of the parameter of interest within the parameter list.
minimum_poi (float) –
minimum value parameter of interest can take to ensure \(N^{\rm bkg}+\mu N^{\rm sig}\geq0\). This value can be set to \(-\infty\) but note that optimiser will take it as a lower bound so such low value might effect the convergence of the optimisation algorithm especially for relatively flat objective surfaceses. Hence we suggest setting the minimum value to

\[\mu_{\rm min} = - \min\left( \frac{N^{\rm bkg}_i}{N^{\rm sig}_i} \right)\ ,\ i\in {\rm bins}\]
suggested_init (List[float]) – suggested initial parameters for the optimiser.
suggested_bounds (List[Tuple[float, float]]) – suggested parameter bounds for the optimiser.

Returns:

obj:~spey.base.model_config.ModelConfig: Model configuration container for optimiser.

fixed_poi_bounds(poi_value: float | None = None) → List[Tuple[float, float]][source]#

Adjust the bounds for the parameter of interest for fixed POI fit.

Parameters:: poi_value (Optional[float], default None) – parameter of interest, \(\mu\).
Returns:: Updated bounds.
Return type:: List[Tuple[float, float]]

minimum_poi: float#: minimum value parameter of interest can take to ensure \(N^{\rm bkg}+\mu N^{\rm sig}\geq0\). This value can be set to \(-\infty\) but note that optimiser will take it as a lower bound so such low value might effect the convergence of the optimisation algorithm especially for relatively flat objective surfaceses. Hence we suggest setting the minimum value to

\[\mu_{\rm min} = - \min\left( \frac{N^{\rm bkg}_i}{N^{\rm sig}_i} \right)\ ,\ i\in {\rm bins}\]

property npar: int#: Number of parameters

parameter_names: List[str] | None = None#: Names of the parameters

poi_index: int#: Index of the parameter of interest wihin the parameter list

rescale_poi_bounds(allow_negative_signal: bool = True, poi_upper_bound: float | None = None) → List[Tuple[float, float]][source]#

Rescale bounds for POI.

Parameters:

allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
poi_upper_bound (float, default None) – Maximum value POI can take during optimisation.

Returns:

Updated bounds.

Return type:

List[Tuple[float, float]]

suggested_bounds: List[Tuple[float, float]]#: Suggested upper and lower bounds for parameters

suggested_fixed: List[bool] | None = None#: Suggested fixed values

suggested_init: List[float]#: Suggested initialisation for parameters

Statistical Model Combiner#

class spey.UnCorrStatisticsCombiner(*args, ntoys: int = 1000)[source]#

Bases: HypothesisTestingBase

Module to combine uncorrelated statistical models. It takes serries of StatisticalModel object as input. These statistical models does not need to have same backend. However, this class assumes that all the input statistical model’s are completely independent from each other.

Warning

UnCorrStatisticsCombiner assumes that all input are uncorrelated and non of the statistical models posesses the same set of nuisance parameters. Thus each statistical model is optimised independently.

Parameters:

arguments (input) – Uncorrelated statistical models
ntoys (int, default 1000) – Number of toy samples for hypothesis testing. (Only used for toy-based hypothesis testing)

Raises:

AnalysisQueryError – If multiple StatisticalModel has the same analysis attribute.
TypeError – If the input type is not StatisticalModel.

property analyses: List[str]#

List of the unique identifiers of the statistical models within the stack.

Returns:: List of analysis names.
Return type:: List[Text]

append(statistical_model: StatisticalModel) → None[source]#

Append new independent StatisticalModel to the stack.

Parameters:

statistical_model (StatisticalModel) – new statistical model to be added to the stack.

Raises:

AnalysisQueryError – If multiple StatisticalModel has the same analysis attribute.
TypeError – If the input type is not StatisticalModel.

asimov_likelihood(poi_test: float = 1.0, expected: ExpectationType = ExpectationType.observed, return_nll: bool = True, test_statistics: str = 'qtilde', statistical_model_options: Dict[str, Dict] | None = None, **kwargs) → float[source]#

Compute likelihood of the statistical model stack generated with the Asimov data.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
test_statistics (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
statistical_model_options (Dict[Text, Dict], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default_pdf.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

likelihood computed for asimov data

Return type:

float

chi2(poi_test: float = 1.0, poi_test_denominator: float | None = None, expected: ExpectationType = ExpectationType.observed, allow_negative_signal: bool = False, **kwargs) → float#

If poi_test_denominator=None computes

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\hat\mu,\hat\theta)}\right)\]

else

\[\chi^2 = -2\log\left(\frac{\mathcal{L}(\mu,\theta_\mu)}{\mathcal{L}(\mu_{\rm denom},\theta_{\mu_{\rm denom}})}\right)\]

where \(\mu_{\rm denom}\) is poi_test_denominator which is typically zero to compare signal model with the background only model.

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
poi_test_denominator (float, default None) – parameter of interest for the denominator, \(\mu\). If None maximum likelihood will be computed.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative. Only valid when poi_test_denominator=None.
kwargs – keyword arguments for the optimiser.

Returns:

value of the \(\chi^2\).

Return type:

float

Compute exclusion confidence level (\(1-CL_s\)) at a given POI, \(\mu\).

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed. If expected is set to "all" it will return both p-values and expected p-values where the likelihood will be fitted to observations.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
  
  Note
  
  In case of observed, function will return one value which has been fit to the observed data.
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
  
  Note
  
  In case of aposteriori, function will return five value for expected p-values which has been fit to the observed data. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The order of the output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  In case of apriori, function will return five value for expected p-values which has been fit to the SM background. Values represent \(1\sigma\) and \(2\sigma\) fluctuations from the background. The output order is \(-2\sigma\) value, \(-1\sigma\) value, central value, \(1\sigma\) and \(2\sigma\) value.
allow_negative_signal (bool, default False) – If True \(\hat\mu\) value will be allowed to be negative.
calculator (Literal["asymptotic", "toy", "chi_square"], default "asymptotic") –
Chooses the computation basis for hypothesis testing
- "asymptotic": Uses asymptotic hypothesis testing to compute p-values.
- "toy": Uses generated toy samples to compute p-values.
- "chi_square": Computes p-values via chi-square; \(\chi^2=-2\log\frac{\mathcal{L}(1,\theta_1)}{\mathcal{L}(0,\theta_0)}\).
poi_test_denominator (float, default None) – Set the POI value for the null hypothesis. if None, signal hypothesis will be compared against maximum likelihood otherwise with respect to the hypothesis determined with the POI value provided with this input. Only used when calculator="chi-square".
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Raises:

CalculatorNotAvailable – If calculator is not available.

Returns:

Exclusion confidence level i.e. \(1-CL_s\).

Return type:

List[float]

find_most_sensitive() → StatisticalModel[source]#

If the cross sections are defined, finds the statistical model with minimum prefit excluded cross section value. See s95exp.

Returns:: Statistical model with minimum prefit excluded cross section value.
Return type:: StatisticalModel

Sample data from the statistical model with fixed parameter of interest.

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
size (int, default None) – sample size. If None a callable function will be returned which takes sample size as input.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
init_pars (List[float], default None) – initial parameters for the optimiser
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
kwargs – keyword arguments for the optimiser.

Raises:

MethodNotAvailable – If bacend does not have sampler implementation.

Returns:

Sampled data with shape of (size, number of bins) or callable function to sample from directly.

Return type:

Union[np.ndarray, Callable[[int], np.ndarray]]

generate_asimov_data(expected: ExpectationType = ExpectationType.observed, test_statistic: str = 'qtilde', statistical_model_options: Dict[str, Dict] | None = None, **kwargs) → Dict[str, List[float]][source]#

Generate Asimov data for the statistical model. This function calls generate_asimov_data() function for each statistical model with appropriate statistical_model_options and generates Asimov data for each statistical model independently.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistic (Text, default "qtilde") –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
statistical_model_options (Dict[Text, Dict], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default_pdf.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

Returns a dictionary for data specific to each analysis. keywords will be analysis names and the items are data.

Return type:

Dict[Text, List[float]]

property is_alive: bool#

Returns true if there is at least one statistical model with at least one non zero bin with signal yield.

Returns:: bool

property is_asymptotic_calculator_available: bool#: Check if Asymptotic calculator is available for the backend

property is_chi_square_calculator_available: bool#: Check if \(\chi^2\) calculator is available for the backend

property is_toy_calculator_available: bool#: Check if Toy calculator is available for the backend

items() → Iterator[Tuple[str, StatisticalModel]][source]#

Returns a generator that returns analysis name and StatisticalModel every iteration.

Returns:: Iterator[Tuple[Text, StatisticalModel]]

likelihood(poi_test: float = 1.0, expected: ExpectationType = ExpectationType.observed, return_nll: bool = True, data: Dict[str, List[float]] | None = None, statistical_model_options: Dict[str, Dict] | None = None, **kwargs) → float[source]#

Compute the likelihood of the statistical model stack at a fixed parameter of interest

Parameters:

poi_test (float, default 1.0) – parameter of interest or signal strength, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
data (Dict[Text, List[float]], default None) – Input data to be used for fit. It needs to be given as a dictionary where the key argument is the name of the analysis and item is the statistical model specific data.
statistical_model_options (Optional[Dict[Text, Dict]], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default_pdf.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

value of the likelihood of the stacked statistical model.

Return type:

float

maximize_asimov_likelihood(return_nll: bool = True, expected: ExpectationType = ExpectationType.observed, test_statistics: str = 'qtilde', initial_muhat_value: float | None = None, par_bounds: List[Tuple[float, float]] | None = None, statistical_model_options: Dict[str, Dict] | None = None, **optimiser_options) → Tuple[float, float][source]#

Find the maximum of the likelihood which computed with respect to Asimov data.

Parameters:

return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default "qtilde") –
test statistic.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
initial_muhat_value (float, default None) – Initial value for \(\hat\mu\).
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
statistical_model_options (Dict[Text, Dict], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default_pdf.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

maximize_likelihood(return_nll: bool = True, expected: ExpectationType = ExpectationType.observed, allow_negative_signal: bool = True, data: Dict[str, List[float]] | None = None, initial_muhat_value: float | None = None, par_bounds: List[Tuple[float, float]] | None = None, statistical_model_options: Dict[str, Dict] | None = None, **optimiser_options) → Tuple[float, float][source]#

Find the maximum of the likelihood.

Parameters:

return_nll (bool, default True) – If True, returns negative log-likelihood value. if False returns likelihood value.
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
allow_negative_signal (bool, default True) – If True \(\hat\mu\) value will be allowed to be negative.
data (Dict[Text, List[float]], default None) – Input data to be used for fit. It needs to be given as a dictionary where the key argument is the name of the analysis and item is the statistical model specific data.
initial_muhat_value (float, default None) –
Initialisation for the \(\hat\mu\) for the optimiser. If None the initial value will be estimated by weighted combination of \(\hat\mu_i\) where \(i\) stands for the statistical models within the stack.

\[\mu_{\rm init} = \frac{1}{\sum_i \sigma_{\hat\mu, i}^2} \sum_i \frac{\hat\mu_i}{\sigma_{\hat\mu, i}^2}\]

where the value of \(\sigma_{\hat\mu}\) has been estimated by sigma_mu() function.
par_bounds (List[Tuple[float, float]], default None) – parameter bounds for the optimiser.
statistical_model_options (Dict[Text, Dict], default None) –
backend specific options. The dictionary key needs to be the backend name and the item needs to be the dictionary holding the keyword arguments specific to that particular backend.
```
>>> statistical_model_options = {"default_pdf.uncorrelated_background" : {"init_pars" : [1., 3., 4.]}}
```
kwargs – keyword arguments for the optimiser.

Returns:

\(\hat\mu\) value and maximum value of the likelihood.

Return type:

Tuple[float, float]

property minimum_poi: float#

Find the maximum minimum poi, \(\mu\) value within the analysis stack.

Returns:: minimum poi value that the combined stack can take.
Return type:: float

ntoys#: Number of toy samples for sample generator during hypothesis testing

Compute the upper limit for the parameter of interest i.e. \(\mu\).

Note

This function uses "qtilde" test statistic which means signal values are always assumed to be positive i.e. \(\hat\mu>0\).

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
confidence_level (float, default 0.95) – Determines the confidence level of the upper limit i.e. the value of \(1-CL_s\). It needs to be between [0,1].
low_init (Optional[float], default 1.0) –
Lower limit for the search algorithm to start If None it the lower limit will be determined by \(\hat\mu + 1.5\sigma_{\hat\mu}\).

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
hig_init (Optional[float], default 1.0) –
Upper limit for the search algorithm to start If None it the upper limit will be determined by \(\hat\mu + 2.5\sigma_{\hat\mu}\).

Note

\(\sigma_{\hat\mu}\) is determined via sigma_mu() function.
expected_pvalue (Literal["nominal", "1sigma", "2sigma"], default "nominal") –
In case of aposteriori and apriori expectation, gives the choice to find excluded upper limit for statistical deviations as well.
- "nominal": only find the upper limit for the central p-value. Returns a single value.
- "1sigma": find the upper limit for central p-value and \(1\sigma\) fluctuation from background. Returns 3 values.
- "2sigma": find the upper limit for central p-value and \(1\sigma\) and \(2\sigma\) fluctuation from background. Returns 5 values.
Note

For expected=spey.ExpectationType.observed, expected_pvalue argument will be overwritten to "nominal".
maxiter (int, default 10000) – Maximum iteration limit for the optimiser.
optimiser_arguments (Dict, default None) – Arguments for optimiser that is used to compute likelihood and its maximum.

Returns:

Return type:

Union[float, List[float]]

remove(analysis: str) → None[source]#

Remove an analysis from the stack.

Parameters:: analysis (Text) – unique identifier of the analysis to be removed.
Raises:: AnalysisQueryError – If the unique identifier does not match any of the statistical models in the stack.

sigma_mu(poi_test: float, expected: ExpectationType = ExpectationType.observed, test_statistics: Literal['qtilde', 'q', 'q0'] = 'qtilde', **kwargs) → float#

\[\sigma^2_A = \frac{(\mu - \mu^\prime)^2}{q_{\mu,A}}\quad , \quad q_{\mu,A} = -2\ln\lambda_A(\mu)\]

see eq. (31) in [arXiv:1007.1727]

Parameters:

poi_test (float, default 1.0) – parameter of interest, \(\mu\).
expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- aposteriori: Computes the expected p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth.
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
test_statistics (Text, default “qtilde”) –
test statistics.
- 'qtilde': (default) performs the calculation using the alternative test statistic, \(\tilde{q}_{\mu}\), see eq. (62) of [arXiv:1007.1727] (qmu_tilde()).
  
  Warning
  
  Note that this assumes that \(\hat\mu\geq0\), hence allow_negative_signal assumed to be False. If this function has been executed by user, spey assumes that this is taken care of throughout the external code consistently. Whilst computing p-values or upper limit on \(\mu\) through spey this is taken care of automatically in the backend.
- 'q': performs the calculation using the test statistic \(q_{\mu}\), see eq. (54) of [arXiv:1007.1727] (qmu()).
- 'q0': performs the calculation using the discovery test statistic, see eq. (47) of [arXiv:1007.1727] \(q_{0}\) (q0()).
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

value of the variance on \(\mu\).

Return type:

float

significance(expected: ExpectationType = ExpectationType.observed, **kwargs) → Tuple[float, float, List[float], List[float]]#

Compute the discovery of a positive signal. See [arXiv:1007.1727] eq. (53). and sec. 5.1.

Parameters:

expected (ExpectationType) –
Sets which values the fitting algorithm should focus and p-values to be computed.
- observed: Computes the p-values with via post-fit prescriotion which means that the experimental data will be assumed to be the truth (default).
- apriori: Computes the expected p-values with via pre-fit prescription which means that the SM will be assumed to be the truth.
  
  Note
  
  Since aposteriori and observed are both represent post-fit computation the result will be the same. The only difference can be seen via prefit, apriori, computation.
kwargs –
keyword arguments for the optimiser.
- init_pars (List[float], default None): initial parameters for the optimiser
- par_bounds (List[Tuple[float, float]], default None): parameter bounds for the optimiser.

Returns:

(\(\sqrt{q_{0,A}}\), \(\sqrt{q_0}\), p-values and expected p-values)

Return type:

Tuple[float, float, List[float], List[float]]

property statistical_models: Tuple[StatisticalModel]#

Accessor to the statistical model stack

Returns:: statistical model stack as an ntuple.
Return type:: Tuple[StatisticalModel]

Hypothesis testing#

`test_statistics.qmu`(mu, muhat, max_logpdf, ...)	Test statistic \(q_{\mu}\), see eq.
`test_statistics.qmu_tilde`(mu, muhat, ...)	Alternative test statistics, \(\tilde{q}_{\mu}\), see eq.
`test_statistics.q0`(mu, muhat, max_logpdf, logpdf)	Discovery test statistics, \(q_{0}\) see eq.
`test_statistics.get_test_statistic`(test_stat)	Retreive the test statistic function
`test_statistics.compute_teststatistics`(mu, ...)	Compute test statistics
`upper_limits.ComputerWrapper`(computer)	Wrapper for the computer function to track inputs and outputs
`upper_limits.find_poi_upper_limit`(...[, ...])	Find upper limit for parameter of interest, \(\mu\)
`upper_limits.find_root_limits`(computer[, ...])	Find upper and lower bracket limits for the root finding algorithm
`utils.pvalues`(delta_test_statistic, ...)	Calculate the p-values for the observed test statistic under the signal + background and background-only model hypotheses.
`utils.expected_pvalues`(...)	Calculate the \(p\) values corresponding to the median significance of variations of the signal strength from the background only hypothesis \(\mu=0\) at \((-2,-1,0,1,2)\sigma\).
`distributions.AsymptoticTestStatisticsDistribution`(shift)	The distribution the test statistic in the asymptotic case.
`distributions.EmpricTestStatisticsDistribution`(samples)	Create emprical distribution.
`asymptotic_calculator.compute_asymptotic_confidence_level`(...)	Compute p values i.e. \(p_{s+b}\), \(p_b\) and \(p_s\).
`toy_calculator.compute_toy_confidence_level`(...)	Compute confidence limits i.e. \(CL_{s+b}\), \(CL_b\) and \(CL_s\).

Gradient Tools#

`value_and_grad`(statistical_model[, ...])	Retreive function to compute negative log-likelihood and its gradient.
`hessian`(statistical_model[, expected, data])	Retreive the function to compute Hessian of negative log-likelihood

Default Backends#

Distributions#

`Poisson`(loc)	Poisson distribution
`Normal`(loc, scale[, weight, domain])	Normal distribution
`MultivariateNormal`(mean, cov[, weight, domain])	Multivariate normal distribution
`MainModel`(loc)	Main statistical model, modelled as Poisson distribution which has a variable lambda.
`ConstraintModel`(pdf_descriptions)	Constraint term modelled as a Gaussian distribution.

Default PDFs#

`DefaultPDFBase`(signal_yields, ...[, ...])	Default PDF backend base
`UncorrelatedBackground`(signal_yields, ...[, ...])	Interface for uncorrelated background uncertainties.
`CorrelatedBackground`(signal_yields, ...[, ...])	Correlated multi-region statistical model.
`ThirdMomentExpansion`(signal_yields, ...[, ...])	Simplified likelihood interface with third moment expansion.
`EffectiveSigma`(signal_yields, ...[, ...])	Simplified likelihood interface with variable Gaussian.
`third_moment.third_moment_expansion`(...[, ...])	Construct the terms for third moment expansion.
`third_moment.compute_third_moments`(...[, ...])	Assuming that the uncertainties are modelled as Gaussian, it computes third moments using Bifurcated Gaussian with asymmetric uncertainties.

Simple PDFs#

simple_pdf.Poisson(signal_yields, ...)

Poisson distribution without uncertainty implementation.

Exceptions#

`AnalysisQueryError`	Analysis query exception
`MethodNotAvailable`	If the method is not available for a given backend
`NegativeExpectedYields`	Negative expected yields exception
`PluginError`	Invalid plugin exception
`UnknownCrossSection`	Unknown cross-section exception
`UnknownTestStatistics`	Unknown test statistics exception
`CalculatorNotAvailable`	Unavailable calculator Exception
`CanNotFindRoots`	Unable to find roots of the function
`UnknownComputer`	Unknown computation base
`CombinerNotAvailable`	Unavailable combination routine exception

Description of all functions and classes

Contents

Description of all functions and classes#

Top-Level#

Main Classes#

Base Classes#

Statistical Model Combiner#

Hypothesis testing#

Gradient Tools#

Default Backends#

Distributions#

Default PDFs#

Simple PDFs#

Exceptions#