Skip to content

# Brier Score and Integrated Brier Score

## Brier Score

The Brier score is used to evaluate the accuracy of a predicted survival function at a given time $t$; it represents the average squared distances between the observed survival status and the predicted survival probability and is always a number between 0 and 1, with 0 being the best possible value.

Given a dataset of $N$ samples, $\forall i \in [\![1, N ]\!], \left(\vec{x}_i, \delta_i, T_i \right)$ is the format of a datapoint, and the predicted survival function is $\hat{S}(t, \vec{x}_i), \forall t \in \mathbb{R^+}$:

In the absence of right censoring, the Brier score can be calculated such that:

However, if the dataset contains samples that are right censored, then it is necessary to adjust the score by weighting the squared distances using the inverse probability of censoring weights method. Let $\hat{G}(t) = P[C > t ]$ be the estimator of the conditional survival function of the censoring times calculated using the Kaplan-Meier method, where $C$ is the censoring time.

In terms of benchmarks, a useful model will have a Brier score below $0.25$. Indeed, it is easy to see that if $\forall i \in [\![1, N]\!], \hat{S}(t, \vec{x}_i) = 0.5$, then $BS(t) = 0.25$.

### Location

The function can be found at pysurvival.utils.metrics.brier_score.

### API

brier_score - Brier score computations

    brier_score(model, X, T, E, t_max=None)


Parameters:

• model : Pysurvival object -- Pysurvival model

• X : array-like -- input samples; where the rows correspond to an individual sample and the columns represent the features (shape=[n_samples, n_features]).

• T : array-like -- target values describing the time when the event of interest or censoring occurred.

• E : array-like -- values that indicate if the event of interest occurred i.e.: E[i]=1 corresponds to an event, and E[i] = 0 means censoring, for all i.

• t_max: float ( default=None ) Maximal time for estimating the prediction error curves. If missing the largest value of the response variable is used.

Returns:

• times: array-like. A vector of timepoints. At each timepoint the brier score is estimated

• brier_scores: array-like. A vector of brier scores

## Integrated Brier Score

The Integrated Brier Score (IBS) provides an overall calculation of the model performance at all available times.

### Location

The function can be found at pysurvival.utils.metrics.integrated_brier_score to output the values and pysurvival.utils.display.integrated_brier_score to display the predictive error curve.

### API

integrated_brier_score - Integrated Brier score computations

integrated_brier_score(model, X, T, E, t_max=None, figure_size=(20, 10))


Parameters:

• model : Pysurvival object -- Pysurvival model

• X : array-like -- input samples; where the rows correspond to an individual sample and the columns represent the features (shape=[n_samples, n_features]).

• T : array-like -- target values describing the time when the event of interest or censoring occurred.

• E : array-like -- values that indicate if the event of interest occurred i.e.: E[i]=1 corresponds to an event, and E[i] = 0 means censoring, for all i.

• t_max: float ( default=None ) Maximal time for estimating the prediction error curves. If missing the largest value of the response time variable is used.

• figure_size: tuple of double ( default=(20, 10) ) width, height in inches representing the size of the chart of the survival function. Option available if the function is being called from pysurvival.utils.display

Returns:

• ibs: double The integrated brier score