On the C-statistics for Evaluating Overall Adequacy of Risk Prediction Procedures with Censored Survival Data
关于使用删失生存数据评估风险预测程序总体充分性的 C 统计量

1. INTRODUCTION 1. 引言

For modern clinical medicine, risk prediction procedures are valuable tools for disease prevention and management. Pioneered by the Framingham study, risk score systems have been established for assessing individual risks of developing cardiovascular diseases, cancer or many other conditions within a certain time period [1, 2, 3, 4]. A key component in the assessment of risk algorithm performance is its ability to distinguish subjects who will develop an event (“cases”) from those who will not (“controls”). This concept, known as discrimination, has been well studied and quantified for binary outcomes using measures such as the estimated area under the Receiver Operating Characteristics (ROC) curve (AUC), which is also referred to as a “C-statistic” [5]. Such a statistic is an estimated conditional probability that for any pair of “case” and “control,” the predicted risk of an event is higher for the “case” [6].
对于现代临床医学，风险预测程序是疾病预防和管理的宝贵工具。由 Framingham 研究开创，已经建立了风险评分系统，用于评估在一定时间内患心血管疾病、癌症或许多其他疾病的个体风险 [ 1， 2， 3， 4]。评估风险算法性能的一个关键组成部分是它能够区分将发生事件的受试者（“病例”）和不会发生事件的受试者（“对照”）。这个概念被称为歧视，已经使用诸如受试者工作特征 （ROC） 曲线下估计面积 （AUC） 等措施对二元结果进行了充分研究和量化，该曲线也称为“C 统计量”[ 5]。这样的统计量是一个估计的条件概率，即对于任何一对 “案例” 和 “对照” ，“案例” 的事件预测风险更高 [ 6]。

If the primary response variable is the time to a certain event, the aforementioned procedure for binary outcomes can be used to quantify the ability of the risk score system to differentiate cases from controls at a time point t. If one is not interested in a particular time point, a standard concordance measure may be used to evaluate the overall performance of the risk scoring system. Specifically, let T be the event time, Z be a p × 1 covariate vector, and g(Z) be an estimated risk score for subjects with Z. There is a large class of measures to quantify how well the risk score g(Z) predicts the distribution of T or a function thereof. A good review paper is given by Korn & Simon [7] or more recently by Hielscher et al. [8] Such prediction measures can be classified into two broad classes, one based on explicit loss functions between the risk score and the survival time and the other based on rank correlations between these two quantities. The C-statistic proposed by Harrell et al. [9, 10, 11] is essentially a rank-correlation measure, motivated by Kendall’s tau for censored survival data [12]. A critical issue for rank correlation methods is how to order survival times in the presence of censoring. Brown et al. [12] used all observations and assigned probability scores to pairs in which ordering is not obvious due to censoring, based on the pooled Kaplan-Meier estimate for T. However, the score based on the pooled Kaplan-Meier estimate may not be appropriate when the covariates are associated with T. Alternative forms of C-statistic considered in [9, 10, 11, 13] use only so-called “useable” pairs and calculate the proportion of concordant pairs among them. However, such C-statistics estimate population parameters that may depend on the current study-specific censoring distribution. In this article, we propose a modified C-statistic which is consistent for a population concordance measure that is free of censoring.
如果主要响应变量是某个事件发生的时间，则上述二元结果程序可用于量化风险评分系统在某个时间点 t 区分病例和对照的能力。如果对某个特定时间点不感兴趣，可以使用标准一致性度量来评估风险评分系统的整体性能。具体来说，设 T 为事件时间，Z 为 p × 1 协变量向量，g（Z） 为 Z 为主体的估计风险评分。有一大类度量可以量化风险评分 g（Z） 预测 T 或其函数分布的程度。Korn & Simon [ 7] 或最近的 Hielscher 等人 [ 8] 给出了一篇很好的综述论文。这种预测措施可以分为两大类，一类基于风险评分和生存时间之间的显式损失函数，另一类基于这两个量之间的等级相关性。Harrell 等人 [ 9， 10， 11] 提出的 C 统计量本质上是一种秩相关度量，由 Kendall 的 tau 驱动，用于删失生存数据 [ 12]。秩相关方法的一个关键问题是如何在存在删失的情况下对生存时间进行排序。Brown等[12]使用了所有观测值，并根据T的合并Kaplan-Meier估计，将概率分数分配给由于删失而排序不明显的对。然而，当协变量与 T 相关联时，基于合并的 Kaplan-Meier 估计的分数可能不合适。[ 9， 10， 11， 13] 中考虑的 C 统计量的替代形式仅使用所谓的“可用”对并计算它们之间一致对的比例。 然而，这种 C 统计量估计的总体参数可能取决于当前研究特定的删失分布。在本文中，我们提出了一种修改后的 C 统计量，它与没有删失的总体一致性测量是一致的。

More specifically, for two independent copies {(T₁, g(Z₁))′, (T₂, g(Z₂))′} of (T, g(Z))′, a commonly used concordance measure is
更具体地说，对于 （T， g（Z））′ 的两个独立副本 {（T ₁ ， g（Z ₁ ））′， （T ₂ ， g（Z ₂ ））′}，常用的一致性度量是

[14]. When T is subject to right censoring, as discussed in Heagerty & Zheng [14] one would typically consider a modified C_τ with a fixed, prespecified follow-up period (0, τ), where
当T受到右删失时，如Heagerty和Zheng [14]中所讨论的那样，人们通常会考虑 _τ 具有固定的、预先指定的随访期（0， τ）的修改C，其中

Estimation of (1.1) or (1.2) when the event time may be censored, however, is not straightforward [11, 13, 15]. The estimator for C or C_τ proposed by Heagerty & Zheng [14] is derived under a proportional hazards model. If this semi-parametric working model is not correctly specified, the resulting estimator may be biased. A popular nonparametric C-statistic for estimating C or C_τ was proposed by Harrell et al. [9, 10, 11] and extensively studied by Pencina & D’Agostino [13]. Note that this generalization is a weighted area under an “incident/dynamic” ROC curve [14, 16] with weights depending on the study-specific censoring distribution.
然而，当事件时间可以被删失时，估计 （ 1.1） 或 （ 1.2） 并不简单 [ 11， 13， 15]。Heagerty和Zheng [14]提出的C或C _τ 的估计量是在比例风险模型下推导出来的。如果未正确指定此半参数工作模型，则生成的估计量可能会有偏差。Harrell等人提出了一种流行的非参数C统计量来估计C或C _τ [ 9， 10， 11] 并被Pencina & D'Agostino [ 13]广泛研究。请注意，这种泛化是“事件/动态”ROC 曲线 [ 14， 16] 下的加权区域，其权重取决于特定于研究的删失分布。

When the study individuals have different follow-up times, the C-statistic studied by Harrell et al. [9, 10, 11] converges to an association measure that involves the study censoring distribution. In this article, under the general random censorship assumption, we provide a simple non-parametric estimator for the concordance measure in (1.2), which is free of censoring. Furthermore, we study the large sample properties of the new estimation procedure. Our proposal is illustrated with two real examples. The performance of the new proposal under various practical settings is also examined via a simulation study. Note that Gönen & Heller [17] proposed a method for censored survival data to estimate pr(T₂ > T₁ ∣ g(Z₁) > g(Z₂)), which is also a concordance measure and has a similar form to (1.1), but we do not focus on this type of measures in this paper.
当研究个体的随访时间不同时，Harrell 等人 [ 9， 10， 11] 研究的 C 统计量会收敛到涉及研究删失分布的关联度量。在本文中，在一般的随机删失假设下，我们为 （ 1.2 中的一致性测度提供了一个简单的非参数估计器），它是无删失的。此外，我们研究了新估计程序的大样本特性。我们的提案用两个真实的例子来说明。还通过模拟研究检查了新提案在各种实际设置下的性能。请注意，Gönen & Heller [ 17] 提出了一种删失生存数据估计 pr（T ₂ > T ₁ ∣ g（Z ₁ ） > g（Z ₂ ）） 的方法，这也是一种一致性测量，与（ 1.1）的形式相似，但我们在本文中不关注这种类型的测量。

2. INFERENCE PROCEDURES FOR DEGREE OF ASSOCIATION BETWEEN EVENT TIMES AND ESTIMATED RISK SCORES
2. 事件时间与估计风险评分之间关联程度的推理程序

In this section, we consider a non-trivial case that at least one component of the covariate vector Z is continuous. For the survival time T, let D be the corresponding censoring variable. Assume that D is independent of T and Z. Let {(T_i, Z_i, D_i), i = 1, …, n} be n independent copies of {(T,Z,D)}. For the ith subject, we only observe (X_i, Z_i, Δ_i), where X_i = min (T_i, D_i), and Δ_i equals 1 if X_i = T_i and 0 otherwise.
在本节中，我们考虑一个非平凡的情况，即协变量向量 Z 的至少一个分量是连续的。对于生存时间 T，设 D 为相应的删失变量。假设 D 独立于 T 和 Z。设 {（T， Z， D）， i = 1， ...， n} 是 {（T，Z，D）} 的 n 个独立副本。对于第 i 个主题，我们只观察到 （X， Z， Δ），其中 X = min （T， D），如果 X = T，则 Δ 等于 1，否则为 0。

Suppose that we fit the data with a working parametric or semi-parametric regression model, for example, a standard Cox proportional hazards model [18]:
假设我们使用有效的参数或半参数回归模型拟合数据，例如，标准 Cox 比例风险模型 [ 18]：

where Λ_Z(·) is the cumulative hazard function for subjects with covariate vector Z, Λ₀(·) is the unknown baseline cumulative hazard function and β is the unknown p × 1 parameter vector. Let the maximum partial likelihood estimator for β be denoted by β̂. Note that even when the model (2.1) is not correctly specified, under a rather mild non-separable condition that there does not exist vector ζ such that pr(T₁ > T₂ ∣ ζ′Z₁ < ζ′Z₂) = 1, β̂ converges to a constant vector, say, β₀, as n → ∞. This stability property is important for deriving the new inference procedure.
其中 Λ _Z （·） 是具有协变量向量 Z 的主体的累积风险函数，Λ ₀ （·） 是未知的基线累积风险函数，β 是未知的 p × 1 参数向量。设 β 的最大偏似然估计量用 β̂ 表示。请注意，即使模型 （ 2.1） 没有正确指定，在不存在向量ζ的相当温和的不可分条件下，使得 pr（T ₁ > T ₂ ∣ ζ′Z ₁ < ζ′Z ₂ ） = 1， β̂ 收敛到一个常数向量，比如 β ₀ ，作为 n → ∞。此稳定性属性对于派生新的推理过程非常重要。

For a pair of future patients with covariate vectors {𝑍0𝑘,𝑘=1,2} and the potential survival times {𝑇0𝑘,𝑘=1,2}, their corresponding risk scores are {̂𝛽′⁢𝑍0𝑘,𝑘=1,2}. To evaluate this risk score system, one may use the concordance measure discussed in Section 1:
对于一对具有协变量向量 {𝑍0𝑘,𝑘=1,2} 和潜在生存时间 {𝑇0𝑘,𝑘=1,2} 的未来患者，他们相应的风险评分为 {̂𝛽′⁢𝑍0𝑘,𝑘=1,2} 。要评估这个风险评分系统，可以使用第 1 节中讨论的一致性测量：

where the probability is evaluated with respect to the data, and ( 𝑇01,𝑍01) and ( 𝑇02,𝑍02). Note that C_n depends on the sample size. Let the limit of C_n be denoted by
其中，根据数据评估概率，以及 （ 𝑇01,𝑍01 ） 和 （ 𝑇02,𝑍02 ）。请注意，C _n 取决于样本大小。设 C _n 的极限表示为

Now, since the support of the censoring variable D is usually shorter than that of the failure time T, the tail part of the estimated survival function of T is rather unstable. Therefore, we consider a truncated version of C in (2.2), that is,
现在，由于删失变量 D 的支持通常短于失效时间 T 的支持，因此 T 的估计生存函数的尾部相当不稳定。因此，我们在 （ 2.2） 中考虑 C 的截断版本，即

where τ is a prespecified time point such that pr(D > τ) > 0.
其中 τ 是预先指定的时间点，使得 pr（D > τ） > 0。

It follows from an “inverse probability weighting” technique as employed in Cheng et al. [19] for dealing with a completely different problem in survival analysis that C_τ can be consistently, nonparametrically estimated by
它遵循 Cheng 等人 [ 19] 采用的“逆概率加权”技术，用于处理生存分析中一个完全不同的问题，即 C _τ 可以由

where I(·) is the indicator function and Ĝ(·) is the Kaplan-Meier estimator for the censoring distribution G(t) = pr(D > t). Heuristically, the consistency of the above estimator follows from the fact that as n → ∞, the denominator of (2.4) divided by n² converges to
其中 I（·） 是指示函数，Ĝ（·） 是删失分布 G（t） = pr（D > t） 的 Kaplan-Meier 估计量。从启发式的角度来看，上述估计量的一致性源于以下事实：当 n → ∞时，（ 2.4） 除以 n ² 的分母收敛为

Note that E {I(D > T₁){G(T₁)}⁻¹∣T₁} = 1 in the above equation, where the expectation is taken with respect to D. Thus, the denominator and the numerator of (2.4) divided by n² converge to pr⁡(𝑇01<𝑇02,𝑇01<𝜏) and pr⁡(𝛽′0⁢𝑍01>𝛽′0⁢𝑍02,𝑇01<𝑇02,𝑇01<𝜏), respectively.
请注意，在上述方程中，E {I（D > T ₁ ）{G（T ₁ ）} ∣ ⁻¹ T ₁ } = 1，其中期望值是相对于 D 的。因此，（ 2.4） 除以 n ² 的分母和分子分别收敛到 pr⁡(𝑇01<𝑇02,𝑇01<𝜏) 和 pr⁡(𝛽′0⁢𝑍01>𝛽′0⁢𝑍02,𝑇01<𝑇02,𝑇01<𝜏) 。

In Appendix A, we show that
在附录 A 中，我们展示了

is asymptotically normal with mean 0. Moreover, in the Appendix, we show how to use a perturbation-resampling method to approximate the distribution of W. Specifically, we show that the asymptotic distribution of W̃ given in (A.3) is the same as that of W. The realizations of W̃ can be generated easily by simulating a large number, M, of random samples from, for instance, the unit exponential. Inferences about C_τ can then be made via the normal approximation to the distribution of Ĉ_τ and these realizations of W̃ . For instance, a two-sided 0.95 confidence interval for C_τ would be Ĉ_τ ± 1.96n^−1/2σ, where σ² is the standard sample variance or a robust version thereof based on the above M realizations of W̃.
是渐近正态的，均值为 0。此外，在附录中，我们展示了如何使用扰动重采样方法来近似 W 的分布。具体来说，我们展示了 （A.3） 中给出的 W̃ 的渐近分布与 W 的渐近分布相同。W̃ 的实现可以通过模拟大量随机样本 M 来轻松生成，例如，单位指数。然后可以通过对 Ĉ 的分布 _τ 和 W̃ 的这些实现的正态近似来做出关于 C _τ 的推断。例如，C _τ 的双侧 0.95 置信区间将是 Ĉ _τ ± 1.96n ^−1/2 σ，其中 σ ² 是标准样本方差或基于上述 W 的 M 实现的稳健版本。

It is important to note that the C-statistic proposed by Harrell et al. [11] is
需要注意的是，Harrell 等人 [ 11] 提出的 C 统计量是

which converges to a censoring-dependent quantity,
它收敛到一个与删失相关的量，

Pencina & D’Agostino [13] formulated an alternative form of C-statistic,
Pencina & D'Agostino [ 13] 提出了一种替代形式的C-统计量，

for various τ. The limiting value of this statistic,
对于各种 τ.此统计数据的极限值

also involves the censoring distribution.
还涉及删失分布。

When there are two competing survival regression models, say A and B, one may compare the overall predictive performances of these models based on their C-statistics. Specifically, let 𝐶(𝐴)𝜏 and 𝐶(𝐵)𝜏 be C_τ for Model A and Model B, respectively. Let 𝜉=𝐶(𝐴)𝜏−𝐶(𝐵)𝜏. Then a consistent estimator for ξ is ̂𝜉=̂𝐶(𝐴)𝜏−̂𝐶(𝐵)𝜏, where ̂𝐶(𝐴)𝜏 and ̂𝐶(𝐵)𝜏 are the corresponding C-statistics (2.4). Note that these two C-statistics are correlated. It follows from a similar argument for deriving the large sample properties of Ĉ_τ for a single model, that the distribution of
当有两个竞争的生存回归模型（比如 A 和 B）时，人们可以根据它们的 C 统计量来比较这些模型的整体预测性能。具体来说，模型 A 和模型 B 分别为 let 𝐶(𝐴)𝜏 和 𝐶(𝐵)𝜏 be C _τ 。设 𝜉=𝐶(𝐴)𝜏−𝐶(𝐵)𝜏 .那么 ξ 的一致估计量是 ̂𝜉=̂𝐶(𝐴)𝜏−̂𝐶(𝐵)𝜏 ，其中 ̂𝐶(𝐴)𝜏 和 ̂𝐶(𝐵)𝜏 是相应的 C 统计量 （ 2.4）。请注意，这两个 C 统计量是相关的。从为单个模型推导出 Ĉ _τ 的大样本属性的类似参数中，可以得出

can be approximated by a normal with mean zero. Its variance can be obtained via the above perturbation-resampling method. The details of this normal approximation are given in Appendix B. To make inference about ξ, a two-sided 0.95 confidence interval is ξ̂ ± 1.96n^−1/2σ̂_Wξ , where ̂𝜎2𝑊𝜉 denotes the estimated variance of W_ξ.
可以用均值为零的正态近似。它的方差可以通过上述扰动重采样方法获得。附录 B 中给出了这种正态近似的详细信息。为了推断 ξ，双侧 0.95 置信区间是 ξ̂ ± 1.96n ^−1/2 σ̂ _Wξ ，其中 ̂𝜎2𝑊𝜉 表示 W _ξ 的估计方差。

3. NUMERICAL STUDIES 3. 数值研究

4. REMARKS 4. 备注

In this article, we show that the overall performance of a parametric or semi-parametric model in predicting the subject-level survival over an interval (0, τ) can be evaluated via a simple, unbiased estimation procedure for C_τ. Based on our numerical study, we find that the new estimation procedure is robust with respect to the choice of τ. However, if the pre-specified τ is “too” large such that very few events were observed or very few study subjects were followed beyond this time point, the standard error estimate for Ĉ_τ can be quite large, reflecting a high degree of uncertainty of our inferences about C_τ. For this case, one should cautiously utilize the fitted model for prediction over this large time interval.
在本文中，我们表明参数或半参数模型在预测区间 （0， τ） 内受试者水平生存率的整体性能可以通过简单、无偏的 C _τ 估计程序来评估。根据我们的数值研究，我们发现新的估计程序在 τ 的选择方面是稳健的。但是，如果预先指定的 τ “太”大，以至于观察到的事件非常少，或者在此时间点之后跟踪的研究对象非常少，则 Ĉ 的标准误差估计 _τ 可能相当大，反映了我们对 C _τ 的推断的高度不确定性。对于这种情况，应该谨慎地利用拟合模型在这个大时间间隔内进行预测。

There are various C-statistics proposed in the literature. With the same technique utilized in this article, one may modify these statistics accordingly so that they estimate concordance measures which are free of the study-specific censoring distribution. The computer code for implementing the new inference procedure can be downloaded from (http://bcb.dfci.harvard.edu/~huno).
文献中提出了各种 C 统计量。使用本文中使用的相同技术，可以相应地修改这些统计数据，以便他们估计不受研究特定删失分布影响的一致性度量。用于实现新推理过程的计算机代码可以从 （ http://bcb.dfci.harvard.edu/~huno） 下载。

For the new proposal, we assume that the censoring distribution is independent of the covariates. This assumption is not unreasonable in a well-executed clinical study, especially when there are no competing risks for observing the endpoint (for example, death or a composite clinical endpoint) and the event times are possibly censored mainly due to administrative censoring. When the covariate vector can be discretized, our new procedure can be modified easily using stratified Kaplan-Meier estimates for the censoring. When some of the covariates are continuous, one needs to assume a model for censoring distribution and use the resulting estimated distribution of the censoring to construct a new C-statistic. In theory, the validity of the resulting estimators depends on the adequacy of the model. On the other hand, based on the results from our simulation study, it appears that our proposal is quite robust even when the censoring is dependent on the covariates.
对于新提案，我们假设删失分布独立于协变量。在执行良好的临床研究中，这种假设并非不合理，尤其是当观察终点没有竞争风险（例如，死亡或复合临床终点）并且事件时间可能主要由于行政审查而被删失时。当协变量向量可以离散化时，我们的新程序可以使用分层 Kaplan-Meier 估计进行删失轻松修改。当某些协变量是连续的时，需要假设一个删失分布模型，并使用删失的结果估计分布来构建新的 C 统计量。从理论上讲，所得估计量的有效性取决于模型的充分性。另一方面，根据我们的仿真研究结果，即使删失取决于协变量，我们的提议似乎也相当稳健。

In this article, we showed an inference procedure not only for C_τ but also for the difference between two C_τ ’s for comparing two competing prediction models. Although C-statistics are commonly used for quantifying the predictability of working models, they are not sensitive for capturing overall added values from a new marker on top of conventional predictors [11, 13]. Alternatively, one might use measures such as explained variation for survival data to compare two models [7]. If the model is correct, the likelihood based measures may be more sensitive in detecting differences in prediction ability, compared to rank-based measures such as C-indexes. Recently, alternative statistical estimation procedures were proposed for comparing prediction models [23, 24], which may be utilized to quantify incremental values. Further research is needed for developing intuitively interpretable and sensitive methods for comparing prediction models.
在本文中，我们不仅展示了 C _τ 的推理程序，还展示了两个 C _τ 之间的差异，用于比较两个竞争的预测模型。尽管 C 统计量通常用于量化工作模型的可预测性，但它们对于在传统预测变量之上捕获新标记的总体附加值并不敏感 [ 11， 13]。或者，人们可以使用诸如生存数据的解释变异等措施来比较两个模型 [ 7]。如果模型正确，则与基于排名的度量（如 C 指数）相比，基于似然的度量在检测预测能力的差异时可能更敏感。最近，提出了替代统计估计程序来比较预测模型 [ 23， 24]，可用于量化增量值。需要进一步的研究来开发直观可解释和敏感的预测模型比较方法。