Conditional error spending functions

Introduction

We describe conditional error spending functions for group sequential designs. These functions are used to calculate conditional error spending boundaries for group sequential designs using spending functions proposed by Xi and Gallo (2019). Note that for all spending functions, for t > 1 we define the spending function as if t = 1. For γ ∈ [0, 1], we define

z_γ = Φ⁻¹(1 − γ)

where Φ is the standard normal cumulative distribution function.

There are 3 spending functions proposed by Xi and Gallo (2019), which we will refer to as Method 1 (sfXG1()), Method 2 (sfXG2()), and Method 3 (sfXG3()). When there is a single interim analysis, conditional error from Method 1 is almost exactly the same as the parameter γ but it allows a narrower range of γ. Method 2 provides less accurate approximation but allows a wider range of γ. Method 3 was proposed to approximate Pocock bounds with equal bounds on the Z-scale. We replicate spending function bounds of Xi and Gallo (2019) along with corresponding conditional error computations below. We also compare results to other commonly used spending functions.

Implementation in gsDesign

library(gsDesign)
library(tibble)
library(dplyr)
library(gt)

Method 1

For γ ∈ [0.5, 1), the spending function is defined as

$$ \alpha_\gamma(t) = 2 - 2\times \Phi\left(\frac{z_{\alpha/2} - z_\gamma\sqrt{1-t}}{\sqrt t} \right). $$

Recalling the range γ ∈ [0.5, 1), we plot this spending function for γ = 0.5, 0.6, 0.75, 0.9.

pts <- seq(0, 1.2, 0.01)
pal <- palette()

plot(
  pts,
  sfXG1(0.025, pts, 0.5)$spend,
  type = "l", col = pal[1],
  xlab = "t", ylab = "Spending", main = "Xi-Gallo, Method 1"
)
lines(pts, sfXG1(0.025, pts, 0.6)$spend, col = pal[2])
lines(pts, sfXG1(0.025, pts, 0.75)$spend, col = pal[3])
lines(pts, sfXG1(0.025, pts, 0.9)$spend, col = pal[4])
legend(
  "topleft",
  legend = c("gamma=0.5", "gamma=0.6", "gamma=0.75", "gamma=0.9"),
  col = pal[1:4],
  lty = 1
)

Method 2

For γ ∈ [1 − Φ(z_α/2/2), 1), the spending function for Method 2 is defined as

$$ \alpha_\gamma(t)= 2 - 2\times \Phi\left(\frac{z_{\alpha/2} - z_\gamma(1-t)}{\sqrt t} \right) $$

For α = 0.025, we restrict γ to [0.131, 1) and plot the spending function for γ = 0.14, 0.25, 0.5, 0.75, 0.9.

plot(
  pts,
  sfXG2(0.025, pts, 0.14)$spend,
  type = "l", col = pal[1],
  xlab = "t", ylab = "Spending", main = "Xi-Gallo, Method 2"
)
lines(pts, sfXG2(0.025, pts, 0.25)$spend, col = pal[2])
lines(pts, sfXG2(0.025, pts, 0.5)$spend, col = pal[3])
lines(pts, sfXG2(0.025, pts, 0.75)$spend, col = pal[4])
lines(pts, sfXG2(0.025, pts, 0.9)$spend, col = pal[5])
legend(
  "topleft",
  legend = c("gamma=0.14", "gamma=0.25", "gamma=0.5", "gamma=0.75", "gamma=0.9"),
  col = pal[1:5],
  lty = 1
)

Method 3

For γ ∈ (α/2, 1)

$$ \alpha_\gamma(t)= 2 - 2\times \Phi\left(\frac{z_{\alpha/2} - z_\gamma(1-\sqrt t)}{\sqrt t} \right). $$

For α = 0.025, we restrict γ to (0.0125, 1) and plot the spending function for γ = 0.013, 0.02, 0.05, 0.1, 0.25, 0.5, 0.75, 0.9.

plot(
  pts,
  sfXG3(0.025, pts, 0.013)$spend,
  type = "l", col = pal[1],
  xlab = "t", ylab = "Spending", main = "Xi-Gallo, Method 3"
)
lines(pts, sfXG3(0.025, pts, 0.02)$spend, col = pal[2])
lines(pts, sfXG3(0.025, pts, 0.05)$spend, col = pal[3])
lines(pts, sfXG3(0.025, pts, 0.1)$spend, col = pal[4])
lines(pts, sfXG3(0.025, pts, 0.25)$spend, col = pal[5])
lines(pts, sfXG3(0.025, pts, 0.5)$spend, col = pal[6])
lines(pts, sfXG3(0.025, pts, 0.75)$spend, col = pal[7])
lines(pts, sfXG3(0.025, pts, 0.9)$spend, col = pal[8])
legend(
  "bottomright",
  legend = c(
    "gamma=0.013", "gamma=0.02", "gamma=0.05", "gamma=0.1",
    "gamma=0.25", "gamma=0.5", "gamma=0.75", "gamma=0.9"
  ),
  col = pal[1:8],
  lty = 1
)

Replicating published examples

We replicate spending function bounds of Xi and Gallo (2019) along with corresponding conditional error computations. We have two utility functions. Transposing a tibble and a custom function to compute conditional error.

# Custom function to transpose while preserving names
# From https://stackoverflow.com/questions/42790219/how-do-i-transpose-a-tibble-in-r
transpose_df <- function(df) {
  t_df <- data.table::transpose(df)
  colnames(t_df) <- rownames(df)
  rownames(t_df) <- colnames(df)
  t_df <- t_df %>%
    tibble::rownames_to_column(.data = .) %>%
    tibble::as_tibble(.)
  return(t_df)
}

ce <- function(x) {
  k <- x$k
  ce <- c(gsCPz(z = x$upper$bound[1:(k - 1)], i = 1:(k - 1), x = x, theta = 0), NA)
  t <- x$timing
  ce_simple <- c(pnorm((last(x$upper$bound) - x$upper$bound[1:(k - 1)] * sqrt(t[1:(k - 1)])) / sqrt(1 - t[1:(k - 1)]),
    lower.tail = FALSE
  ), NA)
  Analysis <- 1:k
  y <- tibble(
    # Analysis = Analysis,
    Z = x$upper$bound,
    "CE simple" = ce_simple,
    CE = ce
  )
  return(y)
}

Method 1

The conditional error spending functions of Xi and Gallo (2019) for Method 1 and Method 2 are designed to derive interim efficacy bounds with conditional error approximately equal to the spending function parameter γ. The conditional probability of crossing the final bound u_K given an interim result Z_k = u_k at analysis k < K under the assumption of no treatment effect is

$$ p_0(Z_K\ge u_K|Z_k=u_k) = 1 - \Phi\left(\frac{u_K - u_k\sqrt t}{\sqrt{1-t}}\right). $$

Conditional rejection probabilities accounting for all future analyses as well as under any assumed treatment effect are explained further in the gsDesign technical manual. We will see that where there are future interim analyses below, the conditional error for crossing at least one future efficacy bound is substantially greater than the simple conditional error ignoring future interims.

We will compare the Method 1 conditional error spending functions of Xi and Gallo (2019) with O’Brien-Fleming bounds (sfu = "OF"), exponential spending (sfu = sfExponential), and the Lan-DeMets spending function to approximate O’Brien-Fleming bounds (sfu = sfLDOF). The O’Brien-Fleming bounds are specifically known to the 0.5 (simple) conditional error as seen in the table below. The exponential spending function provides the closest approximation of O’Brien-Fleming bounds with a parameter of 0.76; this was suggested previously by Anderson and Clark (2010). The other Method 1 spending functions generally have higher than the targeted simple conditional error; thus, if you wish to use a particular conditional error at bounds, it may be better to see if a smaller γ than the targeted conditional error provides a better match.

xOF <- gsDesign(k = 4, test.type = 1, sfu = "OF")
xLDOF <- gsDesign(k = 4, test.type = 1, sfu = sfLDOF)
xExp <- gsDesign(k = 4, test.type = 1, sfu = sfExponential, sfupar = 0.76)
x1.8 <- gsDesign(k = 4, test.type = 1, sfu = sfXG1, sfupar = 0.8)
x1.7 <- gsDesign(k = 4, test.type = 1, sfu = sfXG1, sfupar = 0.7)
x1.6 <- gsDesign(k = 4, test.type = 1, sfu = sfXG1, sfupar = 0.6)
x1.5 <- gsDesign(k = 4, test.type = 1, sfu = sfXG1, sfupar = 0.5)
xx <- rbind(
  transpose_df(ce(xOF)) %>% mutate(gamma = "O'Brien-Fleming"),
  transpose_df(ce(xExp)) %>% mutate(gamma = "Exponential, nu=0.76 to Approximate O'Brien-Fleming"),
  transpose_df(ce(xLDOF)) %>% mutate(gamma = "Lan-DeMets to Approximate O'Brien-Fleming"),
  transpose_df(ce(x1.5)) %>% mutate(gamma = "gamma = 0.5"),
  transpose_df(ce(x1.6)) %>% mutate(gamma = "gamma = 0.6"),
  transpose_df(ce(x1.7)) %>% mutate(gamma = "gamma = 0.7"),
  transpose_df(ce(x1.8)) %>% mutate(gamma = "gamma = 0.8")
)
xx %>%
  gt(groupname_col = "gamma") %>%
  tab_spanner(label = "Analysis", columns = 2:5) %>%
  fmt_number(columns = 2:5, decimals = 3) %>%
  tab_options(data_row.padding = px(1)) %>%
  tab_header(
    title = "Xi-Gallo, Method 1 Spending Function",
    subtitle = "Conditional Error Spending Functions"
  ) %>%
  tab_footnote(
    footnote = "Conditional Error not accounting for future interim bounds.",
    locations = cells_stub(rows = seq(2, 20, 3))
  ) %>%
  tab_footnote(
    footnote = "CE = Conditional Error accounting for all analyses.",
    locations = cells_stub(rows = seq(3, 21, 3))
  )

	Analysis
Xi-Gallo, Method 1 Spending Function
Conditional Error Spending Functions
	1	2	3	4
O'Brien-Fleming
Z	4.049	2.863	2.337	2.024
¹ CE simple	0.500	0.500	0.500	NA
² CE	0.687	0.625	0.500	NA
Exponential, nu=0.76 to Approximate O'Brien-Fleming
Z	4.052	2.890	2.346	2.020
¹ CE simple	0.502	0.513	0.509	NA
² CE	0.682	0.636	0.509	NA
Lan-DeMets to Approximate O'Brien-Fleming
Z	4.333	2.963	2.359	2.014
¹ CE simple	0.570	0.546	0.523	NA
² CE	0.747	0.668	0.523	NA
gamma = 0.5
Z	4.333	2.963	2.359	2.014
¹ CE simple	0.570	0.546	0.523	NA
² CE	0.747	0.668	0.523	NA
gamma = 0.6
Z	4.784	3.230	2.508	1.983
¹ CE simple	0.682	0.665	0.647	NA
² CE	0.804	0.749	0.647	NA
gamma = 0.7
Z	5.265	3.514	2.671	1.969
¹ CE simple	0.778	0.767	0.754	NA
² CE	0.858	0.821	0.754	NA
gamma = 0.8
Z	5.826	3.845	2.863	1.963
¹ CE simple	0.864	0.857	0.849	NA
² CE	0.908	0.887	0.849	NA
¹ Conditional Error not accounting for future interim bounds.
² CE = Conditional Error accounting for all analyses.

Method 2

Method 2 provides a wider range of γ values targeting conditional error at bounds. Again, choice of γ to get the targeted conditional error may be worth some evaluation.

x1.8 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.8)
x1.7 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.7)
x1.6 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.6)
x1.5 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.5)
x1.4 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.4)
x1.3 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.3)
x1.2 <- gsDesign(k = 4, test.type = 1, sfu = sfXG2, sfupar = 0.2)
xx <- rbind(
  transpose_df(ce(x1.2)) %>% mutate(gamma = "gamma = 0.2"),
  transpose_df(ce(x1.3)) %>% mutate(gamma = "gamma = 0.3"),
  transpose_df(ce(x1.4)) %>% mutate(gamma = "gamma = 0.4"),
  transpose_df(ce(x1.5)) %>% mutate(gamma = "gamma = 0.5"),
  transpose_df(ce(x1.6)) %>% mutate(gamma = "gamma = 0.6"),
  transpose_df(ce(x1.7)) %>% mutate(gamma = "gamma = 0.7"),
  transpose_df(ce(x1.8)) %>% mutate(gamma = "gamma = 0.8")
)
xx %>%
  gt(groupname_col = "gamma") %>%
  tab_spanner(label = "Analysis", columns = 2:5) %>%
  fmt_number(columns = 2:5, decimals = 3) %>%
  tab_options(data_row.padding = px(1)) %>%
  tab_footnote(
    footnote = "Conditional Error not accounting for future interim bounds.",
    locations = cells_stub(rows = seq(2, 20, 3))
  ) %>%
  tab_footnote(
    footnote = "CE = Conditional Error accounting for all analyses.",
    locations = cells_stub(rows = seq(3, 21, 3))
  ) %>%
  tab_header(
    title = "Xi-Gallo, Method 2 Spending Function",
    subtitle = "Conditional Error Spending Functions"
  )

	Analysis
Xi-Gallo, Method 2 Spending Function
Conditional Error Spending Functions
	1	2	3	4
gamma = 0.2
Z	3.016	2.350	2.208	2.224
¹ CE simple	0.204	0.213	0.267	NA
² CE	0.475	0.368	0.267	NA
gamma = 0.3
Z	3.516	2.574	2.239	2.097
¹ CE simple	0.348	0.348	0.376	NA
² CE	0.591	0.498	0.376	NA
gamma = 0.4
Z	3.940	2.774	2.295	2.044
¹ CE simple	0.466	0.454	0.455	NA
² CE	0.677	0.592	0.455	NA
gamma = 0.5
Z	4.333	2.963	2.359	2.014
¹ CE simple	0.570	0.546	0.523	NA
² CE	0.747	0.668	0.523	NA
gamma = 0.6
Z	4.724	3.152	2.429	1.995
¹ CE simple	0.664	0.629	0.586	NA
² CE	0.807	0.734	0.586	NA
gamma = 0.7
Z	5.141	3.353	2.509	1.982
¹ CE simple	0.751	0.709	0.648	NA
² CE	0.861	0.795	0.648	NA
gamma = 0.8
Z	5.627	3.588	2.604	1.973
¹ CE simple	0.834	0.788	0.714	NA
² CE	0.909	0.853	0.714	NA
¹ Conditional Error not accounting for future interim bounds.
² CE = Conditional Error accounting for all analyses.

Method 3

Method 3 spending functions are designed to approximate Pocock bounds with equal bounds on the Z-scale. Two common approximations used for this is the Hwang, Shih, and De Cani (1990) spending function with γ = 1

$$ \alpha_{HSD}(t, \gamma) = \alpha\frac{1-e^{-\gamma t}}{1 - e^{-\gamma}}. $$

and the Lan and DeMets (1983) spending function to approximate Pocock bounds

α_LDP(t) = αlog (1 + (e − 1)t).

We compare these methods in the following table. While the γ = 0.025 conditional error spending bounds are close to the targeted Pocock bounds, the γ = 0.05 conditional error spending bound is substantially higher than the targeted value at the first interim. The traditional Lan-DeMets and Hwang-Shih-DeCani approximations are quite good approximations of the Pocock bounds. The one number not reproduced from Xi and Gallo (2019) is the conditional error at the first analysis for γ = 0.05; while here we have computed 0.132, in the paper this value was 0.133. There are differences in the computation algorithms that may account for this difference. The method used in gsDesign is from Chapter 19 of Jennison and Turnbull (2000), specifically designed for numerical integration for group sequential trials. The method used in Xi and Gallo (2019) is a more general method approximating multivariate normal probabilities.

xPocock <- gsDesign(k = 4, test.type = 1, sfu = "Pocock")
xLDPocock <- gsDesign(k = 4, test.type = 1, sfu = sfLDPocock)
xHSD1 <- gsDesign(k = 4, test.type = 1, sfu = sfHSD, sfupar = 1)
x3.025 <- gsDesign(k = 4, test.type = 1, sfu = sfXG3, sfupar = 0.025)
x3.05 <- gsDesign(k = 4, test.type = 1, sfu = sfXG3, sfupar = 0.05)
xx <- rbind(
  transpose_df(ce(xPocock)) %>% mutate(gamma = "Pocock"),
  transpose_df(ce(xLDPocock)) %>% mutate(gamma = "Lan-DeMets to Approximate Pocock"),
  transpose_df(ce(xHSD1)) %>% mutate(gamma = "Hwang-Shih-DeCani, gamma = 1"),
  transpose_df(ce(x3.025)) %>% mutate(gamma = "gamma = 0.025"),
  transpose_df(ce(x3.05)) %>% mutate(gamma = "gamma = 0.05 ")
)
xx %>%
  gt(groupname_col = "gamma") %>%
  tab_spanner(label = "Analysis", columns = 2:5) %>%
  fmt_number(columns = 2:5, decimals = 3) %>%
  tab_options(data_row.padding = px(1)) %>%
  tab_footnote(
    footnote = "Conditional Error not accounting for future interim bounds.",
    locations = cells_stub(rows = seq(2, 11, 3))
  ) %>%
  tab_footnote(
    footnote = "CE = Conditional Error accounting for all analyses.",
    locations = cells_stub(rows = seq(3, 12, 3))
  ) %>%
  tab_header(
    title = "Xi-Gallo, Method 3 Spending Function",
    subtitle = "Conditional Error Spending Functions"
  )

	Analysis
Xi-Gallo, Method 3 Spending Function
Conditional Error Spending Functions
	1	2	3	4
Pocock
Z	2.361	2.361	2.361	2.361
¹ CE simple	0.086	0.164	0.263	NA
² CE	0.228	0.283	0.263	NA
Lan-DeMets to Approximate Pocock
Z	2.368	2.368	2.358	2.350
¹ CE simple	0.089	0.170	0.269	NA
² CE	0.230	0.289	0.269	NA
Hwang-Shih-DeCani, gamma = 1
Z	2.376	2.357	2.350	2.357
¹ CE simple	0.088	0.164	0.260	NA
² CE	0.235	0.286	0.260	NA
gamma = 0.025
Z	2.269	2.339	2.422	2.483
¹ CE simple	0.060	0.120	0.220	NA
² CE	0.196	0.230	0.220	NA
gamma = 0.05
Z	2.609	2.330	2.281	2.270
CE simple	0.132	0.189	0.278	NA
CE	0.328	0.318	0.278	NA
¹ Conditional Error not accounting for future interim bounds.
² CE = Conditional Error accounting for all analyses.

Summary

Xi and Gallo (2019) proposed conditional error spending functions for group sequential designs are implemented in the gsDesign package. When there is a single interim analysis, conditional error from Method 1 is almost exactly the same as the parameter γ but it allows a narrower range of γ. Method 2 provides less accurate approximation but allows a wider range of γ. Using γ = 0.5 replicates the Lan-DeMets spending function to approximate O’Brien-Fleming bounds. An exponential spending function provides a possibly better approximation of O’Brien-Fleming bounds. Simply selecting a smaller γ than the targeted conditional error may provide a better match for the targeted bounds. While Method 3 provides a reasonable approximation of a Pocock bound with equal bounds on the Z-scale, its stated objective, traditional approximations of Pocock bounds with the Lan-DeMets and Hwang-Shih-DeCani spending functions may be slightly better.

Results duplicated findings from the original paper.

References

Anderson, K. M., and J. Clark. 2010. “Group Sequential Design: Selecting Appropriate Bounds.” Journal of Clinical Oncology 28 (15_suppl): e13159–59.

Hwang, Irving K, Weichung J Shih, and John S De Cani. 1990. “Group Sequential Designs Using a Family of Type i Error Probability Spending Functions.” Statistics in Medicine 9 (12): 1439–45.

Jennison, Christopher, and Bruce W. Turnbull. 2000. Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman; Hall/CRC.

Lan, K. K. G., and David L. DeMets. 1983. “Discrete Sequential Boundaries for Clinical Trials.” Biometrika 70: 659–63.

Xi, Dong, and Paul Gallo. 2019. “An Additive Boundary for Group Sequential Designs with Connection to Conditional Error.” Statistics in Medicine 38 (23): 4656–69.