tetano
Editor, Senior Moderator
JAMA Netw Open
. 2025 May 1;8(5):e2512763.
doi: 10.1001/jamanetworkopen.2025.12763. Evaluating the Test-Negative Design for COVID-19 Vaccine Effectiveness Using Randomized Trial Data: A Secondary Cross-Protocol Analysis of 5 Randomized Clinical Trials
Leah I B Andrews[SUP] 1 [/SUP], M Elizabeth Halloran[SUP] 1 2 [/SUP], Kathleen M Neuzil[SUP] 3 [/SUP], Lars van der Laan[SUP] 4 [/SUP], Yunda Huang[SUP] 2 [/SUP], Jessica Andriesen[SUP] 2 [/SUP], Mayur Patel[SUP] 2 [/SUP], Leigh H Fisher[SUP] 2 [/SUP], Holly Janes[SUP] 2 [/SUP], Nadine Rouphael[SUP] 5 [/SUP], Stephen R Walsh[SUP] 6 [/SUP], Deborah A Theodore[SUP] 7 [/SUP], Hong-Van Tieu[SUP] 7 8 [/SUP], Magdalena Sobieszczyk[SUP] 7 [/SUP], Hana M El Sahly[SUP] 9 [/SUP], Lindsey R Baden[SUP] 6 [/SUP], Ann R Falsey[SUP] 10 [/SUP], Thomas B Campbell[SUP] 11 [/SUP], Colleen F Kelley[SUP] 5 [/SUP], Catherine Mary Healy[SUP] 12 [/SUP], Lilly Immergluck[SUP] 13 [/SUP], Benjamin Luft[SUP] 14 [/SUP], Ian Hirsch[SUP] 15 [/SUP], Guy de Bruyn[SUP] 16 [/SUP], Carla Truyers[SUP] 17 [/SUP], Frances Priddy[SUP] 18 [/SUP], Kelsey M Sumner[SUP] 19 [/SUP], Brendan Flannery[SUP] 19 [/SUP], Dean Follmann[SUP] 20 [/SUP], Peter B Gilbert[SUP] 1 2 [/SUP]; COVID-19 Prevention Network (CoVPN)
Collaborators, Affiliations
Importance: The test-negative design (TND) has been widely used to assess postmarketing COVID-19 vaccine effectiveness but requires further evaluation for this application.
Objective: To determine whether the TND reliably evaluates vaccine effectiveness against symptomatic COVID-19 using placebo-controlled vaccine efficacy randomized clinical trials (RCTs).
Design, setting, and participants: This secondary cross-protocol analysis constructed TND study datasets from study sites in 16 countries across 5 continents using the blinded phase cohorts of 5 harmonized phase 3 COVID-19 Prevention Network RCTs: COVE (Coronavirus Vaccine Efficacy and Safety), AZD1222, ENSEMBLE, PREVENT-19 (Prefusion Protein Subunit Vaccine Efficacy Novavax Trial COVID-19), and VAT00008. Participants included adults who received the intended number of doses, experienced COVID-19-like symptoms, and obtained SARS-CoV-2 testing. Start dates ranged from July 27, 2020, to October 19, 2021; data cutoff dates ranged from March 26, 2021, to March 15, 2022. Statistical analysis was performed from May 11, 2023, to February 25, 2025.
Interventions: Participants received vaccines consisting of messenger RNA-1273 (COVE; 2 doses 28 days apart), ChAdOx1 nCoV-19 (AZD1222; 2 doses 28 days apart), Ad26.COV2.S (ENSEMBLE; 1 dose), NVX-CoV2373 (PREVENT-19; 2 doses 21 days apart), CoV2 preS dTM-AS03 (VAT00008; D614) (2 doses 21 days apart), or CoV2 preS dTM-AS03 (D614 plus B.1.351) (VAT00008; 2 doses 21 days apart) or placebo.
Main outcomes and measures: Main outcomes were symptomatic COVID-19 according to each trial's primary efficacy definition and the Centers for Disease Control and Prevention definition. Vaccine effectiveness was estimated using targeted maximum likelihood estimation under a semiparametric logistic regression model and ordinary logistic regression. Noncase exchangeability, a core TND assumption for unbiased estimation, was also assessed by estimating vaccine efficacy against non-COVID-19 illness.
Results: Among the 12 157 participants included in the analysis, mean (SD) age was 45 (15) years, 6414 were female (53%), 5858 were vaccinated (48%), 2835 experienced primary COVID-19 (23%), and 2992 experienced Centers for Disease Control and Prevention-defined COVID-19 (25%). TND vaccine effectiveness estimates were concordant with RCT vaccine efficacy estimates (concordance correlation coefficient, 0.86 [95% CI, 0.58-0.96] for both outcomes). The semiparametric method had 48% smaller variance estimates than ordinary logistic regression. Noncase exchangeability was generally supported with a median vaccine efficacy against non-COVID-19 illness of 7.7% (IQR, 2.7%-16.8%) across trial cohorts and most 95% CIs including 0.
Conclusions and relevance: In this cross-protocol analysis, the TND provided reliable inferences on COVID-19 vaccine effectiveness in health care-seeking populations for multiple vaccines and symptom definitions when confounding and selection bias were absent. A machine-learning approach for robust confounding control in postmarketing TND studies was also introduced.
. 2025 May 1;8(5):e2512763.
doi: 10.1001/jamanetworkopen.2025.12763. Evaluating the Test-Negative Design for COVID-19 Vaccine Effectiveness Using Randomized Trial Data: A Secondary Cross-Protocol Analysis of 5 Randomized Clinical Trials
Leah I B Andrews[SUP] 1 [/SUP], M Elizabeth Halloran[SUP] 1 2 [/SUP], Kathleen M Neuzil[SUP] 3 [/SUP], Lars van der Laan[SUP] 4 [/SUP], Yunda Huang[SUP] 2 [/SUP], Jessica Andriesen[SUP] 2 [/SUP], Mayur Patel[SUP] 2 [/SUP], Leigh H Fisher[SUP] 2 [/SUP], Holly Janes[SUP] 2 [/SUP], Nadine Rouphael[SUP] 5 [/SUP], Stephen R Walsh[SUP] 6 [/SUP], Deborah A Theodore[SUP] 7 [/SUP], Hong-Van Tieu[SUP] 7 8 [/SUP], Magdalena Sobieszczyk[SUP] 7 [/SUP], Hana M El Sahly[SUP] 9 [/SUP], Lindsey R Baden[SUP] 6 [/SUP], Ann R Falsey[SUP] 10 [/SUP], Thomas B Campbell[SUP] 11 [/SUP], Colleen F Kelley[SUP] 5 [/SUP], Catherine Mary Healy[SUP] 12 [/SUP], Lilly Immergluck[SUP] 13 [/SUP], Benjamin Luft[SUP] 14 [/SUP], Ian Hirsch[SUP] 15 [/SUP], Guy de Bruyn[SUP] 16 [/SUP], Carla Truyers[SUP] 17 [/SUP], Frances Priddy[SUP] 18 [/SUP], Kelsey M Sumner[SUP] 19 [/SUP], Brendan Flannery[SUP] 19 [/SUP], Dean Follmann[SUP] 20 [/SUP], Peter B Gilbert[SUP] 1 2 [/SUP]; COVID-19 Prevention Network (CoVPN)
Collaborators, Affiliations
- PMID: 40434773
- DOI: 10.1001/jamanetworkopen.2025.12763
Importance: The test-negative design (TND) has been widely used to assess postmarketing COVID-19 vaccine effectiveness but requires further evaluation for this application.
Objective: To determine whether the TND reliably evaluates vaccine effectiveness against symptomatic COVID-19 using placebo-controlled vaccine efficacy randomized clinical trials (RCTs).
Design, setting, and participants: This secondary cross-protocol analysis constructed TND study datasets from study sites in 16 countries across 5 continents using the blinded phase cohorts of 5 harmonized phase 3 COVID-19 Prevention Network RCTs: COVE (Coronavirus Vaccine Efficacy and Safety), AZD1222, ENSEMBLE, PREVENT-19 (Prefusion Protein Subunit Vaccine Efficacy Novavax Trial COVID-19), and VAT00008. Participants included adults who received the intended number of doses, experienced COVID-19-like symptoms, and obtained SARS-CoV-2 testing. Start dates ranged from July 27, 2020, to October 19, 2021; data cutoff dates ranged from March 26, 2021, to March 15, 2022. Statistical analysis was performed from May 11, 2023, to February 25, 2025.
Interventions: Participants received vaccines consisting of messenger RNA-1273 (COVE; 2 doses 28 days apart), ChAdOx1 nCoV-19 (AZD1222; 2 doses 28 days apart), Ad26.COV2.S (ENSEMBLE; 1 dose), NVX-CoV2373 (PREVENT-19; 2 doses 21 days apart), CoV2 preS dTM-AS03 (VAT00008; D614) (2 doses 21 days apart), or CoV2 preS dTM-AS03 (D614 plus B.1.351) (VAT00008; 2 doses 21 days apart) or placebo.
Main outcomes and measures: Main outcomes were symptomatic COVID-19 according to each trial's primary efficacy definition and the Centers for Disease Control and Prevention definition. Vaccine effectiveness was estimated using targeted maximum likelihood estimation under a semiparametric logistic regression model and ordinary logistic regression. Noncase exchangeability, a core TND assumption for unbiased estimation, was also assessed by estimating vaccine efficacy against non-COVID-19 illness.
Results: Among the 12 157 participants included in the analysis, mean (SD) age was 45 (15) years, 6414 were female (53%), 5858 were vaccinated (48%), 2835 experienced primary COVID-19 (23%), and 2992 experienced Centers for Disease Control and Prevention-defined COVID-19 (25%). TND vaccine effectiveness estimates were concordant with RCT vaccine efficacy estimates (concordance correlation coefficient, 0.86 [95% CI, 0.58-0.96] for both outcomes). The semiparametric method had 48% smaller variance estimates than ordinary logistic regression. Noncase exchangeability was generally supported with a median vaccine efficacy against non-COVID-19 illness of 7.7% (IQR, 2.7%-16.8%) across trial cohorts and most 95% CIs including 0.
Conclusions and relevance: In this cross-protocol analysis, the TND provided reliable inferences on COVID-19 vaccine effectiveness in health care-seeking populations for multiple vaccines and symptom definitions when confounding and selection bias were absent. A machine-learning approach for robust confounding control in postmarketing TND studies was also introduced.