Limited impact of lifting universal masks on SARS-CoV-2 transmission in schools: The crucial role of outcome measurements (2024)

,

Article history

Abstract

Introduction

Nonpharmaceutical interventions (NPIs) have played a critical role in reducing transmission during the COVID-19 pandemic. One such NPI, face mask mandates, was commonly used during the COVID-19 pandemic. Although challenges exist in examining the effectiveness of face masks in preventing disease transmissions, they were later suggested to be effective in mitigating COVID-19 transmissions and adopted by schools (1–3). As the pandemic progresses, many local authorities relaxed mask mandates during periods when case counts appeared to be declining, while the impact of lifting these mandates on transmission remains under-investigated.

However, oversights of transmission outcomes have been identified in studies about effectiveness of other COVID-19 control measures (4, 5), which may have impacted the accuracy of the findings and their implications. Using incidence-related outcome measures alone may not fully reflect the transmission process, since it does not account for the exponential changes in case numbers during an epidemic (Fig. 1). Incidence discrepancies between two locations can magnify over time, even if the reproductive number remains constant. To demonstrate the potential impact of choices of outcome measurements, we used the same data source as Cowger et al. to examine the impact of outcome measure (i.e. transmission and/or incidence) on the estimated effectiveness of lifting masks mandates on COVID-19 transmission (6, 7). After replicating the original analyses using a difference-in-difference (DID) model on incidence rate, we replaced the outcome measurement with the effective reproductive number (R_t), which measures the within school transmission using the school population and case data. R_t is estimated as the average number of secondary infections resulting from one infected individual, measuring the transmissibility in a population (8). R_t below 1 indicates nonsustainable transmission under existing measures, and its reduction demonstrates how effectively the mitigation measure can limit the virus transmission.

Methodology

We derived the data on masking lifting policies, student population, and weekly COVID-19 cases from the Massachusetts Department of Elementary and Secondary Education, following the data cleaning process outlined previously (5). We collected data from 72 school districts in Massachusetts, with 46 districts (type 1) lifting the mask mandates in the first reporting week (starting on 2022 March 3), and 15 (type 2) and 9 (type 3) districts lifting in the second (starting on 2022 March 10) and third (starting on 2022 March 17) reporting week, respectively. Chelsea and Boston (type 4) retained their mandates throughout entire study period. We only replicated results for students, as data on school staff were not available. In this case, our finding can only reflect the within school transmission rather than the community and household transmission.

We first replicated Cowger et al.'s work by implementing a DID method to gauge the impact of lifting mask mandates on the cumulative incidence of COVID-19 among students. The average treatment effect of the staggered intervention implementation across different groups was estimated using the DID model (9, 10). Although the DID model is unable to capture time-varying covariates, related covariates, including community transmission rate and vaccination rate, were similar across all four types of districts (5). The “did” package in R was used for the analysis.

We replicated the DID analysis by replacing the outcome measurement with time-varying reproductive number (R_t). To estimate R_t, we back projected the incidence using the surveillance package in R, assuming the delay of reporting is a composite of the incubation period and the onset of reporting delay (11). Based on the back projected incidence, we used the EpiEstim package to estimate R_t. We take the estimated daily median R_t as the measurement variable in our DID model. Our main analysis employed a serial interval of 4.4 days (95% CI: 2.9–6.7 days) (12). To examine the impact of changing serial intervals and incubation period on our results, we considered two alternative serial intervals: 5.3 days (95% CI: 4.7–5.9 days) and 6.3 days (95% CI: 5.2–7.6 days) and another incubation period distribution used for case-back projection of 6.4 days (95% CI: 3.04–11.9 days) (13–16). In our sensitivity analysis, we also performed a population-weighted DID, with staggered lifting of mask mandates as the intervention and student R_t as the outcome variable.

Results

Our replication results were consistent with those reported by Cowger et al. We found that lifting the mask mandates in schools was associated with a notable increase in COVID-19 incidence across all district types (Fig. 2A), with the lowest incidence in type 4 district that did not lift the mask mandate. We observed an average treatment effect on treated (ATT) of 39.1 (95% CI: 20.4–57.4) COVID-19 cases per 1,000 students associated with lifting mask mandates, compared with 39.9 (95% CI: 24.3–55.4) in the original analysis (5). Analyses using incidence as outcomes suggested that lifting the mask mandates in schools was associated with 39.1 additional cases per 1,000 students within a 15-week period (Fig. 2C).

We further analyzed the impact of lifting the mask mandates on changes in R_t, which measures COVID-19 transmission intensity. Contrary to the original results using incidence rates, we found no significant difference in R_t across district types, except for type 4 district, which showed a slightly higher R_t in the first 2 weeks after lifting the mask mandates (Fig. 2B). There was no association between lifting mask mandates and reductions in R_t (ATT 0.04, 95% CI: −0.11 to 0.20) throughout the entire postlifting period (Fig. 2D). Moreover, R_t remained below 1 from February to May, indicating a general decline in the epidemic during that time until a new wave of the pandemic in late April (Fig. 2B). Our primary analysis relied on the mean value of R_t from its range. We further assessed the robustness of our findings by implementing a bootstrapping method, generating 100 samples from the confidence interval of R_t estimations for DID analysis. The ATT of the bootstrapping DID was 0.04 (95% CI: −0.10 to 0.19) and 0.01 (95% CI: −0.02 to 0.03) after population weighting, which was consistent with our main analysis. This confirmed that the variability in R_t estimations does not undermine our main results as the result of DID is still insignificant.

Sensitivity analysis yielded consistent results when using longer serial intervals (13, 16), weighting DID model with populations (9, 10) and a different delay distribution (Table S1). Our DID analysis satisfied the parallel trend assumption and, according to Cowger et al., other time-varying variables, including vaccination rate and community transmission, remained stable during the study period.

Discussion

We found no evidence that lifting mask mandates in Massachusetts schools significantly affected COVID-19 transmission rates, which is contrary to the findings reported by Cowger et al. Our findings demonstrate that substantial changes in incidence or case numbers do not necessarily reflect substantial changes in underlying transmission dynamics. Additionally, we found nonsustainable transmission (i.e. R_t < 1) across all school districts before mask mandates were lifted, suggesting that factors other than lifting mask mandates impacted COVID-19 transmission in these schools, such as community transmission and the implementation of other measures (e.g. extensive testing) (17–20).

Our findings highlight the importance of considering transmissibility outcomes when assessing the effectiveness of interventions against disease transmission (5). While as count-based outcomes (e.g. incidence) can serve as proxies for the difficult-to-measure transmission process, it is crucial to note that NPIs work by reducing person-to-person transmission and subsequent incidences. Due to exponential case growth and delays in disease development (e.g. incubation periods), changes in case counts may exaggerate and lag changes in transmission. Therefore, caution is necessary when interpreting the effects of interventions based solely on analyses of incidence rates.

Estimating the force of infection from a transmission model is ideal to assess changes in but is challenging due to unknown preexisting population immunity. However, study (21) has suggested that R_t could be a feasible and reliable measurement in DID analysis, when there is no trivial depletion of susceptible in a short period (e.g. no major outbreaks).

Our study has some limitations. Firstly, we were unable to account for unobserved heterogeneities, such as mask types and compliance with mask mandates. Therefore, our findings reflect the impact of the lifting of mask mandate policies rather than the direct effects of mask wearing (22, 23). Secondly, we only estimated R_t using data on student cases, and we were unable to account for factors such as transmission between students and staff and external community due to data availability.

In summary, we estimated a limited impact of lifting mask mandates on reducing COVID-19 transmission in Massachusetts schools in February 2021. Future assessments of the effects of interventions against transmission should consider including transmissibility outcomes.

Supplementary Material

Supplementary material is available at PNAS Nexus online.

Funding

This work was financially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region, China (project no. T11-705/21-N).

Author Contributions

All authors meet the ICMJE criteria for authorship. The study was conceived by M.L., B.Y., and B.J.C. Data analyses were done by M.L. M.L. and B.Y. wrote the first draft of the manuscript, and all authors provided critical review and revision of the text and approved the final version.

Preprints

This manuscript was posted on a preprint: https://www.medrxiv.org/content/10.1101/2023.08.26.23294658v1

Data Availability

The data of positive cases in school are available at https://archives.lib.state.ma.us/collections/ff084038-2e48-47ef-8bd0-6b6e3281fe3c. The data of student population in Massachusetts school districts are available at https://profiles.doe.mass.edu/statereport/#Enrollment.

References

Author notes

Competing Interests: B.J.C. consults for AstraZeneca, Fosun Pharma, GlaxoSmithKline, Haleon, Moderna, Novavax, Roche, Sanofi Pasteur, and Pfizer. M.L. and B.Y. report no potential conflicts of interest.

© The Author(s) 2024. Published by Oxford University Press on behalf of National Academy of Sciences.

Issue Section:

Brief Report

Email alerts

Citing articles via

More from Oxford Academic