Giuseppe
Emeritus
[Source: PLoS ONE, full text: (LINK). Abstract, edited.]
Assessing Google Flu Trends Performance in the United States during the 2009 Influenza Virus A (H1N1) Pandemic
Samantha Cook<SUP>1</SUP>, Corrie Conrad<SUP>2</SUP><SUP>*</SUP>, Ashley L. Fowlkes<SUP>3</SUP>, Matthew H. Mohebbi<SUP>1</SUP>
1 Google, Inc., New York, New York, United States of America, 2 Google, Inc., London, United Kingdom, 3 Influenza Division, Centers for Disease Control and Prevention, Atlanta, Georgia, United States of America
Abstract
Background
Google Flu Trends (GFT) uses anonymized, aggregated internet search activity to provide near-real time estimates of influenza activity. GFT estimates have shown a strong correlation with official influenza surveillance data. The 2009 influenza virus A (H1N1) pandemic [pH1N1] provided the first opportunity to evaluate GFT during a non-seasonal influenza outbreak. In September 2009, an updated United States GFT model was developed using data from the beginning of pH1N1.
Methodology/Principal Findings
We evaluated the accuracy of each U.S. GFT model by comparing weekly estimates of ILI (influenza-like illness) activity with the U.S. Outpatient Influenza-like Illness Surveillance Network (ILINet). For each GFT model we calculated the correlation and RMSE (root mean square error) between model estimates and ILINet for four time periods: pre-H1N1, Summer H1N1, Winter H1N1, and H1N1 overall (Mar 2009?Dec 2009). We also compared the number of queries, query volume, and types of queries (e.g., influenza symptoms, influenza complications) in each model. Both models' estimates were highly correlated with ILINet pre-H1N1 and over the entire surveillance period, although the original model underestimated the magnitude of ILI activity during pH1N1. The updated model was more correlated with ILINet than the original model during Summer H1N1 (r = 0.95 and 0.29, respectively). The updated model included more search query terms than the original model, with more queries directly related to influenza infection, whereas the original model contained more queries related to influenza complications.
Conclusions
Internet search behavior changed during pH1N1, particularly in the categories ?influenza complications? and ?term for influenza.? The complications associated with pH1N1, the fact that pH1N1 began in the summer rather than winter, and changes in health-seeking behavior each may have played a part. Both GFT models performed well prior to and during pH1N1, although the updated model performed better during pH1N1, especially during the summer months.
Citation: Cook S, Conrad C, Fowlkes AL, Mohebbi MH (2011) Assessing Google Flu Trends Performance in the United States during the 2009 Influenza Virus A (H1N1) Pandemic. PLoS ONE 6(8): e23610. doi:10.1371/journal.pone.0023610
Editor: Benjamin J. Cowling, University of Hong Kong, Hong Kong
Received: April 15, 2011; Accepted: July 21, 2011; Published: August 19, 2011
This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Funding: This research was funded by Google.org, the non-profit arm of Google Inc. This funder played no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Three of the authors (SC, CC, MM) are employees of one of the funders of the study (Google Inc.) and were involved with the study design, data collection and analysis, decision to publish, and preparation of the manuscript.
Competing interests: Yes, the authors have the following competing interest. This study was supported by funding from Google Inc., and three of the authors (SC, CC, MM) are employees of Google Inc. There are no patents, products in development or marketed products to declare. This does not alter the authors' adherence to all the PLoS ONE policies on sharing data and materials, as detailed online in the guide for authors.
* E-mail: cconrad@google.com
- -------Samantha Cook<SUP>1</SUP>, Corrie Conrad<SUP>2</SUP><SUP>*</SUP>, Ashley L. Fowlkes<SUP>3</SUP>, Matthew H. Mohebbi<SUP>1</SUP>
1 Google, Inc., New York, New York, United States of America, 2 Google, Inc., London, United Kingdom, 3 Influenza Division, Centers for Disease Control and Prevention, Atlanta, Georgia, United States of America
Abstract
Background
Google Flu Trends (GFT) uses anonymized, aggregated internet search activity to provide near-real time estimates of influenza activity. GFT estimates have shown a strong correlation with official influenza surveillance data. The 2009 influenza virus A (H1N1) pandemic [pH1N1] provided the first opportunity to evaluate GFT during a non-seasonal influenza outbreak. In September 2009, an updated United States GFT model was developed using data from the beginning of pH1N1.
Methodology/Principal Findings
We evaluated the accuracy of each U.S. GFT model by comparing weekly estimates of ILI (influenza-like illness) activity with the U.S. Outpatient Influenza-like Illness Surveillance Network (ILINet). For each GFT model we calculated the correlation and RMSE (root mean square error) between model estimates and ILINet for four time periods: pre-H1N1, Summer H1N1, Winter H1N1, and H1N1 overall (Mar 2009?Dec 2009). We also compared the number of queries, query volume, and types of queries (e.g., influenza symptoms, influenza complications) in each model. Both models' estimates were highly correlated with ILINet pre-H1N1 and over the entire surveillance period, although the original model underestimated the magnitude of ILI activity during pH1N1. The updated model was more correlated with ILINet than the original model during Summer H1N1 (r = 0.95 and 0.29, respectively). The updated model included more search query terms than the original model, with more queries directly related to influenza infection, whereas the original model contained more queries related to influenza complications.
Conclusions
Internet search behavior changed during pH1N1, particularly in the categories ?influenza complications? and ?term for influenza.? The complications associated with pH1N1, the fact that pH1N1 began in the summer rather than winter, and changes in health-seeking behavior each may have played a part. Both GFT models performed well prior to and during pH1N1, although the updated model performed better during pH1N1, especially during the summer months.
Citation: Cook S, Conrad C, Fowlkes AL, Mohebbi MH (2011) Assessing Google Flu Trends Performance in the United States during the 2009 Influenza Virus A (H1N1) Pandemic. PLoS ONE 6(8): e23610. doi:10.1371/journal.pone.0023610
Editor: Benjamin J. Cowling, University of Hong Kong, Hong Kong
Received: April 15, 2011; Accepted: July 21, 2011; Published: August 19, 2011
This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Funding: This research was funded by Google.org, the non-profit arm of Google Inc. This funder played no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Three of the authors (SC, CC, MM) are employees of one of the funders of the study (Google Inc.) and were involved with the study design, data collection and analysis, decision to publish, and preparation of the manuscript.
Competing interests: Yes, the authors have the following competing interest. This study was supported by funding from Google Inc., and three of the authors (SC, CC, MM) are employees of Google Inc. There are no patents, products in development or marketed products to declare. This does not alter the authors' adherence to all the PLoS ONE policies on sharing data and materials, as detailed online in the guide for authors.
* E-mail: cconrad@google.com