Balancing the response and adjusting estimates for nonresponse bias: complementary activities

Särndal, Carl-Erik; Lundquist, Peter

Journal de la société française de statistique, Tome 155 (2014) no. 4, pp. 28-50.

Un des objectifs d’une collecte adaptative de données est de profiter d’une planification et d’une intervention appropriées, afin d’obtenir au final un ensemble de répondants bien équilibré ou bien représentatif. A ce stade, l’information auxiliaire, qui inclue les paradonnées, joue un rôle central. Mais quoique l’on puisse accomplir durant la période de la collecte, le but ultime est d’obtenir des estimations précises. Au stade de l’estimation, les variables auxiliaires jouent également un rôle important, comme lorsque des poids calés sont utilisés pour réduire le biais de non-réponse qui affecte néanmoins les estimations.

Le concept de déséquilibre de la réponse est central dans cet article. Nous définissons et nous mesurons ses composantes, le déséquilibre total, marginal ou conditionnel. Nous proposons des méthodes basées sur la propension (ou l’intensité) de la réponse, observable de façon continue pendant la collecte de données, dans le but d’obtenir une réponse ultime bien équilibrée. Nous appliquons ces méthodes à des données d’une importante enquête suédoise, et nous examinons dans quelle mesure une réduction bien réussie du déséquilibre peut contribuer à réduire le biais, au-delà de ce qu’un ajustement par calage peut apporter.

One objective of Responsive Design is to manage the data collection through appropriate planning and intervention, so as to promote in the end a well-balanced or well representative set of respondents. At that stage, auxiliary information, including paradata, plays a crucial role. But regardless of what can be accomplished during data collection, accurate estimation is the ultimate goal. The auxiliary variables play an important role at that stage as well, as when calibrated weights are used for adjustment in order to reduce the nonresponse bias that nevertheless affects the estimates.

The concept of imbalance of the survey response is central in this article. We define and measure its components, total, marginal and conditional imbalance. We propose methods based on response propensity, observed continuously throughout the data collection, for obtaining a well-balanced ultimate response. We apply the methods to data from a major Swedish survey, and we explore how a successful reduction of imbalance may contribute further to reducing the bias of estimates, over and beyond what calibration adjustment will accomplish in that regard.

[1] Andridge, R.R.; Little, R.J. Proxy pattern-mixture analysis for survey nonresponse, Journal of Official Statistics, Volume 27 (2011) no. 2, pp. 153-180

[2] Bethlehem, J.; Cobben, F.; Schouten, B. Handbook of nonresponse in household surveys, John Wiley & Sons, 2011

[3] Brick, M.E.; Jones, J.M. Propensity to respond and nonresponse bias, Metron, Volume 66 (2008) no. 1, pp. 51-73 | Zbl

[4] Couper, M.P.; Wagner, J. Using paradata and responsive design to manage survey nonresponse, Unpublished manuscript (2012)

[5] Groves, R.M.; Heeringa, S.G. Responsive design for household surveys: tools for actively controlling survey errors and costs, Journal of the Royal Statistical Society: Series A (Statistics in Society), Volume 169 (2006) no. 3, pp. 439-457 | MR

[6] Groves, R.M. Nonresponse rates and nonresponse bias in household surveys, Public Opinion Quarterly, Volume 70 (2006) no. 5, pp. 646-675

[7] Heckman, J.J. Sample selection bias as a specification error, Econometrica, Volume 47 (1979) no. 1, pp. 153-161 | MR | Zbl

[8] Kreuter, F.; Olson, K.; Wagner, J.; Yan, T.; Ezzati-Rice, T.M.; Casas-Cordero, C.; Lemay, M.; Peytchev, A.; Groves, R.M.; Raghunathan, T.E. Using proxy measures and other correlates of survey outcomes to adjust for non-response: examples from multiple surveys, Journal of the Royal Statistical Society: Series A (Statistics in Society), Volume 173 (2010) no. 2, pp. 389-407 | MR

[9] Laflamme, F. Experiences in assessing, monitoring and controlling survey productivity and costs at Statistics Canada, Proceedings from the 57th International Statistical Institute Conference (2009)

[10] Lundquist, P.; Särndal, C.-E. Aspects of responsive design with applications to the Swedish Living Conditions Survey, Journal of Official Statistics, Volume 29 (2013) no. 4, pp. 557-582

[11] Little, R.J.; Vartivarian, S. Does Weighting for Nonresponse Increase the Variance of Survey Means?, Survey Methodology, Volume 31 (2005) no. 2, pp. 161-168

[12] Mohl, C.; Laflamme, F. Research and responsive design options for survey data collection at Statistics Canada, Proceedings of the American Statistical Association, Section on Survey Research Methods (2007)

[13] Peytcheva, E.; Groves, R.M. Using variation in response rates of demographic subgroups as evidence of nonresponse bias in survey estimates, Journal of Official Statistics, Volume 25 (2009) no. 2, pp. 193-201

[14] Peytchev, A.; Riley, S.; Rosen, J.; Murphy, J.; Lindblad, M. Reduction of nonresponse bias through case prioritization, Survey Research Methods, Volume 4 (2010) no. 1, pp. 21-29

[15] Rao, R.S.; Glickman, M.E.; Glynn, R.J. Stopping rules for surveys with multiple waves of nonrespondent follow-up, Statistics in Medicine, Volume 27 (2008) no. 12, pp. 2196-2213 | MR

[16] Schouten, B.; Bethlehem, J. Representativeness indicators for measuring and enhancing the composition of survey response, RISQ deliverables, Work package 8, deliverable 9, www.risq-project.eu (2009)

[17] Schouten, B.; Cobben, F.; Bethlehem, J. Indicators for the representativeness of survey response, Survey Methodology, Volume 35 (2009) no. 1, pp. 101-113

[18] Särndal, C.-E.; Lundström, S. Assessing auxiliary vectors for control of nonresponse bias in the calibration estimator, Journal of Official Statistics, Volume 24 (2008) no. 2, pp. 167-191

[19] Särndal, C.-E.; Lundström, S. Design for estimation: Identifying auxiliary vectors to reduce nonresponse bias, Survey Methodology, Volume 36 (2010) no. 2, pp. 131-144

[20] Schouten, B.; Shlomo, N.; Skinner, C. Indicators for Monitoring and Improving Representativeness of Response, Journal of Official Statistics, Volume 27 (2011) no. 2, pp. 1-24

[21] Särndal, C.-E. Dealing with survey nonresponse in data collection, in estimation, Journal of Official Statistics, Volume 27 (2011) no. 1, pp. 1-21

[22] Särndal, C.-E. Three Factors to Signal Non-Response Bias With Applications to Categorical Auxiliary Variables, International statistical review, Volume 79 (2011) no. 2, pp. 233-254 | Zbl

[23] Vartivarian, S.; Little, R.J. On the formation of weighting adjustment cells for unit nonresponse, Proceedings of the American Statistical Association, Section on Survey Research Methods (2002)

[24] Wagner, J.R. Adaptive survey design to reduce nonresponse bias, Ph.D. Thesis, University of Michigan, Ann Arbor, 2008 | MR

[25] Wagner, J.R. A comparison of alternative indicators for the risk of nonresponse bias, Public Opinion Quarterly, Volume 76 (2012), pp. 555-575

[26] Wagner, J.R.; Raghunathan, T.E. A new stopping rule for surveys, Statistics in Medicine, Volume 29 (2010) no. 9, pp. 1014-1024 | MR