van Eeden seminar: An Automatic Finite-Sample Robustness Metric: Can Dropping a Little Data Change Conclusions?

Mardi, 22 mars, 2022

11:00 to 13:00

 

Van Eeden Invited Speaker: Tamara Broderick, Associate Professor, Massachusetts Institute of Technology

Event Type: Statistics Seminar

Location: Please register to receive the Zoom link.  

(If you have any questions about your registration or the seminar, please contact headsec@stat.ubc.ca)

 

Abstract:

One hopes that data analyses will be used to make beneficial decisions regarding people's health, finances, and well-being. But the data fed to an analysis may systematically differ from the data where these decisions are ultimately applied. For instance, suppose we analyze data in one country and conclude that microcredit is effective at alleviating poverty; based on this analysis, we decide to distribute microcredit in other locations and in future years. We might then ask: can we trust our conclusion to apply under new conditions? If we found that a very small percentage of the original data was instrumental in determining the original conclusion, we might expect the conclusion to be unstable under new conditions. So we propose a method to assess the sensitivity of data analyses to the removal of a very small fraction of the data set. Analyzing all possible data subsets of a certain size is computationally prohibitive, so we provide an approximation. We call our resulting method the Approximate Maximum Influence Perturbation. Our approximation is automatically computable, theoretically supported, and works for common estimators – including (but not limited to) OLS, IV, GMM, MLE, MAP, and variational Bayes. We show that any non-robustness our metric finds is conclusive. Empirics demonstrate that while some applications are robust, in others the sign of a treatment effect can be changed by dropping less than 0.1% of the data – even in simple models and even when standard errors are small.

 

Van Eeden speakers:

Professor Tamara Broderick has been invited by our department's graduate students to be this year's van Eeden speaker. A van Eeden speaker is a prominent statistician who is chosen by our graduate students each year to give a lecture, supported by the Constance van Eeden Fund.

 

For more details. please visite: https://www.stat.ubc.ca/van-eeden-seminar-automatic-finite-sample-robust...