NOTE 1: For interested students, we can send, by email, an electronic version of most of the references cited below. Please, write us an email to obtain a copy of the papers.
 

NOTE 2: The Imputation Bulletin is a newsletter produced by Statistics Canada twice a year.

Objective

Despite the best efforts made by survey staff to maximize response, it is almost certain that some degree of nonresponse will occur in large scale surveys. Essentially, survey statisticians distinguish between two types of nonresponse, total or unit nonresponse (when no information is collected on a sampled unit) and partial or item nonresponse (when the absence of information is limited to some variables only). Unit nonresponse occurs, for example, when the sampled unit is not-at-home or refuses to participate in the survey. Item nonresponse may occur if the sampled unit refuses to respond to sensitive items, does not know the answer to some items, or because of edit failures. Generally, weighting adjustment methods are used to compensate for unit nonresponse whereas imputation is used to compensate for item nonresponse. The main idea behind a weighting adjustment is to increase the sampling weights of the respondents in order to compensate for the nonrespondents, while imputation is a process where one or more ”plausible values” are produced to replace a missing value. It is customary in both weighting and imputation to first classify respondents and nonrespondents into classes, formed on the basis of information recorded for all units in the sample. The main effects of (unit or item) nonresponse include: (i) bias of point estimators; (ii) increase of the variance of point estimators (since the observed sample size is smaller than the sample size initially planned) and (iii) bias of the complete-data variance estimators. The main objective when treating (unit or item) nonresponse is the reduction of the nonresponse bias, which occurs if respondents and nonrespondents are different with respect to the survey variables.
 

The student assignment is to find imputation strategies that can reduce the nonresponse bias as much as possible. Also, the student will consider the problem of variance estimation in the presence of imputed data and compare the results with those obtained when the imputed values are treated as if they were observed. A list of questions is given on page 10.

Data:

• Data file (SAS)
• Data file (text)
• Record layout for text data file (text)
• Variable description (MS Excel)

The data set used for the case study is a subset of a sample collected between January 2005 and December 2005 for the Canadian Community Health Survey (CCHS), which is a cross-sectional survey that collects information related to health status, health care utilization and health determinants for the Canadian population. The original data file was a Public Use Microdata File (PUMF) obtained for the CCHS Cycle 3.1 (2005) consisting of 132,221 records containing 1284 variables. We selected a relatively small subset of variables. Records with missing or incomplete responses were dropped, which reduced the number of records. This led to the creation of an artificial population file (consisting of complete records) containing 97,035 records. This population was then stratified by province / territory, and a stratified simple random sample without replacement of 20,000 records was taken for the case study sample.
 

In the data file, the variable samplingweight denotes the sampling weight of an individual, which is defined as the inverse of its inclusion probability in the sample. Let whi be the sampling weight of an individual i in stratum h. We have whi = Nh nh , where Nh is the number of individuals in stratum h and nh is the number of individuals sampled in stratum h.
 

Finally, missing values to the variable BMI (body mass index) were generated according to a specified nonresponse mechanism (that is only known to the organizing team). Note that, in practice, it is highly unusual to have nonresponse to just a single variable. For simplicity, we chose to create missing values to one variable only. Also, the nonresponse mechanism used to create missing values is artificial in the sense that it is likely different from the true unknown nonresponse mechanism observed in practice.
 

For more details on stratified sampling, see Lohr (1999), Särndal, Swensson and Wretman (1992) and Haziza (2007a). Also, you can obtain information about the CCHS on the Statistics Canada website.

Background

1. Single Imputation 
   
   We distinguish between single and multiple imputation. Single imputation consists of creating a single imputed value to replace a missing value which leads to the creation of a single complete data file. Multiple imputation, proposed by Rubin (1978, 1987), consists of creating M ≥ 2 imputed values to fill in a missing value which leads to the creation of M complete data files. Multiple imputation is discussed in section 14.
   
   Single imputation is widely used in surveys for treating item nonresponse because it presents the following advantages: (1) It leads to the creation of a complete data file; (2) unlike weighting adjustment for each item, imputation allows for the use of a single sampling weight for all items and (3) the results of different analyses are consistent with each other.
   
   It is nonetheless important to note that imputation presents certain risks. The most significant risks are as follows: (1) Even though imputation leads to the creation of a complete data file, inferences are valid only if the underlying assumptions about the response mechanism and/or the imputation model are satisfied. (2) Some imputation methods tend to distort the distribution of the variables of interest (i.e., the variables being imputed). (3) Treating the imputed values as if they were observed may lead to a substantial underestimation of the variance of the estimator, especially if the item nonresponse rate is appreciable. (4) Marginal imputation for each item separately has the effect of distorting relationships between variables.
    
2. Where there is no nonresponse… 
   
   In the absence of nonresponse, survey samplers usually try to avoid using estimation procedures whose validity depends on that of a given model. To avoid assumptions on the distribution of the data, the properties of estimators are generally based on the sampling design used to select the sample rather than on a particular model. This approach is the so-called design-based approach or randomization approach to inference. This does not mean that models are useless under the design-based approach. In fact, they play an important role in the determination of efficient sampling and estimation procedures. For details on point and variance estimation in the case of complete data, see Lohr (1999), Särndal, Swensson and Wretman (1992) and Haziza (2007a).
   
   Unlike in the case of complete response, the use of models is unavoidable in the presence of nonresponse, and the properties of (point and variance) estimators (e.g., bias and variance) will depend on the validity of the assumed models. Consequently, imputation is essentially a modeling exercise. The quality of the estimates will thus depend on the availability (at the imputation stage) of good auxiliary information and its judicious use in the construction of imputed values and/or imputation classes.
    
3. Auxiliary information 
   
   Auxiliary information is defined as a set of variables, which is available (at least) for all the sampled units (i.e., variables for which we have complete response). Good auxiliary information refers to a set of variables which is related to the variable being imputed and/or related to the probability of response to the variable being imputed.
    
4. Nonresponse mechanism 
   
   The sample is selected according to a known random mechanism called the sampling mechanism. In the case of complete response, survey statisticians use the knowledge of specific aspects of the sampling mechanism (i.e., first-order and second order inclusion probabilities) to construct point and variance estimators that are unbiased under repeated sampling (or design-unbiased).
   
   In the context of nonresponse, there is another random mechanism that, given the selected sample, divides the sample into a random set of respondents and a random set of nonrespondents. This random mechanism is called thenonresponse mechanism. Unlike the sampling mechanism, the nonresponse mechanism is not known to the survey statistician. Thus, it is customary to make assumptions about the nonresponse mechanism. The reader is referred to Beaumont (2002) for a discussion of the nonresponse mechanism and its impact on the bias of point estimators.
    
5. Imputation model The imputation model is set of assumptions with respect to the distribution of the variable being imputed. It links the variable being imputed to a set of auxiliary variables. For example, if the variable being imputed is continuous, the imputation model could be a multiple linear regression model.
    
6. Nonresponse model The nonresponse model is a set of assumptions with respect to the unknown nonresponse mechanism. Let ri = 1 if unit i responded to a given item of interest and ri = 0, otherwise. Since the response status of a unit is a binary variable, one possible nonresponse model is the usual logistic model linking P(ri= 1) to a set of auxiliary variables.
    
7. Some population parameters of interest and their estimators In practice, important parameters include the population total and/or the population mean of a particular variable. Consider a finite population of N individuals. The population total of a variable of interest y is given by . The population mean is defined as:  Suppose we select a random sample, s, of size n, according to a given sampling design p(s) . If we had complete response to the variable y, we could, for example, use the Horvitz-Thompson estimator of Y given by  where wi = 1πidenotes the sampling weight attached to unit i and πi denotes its inclusion probability in the sample. The estimator  is unbiased for Y with respect to the sampling design and we write , where Ep(.) denotes the expectation with respect to the sampling design. An estimator of the population mean, , is obtained by dividing  by N. In the presence of nonresponse to item y, it is not possible to compute the estimator  since some y-values are missing. We define an imputed estimator of Y given by , where  denotes the imputed value used to replace the missing value yi.  In practice, estimates for various domains (subpopulations) are needed. For example, in the context of CCHS, estimates of the average BMI may be required by age-sex group or by province. Let  be a domain of interest of size Nd. The domain mean, , can be expressed as , where di is a domain indicator such that di = 1 if unit ibelongs to Ud and di = 0, otherwise. In the absence of nonresponse, an asymptotically unbiased estimator of  is given by . That is, . In the presence of nonresponse to item y, we define an imputed estimator by .
   
   A third parameter of interest is the proportion of individuals in the population who have a particular characteristic, for example, the proportion of the population that is overweight or obese. Note that, according to World Health Organization and Health Canada guidelines, a person is classified as overweight if his BMI is between 25.0 and 29.9 while a person with a BMI of 30.0 or more is classified as obese; see Shields and Tjepkema (2006), Tjepkema (2006) and Le Petit and Berthelot (2006).
   
   In the absence of nonresponse, an estimate of the proportion of the population with a particular characteristic is given by  where Ci = 1 if uniti has the characteristic and Ci = 0 if unit i does not have the characteristic. In the presence of nonresponse, the imputed estimator is defined as above, except that we are trying to impute a binary variable instead of a continuous variable (e.g., BMI).
    
8. Imputation methods  In practice, many different imputation methods are used to fill in missing values. Descriptions of some of these methods are given in Kovar and Whitridge (1995), Beaumont (2001), Kalton (2003), Beaumont and Bocci (2005), and Haziza (2005).
    
9. Imputation classes  In practice, imputation is rarely performed at the overall sample level. Instead, imputation classes are formed and imputation is then performed independently within each class. In practice, many methods are used to form imputation classes. The reader is referred to Little (1986), Eltinge and Yansaneh (1997), Haziza (2002), Haziza and Beaumont (2007).
    
10. Nonresponse bias  Nonresponse bias occurs when respondents and nonrespondents have different characteristics with respect to the variables measured in the survey. It is defined as the average difference between the imputed estimator and the estimator we would have obtained had complete response been observed. For a discussion of the nonresponse bias, see, for example, Haziza et Beaumont (2007), Haziza and Kuromi (2007) and Haziza (2005). The first objective of imputation is to reduce the nonresponse bias as much as possible. A secondary objective is also to control the nonresponse variance (see section 11) as much as possible. In order to do so, we need good auxiliary information to construct imputed values and/or imputation classes.
    
    To reduce the nonresponse bias, it is important to identify a set of auxiliary variables that explain the variable being imputed as well as a set of auxiliary variables that explain the response probability to the variable being imputed; see, for example, Haziza and Rao (2006).
    
    In fact, one can eliminate the nonresponse bias if: (i) the imputation model and/or the nonresponse model contain all the appropriate auxiliary variables (i.e., the models are correctly specified) and (ii) the nonresponse mechanism is ignorable. This is discussed in Beaumont (2002).
     
11. Nonresponse and imputation variance  In the presence of nonresponse, the observed sample size is smaller than the sample size initially planned so nonresponse usually has the effect of leading to estimators with larger variance than the variance of estimators that would be attained if complete response was possible. This increase in variance is called the nonresponse variance. When a random imputation is used (e.g., random hot deck imputation within classes), a third random mechanism is applied to randomly select the residuals. Thus, random imputation methods suffer from an additional component of variance (called the imputation variance) due to the use of a random imputation mechanism. For a discussion on the nonresponse and imputation variance, see, for example, Haziza et Beaumont (2007), Haziza and Kuromi (2007).
     
12. Variance estimation  In recent years, variance estimation in the presence of imputed data has been widely treated in the literature. Before the 1990’s, it was customary to treat the imputed values as if they were observed values. As result, the published variance estimates were too low because they failed to account for the nonresponse variance, and imputation variance in the case of random imputation methods. Nowadays, many methods have been developed/adapted to take the nonresponse and the imputation variance into account. Resampling methods such as the jackknife (Rao and Shao, 1992) and the bootstrap (Shao and Sitter, 1996) have been considered in the context of imputation. The reader is also referred to Särndal (1992), Rao (1996, 2003), Shao and Steel (1999), Haziza and Rancourt (2004), Mathews (2004), Haziza (2005) and Haziza (2007b).
     
13. Some software For carrying out single imputation and point estimation a person can use the modeling procedures of any standard software. They include SAS, WESVAR,SUDAAN, SPLUS and R . However, these software do not perform correct variance estimation under single imputation. In this case, the students will need to write their own code for variance estimation.
     
14. Multiple imputation

• Each missing value is replaced by M ≥ 2 imputed values, which leads to the creation of M completed data files.
• The M completed data files are then analyzed using standard SAS procedures.

These results are then combined for inference.
 

Multiple imputation, introduced by Rubin (1978, 1987) involves three distinct steps:
 

For a good overview of multiple imputation, see, for example, Rubin (1996) and Little and Rubin (2002). In the context of survey sampling, it may be important to take the design features into account to define an appropriate imputation strategy. This is discussed in Reiter, Raghunathan, and Kinney (2006) and Little and Ragunathan (2007).
 

In SAS (version 9), two procedures are available to the user: PROC MI and PROC MIANALYZE (for a description of the procedures, see, for example, Haziza (2003)). In WESVAR , variance estimation for multiple imputation is also available. SOLAS is another software for multiple imputation.