Dr. Tajuddeen A. Williams

Data Cleaning and Descriptive Statistics

· Personal Essays

#Week 9 - Discussion 1: Performing Data Analysis

Data analyses cannot be performed until data has been cleaned. In fact, many of the errors found in standard data analyses can be traced directly back to "dirty" data. In a perfect world, collected data would be flawless, but as when working with humans in any capacity, errors occur.

To begin the cleaning process, you first need to check collected data for errors, problems, dubious responses, and other issues. Many such checks may be done electronically using statistical software. Once the proper adjustments are made, you can run the analyses. Which analyses techniques you use should align with your hypothesis. In other words, a survey researcher uses his or her hypotheses to drive the data analyses. The hypotheses dictate the "family" of analyses used for the data. The more parsimonious and testable the theory driving the hypotheses, the more straightforward the data analyses will be.

To prepare for this Discussion, consider why data cleaning, including the assessment of missing data, is important. Then think about the role that descriptive statistics plays in data analyses. Finally, consider the relationship between hypothesis(es) and data analyses and how you would illustrate this relationship using at least one of your hypotheses and data analytic strategies from your Final Project as an example.

With these thoughts in mind:

Post by Day 4 an explanation of the importance of data cleaning, including assessment of missing data. Provide one example of data cleaning and the potential impact it might have on data analyses. Then explain the importance of descriptive statistics in data analyses. Finally, explain the relationship between hypothesis(es) and data analyses using at least one of your hypotheses and data analytic strategies from your Final Project as an example.

On my first survey project, because it was software that was written by hand and had not been tested thoroughly, there were a lot of problems. Part of the issue was the duplication of data. Another issue involved incomplete data due to missing completely at random (MCAR). Data is considered MCAR when missingness is independent of the variables of interest and missing at random if missingness is dependent only on the observed data (Leung & Qin, 2006). Multiple choice was used to collect the data and participants were not supposed to be able to progress to the next section without filling it in. This resulted in several calls to get answers that were missing as well as modification to the survey. This survey did not allow for nonresponses since the data we were collecting was medical in nature. Incorrect inferences may result if contacts refuse to participate (Rogelberg & Stanton, 2007). Nonresponses influences the sample size, which in this case (Abraham, Maitland, & Bianchi, 2006). The issue has to do with coverage bias, and this happens when nonrespondents differ from respondents in the characteristics measured (Singer, 2006).

Cleaning data involves remediating incomplete data as well as duplicate data (Bradburn, Sudman, & Wansink, 2004). However, there are also other implications for data cleaning considering how one is collecting data. For instance, if a researcher is collecting information on gender, one typically expects that there are only two values. If the medium of collecting data allows for custom entries, a researcher’s data cleaning technique may involve finding responses with invalid characters which if left in there could lead to errors (Schafer & Graham, 2002).

The reason descriptive statistics is important is because it allow researchers to present data in a meaningful way. On a project that I worked on recently with regards to intercepting ISIS supply chains, due to the complexity the data it would be hard to visualize it raw. When it came to cultural nuances and showing the effects of sociolinguistics on the data we collect, from the outside looking in, one could not make correlations by viewing it. Descriptive statistics allows researchers to present a simpler interpretation of the data. In my proposed study, I hypothesize that repeated exposure to real and perceived racism and discriminative behaviors and situations have a direct and significant effect on the life satisfaction and emotional well-being of Black Americans. Most of my data collection will be done by preexisting scales. To make sense of the data, I will need to employ correlational analysis.

References

  • Abraham, K. G., Maitland, A., & Bianchi, S. M. (2006). Nonresponse in the american time use survey who is missing from the data and how much does it matter? Public Opinion Quarterly, 70(5), 676–703. http://doi.org/10.1093/poq/nfl037 ↩
  • Bradburn, N. M., Sudman, S., & Wansink, B. (2004). Asking questions. John Wiley & Sons. ↩
  • Leung, D. H. Y., & Qin, J. (2006). Analysing survey data with incomplete responses by using a method based on empirical likelihood. Journal of the Royal Statistical Society: Series C (Applied Statistics), 55(3), 379–396. http://doi.org/10.1111/j.1467-9876.2006.00542.x ↩
  • Rogelberg, S. G., & Stanton, J. M. (2007). Introduction understanding and dealing with organizational survey nonresponse. Organizational Research Methods, 10(2), 195–209. http://doi.org/10.1177/1094428106294693 ↩
  • Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177. http://doi.org/10.1037/1082-989X.7.2.147 ↩
  • Singer, E. (2006). Introduction nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 637–645. http://doi.org/10.1093/poq/nfl034 ↩

← All posts