With Big Data Comes Big Responsibility
By: Joe Marx
Tags: Statistics, Analytics, Information Technology, Data

Big Tech and Big Data have enabled previously unimaginable power when it comes to predicting outcomes and behaviors. Given the availability of highly personal data and raw computing power accessible to us, data scientists are able to create very powerful models that can predict patterns of consumption, travel, and behavior with ever-increasing accuracy. Ethical implications of this are many—far more complicated and nuanced than the scope of this article can cover.

Institutions are turning to data analysis and regression as a solution to a growing number of problems and questions as a result of this nexus of availability and computability of data. This places a large responsibility on the shoulders of those who work with data to provide meaningful, but more importantly, accurate and unbiased analysis to stakeholders.

The most obvious ways a data scientist can provide unhelpful or misleading analysis are through poor statistical analysis or by letting personal bias impact their process. There are many avenues though which a data scientist can engage in intellectual dishonesty—either intentionally or unintentionally—being aware of them can help avoid drawing fallacious conclusions.

Data collection

The first, and possibly most consequential mistake a data scientist can make is collecting or using data that was collected improperly. This can mean collecting data from unrepresentative samples as a result of selection bias, or misrepresenting your target population. If the individuals sampled in a dataset don’t reflect the population you’re seeking to understand with a statistical study, your experiment can result in misleading conclusions being drawn. It’s a possible pitfall to select elements of a population that may skew in favor of the experiment’s hypothesis. It’s therefore important to ensure that a sample is randomly chosen and not subject to common selection biases. And similarly, if using an existing dataset, verify how the data was collected to ensure it’s appropriate to use to predict on your target population.

Also worth noting, for specific data collection instances that involve surveying people, be wary of biased or leading questions. These can heavily skew the data you receive and lead to conclusions that are not based in truth. And likewise, if you’re conducting the surveys personally, be sure not to let those you survey know what hypothesis you’re hoping to prove or disprove, as that can lead to biased responses and similarly cloud your data.

Photo by CDC on Unsplash

Data cleaning

When prepping data for analysis, the ever-loathed process of data cleaning can cause issues if not handled with care. The first step to any data cleaning process is exploratory data analysis (EDA). This involves looking for missing or incorrect values in your data, visualizing elements of the data to gain insight into it’s distribution, and check for outliers.

After EDA is complete, generally we then deal with missing and extreme values. If there are outliers that likely are a result of error in data collection, that data can be dropped to allow modeling to properly capture the majority of the data. However, it is important not to overuse this power, and never drop values because they are in contradiction with your hypothesis. Be mindful of the ways your handling of missing and extreme values could change your model and affect the drawn conclusion. Try to keep everything as neutral as possible.

Photo by Cookie the Pom on Unsplash

Data modeling

Modeling frameworks such as linear/logistic regression, decision trees, XGBoost, et al. allow for very fast and effective modeling of data. It may not be as obvious here as it was for other aspects of the process, but there is still room for bias to creep in when modeling data. In selecting hyper-parameters and in the selection of the model to use, it’s important to not try to find ways to contort your model to fit the data to your hypothesis—the goal should be to uncover the truth within the data, and find what data says objectively, then ask if the original hypothesis should be confirmed or rejected. In the case of predictive modeling, avoiding personal bias in how you define model features is keenly important as well.

And it’s worth stating the obvious here: if your data is biased, your model will be biased. Spend some time considering how and from where your data was collected and what impact on your model that will have.

Presentation

This last area is one that would likely feel patronizing to include if this were written years ago, but given the wave of anti-science and anti-intellectualism that has taken the globe by storm (or taken the disk if you’re already a victim of this), it would feel naive to not include the abuse that can be perpetrated through misleading or false representations of your findings.

Whether it’s an exaggeration of your findings, a bending of your findings, or an outright lie about your findings, it is deeply harmful to present your engagement with data analysis as evidence for something different than the data shows. Especially when there is strong evidence that suggests that false and misleading information spreads faster than the truth. We can see clear examples of the consequences of claiming data supporting harmful or hateful world views by looking at the horrors of eugenics, or at the manipulated data used to justify “broken windows policing.” If you’ve made it this far, it’s likely that it wouldn’t be you who would try to use your findings for the purpose of disinformation or fake news, but hey, just in case!

Thanks for reading, if you found this article helpful, please share this with your peers. Feel free to contact with any questions, clarifications, criticisms, or inquiries at josephlmarx@gmail.com.