Keep it Clean: Why Bad Data Ruins Projects and How to Fix it

Abstract

The Internet is full of examples of how to train models. But the reality is that industrial projects spend the majority of the time working with data. The largest improvements in performance can often be found through improving the underlying data.

Bad data is costing the US economy an estimated 3.1 trillion Dollars and approximately 27% of data is flawed in the world’s top companies. Bad data also contributes to the failure of many Data Science projects. Who can forget Tay.ai, Microsoft’s twitter-bot that learned to be genocidal when user’s tweets were not cleaned.

This presentation will discuss in what circumstances bad data can affect your project along with some high profile case studies. We will then spend as much time as we have to go through some of the techniques you will need to fix that bad data. This is aimed towards those with intermediate-level Data Science experience.

What will the audience learn from this talk? This presentation will discuss in what circumstances bad data can affect your project along with some high profile case studies. We will then spend as much time as we have to go through some of the techniques you will need to fix that bad data.

Photos