Your job depends on your data
The goal of this section is to:
???
If you have inadequate data you will not be able to succeed in any data science task.
More generally, I want you to focus on your data. It is necessary to understand your data before building a model.
Read moreThere’s a fairly famous dataset called the “mushroom dataset”.
It describes whether mushrooms are edible or not, depending on an array of features.
The nice thing about this dataset is that the features are all catagorical.
So we can go through and segment the data for each value in a feature.
This is some example data:
| poisonous | cap-shape | cap-surface | cap-color | bruises? |
|---|---|---|---|---|
| p | x | s | n | t |
| e | x | s | y | t |
| e | b | s | w | t |
| p | x | y | w | t |
| e | x | s | g | f |
etc.
Read moreSo let’s walk through a very visual, intuitive example to help describe what all data science algorithms are trying to do.
This will seem quite complicated if you’ve never done anything like this before. That’s ok!
I want to do this to show you that all algorithms that you’ve every heard of have some very basic assumption of what they are trying to do.
At the end of this, we will have completely derived one very important type of classifier.
Read moreFirst lets discuss what the goal is. What is the goal?
Based upon what?
How can we improve the quality of the decision or prediction?
Think about this for a moment. It’s a key insight. Think about your projects. Your research. The decisions you make. They are all based upon some information. And you can make better decisions when you have more good quality information.
Read moreUnderstanding the problem
“the five-whys”
Different questions dramatically effect the tools and techniques used to solve the problem.
By Kenneth Jensen CC BY-SA 3.0, via Wikimedia Commons
Read moreSoftware Engineering, Maths, Automation, Data
A.k.a: Machine Learning, AI, Big Data, etc.
It’s current rise in popularity is due to more data and more computing power.
For more information: https://winderresearch.com/what-is-data-science/
US Supermarket Giants
Target: Optimising Marketing using customer spending data.
Walmart: Predicting demand ahead of a natural disaster.
Most projects are “Discovery Projects”.
Primary Business goals: Increase Revenue, save costs, save time.
Budgets can come from other parts of the business.
Read more
Welcome! This workshop is from Winder.ai. Sign up to receive more free workshops, training and videos.
To test whether your models are robust to changes, one simple test is to add some noise to the test data. When we alter the magnitude of the noise, we can infer how well the model will perform with new data and different sources of noise.
In this example we’re going to add some random, normally-distributed noise, but it doesn’t have to be normally distributed! Maybe you could add some bias, or add some other type of trend!
Read moreWelcome! This workshop is from Winder.ai. Sign up to receive more free workshops, training and videos.
We need to be able to compare models for a range of tasks. The most common use case is to decide whether changes to your model improve performance. Typically we want to visualise this, and we will in another workshop, but first we need to establish some quantitative measures of performance.
Read more
Welcome! This workshop is from Winder.ai. Sign up to receive more free workshops, training and videos.
Being able to evaluate models numerically is really important for optimisation tasks. However, performing a visual evaluation provides two main benefits:
It is so easy to miss a gross error when looking at summary statistics alone. Always visualise your data/results!
Read more
Welcome! This workshop is from Winder.ai. Sign up to receive more free workshops, training and videos.
Clustering is an unsupervised task. In other words, we don’t have any labels or targets. This is common when you receive questions like “what can we do with this data?” or “can you tell me the characteristics of this data?”.
There are quite a few different ways of performing clustering, but one way is to form clusters hierarchically. You can form a hierarchy in two ways: start from the top and split, or start from the bottom and merge.
Read more