Data Science is Like Panning for Gold

In my previous role, I created the infrastructure used to process and visualise data about the company I worked for.

When someone wanted access to a new data set, the scripts were written to extract and clean the data, and make it available for analysis.

This process can be fairly time consuming, so the ability to richly visualise and play with the resultant data set at the end was very rewarding.

Unfortunately most data sets are almost entirely predictable. For instance, a dataset about online ad spend might be flat for months on end. It’s rewarding in a way, because it means that the data you’ve extracted is accurate.

What every data scientist hopes for are unpredictable outcomes - little wrinkles and quirks in the dataset which lead to insight. For example, why was the adspend so high on that day? Why did so many people click on that particular ad?

Insights like that are very valuable to a business. In this way, data science is like panning for gold. There’s a lot of sifting and cleaning, and hard work, and sometimes there’s nothing valuable at the end of it. But sometimes there are hidden nuggets, and that’s the appeal of data science.