What is the Data Science Process?
The data science process begins with a question or hypothesis, followed by the collection of relevant raw data, data cleaning and exploration, modeling and evaluation, deployment, visualization, and the communication of results through reports, as illustrated in Figure 1-1.
Figure 1-1. Data science process
Questions vary according to the field, for example:
· Politics: Will Trump win in Maine in 2016 and 2020?
· Facebook: How to make people stay on Facebook longer?
· Medical: Is this tumor cancer or not?
· Hospital Management: How to decrease patients’ wait lines to increase patients’ satisfaction?
The second step is collecting raw data. For example, in the politics question: will a particular candidate win in a specific state?
Collecting all the voters’ information, including age, race, education, income, gender, and industry, is a crucial step. The process involves collecting ballot data and voting results over the years. The more historical data we have, the more accurate our predictions are. Furthermore, we collect population distribution data throughout the years.
The third step involves cleaning the raw data by handling missing values, outliers, and duplicate rows, correcting misspellings, adjusting column data types, and standardizing value formats.
The fourth step involves evaluating several models and comparing their results, depending on the nature of the problem. In the presidential election project, I used Monte Carlo and Bayes algorithms.
The fifth and final step involves visualizing the results and communicating them in plain language in our reports. This step is the primary goal of the entire process because it aligns the predictions with the answer to the first question that initiated the process.
Note that the data science process is iterative, and after reaching the deployment stage, we may need to collect additional data to gain further insights and address the question that initiated the process.
Acknowldgement
Thank you, Andy Williamson, for the fantastic infographic!
This snippet is from my book, Learn Data Science Using SAS Studio, From Clicks to Code, Second Edition
