3-2.Data Science Methodology 2주차

Sorting data is not part of the Data Understanding stage.

 

Data preparation - Cleansing data 

Transforming data in the data preparation phase is the process of getting the data into a state where it may be easier to work with.

The Data Preparation stage is in fact the most time-consuming phase of a data science project.

Data preparation is similar to washing freshly picked vegetables insofar as unwanted elements, such as dirt or imperfections, are removed.

The target variable was congestive heart failure (CHF) with 30 days following discharge from CHF hospitalization.

 

 

# 2주차- (1) 문제풀이

 

Q. The Data Understanding stage refers to the stage of removing redundant data.

-> False

 

Q. In the case study, working through the Data Preparation stage, it was revealed that the initial definition was not capturing all of the congestive heart failure admissions that were expected, based on clinical experience.

-> False

 It was working through the Data Understanding staging that the initial definition was found to be incomplete.

 

Q. The Data Preparation stage involves correcting invalid values and addressing outliers.

-> True

- The Data Preparation stage involves addressing missing values.

- The Data Preparation stage involves removing duplicate data.

- The Data Preparation stage involves properly formatting the data.

 

Q. During the Data Preparation stage, clients and stakeholders aggregate the data and merge them from different sources, enabling data scientists to use clean data in the analysis.

->False

- During the Data Preparation stage, data scientists define the variables to be used in the model.

- During the Data Preparation stage, data scientists determine the timing of events.

- During the Data Preparation stage, data scientists aggregate the data and merge them from different sources.

- During the Data Preparation stage, data scientists identify missing data.

 

Q. The Data Preparation stage is a very iterative and complicated stage that cannot be accelerated through automation.

-> False

 

---

 

# Modeling

Modelling is the stage in the data science methodology where the data scientist has the chance to sample the sauce and determine if it's bang on or in need of more seasoning! 

 

First, what is the purpose of data modeling, and second, what are some characteristics of this process?

Data Modelling focuses on developing models that are either descriptive or predictive.

In John Rollins' descriptive Data Science Methodology, the framework is geared to do 3 things:

- First, understand the question at hand.

- Second, select an analytic approach or method to solve the problem, and 

- third, obtain, understand, prepare, and model the data. 

 

The end goal is to move the data scientist to a point where a data model can be built to answer the question.

A training set is used to build a predictive model.

 

The question then becomes: How could the accuracy of the model be improved in predicting the 

yes outcome?

-> For decision tree classification, the best parameter to adjust is the relative cost of misclassified yes and no outcomes.

 

# Evaluation

Evaluation allows the quality of the model to be assessed but it's also an opportunity to see if it meets the initial request.

The ROC curve is a useful diagnostic tool in determining the optimal classification model.

This curve quantifies how well a binary classification model performs, declassifying the yes and no outcomes when some discrimination criterion is varied. 

By plotting the true-positive rate against the false-positive rate for different values of the relative misclassification cost, the ROC curve helped in selecting the optimal model. 

 

Model evaluation can have two main phases: a diagnostic measures phase and statistical significance testing.

 

 

# 2주차- (2) 문제풀이

 

Q. A training set is used for predictive modeling.

-> True

- not descriptive model

 

Q. A false-positive is when a true, non-readmission is misclassified. If action is taken to reduce the patient's risk, the cost of the misclassification is wasted intervention. This is also called a type 1 error.

 

Q. Which statement best describes the Modeling Stage of the data science methodology?

-> Depending on the the posed question, the problem, and the data, different modeling methods can be used.

 

Q. Model Evaluation includes ensuring that the data are properly handled and interpreted.

-> True

- Model Evaluation includes ensuring that the data are properly handled and interpreted.

- Model Evaluation includes ensuring the model is designed as intended.

 

Q. Select the correct statements about the ROC curve.

-> ROC stands for Receiver Operating Characteristic curve, which was originally developed to detect enemy aircrafts on radar.

-> The ROC curve is a useful diagnostic tool for determining the optimal classification model.

-> By plotting the true-positive rate against the false-positive rate for different values of the relative misclassification cost, the ROC curve can be used to select the optimal model.