"Why you should preprocess your data. All about normalization, removing outliers, imputing missing data and variable transformations.
___________________________________________
___________________________________________
Part 1: Regression in Machine Learning and the Linear Regression Model
Part 2: The Different Cost Functions for Regression - Understand MAE, MSE, WMAE
Part 3: How Model Training with Gradient Based Minimization works
Part 4: Data Preprocessing - Normalization, Outliers, Missing Data, Variable Transformation
Part 5: Cross Validation - How to Select the Best Machine Learning Model?
Part 6: Regularization - Early Stopping, Ridge Regression (L2) and Lasso Regression (L1)
___________________________________________
- Normalization is especially important for a gradient based minimization. Data can be normalized via min-max normalization or standardization.
- Outliers detection can be used to remove outliers. Outliers can be identified via boxplots. A boxplot displays the median and interquartile range with whiskers.
- Missing data can be either dropped or imputed via for example mean or median.
- Variable transformation can be applied to numeric variables. For example a functional modification, combination of variables or complex models can be used. A variable transformation for categirical variables can transform categorical features into binary variables. This is called one-hot encoding.