Understanding z-score normalization in machine learning

Understanding z-score normalization in machine learning

Machine learning is making its way into almost every field of study, including finance, biology, and social sciences. It is a data-driven method that allows the computer to learn and improve over time. Since the data has to be in numerical form for the algorithm to work, preprocessing is an essential step in the machine learning pipeline. One such preprocessing technique is z-score normalization. In this article, we will dive deep into understanding what z-score normalization is and how it works.

What is z-score normalization?

Z-score normalization, also known as standardization, is a statistical technique used to transform the data into a standard normal distribution. It does so by subtracting the mean of the data and dividing by its standard deviation. This standardization process results in a zero mean and unit variance of the distribution. The formula for z-score normalization is:

z = (x – μ) / σ

where,
z = normalized value
x = original value
μ = mean of the data
σ = standard deviation of the data

Why is z-score normalization important?

Z-score normalization is essential in machine learning as it brings the data to a common scale, making it easier for the model to learn. The normalization process prevents the features with higher magnitudes from dominating the training process, ensuring that each feature contributes equally to the final output. It also helps to improve the model’s stability, as it reduces the chances of numerical instabilities during the computation.

How does z-score normalization work?

Let’s take an example to understand how z-score normalization works. Suppose we have a dataset of the height of students in two different schools, A and B.

| Schools | Students | Height (in cm) |
|———|———-|—————-|
| A | 1 | 155 |
| A | 2 | 162 |
| A | 3 | 163 |
| A | 4 | 159 |
| A | 5 | 166 |
| B | 1 | 169 |
| B | 2 | 161 |
| B | 3 | 172 |
| B | 4 | 175 |
| B | 5 | 157 |

We want to normalize the data to bring it into a standard distribution. The first step is to find the mean and standard deviation of the data.

μ = 163.2
σ = 7.68

Now, we can use the formula to calculate the z-score for each data point.

| Schools | Students | Height (in cm) | Z-score |
|———|———-|—————-|———|
| A | 1 | 155 | -0.661 |
| A | 2 | 162 | -0.104 |
| A | 3 | 163 | 0.000 |
| A | 4 | 159 | -0.521 |
| A | 5 | 166 | 0.313 |
| B | 1 | 169 | 0.835 |
| B | 2 | 161 | -0.208 |
| B | 3 | 172 | 1.240 |
| B | 4 | 175 | 1.760 |
| B | 5 | 157 | -1.033 |

As we can see, the mean of the data is now zero, and the standard deviation is one, which brings the data into a standard distribution.

Conclusion

Z-score normalization is a powerful technique that is widely used in data preprocessing. It standardizes the data into a standard normal distribution, making it easier for the machine learning algorithm to learn. The normalization process brings the data to a common scale and helps to improve the model’s stability by preventing data with higher magnitudes from dominating the training process. With the help of z-score normalization, we can make our machine learning models more accurate and robust.