scikit learn hyperparameter optimization for MLPClassifier

tune/adjust hyperparameters MLPClassifier in scikit learn

Two simple strategies to optimize/tune the hyperparameters:

Models can have many hyperparameters and finding the best combination of parameters can be treated as a search problem.

Although there are many hyperparameter optimization/tuning algorithms now, this post shows a simple strategy which is grid search. Read more here

How to tune hyperparameters in scikit learn

In scikit learn, there is GridSearchCV method which easily finds the optimum hyperparameters among the given values.

As an example:

I fixed max_iter or the number of epochs.

Image for post
Image for post
another example

As you see, we first define the model (mlp_gs) and then define some possible parameters. GridSearchCV method is responsible to fit() models for different combinations of the parameters and give the best combination based on the accuracies.

cv=5 is for cross validation, here it means 5-folds Stratified K-fold cross validation. Read more here

n_jobs=-1 , -1 is for using all the CPU cores available.

After running the code, the results will be like this:

Image for post
Image for post

To see the perfect/best hyperparameters, we need to run this:

Image for post
Image for post

and we can run this part to see all the scores for all combinations:

The final step is to test the best model on the test set.

If the test set is X_test and corresponding labels is y_test we can do:

Image for post
Image for post
In my example, there are 10 labels (MNIST data set).

Also read more here.

Very soon I will publish another good example which help us to tune hyperparameters professionally.

Written by

Web geek, Self-taught full-stack web developer, Learning Python, Laravel, Vuejs, UX/UI design, Nuclear Physicist PhD

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store