# scikit learn hyperparameter optimization for MLPClassifier

## tune/adjust hyperparameters MLPClassifier in scikit learn

## Two simple strategies to optimize/tune the hyperparameters:

Models can have many hyperparameters and finding the best combination of parameters can be treated as a search problem.

Although there are many hyperparameter optimization/tuning algorithms now, this post shows a simple strategy which is grid search. Read more here

## How to tune hyperparameters in scikit learn

In scikit learn, there is **GridSearchCV** method which easily finds the optimum hyperparameters among the given values.

**As an example:**

mlp_gs= MLPClassifier(max_iter=100)parameter_space = {

'hidden_layer_sizes': [(10,30,10),(20,)],

'activation': ['tanh', 'relu'],

'solver': ['sgd', 'adam'],

'alpha': [0.0001, 0.05],

'learning_rate': ['constant','adaptive'],

}from sklearn.model_selection importGridSearchCVclf =GridSearchCV(mlp_gs, parameter_space, n_jobs=-1, cv=5)

clf.fit(X, y) # X is train samples and y is the corresponding labels

I fixed max_iter or the number of epochs.

As you see, we first define the model (**mlp_gs**) and then define some possible parameters. **GridSearchCV **method is responsible to fit() models for different combinations of the parameters and give the best combination based on the accuracies.

cv=5 is for cross validation, here it means 5-folds Stratified K-fold cross validation. Read more here

n_jobs=-1 , -1 is for using all the CPU cores available.

After running the code, the results will be like this:

To see the perfect/best hyperparameters, we need to run this:

`print('Best parameters found:\n', clf.best_params_)`

and we can run this part to see all the scores for all combinations:

`means = clf.cv_results_['mean_test_score']`

stds = clf.cv_results_['std_test_score']

for mean, std, params in zip(means, stds, clf.cv_results_['params']):

print("%0.3f (+/-%0.03f) for %r" % (mean, std * 2, params))

The final step is to test the best model on the test set.

If the test set is **X_test** and corresponding labels is **y_test** we can do:

y_true, y_pred = y_test ,clf.predict(X_test)from sklearn.metrics import classification_report

print('Results on the test set:')

print(classification_report(y_true, y_pred))

Also read more here.

Very soon I will publish another good example which help us to tune hyperparameters professionally.