Comparative Analysis on Ensemble Learning: Bagging and Boosting

Abstract

Combine the prediction from multiple models to improve the overall performance of model is an ultimate task of Ensemble learning. Bagging and Boosting are two widely used ensemble learning techniques works based on numbers of classifiers combination to aggregate prediction. Performance of single classifier has limitation due to noise, bias and variance in data set. By applying divide and conquer approach on ensemble methods helps to minimize those limitation which ultimately leads to performance improvement. Bagging is a bootstrap aggregation while boosting attempts to fit a sequence of weak learner's models to build a strong classifier. The performance of bagging and boosting has been analyzed on the basis of Accuracy, Precision, Recall and F1-Measures for Adult data set with and without noise. The Gaussian noise distribution has used for noise addition on data set due to CLT. The results show that on the basis of Accuracy, Recall, F1-Measures boosting outperforms bagging whereas in terms of Precision, bagging has better result.

Keywords: Ensemble Learning, Bagging, Boosting, Gaussian Noise