Hybrid Feature Selection and Feature Extraction Based Ensemble Method in Classification
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Department of Computer Science and Information Technology
Abstract
Ensemble methods have been called the most influential development in Data Mining and
Machine Learning in the past decade. The idea of ensemble learning is to employ multiple
learners and combine their predictions.
In this thesis, a novel method is proposed to build an ensemble of classifiers based on feature
selection: Random selection, Relief and feature extraction: Principal component analysis
method. The feature selection process chooses optimal subset of features according to
objective function whereas feature extraction process maps the high dimensional dataset into
lower dimensional dataset using the linear combination of original features. These feature
selection and extraction method helps to produce diverse as well as accurate set of ensemble
classifiers. A comparison of proposed method is made with the Bagging, AdaBoost, feature
selection based NN, feature extraction based NN and also with plain NN using 22 benchmark
dataset. The result obtained by the proposed method outperformed other algorithms with the
following distribution: NN (14 cases), Random-NN (13 cases), Relief-NN (15 cases), PCANN
(19
cases),
AdaBoost
(14
cases),
Bagging
(15
cases).
Keywords:
Ensemble methods, feature selection, feature extraction, Relief, Principal
component analysis, AdaBoost, Bagging, NN, Random-NN, Relief-NN, PCA-NN