A MapReduce-based Deep Belief Network for Intrusion Detection

Abstract

The network security is being a challenging task in this modern era of IT with increasing number of hacking tools, complexity of networks and network security threats. The efficient and reliable intrusion detection system is an essential need in recent growing digital world. Deep Belief Networks (DBNs) with Restricted Boltzmann Machines (RBMs) as the building block have recently attracted wide attention to the field of Network Intrusion Detection System (NIDS) due to their great performance. DBN consists of two phase of training: first is pre-training of stack of RBMs followed by fine-tuning using backpropagation However, the sequential implementation of DBN is computationally very time consuming to process large amount of data sets. So, this thesis research is focused on implementing a MapReduce-based Deep Belief Network (MRDBN) for distributed computation for efficient NIDS using Hadoop ecosystem. The performance of system has been evaluated using two different datasets: UNSW-NB15 and NSL-KDD. First, the dataset is preprocessed to convert all attributes into numerical values and normalized it. Then, the distributed DBN based on MapReduce framework is implemented and evaluated in preprocessed datasets. The scalability test of system showed that training time for 10 nodes cluster sized MRDBN for intrusion detection is 2.2 times faster than 4 nodes cluster sized MRDBN. The overall accuracy of the MRDBN intrusion detection system for multiclass classification is 82.61% with 3.9% false alarm rate for UNSW- NB15 dataset. The system is found to be more precise than existing Artificial Neural Network (ANN) and Support Vector Machine (SVM) in the field of intrusion detection.