Browsing by Subject "MapReduce"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item A Comparative Analysis of Cloud based Recommendation System on Mapreduce and Spark(Pulchowk Campus, 2017-11) Ghimire, SaralaToday, Big Data is a hot issue both in industrial and academic fields. The need of data processing is changing with the gradual increase in data volume and with the mass of sources leading to a diversity of structures. Although relational database management system (RDBMS) remaining the primary technology for data management of structured data and been proven best for more than 40 years, it has reached its limit, and the reason is massive growth in the diverged volume of data. Several researchers and organizations now focused on MapReduce and Spark framework that has discovered huge success in processing and analyzing a large volume of data on several clusters. In this study, the performance of MapReduce, RDBMS, and Spark with various comparison measures are evaluated. To conduct a comparison and analysis, three processes are computed: (a) developed recommendation system with all three algorithms, (b) run that system on various data networks and data sizes, and (c) the output is then analyzed and compared on the basis of time computation, memory consumption, and CPU usage. Moreover, statistical validation of the observed results from all the algorithms with respective node and network configuration using Friedman rank test and Holm post-hoc test are performed. Overall, observations show that Spark is about 2.5x and 5x faster than MapReduce, and 10/20 times faster than RDBMS. The reason for these speedups is the efficiency of the alternative least square algorithm and reduced CPU and disk overheads due to RDD caching in spark.Item A MapReduce-based Deep Belief Network for Intrusion Detection(Pulchowk Campus, 2017-11) Shrestha, RajuThe network security is being a challenging task in this modern era of IT with increasing number of hacking tools, complexity of networks and network security threats. The efficient and reliable intrusion detection system is an essential need in recent growing digital world. Deep Belief Networks (DBNs) with Restricted Boltzmann Machines (RBMs) as the building block have recently attracted wide attention to the field of Network Intrusion Detection System (NIDS) due to their great performance. DBN consists of two phase of training: first is pre-training of stack of RBMs followed by fine-tuning using backpropagation However, the sequential implementation of DBN is computationally very time consuming to process large amount of data sets. So, this thesis research is focused on implementing a MapReduce-based Deep Belief Network (MRDBN) for distributed computation for efficient NIDS using Hadoop ecosystem. The performance of system has been evaluated using two different datasets: UNSW-NB15 and NSL-KDD. First, the dataset is preprocessed to convert all attributes into numerical values and normalized it. Then, the distributed DBN based on MapReduce framework is implemented and evaluated in preprocessed datasets. The scalability test of system showed that training time for 10 nodes cluster sized MRDBN for intrusion detection is 2.2 times faster than 4 nodes cluster sized MRDBN. The overall accuracy of the MRDBN intrusion detection system for multiclass classification is 82.61% with 3.9% false alarm rate for UNSW- NB15 dataset. The system is found to be more precise than existing Artificial Neural Network (ANN) and Support Vector Machine (SVM) in the field of intrusion detection.