Comparative Analysis of TF-IDF and Word2vec Algorithm for Content-based Job Recommendation System

Abstract

Recommender systems are one of the most successful and widely used application of machine learning in business as they provide users ability to gather and obtain result much quicker than ever. In the current world scenario more people are using online job portal for searching jobs which require time and effort for both employer and employee to find the right person. In order to recommend relevant jobs this research presents a content-based recommendation-based approach to provide job recommendations based on TF-IDF and Word2vec in order to evaluate the performance between the two. In order to determine whether semantic relationships of words have impact on job recommendations, WMD with Word2vec are developed and compared with TF-IDF. The Word2vec model is developed using a dataset provided by Kaggle. A series of experiments are designed starting from TF-IDF vectors and cosine similarity which is set as a baseline. Further experiments were designed after observing the results of the current experiment for Word2vec and wordvec with WMD. Keywords:Job recommendation, TF-IDF, Word2vec, Word Embedding, Cosine Similarity, Word Mover’s Distance, KNN