KEYPHRASE DETECTION AND QUESTION GENERATION FROM TEXT USING MACHINE LEARNING

Abstract

Question Generation may not be as prominent as Question Answering but it still remains a relevant task in NLP. The ability to ask meaningful questions provides evidence towards comprehension within an Artificial Intelligence (AI) model. This makes the task of question generation important in the bigger picture of AI. While existing question generation techniques rely on complex model architectures and additional mechanisms to boost performance, we show that transformer-based fine-tuning techniques can create robust question generating systems using only a single language model,, without the use of additional mechanisms, answer metadata, and extensive features. Some training parameters of our project are : epoch :10, batchsize : 4, learning rate 10e-3.Lastly, we also look into the model’s failure modes and identify possible reasons why the model fails.