Automating Response Evaluation: A tool for automatic coding responses to open-ended questions.

The goal of the project is to create a tool leveraging natural language processing (NLP) techniques to evaluate the responses to the open-ended questions. In common practice, researchers evaluate the responses by thematic coding. The process is time-consuming and labor-intensive. Further, in order to control the quality of the work, it relies on at least two experts to manually code the responses. With the development of NLP technology, I created a tool to make the machine as the coder. The key technologies of this project are NLP, aspect-extraction model, classification model, and transformer. With the help of the tool, the users can use this tool to generate a code book and customize it on top of that. Moreover, the application can code the responses. To evaluate the result, I implemented Cohen’s kappa to calculate the inner-annotator agreement.