Chatbot Evaluation Metrics Collection

Chat as a traditional way of communication largely happens among people. AT&T, as a large corporation realize the importance to automate traditional customer support handling, encouraged more than 1500 chatbots built internally at one platform.The problem recently is that the chatbot platform needs to manage the status or activities of all bots while chatbot developers need more feedback about the user experience and bot performance. As a part of chatbot analytics project, I started by generating evaluation metrics from the perspective of chatbot platform, chatbot developers and chatbot project managers and made a MVP(minimal viable product). I teamed up with project managers, system architecture people and analytics experts to create a metric capability list from raw data source to analytics method. The metrics capability list mainly includes bot usage metrics, bot user metrics, access metrics, client metrics, financial metrics and efficiency metrics.