A Portuguese Dataset for Evaluation of Semantic Question Answering

Abstract

Research on question answering tools over open linked data is increas- ing, and that brings the necessity of resources to allow for the evaluation and comparison of such systems. Question Answering over Linked Data (QALD) is a traditional benchmark event that occurs annually since 2011. However, although the multilingual task is available since its third edition, there is a necessity to foster the actual Portuguese Language resources present in this event benchmark. In this paper, we describe the development of the Portuguese language as a QALD corpus complement. The corpus is based on an existing QALD multilingual corpus and comprises 258 sentences used for the event challenge in 2017. We constructed a second corpus to allow direct comparison with the DBPedia Portuguese content. The main topics to highlight are the adopted methodology, which results in corpus related to frequent Brazilian Portuguese use of the language, and the work on adapting the answers to the DBPedia PT knowledge base, providing a corpus to evaluate Portuguese QA systems accurately.