Document Visual Question Answering with CIVQA: Czech Invoice Visual Question Answering Dataset

Investor logo

Warning

This publication doesn't include Faculty of Education. It includes Faculty of Informatics. Official publication website can be found on muni.cz.
Authors

ŠČAVNICKÁ Šárka ŠTEFÁNIK Michal SOJKA Petr

Year of publication 2023
Type Article in Proceedings
Conference Recent Advances in Slavonic Natural Language Processing (RASLAN 2023)
MU Faculty or unit

Faculty of Informatics

Citation
Web fulltext PDF
Keywords Question Answering; Visual Question Answering; Document Visual Question Answering; CIVQA; Czech Invoice Visual Question Answering
Description Applications of document processing become increasingly popular across multiple industries, resulting in a growing amount of research on the applications of artificial intelligence in document processing (Document AI). This paper focuses on a subtask of Document AI, Document Visual Question Answering (DVQA), recently getting well-deserved attention thanks to its universality. However, the limited availability of data sources for languages outside English restrains the applicability of DVQA in non-English languages.

For this reason, we created the CIVQA (Czech Inovice Visual Question Answering) dataset covering 15 entities of financial documents, consisting of more than 6,000 invoices in the Czech language.

We used the CIVQA dataset to create the first-of-its-kind DVQA models specifically tailored for applications to Czech documents. Striving to create DVQA models able to generalize, we specifically evaluate our models on the entities not covered in the training mix and find that multilingual LayoutLM models are able to respond to questions about previously unseen entities substantially more accurately than other models.

The CIVQA dataset and experiment observations offer new opportunities for Document AI in the Czech Republic, with potential applications in research and commercial fields.

Related projects:

You are running an old browser version. We recommend updating your browser to its latest version.