Wally Referring Expression Corpus (WREC) v0.1.0

Abstract

The data in this corpus consists of descriptions of individuals in visually complex scenes. The descriptions were collected and annotated as part of the Wally project, an ongoing research project between collaborators at the University of Edinburgh and the Ohio State University. The project aims to combine insights from the research domains of vision and language in order to better model the way language is used to describe entities in visually complex scenes. In this context, we used crowd-sourcing techniques to collect a large number of descriptions of individuals in scenes. The individuals appeared in scenes from the book 'Where's Wally' (known in the US as 'Where's Waldo') and participants wrote descriptions to identify for a naive listener which individual was the target (indicated to the participant with a box around the target individual). A subset of those descriptions were annotated (1672 out of 4256 total descriptions, representing 11 of the 28 total scenes). Descriptions were annotated to indicate which words referenced the target and which words were used to introduce other entities as landmarks and mark relationships between entities in the scene. Further details can be found in the README file.