Computational Linguistics & Phonetics Computational Linguistics & Phonetics Fachrichtung 4.7 Universität des Saarlandes

The TACoS Corpus

The Saarbrücken Corpus of Textually Annotated Cooking Scenes (short: TACoS) contains a set of video descriptions (in natural language) and timestamp-based alignment with the videos. The videos and a low-level activity annotation of them is available as part of the MPII Composites corpus. We provide links to the subset of MPII Composites that is incorporated in the TACoS corpus.

We also provide "ASim", the Action Similarity Dataset. This set contains 900 pairs of action descriptions from the TACoS corpus, annotated with similarity scores. For each of the sentences, we provide links to the video it annotates and the specific timeframe described by the action. We further provide the visual features used for our experiments described in Grounding Action Descriptions in Videos (TACL Vol. 1). You can find descriptions & downloads of all corpus parts. Please take note of the license agreement linked below.

If you have any questions about the corpus, please contact Michaela Regneri:
regneri [AT]


By downloading the TACoS corpus, you agree to our terms of use in our License Agreement. Basically, this agreement states that you are free to use the corpus for research or educational purposes, and asks you to cite this paper if you do so:

Michaela Regneri, Marcus Rohrbach, Dominikus Wetzel, Stefan Thater, Bernt Schiele and Manfred Pinkal (2013):
Grounding Action Descriptions in Videos.
Transactions of the Association for Computational Linguistics (TACL), Vol. 1. [PDF]

   Author = {Regneri, Michaela and Rohrbach, Marcus and 
             Wetzel, Dominikus and Thater, Stefan and 
             Schiele, Bernt and Pinkal, Manfred}, 
   Journal = {Transactions of the Association for Computational 
              Linguistics (TACL)},
   Title = {Grounding Action Descriptions in Videos},
   Year = {2013},
   Volume = {1},
   Pages = {25-36}}


If you want to download the TACoS corpus, please fill in the download form. The access data will be sent to you immediately.