Annotation Guidelines for Mechanical Turk

Important Instructions

Note: This time you may only submit one HIT from this group. (only displayed in the HIT group with the few remaining last videos)

We will show you a short video of somebody who performs a basic kitchen task (e.g. somebody who prepares an eggplant). Your task is to label the video with sentences that describe the basic actions happening in the video (e.g. 'The person washes the eggplant.', 'The person peels the eggplant.'...). It is important that you put your label down at the right point in time, namely at the end of the respective action. We want to find out what happens in the video, and how long the single steps take.

Please read the following advice carefully.

The task is performed as follows:

1. First watch the whole video once, to see what happens. As long as you do not click on the 'new sentence' button, you can watch the video as often as you think is useful for you. The next steps will be easier if you already take notes on what happens in the video and in which order. If you think you are familiar enough with the video's contents, go to step 2.
2. Press the 'rewind' button and then the 'play' button. Watch the person in the video while he or she is doing some basic step. Wait until he or she is finished with that current action, and then pause the video (by pressing the space bar or the 'Pause' button).
3. Press the 'new sentence' button and enter a single sentence that describes the action that just ended in the video (like 'The person peels the eggplant.'). It's important that you do that when the action just ended, because we need to record how long the action took!
4. Resume the video (by pressing the space bar again, or the 'Play' button) and wait until the next action is finished. Then repeat step 2. and 3. until the end of the video.
5. Remember that you can also label the very last step in the video (that might end when the whole video ends).

Please NOTE:

• As soon as you start writing the labels, you have to finish the whole video in one go. You can't rewind or fast-forward the video! Before you have annotated something, you can watch the video and scroll through it as often as you want to. But when you start putting the labels in, there will be just one run.
• You need to label at least 5 steps in the video, and at most 15
• Please remember to enter your labels for a step in the video right after they are finished in the video!
• Use complete English sentences.
• There is no clear definition of a 'basic step', we leave that to your intuition. A good hint is that you should be able to easily name the step in one sentence. If you can't, the step is probably too complicated or too short.

Technical information

• Download speed and fluent video playback depend on your internet bandwidth and the working memory of your computer. If things seem too slow or brittle for you, we recommend you not to accept the HIT.
• Before accepting the HIT, please make sure that the download proceeds beyond a few percent. In case the download of the video stops after a few percent or does not even start, you should refresh your Browser window or skip to the next HIT.
• Some of the videos in the HITs can be large. For those you might have to set a higher Browser cache limit. In Firefox you can find this setting under 'Preferences -> Advanced -> Network'; then activate 'Overwrite automatic cache management' and set a value of 1024 MB.
• Note: The current HIT contains a large video. (only displayed if the video contained more than 8000 frames)

Our goal here is to build software than can make recipe advice better by estimating more precisely how complicated a recipe is and how long it actually takes to cook the meal. Therefore we need to know how the different ingredients are processed (e.g. a tomato is easier to prepare than a pomegranate) and how long the single steps take (e.g. peeling an onion takes longer than peeling a banana).