However, "understanding" is a difficult problem because it cannot be defined easily. We address this problem by researching procedural text. A procedural text is a set of instructions that describe how to make something, and we can represent this as a graph structure.
Our research focuses on recipes as procedural texts. Millions of recipes have been published on the Internet and in books. It is also easy to find cooking videos.
To represent the "understanding" of cooking recipes we use a flow graph structure. We construct a recipe flow graph corpus (r-FG corpus) that represents the structure of recipes and publish them. The following are possible future applications of the corpus.
|F||Food||Include pronoun expression and intermediate/completed food.|
|T||Tool||Cooking tools, dishes, etc.|
|D||Duration||Include round numbering.|
|Q||Quantity||Include round numbering.|
|Ac||Action by the chef||Separate inflectional endings.|
|Af||Action by foods||Separate inflectional endings.|
|Sf||State of foods||Taste, minced, sliced, etc.|
|St||state of tools||Temperature, size of tools, etc|
r-NEs are non-nested and non-overlapping as well as general named entity.
r-NEs are annotated with not only naun phrases (Foods, Tools, etc.) but also verb and adjective phrases (Action by the chef, State of food, etc.). We separate inflectional endings from adjectives, verbs, and auxiliary verbs (Ac, Af).
We annotate r-FG corpus with r-NEs (verticies) and relations of two r-NE (arcs), and we publish the corpus which are annotated with only r-NEs. You can use r-NE corpus as training corpus for our named entity recognizer (PWNER), that can recognize r-NE automatically.
|Targ||Target (direct object)|
|Dest||Destination (indirect object)|
|V-tm||Head of a clause for timing, etc.|
Predicate argument structure is the most important because of the main component of steps of recipes is the relationship between "foods" and "actions" (Agent, Targ, Dest, T-comp, F-comp).
This type associate a food/tool which has emerged the past once or has processed and first appeared food/tool (F-eq, F-part-of, F-set, T-eq, T-part-of).
Sometimes an action verb is repeatedly written just to specify an object such as "carrot you've cut." This verb to the verb which the chef has to really execute as follows (A-eq).
Some clauses, whose head is a verb phrase annotated with Ac or Af, specify the timing or the condition of another action (V-tm).
All the other modification relationships fall into this category (Other-mod).
Notice: Arc label names [subj, d-obj, i-obj] in past papers are changed to [Agent, Targ, Dest] respectively.
This is a toolset for r-FG corpus. Note that this toolset does not include original recipe sentences but only annotation information. You can get original recipe sentences from "Cookpad Data Set". You can make r-FG corpus and r-NE corpus by using Both of them. Please see 00readme.txt in the toolset for details.
If this toolset does'nt work, please contact the author.
Steps of recipe are represented by a flow graph (r-FG/r-NE corpus does not include title and ingredients). Real files are shown in the right figure (csv format). One recipe corresponds to one file, one line corresponds to one vertex.
The followings are elements of a recipe flow graph file.
We annotated following information to the preparation of recipe text. Real files are shown in the right figure (tree format, available for PNAT). One recipe corresponds to one file, one line corresponds to one word. The followings are elements of a sentence in r-NE corpus file.
KUSK Dataset 2014RC
Kyoto University Smart Kitchen dataset (KUSK dataset) is a dataset of activity observation in a kitchen.
KUSK dataset have 20 real-cooking videos from 20 recipe.
, which are the data source of KUSK dataset 2014RC, are included in our corpus.
- A Framework for Procedural Text Understanding,
- Hirokuni Maeta, Tetsuro Sasada, Shinsuke Mori
- IWPT, 2015.
- KUSK Dataset: Toward a Direct Understanding of Recipe Text and Human Cooking Activity Information
- Atsushi Hashimoto, Tetsuro Sasada, Yoko Yamakata, Shinsuke Mori, Michihiko Minoh,
- CEA, 2014.
- Flow Graph Corpus from Recipe Texts
- Shinsuke Mori, Hirokuni Maeta, Yoko Yamakata, Tetsuro Sasada
- to appear in LREC, 2014.
- Structural Analysis of Cooking Preparation Steps in Japanese.
- Reiko HAMADA, Ichiro IDE, Shuichi SAKAI, Hidehiko TANAKA.
- The fifth International Workshop on Information Retrieval with Asian Languages, pp. 157–164. 2000.
- Control Structures for Actions in Procedural Texts and PT-Chart
- Yoshio Momouchi.
- The 8th International Conference on Computational Linguistics, Vol.1, pp. 108–114. 1980.