Patent application EP19172163A (EP), filed 2019-05-02
Publication number: EP3734498A1
Title: AN APPARATUS AND METHOD FOR PERFORMING IMAGE-BASED DISH RECOGNITION
Abstract: There is provided a computer-implemented method for performing image-based dish recognition. The method comprises: acquiring (202) a first image depicting a dish to be recognized; analyzing (204) the first image using a predictive model to determine a first candidate theme; acquiring (206) a reference set of dish identifiers; calculating (208), for each of the dish identifiers in the reference set, an association score indicative of a degree of similarity between the dish represented by the respective dish identifier and the first candidate theme; selecting (210) one or more dish identifiers in the reference set having the highest association scores as one or more variant dish identifiers of the first candidate theme; and outputting (212) the centroid dish identifier of the first candidate theme and the one or more variant dish identifiers of the first candidate theme.
IPC codes: G06V  20/    68            A I
Claims: number: 1
text: A computer-implemented method for performing image-based dish recognition, the method comprising:
number: 2
text: A computer-implemented method according to claim 1, wherein outputting (212) the centroid dish identifier (310) of the first candidate theme (31) and the one or more variant dish identifiers (311, 312, 313, 314) of the first candidate theme comprises:
number: 3
text: A computer-implemented method according to claim 1 or claim 2, wherein the plurality of candidate dish identifiers in the first candidate theme represent dishes that are similar to each other.
number: 4
text: A computer-implemented method according to any one of the preceding claims, further comprising:
number: 5
text: A computer-implemented method according to claim 4, further comprising, for each of the one or more additional candidate themes, the following method steps:
number: 6
text: A computer-implemented method according to claim 4 or claim 5, further comprising:
number: 7
text: A computer-implemented method according to any one of the preceding claims, further comprising:
number: 8
text: A computer-implemented method according to claim 7, further comprising:
number: 9
text: A computer-implemented method according to claim 7 or claim 8, wherein calculating the similarity score between each of the recipes in the core subset is based on one or more synonyms of at least one of: the dish identifiers, the plurality of food ingredients, and the cooking instructions of the two recipes.
number: 10
text: A computer-implemented method according to any one of claims 7 to 9, wherein clustering the plurality of recipes into a plurality of reference themes is based on K-means clustering or singular value decomposition.
number: 11
text: A computer-implemented method according to any one of claims 7 to 10, further comprising:
determining, for each of the plurality of reference themes, a plurality of keywords based on the recipes of the respective reference themes, wherein each of the plurality of keywords is associated with at least one of a cooking technique and a food ingredient;
number: 12
text: A computer-implemented method according to any one of claims 7 to 11, further comprising:
number: 13
text: A computer-implemented method according to any one of the preceding claims, wherein the predictive model is at least one of: a convolutional neural network, a residual neural network, and a dense neural network.
number: 14
text: A computer program product comprising a computer readable medium, the computer readable medium having computer readable code embodied therein, the computer readable code being configured such that, on execution by a suitable computer or processor, the computer or processor is caused to perform the method as claimed in any one of the preceding claims.
number: 15
text: An apparatus (100) for performing image-based dish recognition, the apparatus comprising a processor (102) configured to:
Office action: search_report, dated 2020-11-04
Rejection reasons: inventive_step_epc_56
Prior art cited: ref_id: US2016063692A1
ref_type: patent
severity: background
ref_id: - LUIS HERRANZ ET AL: "Food recognition and recipe analysis: integrating visual content, context and external knowledge", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 22 January 2018 (2018-01-22), XP081208784
ref_type: npl
severity: background
ref_id: US2016063692
ref_type: patent
severity: obviousness
ref_id: - LUIS HERRANZ ET AL, "Food recognition and recipe analysis: integrating visual content, context and external knowledge", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20180122), XP081208784 [Y] 1-15 * abstract *
ref_type: npl
severity: obviousness
Final disposition: pending (2020-11-04)