Patent AU-2016225819-B2 (AU)
Title: Structured knowledge modeling and extraction from images
Inventor: COHEN SCOTT
IPC: G06K9/46
Filed: 2016-09-07
Granted: 2021-04-22
Abstract: Techniques and systems are described to model and extract knowledge 
from images. A digital medium environment is configured to learn and use a 
model to compute a descriptive summarization of an input image automatically 
and without user intervention. Training data is obtained to train a model using 
machine learning in order to generate a structured image representation that serves 
as the descriptive summarization of an input image. The images and associated 
text are processed to extract structured semantic knowledge from the text, which is 
then associated with the images. The structured semantic knowledge is processed 
along with corresponding images to train a model using machine learning such 
that the model describes a relationship between text features within the structured 
semantic knowledge. Once the model is learned, the model is usable to process 
input images to generate a structured image representation of the image.  
Wolfe-SBMC Docket No.: P5726-US-PROVI-ORGI 
60
Inventors- Cohen at al.  
Title- Structured Knowledge Modeling and Extraction from Images 
100 
....................................................................................................................................................................................................................................  
08 
......................... ........ ...............  .................................. ......  ........................ ........ .......  ...............................  ............................. .........  ...........................................  .. .........................  ......................................... ........  .. .......................... ...............  ....................................... ..........  .. .......................... ...............  ...................................... ........  .. ............................... ...............  .............................. .........  . ......................... .... ................  .. ..................................  .. .........................  .....................................  .....................................  ..................................  ..............................  ...............................  .. ............................  ...........................  ...............................  ..............................  ...............................  ..............................  ............................... ....  ..............................  ...............................  ...............  .. ......... .  ..........................  ...........................  
...............................  ........... ..... OF ..  ....................... ............... ........... .. ....  . .................... .................. ............  .............  ................... ........ .....  .....................  .....................  ............................... ... ..  ........................  ...............................  ......................  .................................  .........................  ...............................  ....................  ...............  ............... - -......  .................  .....................  ............................................ ............  .................... ...... .......  .... ........  ..................................  ....................... i i i > * : I I .... I ..................  ..................... ......  -H .. ..............  M , :.:. .1 ... ..... ....  ..............  ............ ...... ..... .................... ........................... ......  .................................. .................. _ .........  ............ ....... ........ . .....  ................ ....... ......  ..................... .......  . .. ..... ................................. ....................  ........ ........ 11 ............ ....... ..................  ............. ....... ..................  .......... . ........................  ............................  ........... ...................  X ...........  .... ..... ......... .. ............ ............  ...........  N 2 ME~ ...  .............  .................  110% ..............  no a 100 ............  ................. NOX A ................ ......... .................  ............... ..................  NEW, ....................  ...... .....................  
............... .....  
...........  ............  ............. .............  .............  ...........  .............  
..............  A: .......... .......  
............................. .......  
Computing Device 102 
Knowledge 
Extraction System Database 
104 112 -------------------------
--------------------
C aption Structured Image 
Generation Representation Network 
Syste m 1 1 8 106 116 
---- --- --
Image Search 0 
Module 110 a