Source: https://patents.google.com/patent/US9563706B2/en
Timestamp: 2019-08-25 12:01:44
Document Index: 741168081

Matched Legal Cases: ['application No. 200810173071', 'application No. 200810173071', 'application No. 200810173071', 'application No. 200810173071', 'application No. 09829705', 'application No. 2011', 'application No. 2011', 'Application No. 2008101730716']

US9563706B2 - Image search apparatus and methods thereof - Google Patents
US9563706B2
US9563706B2 US14/243,439 US201414243439A US9563706B2 US 9563706 B2 US9563706 B2 US 9563706B2 US 201414243439 A US201414243439 A US 201414243439A US 9563706 B2 US9563706 B2 US 9563706B2
US14/243,439
US20140214792A1 (en
2008-11-26 Priority to CN200810173071.6 priority Critical
2008-11-26 Priority to CN200810173071 priority
2008-11-26 Priority to CNA2008101730716A priority patent/CN101414307A/en
2009-11-12 Priority to US12/663,743 priority patent/US8738630B2/en
2014-04-02 Priority to US14/243,439 priority patent/US9563706B2/en
2014-04-02 Application filed by Alibaba Group Holding Ltd filed Critical Alibaba Group Holding Ltd
2014-07-31 Publication of US20140214792A1 publication Critical patent/US20140214792A1/en
2014-09-30 Assigned to ALIBABA GROUP HOLDING LIMITED reassignment ALIBABA GROUP HOLDING LIMITED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LIN, XUANYIN
2017-02-07 Publication of US9563706B2 publication Critical patent/US9563706B2/en
This application claims priority to and is a continuation of U.S. patent application Ser. No. 12/663,743, filed Dec. 9, 2009, entitled “IMAGE SEARCH APPARATUS AND METHODS THEREOF” which is a national stage application of international patent application PCT/US09/64210, filed Nov. 12, 2009, entitled “IMAGE SEARCH APPARATUS AND METHODS THEREOF” claiming priority from Chinese patent application No. 200810173071.6, filed Nov. 26, 2008, entitled “METHOD AND SERVER TO PROVIDE FOR IMAGE SEARCH”, which applications are hereby incorporated in their entirety by reference.
A Tags Relating to Version
Exif Version Exif Version 36864 9000 UNDEFINED 4
Supported Flashpix version FlashpixVersion 40960 A000 UNDEFINED 4
B Tag Relating to Image Data characteristics
Color space information ColorSpace 40961 A001 SHORT 1
C Tags Relating to Image Configuration
Meaning of each components ComponentsConfiguration 37121 9101 UNDEFINED 4
Image compression mode CompressedBitsPerPixel 37122 9102 RATIONAL 1
Valid image width PixelXDimension 40962 A002 SHORT OR LONG 1
Valid image height PixelXDimension 40963 A003 SHORT OR LONG 1
D Tags Relating to User Information
Manufacturer notes MakerNotes 37500 927C UNDEFINED Any
E Tags Relating to Related File Information
Related Audio File RelatedSoundFile 40964 A004 ASCII 13
A practical scenario is used to explain the details of the above description and provided herewith. On an online transaction platform, a buyer may not know a certain product in a certain image, e.g., a shirt worn by Jay Chou on the cover of CD <<I am very busy>>, and wants to obtain information of the product. The user may promulgate the image on the web page (i.e., the cover of the CD <<I am very busy>> by Jay Chou) and adds annotation content (e.g., the annotation content is of a question type, inquiring product information of the shirt in the image). The user may further frame and label the shirt in the image to intuitively point out his/her desire to know the information of the product. Moreover, while questioning, the user may select a category associated with the product in the web page, such as clothes/shirt, for example. Upon seeing the image, other users or the seller may continue to add annotation contents for the shirt in the image. For example, if the annotation content is of an answer type, the annotation content may include answers to the questions (e.g., information such as brand, texture, material and style of the shirt) and may further include an address of the seller of the product such as a URL address. The other users or the seller may also select a category associated with the product when adding the annotation contents or when answering the questions. Furthermore, any interested user may acquire this image through searching to obtain the questions and the answers associated with the image and enter into a transaction web page of the product through the seller's URL link in the annotation content to complete a trans action.
Ring: MP3 ring Camera Resolution: 0.2 M
1. A computer method implemented by one or more computing devices, the method comprising:
receiving one or more pieces of textual annotation content for an image from one or more clients, the one or more pieces of textual annotation content comprising at least a link to a web page associated with merchandise corresponding to the image and an icon for initiating an instant messaging session with a seller of the merchandise;
examining a legitimacy of the textual annotation content, the examining including applying a word filter based on a predetermined word lexicon to filter out illegitimate words;
storing the one or more pieces of the textual annotation content for the image, wherein the storing the one or more pieces of the textual annotation content includes
storing the textual annotation content having a highest score in auxiliary information of the image, wherein the storing the textual annotation content having the highest score in the auxiliary information of the image comprises storing the textual annotation content in a user comments tag in accordance with Exchangeable image file format (Exif) specification, and
providing the one or more pieces of textual annotation content as search indices for the image to an image search engine; and
ordering the search indices specifically in the database, including establishing reverse indices, wherein a format of question and answer and a scoring rule are organized for the one or more pieces of textual annotation content and the textual annotation content having the highest score corresponds to an answer receiving a highest score from one or more users.
2. The method of claim 1, further comprising: examining the one or more pieces of textual annotation content for the image submitted by the one or more clients prior to storing the one or more pieces of textual annotation content.
3. The method of claim 1, wherein storing the textual annotation content having the highest score in the database and associating the textual annotation content having the highest score with the image comprises:
storing in the database merchandise information on the web page that is linked by a Uniform Resource Locator (URL) link of the seller of the merchandise in the textual annotation content and associating the textual annotation content with the image when the textual annotation content includes the URL link, and
wherein storing the textual annotation content having the highest score in the auxiliary information of the image comprises:
storing in the auxiliary information of the image the merchandise information on the web page that is linked by the URL link in the textual annotation content; and storing the image.
4. The method of claim 1, wherein the one or more pieces of textual annotation content comprises one or more of: texture of the merchandise in the image, material of the merchandise in the image, specification of the merchandise in the image, place of production of the merchandise in the image, unit price of the merchandise in the image, use of the merchandise in the image, a Uniform Resource Locator (URL) address of the seller of the merchandise in the image, a contact method of the seller, or a comment for the merchandise.
5. An apparatus that enables image search, comprising:
computer-readable storage media communicatively connected to the one or more processors;
a web page response unit stored in the computer-readable media and configured to receive a request from a user client and return a web page in response to the request, the returned web page allowing an image to be uploaded and textual annotation content to be submitted for the image;
a storage unit communicatively coupled to the one or more processors and configured to:
receive one or more pieces of textual annotation content for the image from one or more clients, the one or more pieces of textual annotation content comprising at least a link to a web page associated with merchandise corresponding to the image and an icon for initiating an instant messaging session with a seller of the merchandise,
examine a legitimacy of the textual annotation content, the examining including applying a word filter based on a predetermined word lexicon to filter out illegitimate words, and
store the one or more pieces of the textual annotation content for the image including storing the textual annotation content having a highest score in auxiliary information of the image, wherein the storing the textual annotation content having the highest score in the auxiliary information of the image comprises storing the textual annotation content in a user comments tag in accordance with Exchangeable image file format (Exif) specification; and
a search engine interface configured to provide the one or more pieces of textual annotation content as search indices to an image search engine, wherein the database performs a specific ordering of the search indices, including establishing reverse indices, and wherein a format of question and answer and a scoring rule are organized for the one or more pieces of textual annotation content and the textual annotation content having the highest score corresponds to an answer receiving a highest score from one or more users.
6. The apparatus of claim 5, wherein the storage unit comprises: the database; and an association unit, wherein the database is configured to store the textual annotation content, and wherein the association unit is configured to associate the textual annotation content stored in the database with the image corresponding thereto.
a best textual annotation content selection unit configured to select the textual annotation content with the highest score after scoring in accordance with the scoring rule, wherein the database stores the textual annotation content with the highest score, and wherein the association unit associates the textual annotation content with the highest score stored in the database with the image.
a Uniform Resource Locator (URL) link parsing unit configured to parse merchandise information on the web page that is linked by a URL link in the textual annotation content, wherein the database is further configured to store the merchandise information on the web page that is linked by the URL link in the textual annotation content.
a Uniform Resource Locator (URL) link parsing unit configured to parse merchandise information on the web page that is linked by a URL link in the textual annotation content; and an annotation parser configured to write in the auxiliary information of the image the merchandise information on the web page that is linked by the link in the textual annotation content.
an annotation parser configured to write the textual annotation content having the highest score into the auxiliary information of the image,
wherein the storage unit stores the image that has the auxiliary information having the textual annotation content having the highest score.
11. The apparatus of claim 10, wherein upon receiving a request for retrieving the one or more pieces of textual annotation content from the image search engine, the annotation parser parses the textual annotation content having the highest score from the auxiliary information of the image and returns the textual annotation content having the highest score to the user client, or returns the textual annotation content having the highest score to the user client through the image search engine.
12. One or more non-transitory computer-readable media storing instructions, which when executed by one or more processors, cause the one or more processors to perform a method comprising:
storing the one or more pieces of textual annotation content for the image, wherein the storing the one or more pieces of textual annotation content includes storing the textual annotation content having a highest score in auxiliary information of the image, wherein the storing the textual annotation content having the highest score in the auxiliary information of the image comprises storing the textual annotation content in a user comments tag in accordance with Exchangeable image file format (Exif) specification; and
US14/243,439 2008-11-26 2014-04-02 Image search apparatus and methods thereof Active US9563706B2 (en)
CN200810173071 2008-11-26
US12/663,743 US8738630B2 (en) 2008-11-26 2009-11-12 Image search apparatus and methods thereof
US14/243,439 US9563706B2 (en) 2008-11-26 2014-04-02 Image search apparatus and methods thereof
US15/376,607 US20170091330A1 (en) 2008-11-26 2016-12-12 Image search apparatus and methods thereof
PCT/US2009/064210 Continuation WO2010062800A1 (en) 2008-11-26 2009-11-12 Image search apparatus and methods thereof
US12/663,743 Continuation US8738630B2 (en) 2008-11-26 2009-11-12 Image search apparatus and methods thereof
US15/376,607 Continuation US20170091330A1 (en) 2008-11-26 2016-12-12 Image search apparatus and methods thereof
US20140214792A1 US20140214792A1 (en) 2014-07-31
US9563706B2 true US9563706B2 (en) 2017-02-07
US12/663,743 Active 2030-07-02 US8738630B2 (en) 2008-11-26 2009-11-12 Image search apparatus and methods thereof
US14/243,439 Active US9563706B2 (en) 2008-11-26 2014-04-02 Image search apparatus and methods thereof
US15/376,607 Abandoned US20170091330A1 (en) 2008-11-26 2016-12-12 Image search apparatus and methods thereof
US20040208482A1 (en) 1995-10-03 2004-10-21 Canon Kabushiki Kaisha Recording apparatus, reproducing apparatus, recording method, and reproducing method
JP2007087409A (en) 2006-10-30 2007-04-05 Noritsu Koki Co Ltd Print order system using network
US20070092142A1 (en) 2005-10-25 2007-04-26 General Electric Company Automatic significant image generation based on image characteristics
US20080141128A1 (en) 2003-03-27 2008-06-12 Casio Computer Co., Ltd. Display processing device, display processing method and display control program
JP2008219091A (en) 2007-02-28 2008-09-18 Canon Inc Image processing system, information processor, its control method, storage medium, and program
US20080249891A1 (en) 2004-06-08 2008-10-09 Picsout Ltd. Method for Presenting Visual Assets for Sale, Using Search Engines
US20080262960A1 (en) 2007-04-18 2008-10-23 Wade Malone Automated Electronic Commerce Data Extracting and Analyzing System
US20090171918A1 (en) 2003-09-23 2009-07-02 Udi Manber Personalized searchable library with highlighting capabilities
US20090175532A1 (en) 2006-08-01 2009-07-09 Konstantin Zuev Method and System for Creating Flexible Structure Descriptions
US20110191211A1 (en) 2008-11-26 2011-08-04 Alibaba Group Holding Limited Image Search Apparatus and Methods Thereof
Bottoni et al., "Storing and Retrieving Multimedia Web Notes," In: "Field Programmable Logic and Application", Jan. 1, 2005, Berlin, Hiedelbert, vol. 3433, pp. 119-137.
Chinese Office Action mailed Apr. 8, 2011 for Chinese patent application No. 200810173071.6, a counterpart foreign application of U.S. Appl. No. 12/663,743, 36 pages.
Chinese Office Action mailed Aug. 23, 2010 for Chinese patent application No. 200810173071.6, a counterpart foreign application of U.S. Appl. No. 12/663,743, 20 pages.
Chinese Office Action mailed Sep. 6, 2011 for Chinese patent application No. 200810173071.6, a counterpart foreign application of U.S. Appl. No. 12/663,743, 38 pages.
Extended European Search Report mailed Nov. 27, 2012 for European patent application No. 09829705.4, 8 pages.
Final Office Action for U.S. Appl. No. 12/663,743, mailed on Jul. 18, 2013, Xuanyin Lin, "Image Search Apparatus and Methods Thereof", 74 pages.
Non-Final Office Action for U.S. Appl. No. 12/663,743, mailed on Apr. 12, 2012, Xuanyin Lin, "Image Search Apparatus and Methods Thereof", 14 pages.
Non-Final Office Action for U.S. Appl. No. 12/663,743, mailed on Mar. 1, 2013, Xuanyin Lin, "Image Search Apparatus and Methods Thereof", 62 pages.
Non-Final Office Action for U.S. Appl. No. 12/663,743, mailed on Sep. 10, 2012, Xuanyin Lin, "Image Search Apparatus and Methods Thereof", 33 pages.
Translated Japanese Office Action mailed Dec. 10, 2013 for Japanese patent application No. 2011-538621, a counterpart foreign application of U.S. Appl. No. 12/663,743, 9 pages.
Translated Japanese Office Action mailed Jun. 17, 2014 for Japanese patent application No. 2011-538621, a counterpart foreign application of U.S. Appl. No. 12/663,743, 4 pages.
Translation of Chinese Office Action from corresponding Chinese Patent Application No. 2008101730716, 33 pages.
EP2353106A1 (en) 2011-08-10
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LIN, XUANYIN;REEL/FRAME:033849/0052