Patent application EP20181762A (EP), filed 2020-06-23
Publication number: EP3757879A1
Title: METHOD AND APPARATUS FOR INDIVIDUALLY APPLYING ENHANCEMENT TO DETECTED OBJECTS IN AN IMAGE
Abstract: Disclosed is an electronic apparatus comprising, a memory configured to store instructions; and at least one processor connected to the memory, and configured to detect at least one object of a first-class object or a second-class object included in a target image by the electronic apparatus using an artificial intelligent algorithm to apply the target image to a learned neural network model, and identify and apply an image-quality processing method to be individually applied to at least one detected object, the neural network model is set to detect an object included in an image, as trained based on learning data such as an image, a class to which the image belongs, information about the first-class object included in the image, and information about the second-class object included in the image.a
IPC codes: G06N   3/    04            A I
G06T   5/    00            A I
G06T   7/    11            A I
G06V  10/    44            A I
G06V  10/    56            A I
G06V  10/   764            A I
G06V  10/   776            A I
G06V  20/    00            A I
Claims: number: 1
text: An electronic apparatus comprising:
number: 2
text: The electronic apparatus according to claim 1, wherein the neural network model is trained to detect the first-class object and the second-class object included in the target image, based on learning data including an learning image, a class to which the learning image belongs, information about a first-class object included in the learning image, and information about a second-class object included in the learning image.
number: 3
text: The electronic apparatus according to claim 2, wherein
the information about the first-class object comprises at least one of information about pixels corresponding to an object, information about a bounding box surrounding the object, and information about a texture of the object, and
the information about the second-class object comprises at least one of information about pixels corresponding to an object, and information about a texture of the object.
number: 4
text: The electronic apparatus according to one of claims 1 or 2, wherein the trained neural network model comprises a feature-extraction neural network to estimate a class to which an image belongs, a first-class object detection neural network to detect the first-class object, and a second-class object detection neural network to detect the second-class object.
number: 5
text: The electronic apparatus according to claim 4, wherein the feature-extraction neural network comprises a plurality of layers comprising a plurality of filters.
number: 6
text: The electronic apparatus according to claim 5, wherein the plurality of layers comprises:
number: 7
text: The electronic apparatus according to claim 5, wherein the at least one processor is configured to perform processing to extract features of the target image through at least some filters among the plurality of filters, and decrease result values through at least some other filters.
number: 8
text: The electronic apparatus according to one of claims 4 to 7, wherein the at least one processor is configured to perform processing to detect the first-class object included in the target image by applying the features of the target image obtained through the feature-extraction neural network to the first-class object detection neural network.
number: 9
text: The electronic apparatus according to claim 7, wherein the at least one processor is configured to perform processing to control the first-class object detection neural network to merge a result of detecting the first-class object based on at least one of the texture information, the color information, or the edge information of the target image, and a result of detecting the first-class object based on at least one of the color information or the edge information about a shape of at least some areas of an object included in the target image, and identify the first-class object having a high confidence level.
number: 10
text: The electronic apparatus according to one of claims 5 to 7, wherein the at least one processor is configured to perform processing to control the second-class object detection neural network to detect the second-class object by increasing result values, which are obtained from applying the plurality of filters to the features of the target image obtained in each layer of the feature-extraction neural network, into a preset size and then merging the result values.
number: 11
text: A method comprising:
number: 12
text: The method according to claim 11, wherein the trained neural network model comprises a feature-extraction neural network configured to estimate a class to which an image belongs, a first-class object detection neural network to detect the first-class object, and a second-class object detection neural network to detect the second-class object.
number: 13
text: The method according to claim 12, wherein the feature-extraction neural network comprises a plurality of layers comprising a plurality of filters.
number: 14
text: The method according to claim 13, wherein the plurality of layers comprises:
number: 15
text: A computer-readable recording medium storing instructions that are executable by at least one processor to:
Office action: search_report, dated 2020-12-30
Rejection reasons: novelty_epc_54
inventive_step_epc_56
Prior art cited: ref_id: EP1318475A1
ref_type: patent
severity: background
ref_id: US2018285679A1
ref_type: patent
severity: background
ref_id: KR20180051367A
ref_type: patent
severity: background
ref_id: EP3531370A2
ref_type: patent
severity: background
ref_id: WO2019072057A1
ref_type: patent
severity: background
ref_id: EP3674967A1
ref_type: patent
severity: background
ref_id: US2018189615A1
ref_type: patent
severity: background
ref_id: - SHU LIU ET AL: "Path Aggregation Network for Instance Segmentation", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 5 March 2018 (2018-03-05), XP081555910
ref_type: npl
severity: background
ref_id: - LI-JIA LI ET AL: "Towards total scene understanding: Classification, annotation and segmentation in an automatic framework", 2009 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION : CVPR 2009 ; MIAMI [BEACH], FLORIDA, USA, 20 - 25 JUNE 2009, IEEE, PISCATAWAY, NJ, 20 June 2009 (2009-06-20), pages 2036 - 2043, XP031607245, ISBN: 978-1-4244-3992-8
ref_type: npl
severity: background
ref_id: EP1318475
ref_type: patent
severity: novelty_destroying
ref_id: KR20180051367
ref_type: patent
severity: novelty_destroying
ref_id: US2018189615
ref_type: patent
severity: background
ref_id: US2018285679
ref_type: patent
severity: novelty_destroying
ref_id: WO2019072057
ref_type: patent
severity: obviousness
ref_id: EP3531370
ref_type: patent
severity: background
ref_id: EP3674967
ref_type: patent
severity: background
ref_id: - SHU LIU ET AL, "Path Aggregation Network for Instance Segmentation", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, (20180305), XP081555910 [A] 1,4,5,7,9-11,13,15 * Section 3.2;; figure 1 *
ref_type: npl
severity: background
Final disposition: pending (2020-12-30)