Patent Description:
With the development of intellectual terminal technology, functions of intellectual terminals are becoming more and more varied. For example, a user can use a camera in a terminal to take an image (e.g., a photo or a video), which has become a new mode of entertainment for people. However, simply taking images can no longer meet needs of users. How to perform special effect processing on captured images to get more fun and interesting images has become the focus of people's attention.

In prior art, the special effect is usually applied to a whole image. In some scenarios, the special effect processing method cannot achieve a good result. For example, a face sticker is usually applied to a face, and if the face sticker is applied to the whole image, which would make the whole image look strange and cannot achieve a good result.

<CIT> discloses an image processing method and apparatus, a terminal device and a storage medium thereof, and provides a scheme for superimposing a plurality of AR special effects image processing in an image so as to satisfy the user's interest in image processing.

<CIT> relates to a presentation application and to extracting a portion of an object displayed on slides of presentations using a presentation application.

<CIT> discloses a technical solution suitable for application in small videos and live videos, providing rich and colourful design elements for small videos and live videos, realizing flexible special effect organization and design, and realizing free combination and switching between different scenes.

<CIT> relates to a system and method for automatically applying special effects based on media content characteristics.

<CIT> provides an electronic device which includes a display, a memory configured to store instructions, and a processor configured to execute the stored instructions to display an image and a user interface (UI) representing depth information of the image through the display in response to a first user input for editing the image.

<CIT> discloses that a messaging application creates a custom sticker based on the processed image and locally stores the custom sticker. <CIT> relates to a mobile device with a camera and an image processing method suitable for a mobile device. <CIT> relates to a special effect display method and device, a terminal and a storage medium, and belongs to the technical field of image processing.

The summary is provided to introduce concepts in a simplified form, and the concepts will be described in detail in the following description of embodiments. The summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

The technical purpose to be achieved by the present disclosure is to provide an image special effect processing method, so as to at least partially solve the technical problem that a good result cannot be achieved through the special effect processing method in the prior art. In addition, an image special effect processing apparatus, an image special effect processing hardware apparatus, a computer-readable storage medium and an image special effect processing terminal are also provided.

In order to achieve the above purpose, according to one aspect of the present disclosure, the following technical solution is provided:
an image special effect processing method, including:.

In order to achieve the above purpose, according to one aspect of the present disclosure, the following technical solution is provided:
an image special effect processing apparatus, including:.

In order to achieve the above purpose, according to one aspect of the present disclosure, the following technical solution is provided:
a computer-readable storage medium, configured to store non-transitory computer-readable instructions, where when the non-transitory computer-readable instructions are executed by a computer, the computer is caused to execute the image special effects processing method according to any one of the above aspects.

A computer program product, including a computer program instruction, where the computer program instructions causes a computer to execute the image special effect processing method according to any one of the above aspects.

A computer program, where when the computer program runs on a computer, the computer is caused to execute the image special effect processing method according to any one of the above aspects.

In order to achieve the above purpose, according to another aspect of the present disclosure, the following technical solution is provided:
an image special effect processing terminal, including the above image special effect processing apparatus.

In embodiments of the present disclosure, by determining a target display region of a special effect object on an initial image, and displaying the part of the special effect object located in the target display region on the initial image, the special effect object can be displayed differently, thereby achieving a better result.

The above description is only an overview of the technical solutions of the present disclosure. In order to understand the technical means of the present disclosure more clearly, it can be implemented according to the content of the description, and to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are provided in particular, in conjunction with the accompanying drawings, and details are as follows.

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent when taken in conjunction with the accompanying drawings and with reference to the following description of embodiments. Throughout the drawings, same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are illustrated in the accompanying drawings, it should be understood that the present disclosure may be implemented in various forms and should not be construed as limited to the embodiments set forth herein. By contrary, the embodiments are provided for the purpose of a more thorough and complete understanding of the present disclosure. It should be understood that the accompanying drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the protection scope of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders, or performed in parallel. Furthermore, method embodiments may include additional steps or execution of the illustrated steps may be omitted. The scope of the present disclosure is not limited in this regard.

As used herein, the term "including" and variations thereof refer to open inclusions, i.e., "including but not limited to". The term "based on" refers to "based at least in part on". The term "one embodiment" represents "at least one embodiment"; the term "another embodiment" represents "at least one additional embodiment"; the term "some embodiments" represents "at least some embodiments". Relevant definitions of other terms will be given in the description below.

In order to solve the technical problem that special effect processing methods in the prior art cannot achieve good results, an embodiment of the present disclosure provides an image special effect processing method. As shown in <FIG>, the image special effect processing method mainly includes the following steps S11 to S12.

Step S11: in response to an instruction for adding a special effect object on an initial image, determine a target display region of the special effect object on the initial image, where the target display region is a foreground region or a background region of the initial image, the foreground region is an image region where a target object is located in the initial image, and the background region is an image region other than the foreground region in the initial image.

Specifically, the initial image may be a video image input in real time, for example, a live video in a short video application, or a video image pre-stored in a terminal. The image can also be a static image, i.e., a picture. The terminal may be a mobile terminal, such as a smart phone or a tablet computer, or may be a fixed terminal, such as a desktop computer.

The instruction for adding a special effect object may include an identifier or a storage location of the special effect object, and the corresponding special effect object can be acquired from the Internet, a server or a terminal locally according to the identifier or the storage location. Alternatively, the instruction for adding a special effect object includes a special effect object, and in this case, there is no need to additionally acquire the corresponding special effect object, instead, the special effect object is directly parsed from the instruction.

There may be one or more special effect objects to be added, and when there are more than one special effect object, the target display region of each special effect object on the initial image is determined respectively. The target display region is the foreground region or the background region of the initial image. Specifically, the user can select a corresponding special effect object according to needs, and determine the target display region according to a type of the selected special effect object. For example, if a special effect object is a special effect (such as, a face sticker) used to be displayed in the foreground region, the target display region of the special effect object is determined to be the foreground region; if a special effect object is a special effect (such as, a bullet screen) used to be displayed in the background region, the target display region of the special effect object is determined to be the background region; or if a special effect object is to mark an edge of the foreground region, the target display region of the special effect object is determined to be the edge of the foreground region. An example is taken where the initial image is a face image, if the special effect object selected by the user is a face sticker, an instruction for adding the special effect is generated according to the face sticker, where the instruction for adding the special effect includes a target display region, and the target display region is the foreground region.

When determining the foreground region and the background region of the initial image, it is possible to separate the foreground region and the background region of the initial image through methods in the prior art. For example, a target recognition algorithm can be used to identify the image to obtain the image region where the target object is located, and the image region where the image region in which the target object is located is removed is the background region. The target object may be a human face, a human body, or the like. For example, if the image is a face image, a face region in the image can be identified through a face recognition algorithm, the face region is then taken as the foreground region, and the rest of the image is the background region.

Step S12: display a part of the special effect object located in the target display region on the initial image to obtain a target image.

Specifically, if the target display region is the foreground region, the part of the special effect object located in the foreground region is displayed on the initial image; if the target display region is the background region, the part of the special effect object located in the background region is displayed on the initial image. If the target display region is the edge of the foreground region, the edge of the foreground region is marked, for example, an edge line of the foreground region is represented by a colored line or dots.

In the embodiment, through determination of a target display region of a special effect object on an initial image, and displaying the part of the special effect object located in the target display region on the initial image, the special effect object can be displayed differently, thereby achieving a better result.

In an optional embodiment, the step S12 specifically includes:
step S121: determine an overlapping region of a position region of the special effect object on the initial image and the target display region.

The special effect object is displayed on parts of multiple target display regions, and the overlapping region of the position region of each part on the initial image and the target display region is determined. For example, if the special effect object includes a part displayed on the foreground region and a part displayed on the background region, the foreground region and the background region on the initial image are determined respectively.

Step S122: cut a part located in the overlapping region from the special effect object as a target display object.

For example, if the special effect object includes a part displayed on the foreground region and a part displayed on the background region, the part displayed on the foreground region and the part displayed on the background region are respectively cut from the special effect.

Step S123: display the target display object on an upper layer of the overlapping region of the initial image to obtain the target image.

For example, if the special effect object includes a part displayed on the foreground region and a part displayed on the background region, then the target display objects are the part displayed on the foreground region and the part displayed on the background respectively region, so that the part displayed on the foreground region and the part displayed on the background region are correspondingly displayed on the foreground region and the background region of the initial image.

In an optional embodiment, the method further includes:
Step S13: in response to an operation by a user on a special effect parameter setting option on a special effect setting interface, generate the instruction for adding the special effect object on the initial image, where the special effect parameter setting option is used for the user to set the special effect object and the target display region, and the instruction is used to indicate the special effect object and the target display region.

Specifically, the special effect parameter setting option can be provided on the special effect setting interface, and the number of the special effect parameter setting options can be more than one, as shown in <FIG>, including a special effect option for displaying on the foreground region (that is, the foreground special effect), a special effect option for displaying on the background region (that is, the background special effect), and a special effect option for marking the edge of the foreground region, etc. The user can select the setting parameters according to these options, and can add multiple background special effects and/or foreground special effects. After the user has finished selecting the setting parameters, an instruction for adding the special effect object can be generated through triggering of an OK button on the special effect setting interface.

Further, the step S13 specifically includes:
Step S131: in response to a selecting operation on a special effect navigation tree on a first interface, determine the special effect object.

Specifically, as shown in <FIG>, a special effect navigation tree can be provided on the first interface, and the special effect navigation tree includes special effect options, and there are at least two special effect options, that is, a foreground special effect option and a background special effect option, where the foreground special effect option includes at least one foreground effect, and the background effect option includes at least one background effect. The user can select the desired special effect object from the special effect options in the special effect navigation tree.

Step S132: in response to a selecting operation on a display region setting option on a second interface, determine the target display region.

The second interface and the first interface may be the same display interface.

Step S133: generate the instruction according to the special effect object and the target display region.

Further, the step S11 specifically includes: when the special effect object to be added includes a plurality of sub-effect objects, determine a target display region of each of the sub-effect objects respectively.

Correspondingly, the step S12 specifically includes: determining a display order of a respective sub-effect object in the special effect object respectively; displaying a part of the respective sub-effect object in the special effect object located in a respective target display region of the respective sub-effect object on the initial image in sequence according to the display sequence of the respective sub-effect object in the special effect object, to obtain the target image.

Further, before the step S12 is performed, the method further includes:.

Specifically, a special effect switch option is provided on the special effect setting interface for users to choose whether to distinguish the foreground region and the background region when the special effect object is added. If the user wants to distinguish between the foreground region and the background region, the special effect switch option needs to be turned on, and at this time, the execution of operations of determining the foreground region and the background region of the image and the subsequent steps is triggered. If the user does not want to distinguish between the foreground region and the background region, the special effect switch option needs to be turned off, at this time, the foreground region will not be distinguished from the background region when the special effect object is displayed, and the special effect object can be displayed anywhere on the initial image.

Those skilled in the art should understand that, on the basis of the above embodiments, obvious modifications (for example, superimposing the listed modes) or equivalent substitutions can also be made.

In the above, although the steps in the embodiments of the image special effect processing method are described in the above-mentioned order, those skilled in the art should understand that the steps in the embodiments of the present disclosure are not necessarily executed in the above-mentioned order, and may also be performed in a reverse order, in parallel, in a crossing sequence and in other sequences. Moreover, on the basis of the above steps, those skilled in the art can also add other steps, these obvious modifications or equivalent alternatives should also be included within the scope of protection of the present disclosure, which will not be repeated here.

The following describes apparatus embodiments of the present disclosure. The apparatus embodiments of the present disclosure can be used to execute the steps implemented by the method embodiments of the present disclosure. For convenience of description, only the parts related to the embodiments of the present disclosure are shown, for the specific technical details not disclosed, reference can be made to the method embodiments of the present disclosure.

In order to solve the technical problem that the special effect processing method in the prior art cannot achieve a good result, an embodiment of the present disclosure provides an image special effect processing apparatus. The apparatus may execute the steps in the embodiments of the image special effect processing method described in Embodiment I. As shown in <FIG>, the apparatus mainly includes: a target display region determining module <NUM> and a special effect displaying module <NUM>; where,.

Further, the special effect displaying module <NUM> is specifically configured to: determine an overlapping region of a position region of the special effect object on the initial image and the target display region; cut a part located in the overlapping region from the special effect object as a target display object; display the target display object on an upper layer of the overlapping region of the initial image to obtain the target image.

Further, the apparatus further includes:
an instruction generating module <NUM>, configured to: in response to an operation by a user on a special effect parameter setting option on a special effect setting interface, generate the instruction for adding the special effect object on the initial image, where the special effect parameter setting option is used for the user to set the special effect object and the target display region, and the instruction is used to indicate the special effect object and the target display region.

Further, the instruction generating module <NUM> is specifically configured to: in response to a selecting operation on a special effect navigation tree on a first interface, determine the special effect object; in response to a selecting operation on a display region setting option on a second interface, determine the target display region; and generate the instruction according to the special effect object and the target display region.

Further, the target display region determining module <NUM> is specifically configured to: when the special effect object to be added includes a plurality of sub-effect objects, determine a target display region of each of the sub-effect objects respectively; correspondingly, the special effect object displaying module <NUM> is specifically configured to: determine a display order of a respective sub-effect object in the special effect object respectively; display a part of the respective sub-effect object in the special effect object located in a respective target display region of the respective sub-effect object on the initial image in sequence according to the display sequence of the respective sub-effect object in the special effect object, to obtain the target image.

Further, the apparatus further includes:.

Further, the special effect object is a sticker and/or a frame marking the target display region.

For detailed descriptions on the working principle and the technical effects achieved by the embodiments of the image special effect processing apparatus, reference may be made to the relevant descriptions in the foregoing image special effect processing method embodiments, which will not be repeated here.

Referred to <FIG> below, it shows a schematic structural diagram of an electronic device <NUM> adapted to implement the embodiments of the present disclosure. The electronic device <NUM> in the embodiments of the present disclosure may include, but is not limited to, a mobile device, such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (Portable Android Device), a PMP (Personal Multimedia Player), a vehicle terminal (e.g., a vehicle navigation terminal), etc., and a fixed terminal such as a digital TV, a desktop computer, etc. The electronic device shown in <FIG> is only an example, and should not bring any limitation to the function and scope of use for the embodiments of the present disclosure.

As shown in <FIG> , the electronic device <NUM> may include a processing apparatus <NUM> (such as a central processing unit, a graphics processor, etc.), which may execute various appropriate actions and processing according to programs stored in a read-only memory (Read-Only Memory, ROM) <NUM> or programs loaded into a random access memory (RAM) <NUM>. In the RAM <NUM>, various programs and data required for the operation of the electronic device <NUM> are also stored. The processing apparatus <NUM>, the ROM <NUM>, and the RAM <NUM> are connected to each other through a bus <NUM>. An input/output (I/O) interface <NUM> is also connected to the bus <NUM>.

Generally, the following apparatuses can be connected to the I/O interface <NUM>: an input apparatus <NUM>, including for example a touch screen, a touch panel, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; an output apparatus <NUM>, including for example a liquid crystal display (LCD), a speaker, a vibrator, and the like; a storage apparatus <NUM>, including for example a magnetic tape, a hard disk, and the like; and a communication apparatus <NUM>. The communication apparatus <NUM> allows the electronic device <NUM> to perform wireless or wired communication with other devices to exchange data. Although <FIG> shows the electronic device <NUM> with multiple kinds of apparatuses, it is not required to implement or have all the apparatuses shown in <FIG>. It may alternatively be implemented or provided with more or fewer apparatuses.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program codes for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication apparatus <NUM>, or installed from the storage apparatus <NUM>, or installed from the ROM <NUM>. When the computer program is executed by the processing apparatus <NUM>, the above-mentioned functions defined in the method of the embodiments of the present disclosure are executed.

It should be noted that, the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the both. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that includes or stores programs, and the programs may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein. This propagated data signal may adopt many forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, the computer-readable signal medium may send, propagate, or transmit the program used by or in combination with the instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted by any suitable medium, including but not limited to: a wire, an optical cable, a RF (Radio Frequency), etc., or any suitable combination of the above.

In some embodiments, the client and server can use any currently known or future developed network protocol, such as HTTP (hypertext transfer protocol), to communicate, and can be communicatively interconnected with digital data (e.g., communication network) in any form or medium. The example of a communication network includes a local area network ("LAN"), a wide area network ("WAN"), the Internet work (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), and any currently known or future developed network.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device.

The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device may: in response to an instruction for adding a special effect object on an initial image, determine a target display region of the special effect object on the initial image, where the target display region is a foreground region or a background region of the initial image, the foreground region is an image region where a target object is located in the initial image, and the background region is an image region other than the foreground region in the initial image; and display a part of the special effect object located in the target display region on the initial image to obtain a target image.

The computer program code used to perform operations of the present disclosure may be written in one or more programming languages or a combination thereof. The above-mentioned programming languages include, but not limited to, an object-oriented programming language - such as Java, Smalltalk, C++, and also include a conventional procedural programming language - such as "C" language or similar programming languages. The program code may be executed entirely on a computer of a user, partly on a computer of a user, executed as an independent software package, partly executed on a computer of a user and partly executed on a remote computer, or entirely executed on a remote computer or server. In a case where a remote computer is involved, the remote computer may be connected to the computer of the user through any kind of network - including a local area network (LAN) or a wide area network (WAN), or, it may be connected to an external computer (for example, connected via the Internet through an Internet service provider).

The flowcharts and block diagrams in the accompanying drawings illustrate possible implementation architecture, functions, and operations of the system, method, and computer program product according to the embodiments of the present disclosure. In this point, each block in the flowchart or block diagram may represent a module, a program segment, or a part of codes, and the module, the program segment, or the part of codes contains one or more executable instructions for implementing a designated logical function. It should also be noted that, in some alternative implementations, the functions marked in the blocks may also occur in an order different from the order marked in the drawings. For example, two blocks shown one after another may actually be executed substantially in parallel, or sometimes may be executed in a reverse order, which depends on the functions involved. It should also be noted that, each block in the block diagram and/or flowchart, and a combination of the blocks in the block diagram and/or flowchart, may be implemented by a dedicated hardware-based system that performs designated functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented in software or hardware. Where a name of a unit does not constitute a limitation on the unit itself in one case.

The functions described herein above may be performed at least in part by one or more hardware logic components. For example, non-restrictively, exemplary types of hardware logic components that may be used include: a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on chip (SOC), a complex programmable logic device (CPLD), etc..

In the context of the present disclosure, a machine-readable medium may be a tangible medium, which may include or store a program for use by or in combination with an instruction execution system, apparatus or device. The machine readable medium may be a machine readable signal medium or a machine readable storage medium. The machine readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the machine readable storage medium will include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

The above description is only for preferred embodiments of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by superposition of the above technical features or equivalent features without departing from the above disclosed concept, for example, technical solutions formed by replacement of the above-mentioned features and the technical features disclosed in the present disclosure (but not limited thereto) having similar functions.

In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order or in a sequential order shown. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of a single embodiment may also be implemented in combination in the single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub combination.

Claim 1:
An image special effect processing method, comprising:
in response to an instruction for adding a special effect object on an initial image, and in response to a switch operation by a user on a switch option on a special effect setting interface provided for the user to choose whether to distinguish between a foreground region and a background region when the special effect object is added:
in a disabling switch operation, displaying the special effect object on the initial image; and
in an enabling switch operation:
determining (S11) a target display region of the special effect object on the initial image, wherein the target display region is the foreground region or the background region of the initial image, the foreground region is an image region where a target object is located in the initial image, and the background region is an image region other than the foreground region in the initial image; and
displaying (S12) a first part of the special effect object located in the target display region of the initial image to obtain a target image, wherein the special effect object comprises the first part located in the target display region of the initial image and a second part located outside the target display region but inside the initial image, wherein the second part of the special effect object is not displayed;
wherein displaying (S12) the first part of the special effect object located in the target display region on the initial image to obtain the target image comprises:
determining an overlapping region of a position region of the special effect object on the initial image and the target display region;
cutting the first part located in the overlapping region from the special effect object as a target display object; and
displaying the target display object on an upper layer of the overlapping region of the initial image to obtain the target image.