Patent Publication Number: US-10769252-B2

Title: Method and apparatus for watermarking of digital content, method for extracting information

Description:
BACKGROUND OF THE INVENTION 
     Field of the Invention 
     The present invention relates to a method and apparatus for watermarking digital content and a data extraction method, and more particularly, to a method and apparatus for watermarking digital content that allow for recording and extracting various user tracking information on digital content and can greatly improve data security, and a data extraction method that allows for extracting data from a watermark in digital content. 
     Related Art 
     In general, digital content such as images, video, and multimedia can be quickly copied and transferred over a network, which facilitates file sharing and distribution between users. Thus, digital content by its nature is vulnerable to copy protection cracking. For example, digital data can be copied in large quantities without restriction in an open space like the internet where data is easily produced and transformed, and a copy still contains the same data as the original and can be distributed without the copyright owner&#39;s permission, making copyright protection difficult. 
     Such copyright infringement is becoming increasingly serious, and many legal, policy, and technical solutions have been proposed to combat copyright infringement. For instance, watermarking is the technology of embedding copyright information of the owner as a watermark, which is not perceptible by human beings, into digital content and identifying the embedded watermark by a detector, which allows the copyright owner to claim owner of the digital content. Using watermarking technology, information of the copyright ownership or distributor can be embedded into digital content and used to prove them right in possible future intellectual property disputes. 
     Particularly, forensic watermarking is the technology of tracing back to the source of leaked digital content by embedding purchaser information, a distribution route, user information, etc. into the digital content. Forensic watermarking provides post-detection capabilities such as allowing a seller of digital content to identify the original purchaser if copies of the digital content are pirated, which provides protection to keep the original purchaser from illegally distributing the digital content. 
     In forensic watermarking, a watermark has to contain user tracking information such as personal information or purchase details, in order to trace users of digital content. However, it is not practically possible to embed a large amount of information as a watermark because the watermark has limited information capacity. For example, in a conventional forensic watermarking system, a watermark may contain approximately 40 to 70 bits of data, which is a minimum of information like a user ID or payment information. Therefore, there is a need for a forensic watermarking technology that can store and manage large volumes of user tracking information. 
     In forensic watermarking, a method for managing user tracking information based on a database server may be proposed. In this case, however, there is a possibility that data recorded in the database might be forged or tampered. Accordingly, there is an urgent need for the development of a new technology that can record and manage large volumes of information for user tracking and guarantee the prevention of forgery and tampering of the database. 
     SUMMARY OF THE INVENTION 
     In view of the above background, the present invention has been made in an effort to provide a method and apparatus for watermarking digital content that allow for recording and extracting various user tracking information in digital content and can greatly improve data security. 
     The present invention provides a data extraction method based on a watermark embedded in digital content that allows for extracting data from a watermark in digital content if the digital content is illegally leaked. 
     In one aspect of the present invention, a watermarking method performed by a user terminal may include: generating private and public keys corresponding to the user terminal; generating watermark data containing user identity information corresponding to the user terminal and user details associated with the watermark data; generating encrypted data by encrypting the watermark data and the user details with the private key; transmitting the watermark data, the encrypted data, and the public key to a server; receiving digital content when authorized by the server; and watermarking the digital content with the watermark data. 
     The user identity information may include a user ID corresponding to the user terminal, and the user details may include at least one among the user&#39;s payment details and credit card information for the digital content and the hardware information of the user terminal. The digital content watermarking method may further include discarding the private key after generating encrypted data. 
     The server may store the transmitted watermark data, encrypted data, and public key in the same way in a plurality of databases, respectively. A different retention period may be set for each database, and the data stored in each database may be discarded when the retention period expires. 
     The server may store the transmitted watermark data, encrypted data, and public key in the same way in a plurality of closed blockchains, respectively. A different retention period may be set for each database, and the data stored in each blockchain may be discarded when the retention period expires. 
     In the digital content watermarking method, the transmission of the watermark data, encrypted data, and public key from the user terminal to the server may be authorized by a third-party or a third-party certificate. 
     In another aspect of the present invention, a digital content watermarking apparatus may include: a key generator that generates private and public keys corresponding to the user terminal; a data generator that generates watermark data containing user identity information corresponding to the user terminal and user details associated with the watermark data; an encryption part that generates encrypted data by encrypting the watermark data and the user details with the private key; a data transmitter that transmits the watermark data, the encrypted data, and the public key to a server; and a watermark embedder that receives digital content when authorized by the server and watermarks the digital content with the watermark data. 
     The user identity information may include a user ID corresponding to the user terminal, and the user details may include at least one among the user&#39;s payment details and credit card information for the digital content and the hardware information of the user terminal. The digital content watermarking method may further include discarding the private key after generating encrypted data. 
     The server may store the transmitted watermark data, encrypted data, and public key in the same way in a plurality of databases, respectively. A different retention period may be set for each database, and the data stored in each database may be discarded when the retention period expires. 
     The server may store the transmitted watermark data, encrypted data, and public key in the same way in a plurality of closed blockchains, respectively. A different retention period may be set for each database, and the data stored in each blockchain may be discarded when the retention period expires. 
     In another aspect of the present invention, a data extraction method may include: extracting first watermark data from a watermark embedded in digital content; extracting second watermark data from a database or blockchain by searching the database or blockchain using the first watermark data; decrypting encrypted data associated with the second watermark data using a public key associated with the second watermark data; extracting third watermark data contained in the decrypted data; comparing the first watermark data, second watermark data, and third watermark data, and, if all of the data matches, extracting user details contained in the encrypted data. 
     As explained above, according to the present invention, a minimum of information for identifying digital content may be watermarked and embedded into the digital content, and user details may be encrypted and stored in a plurality of databases or blockchains. Accordingly, it is possible to store and manage large volumes of information for user tracking while reducing watermarking time and ensuring security. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a block diagram of the configuration of a system for implementing a digital content watermarking method according to an exemplary embodiment of the present invention. 
         FIG. 2  is a block diagram of the configuration of a watermarking engine shown in  FIG. 1 . 
         FIG. 3  is a flowchart illustrating a method for watermarking digital content according to an exemplary embodiment of the present invention. 
         FIG. 4  is an illustration showing data transmitted from a user terminal to a management server. 
         FIG. 5  is an illustration showing a structure for storing watermark data, encrypted data, and a public key using databases. 
         FIG. 6  is an illustration showing a structure for storing watermark data, encrypted data, and a public key using block chains. 
         FIG. 7  is a flowchart of a process for extracting user details based on a watermark embedded in digital content. 
     
    
    
     DESCRIPTION OF EXEMPLARY EMBODIMENTS 
     Hereinafter, an exemplary embodiment of the present invention will be described in further details with reference to the accompanying drawings. In order to help overall understanding, the same reference numerals designate the same elements throughout the drawings, and a redundant description of the same elements is omitted. 
       FIG. 1  is a block diagram of the configuration of a system for implementing a digital content watermarking method according to an exemplary embodiment of the present invention. 
     As shown in  FIG. 1 , a user terminal CL may interface with a management server SP. The user terminal CL may refer to a network device where digital content can be purchased or can be downloaded for use. For example, the user terminal CL may be a set-top box, a smart TV, a mobile terminal, a personal computer, etc. The digital content may be copyrighted multimedia content, images, etc. 
     The user terminal CL may have a watermarking engine  100 . The watermarking engine  100  may be a device that is installed as software on the user terminal CL and operates based on the hardware resources of the user terminal CL, such as a processor, memory, etc., or may be a device that is implemented in hardware within the user terminal CL. 
       FIG. 2  is a block diagram of the configuration of the watermarking engine  100  shown in  FIG. 1 . 
     As shown in  FIG. 2 , the watermarking engine  100  may include a key generator  110 , a data generator  120 , an encryption part  130 , an data transmitter  140 , and a watermark embedder  150 . Each of these components may work in conjunction with one another by sending and receiving signals as required for a process. 
     The key generator  110  may generate private and public keys corresponding to the user terminal CL. The data generator  120  may generate watermark data containing user identity information corresponding to the user terminal CL and user details associated with the watermark data. The encryption part  130  may generate encrypted data by encrypting the watermark data and the user details with the private key. The data transmitter  140  may transmit the watermark data, the encrypted data, and the public key to a server. When authorized by the server, the watermark embedder  150  may receive digital content and watermark the digital content with the watermark data. 
       FIG. 3  is a flowchart illustrating a method for watermarking digital content according to an exemplary embodiment of the present invention. Below is a description of an operation based on the system and machine respectively shown in  FIGS. 1 and 2 . In what follows, the roles and interactions of the components mentioned with respect to  FIGS. 1 and 2  will be clarified. 
     First of all, the user terminal CL may connect to a content market over a communication network and purchase digital content. The content market may be a market interfacing with the management server SP. For example, the management server SP may confirm that a watermarking process is being performed properly on digital content, and, if properly done, may authorize the transmission of the digital content to the person who purchased it. 
     Referring to  FIGS. 1 to 3 , when the user terminal CL purchases digital content, the key generator  110  of the watermark engine  100  on the user terminal CL may generate private and public keys corresponding to the user terminal CL (Step S 1 ). 
     Moreover, the data generator  120  of the watermarking engine  100  may generate watermark data (Step S 2 ). The watermark data may be a minimum of information, e.g., a user ID, used to uniquely identify the user terminal CL. Here, the user terminal CL may be in a logged-in state based on a specific user ID. Alternatively, the watermark data may be the hardware identification information, MAC (media access control) address, etc. of the user terminal CL. 
     The data generator  120  of the watermarking engine  100  may generate user details associated with the watermark data (Step S 3 ). The user details may include the user&#39;s usage details of the digital content. For example, the user details may include the user&#39;s payment details and credit card information for the purchase of the digital content. The user details may further include the hardware information or MAC address of the user terminal CL. 
     Next, the encryption part  130  of the watermarking engine  100  may generate encrypted data by encrypting watermark data and user details with the private key generated by the key generator  110  (Step S 4 ). That is, the encrypted data may refer to data generated by encrypting the watermark data and the user details with the private key. 
     Meanwhile, the encryption part  130  may generate hash codes or error-correcting codes ECC for the watermark data and user details and encrypt the generated hash codes or error-correcting codes, in order to enhance the capabilities for preventing the forgery or tampering of the watermark data and user details. In this case, the encrypted data may include the watermark data, the user details, and the hash codes or error-correcting codes. 
     Upon completion of the encryption, the encryption part  130  may discard the private key used for the encryption (Step S 5 ). 
     The data transmitter  140  of the watermarking engine  100  may transmit the watermark data, the encrypted data, and the public key to the server (Step S 6 ).  FIG. 4  is an illustration showing data transmitted from the user terminal CL to the management server SP. As shown in  FIG. 4 , the user terminal CL may transmit to the management server SP non-encrypted watermark data, encrypted data (i.e., data generated by encrypting the watermark data and the user details with the private key), and the public key. 
     When transmitting the watermark data, encrypted data, and public key, a security protocol may be used between the user terminal CL and the server. Moreover, the transmission of the watermark data, encrypted data, and public key from the user terminal CL to the management server SP may be authorized by a third party or a third-party certificate, e.g., an authentication server. For instance, a security communication protocol using a secure form of third-party certification may be used for communication between the user terminal CL and the management server SP. 
     Meanwhile, the management server SP may receive and store the watermark data, encrypted data, and public key transmitted from the user terminal CL (Step S 7 ). The management server SP may store the watermark data, encrypted data, and public key using a plurality of databases or a plurality of block chains. 
       FIG. 5  is an illustration showing a structure for storing watermark data, encrypted data, and a public key using databases. 
     As shown in  FIG. 5 , the management server SP may store watermark data, encrypted data, and a public key using a plurality of databases—for example, a first database DB 1 , a second database DB 2 , and a third database DB 3 . The watermark data, encrypted data, and public key may be stored in the same way in the first database DB 1 , second database DB 2 , and third database DB 3 , respectively. In this case, the watermark data, encrypted data, and public key stored in their respective databases may be compared to one another. If they match, it is determined that data integrity is verified, and the verified data may be stored. Therefore, the integrity of the data stored in the plurality of databases may be verified. 
     Meanwhile, the data stored in the databases might need to be deleted after a given period of time according to a policy such as a personal data retention period. To this end, a different retention period is set for each database, and the data stored in each database may be discarded when the retention period expires. 
     For example, the first database DB 1  may be retained until May 31, 2018, the second database DB 2  may be retained until Jun. 30, 2018, and the third database DB 3  may be retained until Sep. 30, 2018, and the data stored in each database may be physically or logically discarded when the set retention period expires. Here, the term “physical discarding” may refer to discarding the database from the system using a physical method, like removing the hardware storing the database, and the term “logical discarding” may refer to deleting the data in the database or deleting a link to the database. 
       FIG. 6  is an illustration showing a structure for storing watermark data, encrypted data, and a public key using block chains. 
     As shown in  FIG. 6 , the management server SP may store watermark data, encrypted data, and a public key using a plurality of block chains—for example, a first blockchain BC 1 , a second blockchain BC 2 , and a third block chain BC 3 . The watermark data, encrypted data, and public key may be included in blocks produced in the blockchains and stored in the same way in the first blockchain BC 1 , second blockchain BC 2 , and third blockchain BC 3 , respectively. In this case, the watermark data, encrypted data, and public key, which are included in the blocks connected to each other that are produced in the blockchains, may be compared to one another. If they match, it is determined that data integrity is verified, and the verified data may be stored. Therefore, the integrity of the data stored in the plurality of blockchains may be verified. 
     Meanwhile, the data stored in the blockchains might need to be deleted after a given period of time according to a policy such as a personal data retention period. To this end, a different retention period is set for each database, and the data stored in each blockchain may be discarded when the retention period expires. 
     For example, the first blockchain BC 1  may be retained until May 31, 2018, the second blockchain BC 2  may be retained until Jun. 30, 2018, and the third blockchain BC 3  may be retained until Sep. 30, 2018, and the data stored in each blockchain may be physically or logically discarded when the set retention period expires. Here, the term “physical discarding” may refer to discarding the blockchain from the system using a physical method, like removing the hardware for nodes in the blockchain, and the term “logical discarding” may refer to deleting the data in the blockchain or deleting a link to the blockchain. 
     Once the watermark data, encrypted data, and public key are stored, the management server SP may send to the content market an authorization for the transmission of the digital content purchased on the user terminal CL to the user terminal CL and send the user terminal CL an authorization for the use of the digital content (Step S 8 ). 
     Upon receiving the authorization, the watermark embedder  150  of the watermarking engine  100  may receive the digital content and then watermark the digital content with the watermark data (Step S 9 ). More specifically, the watermark embedder  150  may generate watermarked digital content by generating a watermark based on the watermark data and embedding the generated watermark into the digital content. 
     Upon completion of the watermarking, the user terminal CL may reproduce the watermarked digital content through the user terminal CL or transfer it to an authorized device within legal bounds. However, if the watermarked digital content is illegally leaked, the management server SP may perform user tracking to trace back to the source of the watermarked digital content that has been illegally leaked. 
       FIG. 7  is a flowchart of a process for extracting user details based on a watermark embedded in digital content. 
     As shown in  FIG. 7 , the management server SP may extract watermark data from a watermark embedded in watermarked digital content that has been leaked (Step S 11 ). The watermark data extracted from the watermark embedded in the digital content may be called first watermark data. 
     Then, the management server SP may extract second watermark data from a database or blockchain by searching the database or blockchain using the first watermark data (Step S 12 ). That is, watermark data matching the first watermark data may be detected by searching a database or blockchain. Here, the watermark data detected in the database may be called second watermark data. 
     Next, the management server SP may decrypt encrypted data associated with the second watermark data using a public key associated with the second watermark data (Step S 13 ). As mentioned above, the encrypted data contains watermark data and user details. 
     Accordingly, the management server SP may extract the watermark data contained in the decrypted data (Step S 14 ). The watermark data contained in the decrypted data may be called third watermark data. 
     The management server SP may compare the first watermark data, second watermark data, and third watermark data (Step S 15 ). If all of the first watermark data, second watermark data, and third watermark data match, the management server SP may determine that the integrity of the watermark data is verified, and extract user details contained in the encrypted data. Therefore, the management server SP may detect the user who leaked the digital content, based on the user details. 
     On the other hand, if the first watermark data, second watermark data, and third watermark data do not match, the management server SP may determine that at least one of the watermark data has been forged or tampered and issue an error (Step S 17 ). 
     As explained above, according to an exemplary embodiment of the present invention, a minimum of information for identifying digital content may be watermarked and embedded into the digital content, and user details may be encrypted and stored in a plurality of databases or blockchains. Accordingly, it is possible to store and manage large volumes of information for user tracking while reducing watermarking time and ensuring security. 
     The exemplary embodiments of the present invention are illustrative only, and various modifications, changes, substitutions, and additions may be made without departing from the technical spirit and scope of the appended claims by those skilled in the art, and it will be appreciated that the modifications and changes are included in the appended claims.