Methods and computer systems with provisions for high availability of cryptographic keys

Computer systems and methods ensuring high availability of cryptographic keys using a shared file system. The keys are encrypted with at least one shareable master key to generate corresponding encrypted cryptographic keys, which are stored in a key database in the shared file system. A master key manager with access to the key database is elected from among master key manager candidates and is assigned a common virtual address. All master key manager candidates have the shareable master key such that during a failover event the availability of the encrypted cryptographic keys is not interrupted as a new master key manager takes over the common virtual address from the previous master key manager. Additionally, a message authentication code (MAC) is deployed for testing the integrity of keys during their retrieval.

FIELD OF THE INVENTION

This invention relates to methods and computer systems including those deployed in computer clusters, with provisions and key databases operated in the context of a shared file system to ensure high availability of cryptographic keys.

BACKGROUND ART

Computing systems are becoming more highly distributed and they are required to support many users, each operating one or more client devices. Some of these systems are centered about computer clusters whose machines or nodes execute client requests and serve the desired files. A large number of client devices ranging from smart phones to computers are usually connected to such systems at all times. The users of these devices have come to expect high availability of supported services.

At the same time, growth in the number client devices is putting pressure on security of individual services. In particular, the focus is on the handling of cryptographic keys that are used for file encryption and decryption in such computing systems. The management of these keys has to ensure both their high availability and security. Needless to say, satisfying both requirements at the same time is a challenge.

Most modern computer systems deploy a key manager for storing and handling of cryptographic keys. The key manager, in conjunction with other authentication and authorization functions, determines which parties or clients are authorized to retrieve any particular key. For added safety, several secure key managers are usually deployed together under some type of failover scheme. In other words, there is an agreed-upon process that dictates how the secure key managers act in the event that one or more of them fail.

In some systems, the secure key managers are synchronized and use time-outs when a client happens to be connected to a failing key manager. This means that the client has to wait until after the time-out to be connected to an active key manager, thus negating the requirement of high availability. Furthermore, all key managers have to be synchronized in order to ensure proper operation.

SafeNet's Key Secure is an example of a system with a number of key servers and it is described in the SafeNet KeySecure Product Brief, Feb. 22, 2013, pp. 1-4. This system works well when all key servers are operational and updated, but presents synchronization problems and operation blockage when one of them is down. Thus, a user may have to wait a potentially long time before the system is able to fulfill their key request. A key server may be unavailable when a key operation is performed, require synchronization and possible manual intervention.

Another approach is taken in the Virtual Machine Manager of Microsoft as described in pre-release documents entitled “Configuring Distributed Key Management in VMM”, Oct. 17, 2013, pp. 1-3. This approach uses a multi-master system with a loose criterion for convergence of their replicas when the system reaches steady state. At any particular instant in time, the replicas are not guaranteed to be consistent with each other. That is because in the multi-master paradigm changes can be applied to any replica at any time. When any particular master makes an update, it uses the time stamps of changes in the replication messages to determine whether it should update or discard the replication message. Due to finite response time of such systems, certain corner cases may force failure modes due to key conflict.

Another approach to encryption and key managers is taught in U.S. Pat. No. 7,877,603 to Goodman et al. This system relates to a storage drive that communicates with encryption and key managers. It is useful in the context of encrypting removable media but not extendable to distributed computing environments. Another useful teaching related to a key manager that tracks which assets are encrypted with which keys in the context of a data archiving system is found in U.S. Pat. No. 7,801,871 to Gosnell.

Finally, U.S. Pat. No. 8,385,551 to Pourzandi et al. teaches a highly available cryptographic key storage that deploys a classic active/passive failover model. This approach is better adapted to a highly available computer system but still suffers from a number of shortcomings. One of the main ones is the assumption made during key creation that backup nodes of the system are up. When this assumption does not hold, then the key creation operation blocks.

Thus, although a considerable amount of work has been devoted to devising appropriate computer systems and methods for highly available and secure key management, many challenges remain. In particular, it would be an advance in the art to further increase cryptographic key security in such computer systems while compromising little or not at all on the high availability aspect.

OBJECTS AND ADVANTAGES OF THE INVENTION

In view of the shortcomings of the prior art, it is an object of the invention to provide for computer systems and methods that use key managers in a manner that increases cryptographic key security while ensuring high availability of these keys. More precisely, it is an object of the invention to leverage a shared file system with a key database as well as appropriate key integrity testing techniques to avoid real-time key distribution and synchronization between different machines.

It is another object of the invention to provide for computer systems and methods that are highly secure and provide for a safe failover between master key managers that are in charge of handling the cryptographic keys and access to the key database.

These and many other objects and advantages of the invention will become apparent from the ensuing description.

SUMMARY OF THE INVENTION

A number of objects and advantages of the invention are achieved by methods and computer systems that ensure high availability of cryptographic keys in a computer system that operates a shared file system. The computer system can be implemented in a computer cluster that has a number of nodes that serve the shared file system to clients. According to the invention, the cryptographic keys are encrypted with at least one shareable master key to generate corresponding encrypted cryptographic keys. The encrypted cryptographic keys are stored in a key database belonging to the shared file system. A master key manager is elected from among master key manager candidates that belong to the nodes of the cluster. Of course, many master key manager candidates should be available at any time, but there should be at least two master manager candidates to permit such election to take place. The master key manager thus elected from among the candidates is assigned a common virtual address.

The invention further calls for issuing the at least one shareable master key to the master key manager and to each of the master key manager candidates. By making this provision, a failover event does not cause interruption in the availability of the encrypted cryptographic keys. More precisely, upon a failover event the common virtual address is transferred from the master key manager to a new master key manager elected from among the at least one master key manager candidate available to assume the role of master key manager.

Furthermore, a master message authentication code key or master MAC key is issued to the master key manager. The master MAC key is also issued to the master key manager candidates. The master MAC key is applied over each of the encrypted cryptographic keys to generate corresponding message authentication tags. The tags are used to test or confirm the integrity of the encrypted cryptographic keys to which they correspond.

Integrity is typically tested when encrypted cryptographic keys are retrieved for any given purpose, e.g., upon a client request, from the key database. Thus, the message authentication tags are preferably stored along with the encrypted cryptographic keys for which they were generated in the key database. In a preferred embodiment, the key metadata of each of the encrypted cryptographic keys is also included in the MAC process. In other words, the master MAC algorithm is applied over each encrypted cryptographic key along with its metadata to generate joint message authentication tags. This provision is deployed to prevent unauthorized modification of keys or their metadata. It also prevents attachment of unauthorized metadata to encrypted cryptographic keys during a security breach.

In a typical implementation, it is a hardware security module (HSM) that belongs to the master key manager that produces the shareable master key. The shareable master key is also stored in the same HSM. The one or more master key manager candidates are equipped with their own HSMs and the shareable master key is stored in those.

The shareable master key, upon being generated in the HSM of the master key manager is distributed to HSMs of master key manager candidates in a secure manner. For example, a wrapped key exchange protocol or other secure exchange provisions including mutually authenticated and secure communication lines between the master key manager and the one or more master key manager candidates are deployed. It is preferable that master MAC keys also be securely distributed and stored in HSMs of the master key manager and of the one or more master key manager candidates.

During operation, a requestor, e.g., an external client or an internal node or any authorized resource, may request a new cryptographic key. In some cases, the requestor may submit a new cryptographic key to the master key manager. Alternatively, upon the requestor's order, the new cryptographic key can be generated by the master key manager. This step may be performed by the HSM, for example.

In accordance with the invention, the new cryptographic key is encrypted with the shareable master key. This action produces a new encrypted cryptographic key, which is then stored in the key database. Of course, it is preferable that the new encrypted cryptographic key be treated in the same manner as the other encrypted cryptographic keys. This means that it, along with its key metadata, is MACed with the master MAC key to generate corresponding message authentication tag for subsequent confirmation of integrity.

The shared file system can be any type of system, although deployment in distributed file systems may be most advantageous. Among other, suitable distributed file systems include: Hadoop Distributed File System (HDFS), NFS, SMB, DRBD, XtremeFS, legato, Veritas, GFS, Fusion, GPFS, Ceph, Coda, Solaris ZFS. In many of these applications, the encrypted cryptographic keys will pertain to corresponding file units stored in the distributed file system. It is thus convenient to also store in the key database file metadata pertaining to the file units. The file units themselves can be embodied by blocks in storage resources belonging to the computer system (e.g., block storage resources belonging to the cluster in which the computer system is implemented).

The computer system that operates a shared file system that ensures high availability of cryptographic keys, such as the encrypted cryptographic keys derived in accordance with the invention, does not necessarily have to be deployed in a cluster environment. However, many benefits of high availability during failover accrue in the cluster implementation. Specifically, transfer of the common virtual address between nodes of a cluster during failover is well-adapted to transitions between a failed master key manager and a new master key manager elected from among the master key manager candidates. This process not only ensures uninterrupted availability of encrypted cryptographic keys during failover, but it is also similar to the handling of general failover between a master node in charge of the cluster and its backup master node candidates. The latter are elected upon failover in accordance with various well-known strategies (e.g., quorum vote). Therefore, in some embodiments the master key manager and the master key manager candidates are assigned to nodes of the cluster that may in some cases correspond to the cluster's master node and its backup nodes.

In most embodiments, it is a message authentication code module, or MAC module that performs the MACing and integrity confirmation steps. Each member of the set of master key manager candidates designated among the cluster nodes and thus also the present master key manager is equipped with its own MAC module. Thus, any one of them can perform both the MACing and the verification or integrity confirmation on a previously MACed encrypted cryptographic key and its key metadata.

The present invention, including the preferred embodiment, will now be described in detail in the below detailed description with reference to the attached drawing figures.

DETAILED DESCRIPTION

The following description relates to preferred embodiments of the present invention by way of illustration only. Likewise, the figures depict embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the description and figures that alternative embodiments of the methods and systems illustrated herein may be employed without departing from the principles of the invention described herein.

The present invention will be best understood by initially referring to the high-level diagram ofFIG. 1. This drawing illustrates the main aspects of a computer system100deploying methods that ensure high availability of cryptographic keys in accordance with the invention. Computer system100services a user U through his or her client device102that will henceforth be simply referred as client102. User U and client102are specifically designated for explanatory purposes. In practical situations a large number of users and client devices or clients, some of which may even belong to the same users, are serviced by computer system100.

In the present embodiment, client102is a mobile computer, and more specifically still a laptop computer. In general, however, client102can be any type of device capable of making a network connection and performing any useful function. In most cases, of course, client102is a mobile user device such as a mobile phone, a mobile computer, a tablet computer or any thin client device, including devices ranging from electronic watches to sensors with minimal processing capabilities. The latter are sometimes referred to as networked devices making up the Internet of things by those skilled in the art.

User U employs client102to connect to computer system100over a network connection104established over any suitable medium104′. Connection104can deploy any type of wireless or wired medium104′. In fact, communication network104can include a Wide Area Network (WAN) like the Internet, a Metropolitan Area Network (MAN) or a Local Area Network (LAN). The physical connection can be supported by any communications infrastructure including wired, optical and wireless.

In the preferred embodiment, computer system100is implemented in a computer cluster106encompassed by a dashed line inFIG. 1. Cluster106can be geographically collocated or spread out over several physical locations. For reasons of clarity, only the few parts of cluster106required to explain the invention are shown explicitly. It will be evident to a skilled artisan, however, that computer cluster106can also be implemented on various computer grids and other configurations well known in the art.

Cluster106has a number of machines or computers typically referred to as nodes by those skilled in the art. Of all such nodes, which may number in the hundreds, thousands or even tens of thousands and more, only five, namely nodes108A,108B,108F and108J,108K are expressly shown. In this example, all five drawn nodes108A,108B,108F,108J,108K are provisioned with similar computing or processing resources110A,110B,110F,110J,110K. These include central processing units (CPUs), graphical processing units (GPUs) and any generally acceptable dedicated or generic processors and support resources.

First three visible nodes108A,108B,108F are also provisioned with any suitable type of storage resources and any other node support resources generally indicated by references112A,112B,112F. For example, resources112A,112B,112F are embodied by any local storage hardware capable of storing files. More generally, resources112A,112B,112F can include printers, monitors, application-specific processors, block storage devices including hard drive devices (e.g., parallel or serial, such as SATA), flash drives and any other cluster resources. In fact, any physical or logical component that can be brought on-line and off-line, managed in cluster106and hosted by just one of nodes108A,108B,108F can be represented among the set of resources112A,112B,112F. It should be also noted that nodes108A,108B,108F can each be provisioned with the same or different resources. A person skilled in the art will recognize that any workable allocation of resources and node topology is permissible.

Meanwhile, nodes108J,108K are also provisioned with suitable resources112J,112K that may be analogous or different from those found in nodes108A,108B,108F. In addition, nodes108J,108K are equipped with specific storage resources114J,114K here embodied by hard disk drive arrays that can efficiently store and serve a large number of files F. In fact, storage resources114J,114K support an entire shared file system115that contains files F. One specific file, namely the j-th file Fjis called out from among files F for later explanatory purposes.

In the present embodiment, shared file system115is clearly also a distributed file system. As shown, it is served from storage resources114J,114K of two separate nodes, namely nodes108J,108K. In practice, many of the nodes omitted inFIG. 1are also engaged in serving up file system115and thus file system115is distributed over many nodes of cluster106besides representative nodes108J,108K.

Shared file system115can be any type of distributed file system whose operation can be supported and managed in cluster106. For example, distributed file system115can be any one of the following: Hadoop Distributed File System (HDFS), NFS, SMB, DRBD, XtremeFS, legato, Veritas, GFS, Fusion, GPFS, Ceph, Coda, Solaris ZFS. A person skilled in the art will be able to discern that still other systems can be deployed in the present invention upon any requisite modifications based on the subsequent teachings.

All nodes of cluster106, including nodes108A,108B,108F and108J,108K are interconnected by a corresponding network of interconnections116. In many situations, interconnections116are embodied by a local area network (LAN) and include any special connections, such as heartbeat lines, etc. Furthermore, when cluster106is geographically spread out, interconnections116include the requisite intra- and inter-cluster communication fabric including any requisite wide area network lines (WAN) or dedicated pipes.

In accordance with the invention, nodes108A,108B,108F and omitted nodes between nodes108B and108F indicated by the ellipsis are all members of a set118of master key manager candidates. Set118of master key manager candidates108A,108B, . . . ,108F is preferably much larger than the minimum number of two such candidates. With just two master key manager candidates in set108, when one is elected to serve as master key manager only one is available to assume the role of master key manager during failover. A person skilled in the art will recognize that this situation is reminiscent of failover between a master node in charge of entire cluster106and one of its backup master nodes. In high availability clusters such situation is not desirable and thus more backup master nodes are always on stand-by. The situation with master key managers is similar. Consequently, it is preferred that set118contain at least three or still more master key manger candidates, just as in the case of backup master nodes.

In the example shown inFIG. 1, first node108A of set118has already been elected as master key manager MKM and is correspondingly designated. The election itself was performed by any well-known technique. For example, any of the techniques deployed for election of a master node (not shown) of cluster106can be deployed in electing master key manager MKM. These include election by quorum of candidates in set118or by other voting procedures known to those skilled in the art. It should be noted, that node108A that is now master key manger MKM is typically not the same node as the one elected to be the master of entire cluster106. However, in some embodiments it is possible to assign the duties of managing cluster106as well as being master key manager MKM to the same node.

Computer system100uses a virtual address assignment module120to keep track of the assignment of master key manager MKM. In particular, master key manager MKM elected from among candidates of set118is assigned a common virtual address CVA by module120. In the present case, while first node108A is performing its duties as master key manager MKM, it is assigned common virtual address CVA by module120. A person skilled in the art will recognize that module120can be implemented on any node in cluster106or in a separate unit, as depicted inFIG. 1. In either case, it is important that client102or any requestor of cryptographic keys first obtain from module120common virtual address CVA assigned to current master key manager MKM.

Each one of nodes108A,108B, . . . ,108F in set118of master key manager candidates is provisioned with a hardware security module (HSM). Node108A serving as current master key manager MKM is equipped with hardware security module122A that is herein referred to as primary hardware security module or just primary HSM for short. Meanwhile, nodes108B, . . . ,108F are equipped with hardware security modules122B, . . . ,122F that are henceforth referred to as secondary hardware security modules or secondary HSMs. Of course, only secondary HSMs122B and122F are expressly shown inFIG. 1. As is known in the art, hardware security modules HSMs are typically responsible for generating and storing cryptographic keys in a secure manner.

Each one of nodes108A,108B, . . . ,108F in set118of master key manager candidates is further equipped with a message authentication code module124A,124B, . . . ,124F or MAC module for short. MAC modules124A,124B, . . . ,124F are designed to authenticate or verify the integrity of messages, files, keys or other digital entities in accordance with any known verification technique that may include hashing or keying of the digital entities that are to be checked for integrity. In the present embodiment, MAC modules124A,124B, . . . ,124F are of the type that deploy master message authentication code key MMK or master MAC key for short to carry out their integrity tests.

Master MAC key MMK is generated and stored in primary HSM122A. Thus, master MAC key MMK is issued to acting master key manager MKM. As it is common to devolve the duties of key creation and safekeeping to HSMs, this approach is preferred. However, in alternative embodiments, MAC module124A of acting master key manager MKM can be tasked with generating and safeguarding master MAC key MMK.

According to the invention, primary HSM122A is also entrusted with generating and storing a shareable master key SMK. Thus, in preparation for serving its duties, master key manager MKM is issued both shareable master key SMK and master MAC key MMK. Both keys SMK, MMK are illustrated after their generation and already stored in primary HSM122A.

Nodes108A,108B, . . . ,108F in set118of master key manager candidates also have to be issued shareable master key SMK and master MAC key MMK in preparation for fulfilling their duties.FIG. 2is a more detailed diagram that clarifies this step. Specifically, keys MMK, SMK are transmitted from primary HSM122A of master key manager MKM where they were generated to secondary HSMs122B, . . . ,122F of all members of set118. Transmission of SMK key for storage in HSMs122B, . . . ,122F is indicated by dashed arrows126AB, . . . ,126AF. Transmission of MMK key for storage in HSMs122B, . . . ,122F is indicated by dashed and dotted arrows128AB, . . . ,128AF. As a result of this transmission of keys, each one of master key manager candidates in set118, i.e., nodes108B, . . . ,108F in the present example, is issued SMK and MMK keys.

In principle, any manner of transmitting SMK, MMK keys over interconnections116between node108A serving as master key manager MKM and nodes108B, . . . ,108F of set118is permissible. What is important, however, is to convey them between primary HSM122A and secondary HSMs122B, . . . ,122F in a secure manner. For this last reason, it is preferred that transmission of SMK, MMK keys as indicated by arrows126AB, . . . ,126AF and arrows128AB, . . . ,128AF take place over mutually authenticated and secure connections between primary and secondary HSMs122A,122B, . . . ,122F. The negotiation and establishment of such connections is well known to those skilled in the art.

Mutually authenticated and secure connections126AB, . . . ,126AF and128AB, . . . ,128AF between MKM and the remaining members of set118can be supplemented with additional safeguards. For example, in the preferred embodiment, a wrapped key exchange protocol is deployed to further increase security of key transmission. SMK key is wrapped in a key exchange key130. MMK key is wrapped in a key exchange key132. Key exchange keys130,132can comply with any suitable exchange protocol. Such strategies are based on previously negotiated parameters between all nodes108A,108B, . . . ,108F that belong to set118of master key manager candidates. Suitable processes are well known to those skilled in the art.

We now turn back toFIG. 1to review the provisions of computer system100to ensure high availability of cryptographic keys. First, we note that user U working on a specific file Fqon client102requires a corresponding cryptographic key CKqfor securing file Fq. For purposes of explanation we presume that only a single cryptographic key CKqis necessary. In practice, however, depending on system security type, encryption file size and still other parameters, file Fqmay actually require several such cryptographic keys. For example, if file Fqis large, it may require a series of cryptographic keys; e.g., one cryptographic key for each block in memory resources occupied by file Fq. A person skilled in the art will be able to adapt the present teachings to any specific file encryption strategies.

In accordance with the invention, any cryptographic keys such as cryptographic key CKqin the example depicted inFIG. 1, is encrypted with shareable master key SMK to generate corresponding encrypted cryptographic key ECKq. Encrypted cryptographic key ECKqis then stored in a key database134belonging to shared and in this case also distributed file system115. The same is done with other cryptographic keys CK that are requested by any legitimate client of computer system100, by a cluster resource or by any other legitimate requestor. Key database134is used to store encrypted cryptographic keys ECK for all cryptographic keys CK thus requested or even submitted by a client or a resource for encryption with shareable master key SMK to generate corresponding encrypted cryptographic keys ECK.

Note that in the present embodiment distributed file system115is also used as repository of user-generated files F. These include file Fjpreviously created user U and file Fqthat is presently being operated on by user U and may be stored in file system115when user U is done working on it.

The step of encryption of cryptographic key CKqwith shareable master key SMK is shown taking place within a secure space136. Space136could be hosted by any resource of node108A that is serving as master key manger MKM. It is most convenient, however, for secure space136to be within primary HSM122A. Further, we note that shareable master key SMK is a symmetric key. Symmetric encryption has the advantage that it is more efficient at processing large amounts of file encryptions and is computationally less intensive than encryption with asymmetric keys.

To gain a better understanding of the operation of computer system100we now turn toFIGS. 3A-B.FIG. 3Ashows the process of encryption of cryptographic key CKqwith sharable master key SMK in more detail. This process commences with client102issuing a key request140for encrypting file Fq. Note that client102is appraised that current or acting master key manager MKM is node108A. That is because module120informs client102of common virtual address CVA. Meanwhile, node108A is the one that holds common virtual address CVA. For as long as node108A does not fail, malfunction or become otherwise unreliable so as to provoke failover to another master key manager selected from set118, it holds that common virtual address CVA (seeFIG. 1).

Using common virtual address CVA, client102presents key request140to node108A in its capacity as master key manager MKM. As noted above, it is possible for client102acting as requestor to supply its own cryptographic key CKqto master key manager MKM. In such cases we refer to requestor102as a submitter. Acting as submitter, client102generates its own cryptographic key CKqand encrypts file Fqwith it. Then, submitter102sends cryptographic key CKqto master key manager MKM without retaining a local copy thereof.

After sending away cryptographic key(s) pertaining to any encrypted file(s) client102, whether acting as submitter or only as requestor, can only receive keys upon proper client authentication and thus the security risks are contained. In other words, the comparative security risk of a key compromise is no worse if it is client102that generates its own cryptographic key(s) instead of relying on master key manager MKM to generate cryptographic key(s) upon key request140.

As noted above, cryptographic key CKqcan be a per-block key (i.e., it can be designed to unlock and unlock a particular block in memory space). In the present example, cryptographic key CKqapplies to an entire disk partition rather than just a single block in memory space. In other words, cryptographic key CKqis used to unlock encrypted disk partition(s) where the desired file(s) F, e.g., file Fq, are located.

This threat model is particularly useful in cases where an attacker steals client102belonging to user U from a data center. Once off the data center's network, the attacker will not be able to obtain cryptographic key CKqand decrypt the data in the protected partition(s). That is because client102will not get access to cryptographic key CKqthat has been sent to master key manager MKM for further encryption and ultimate storage in distributed file system115. Introduction of such key serving strategy is advantageous in that cryptographic keys are not stored on client machines and thus cannot be taken by an attacker who takes the client machine. Meanwhile, any person skilled in the art will recognize that it is easier to improve the security for a small number of key servers than a large number of client machines.

In embodiments where requestor102is not a submitter cryptographic key CKqstill needs to be generated. Thus, in response to key request140, master key manager MKM instructs primary HSM122A to generate cryptographic key CKq. Of course, in order to encrypt file Fqthat resides on requestor102, cryptographic key CKqhas to be returned to requestor102. This step is not explicitly shown inFIG. 3A. Just as in the case of client102acting as submitter, requestor102retains no local copy of cryptographic key CKqafter it has successfully encrypted file Fqwith it.

Once file Fqis encrypted, shareable master key SMK is deployed by master key manager MKM to encrypt cryptographic key CKq. The act of encryption of cryptographic key CKqyields encrypted cryptographic key ECKq. The act itself takes place in secure space136, such as within primary HSM122A. A person skilled in the art will recognize that encrypting a key with another key is commonly referred to as wrapping. Wrapping is a standard procedure in the art of cryptography. The application of the key-wrapping step in the present embodiment yields a wrapped key that is referred to as encrypted cryptographic key ECKq.

FIG. 3Bis a diagram illustrating a preferred manner of handling encrypted cryptographic key ECKqprior to its storage in key database134of file system115. First of all, it is preferable to treat encrypted cryptographic key ECKqand its correspondent key metadata142together or as a data unit143. In other words, although the methods and systems of invention are applicable to just encrypted cryptographic keys ECK on their own, for reasons of added security it is preferable to extend these methods and systems to include key metadata142.

Key metadata142typically consists of information about encrypted cryptographic key ECKqand also possibly about its progenitor cryptographic key CKq. In the present embodiment, key metadata142contains the following entries: key owner, owner permissions, creation time, expiration time, last used time. Of course, many other relevant data for encrypted cryptographic key ECKqcan be included in key metadata142in other embodiments. A person skilled in the art will be familiar with metadata formats for encrypted keys and what specific data entries may be required for any given application.

Once ready, data unit143consisting of encrypted cryptographic key ECKqand its key metadata142is passed to a MAC algorithm144running in MAC module124A of acting master key manager MKM (also seeFIG. 1). MAC algorithm144applies master MAC key MMK over entire unit143. In other words, MAC algorithm144applies master MAC key MMK over both encrypted cryptographic key ECKqand its key metadata142to generate a single or a joint message authentication tag146or MAC tag for short. The process of applying master MAC keys over data is well-known in the art and is sometimes also referred to as MACing of data. The output of MACing in the present situation is joint MAC tag146.

Note that although MAC module124A is shown as a separate unit, it is common and indeed preferred that MAC module124A actually reside in HSM122A along with master MAC key MMK. The secure space set up within HSM122A (see, e.g., secure space136inFIG. 3A) ensures a high security level during the MACing process. This is just as important as protecting shareable master key SMK and ensuring its secure application to cryptographic key CKqto generate encrypted cryptographic key ECKq.

In embodiments in which only encrypted cryptographic key ECKqis MACed, MAC tag146obviously only protects the integrity of that key. It is thus not a joint MAC tag that protects the integrity of keys and their metadata. Meanwhile, the MACing of both keys and metadata is safer as it automatically builds in a safety cross-check that enables one to detect and thus prevent tampering by attackers that reshuffle keys and their metadata. The same provision also permits one to discover and therefore prevent attachment of unauthorized metadata to encrypted cryptographic keys.

MAC tags for all MACed data units are stored in key database134along with data units to which they belong. Thus, in the present case, joint MAC tag146is placed along with data unit143containing encrypted cryptographic key ECKqand key metadata142among all of the other MACed data units. For simplicity,FIG. 3Bdesignates the combination of joint MAC tag146and data unit143with encrypted cryptographic key ECKqby primed reference ECK′qdenoting the composite data and a corresponding “MAC” adnotation.

During normal operation of computer system100, joint MAC tag146is used to test or confirm the integrity of encrypted cryptographic key ECKqto which it corresponds. The process of verification or confirmation of integrity includes key metadata142. The corresponding steps are initiated by a request from a requestor. For example, user U may submit a request via client102for encrypted cryptographic key ECKqto access file Fq(seeFIG. 3A) after that file has been previously encrypted with cryptographic key CKqand stored.

The diagram ofFIG. 3Cillustrates the steps undertaken upon receipt of a request for encrypted cryptographic key ECKqin particular. Of course, the same steps are followed when any authorized requestor needs one or more other encrypted cryptographic keys ECK from key database134. In fact, integrity is tested whenever encrypted cryptographic keys ECK are retrieved for any given purpose.

During the first step, the combination of joint MAC tag146and data unit143with encrypted cryptographic key ECKqall referred to as composite data ECK′qare retrieved from database134. Since tampering with composite data ECK′qmay have occurred while being stored in database134, integrity cannot be assumed. Therefore, it is not assumed that joint MAC tag146is still the same. Instead, upon retrieval from database134the MAC tag is presumed to be different and is therefore referred to as joint MAC tag146* to be tested.

In the next step, joint MAC tag146* to be tested is submitted along with the potentially tampered data unit143* to MAC module124A. The master MAC key MMK is now deployed by MAC algorithm144to compute the value of joint MAC tag146* to be tested. This step takes place in MAC module124A that is preferably in primary HSM122A or in some other secure space for reasons explained above.

During the subsequent step, MAC algorithm144is used to calculate joint MAC tag146over metadata142and encrypted key ECKq. The thus calculated MAC tag146is then compared with joint MAC tag146* from the database. The comparison is carried out in a simple comparator148that yields the answer “YES” in the case of equality. Comparator148yields “NO” when joint MAC tag146* to be tested is not equal to the newly calculated MAC value. It should be noted that no specific security measures are taken to protect MAC tag146. The security of any MAC tag has a much lower priority than the security of keys. That is because the correct MAC tag can only be generated by the holder of master MAC key MMK. In accordance with the invention, the holder of master MAC key MMK can only be a member of set118of potential master key manager candidates. All of those are very secure and use their HSMs for keeping master MAC key MMK.

The “NO” answer attests to possible tampering. When confronted with this result, MAC algorithm144reports the result “TAMPERED” thus indicating that either encrypted cryptographic key ECKqor key metadata142or both have possibly been tampered with. At this point, neither one can be trusted. Specifically, metadata142could now be compromised such that it may grant permissions to an attacker. Meanwhile, an attacker could also mount cryptographic attacks against shared master key SMK by using the modified encrypted cryptographic key. Therefore, neither one should be used again.

An attacker's modification to encrypted cryptographic keys or key metadata effectively renders the affected files F that require cryptographic key CKqderived from encrypted cryptographic key ECKqunusable. Differently put, such compromise is tantamount to deletion of affected files F. Given that the number of nodes in set118that handle the serving of keys is usually much smaller than the number of data servers in cluster106, however, these nodes can be secured with relative ease. Furthermore since key database134is also relatively small, it can be easily backed up.

An important aspect of the invention is to ensure that computer system100and its ability to serve cryptographic keys obtained in the manner described above are not compromised by a failure of master key manager MKM. To better understand the failover event triggered by a failure of master key manager MKM, which in the present example is node108A, we turn toFIG. 4. Here we focus on just nodes108A,108B, . . . ,108F that are members of set118of master key manager candidates. The failure of node108A serving as current master key manager MKM is indicated by a break150in one of interconnections116that connects node108A to all other nodes. Of course, connection break150is just one mode of failure that can trigger failover. A person skilled in the art will recognize that problems such as intermittency, high error rate and any other issues leading to unreliability are also valid causes for a failover.

Failure of node108A precipitates the failover event in which a new master key manager MKM is elected from among available master key manager candidates in set118. As pointed out above, the election process can be analogous to the one used in the election of a master node for managing entire cluster106. For example, the election is held based on a quorum vote of available master key manager candidates in set118. In the present case, the available master key manager candidates are nodes108B through108F. Of those, node108B is actually chosen by the election process to be the new master key manager MKM. Failover due to break150thus results in node108B taking over the duties of MKM from node108A as indicated by arrow152inFIG. 4.

As soon as failover occurs, elected node108B acting as master key manager MKM grabs common virtual address CVA assigned by virtual address assignment module120. This new assignment is indicated inFIG. 4. Once again, the skilled artisan will recognize that any method of managing common virtual address CVA is permissible, as long as it promptly reacts to the failover event. In fact, common virtual address CVA can even be implemented based on a simple list of node addresses for all nodes in set118. The current master key manager MKM can thus be placed at the top of the list and use common virtual address CVA for as long as it correctly fulfills its duties, i.e., until the next failover event. In any event, it is important that client102or any requestor of cryptographic keys first obtain from module120common virtual address CVA assigned to current master key manager MKM, which is now node108B.

By equipping all nodes in set118with the same capabilities, the failover event does not cause interruption in the availability of encrypted cryptographic keys ECK. For example, any requestor, e.g., an external client or an internal node or any authorized resource, may now request a previously created cryptographic key through node108B. Such request may be for an already existing cryptographic key encrypting or for a new cryptographic key that is either supplied by the client (client is the submitter) or to be generated in HSM122B of node108B (client is a requestor).

In particular, a client may request previously created cryptographic key CKqthat was already converted to encrypted cryptographic key ECKqin node108A with the aid of shared master key SMK in accordance with the method described above. Acting as active master key manager MKM, node108B now accesses key database134in file system115and pulls encrypted cryptographic key ECKq, as indicated by arrow154. It then executes above-described steps to verify integrity of key ECKq. If MAC tag146* shows no evidence of tampering (seeFIG. 3C), then node108B recovers cryptographic key CKqfrom key ECKqby reversing the process of encryption with shared master key SMK. It then sends cryptographic key CKqto the client. On the other hand, if the client requests or provides a new key, then node108B executes the same steps as already described above for node108A.

In many applications encrypted cryptographic keys ECK will pertain to corresponding file units stored in distributed file system115. In one of the cases, the file units are simply denominated in files F themselves and keys ECK are just per-file keys. In other embodiments, as indicated above, the units can be much larger, e.g., on the order of a memory partition. At the other extreme, small file units such individual memory blocks corresponding to block in block-storage resources can represent the file units. A person skilled in the art will recognize that the granularity of file units will depend on many typical parameters of database and system designs that trade off between security, ease of access, overhead and other well-known performance parameters.

FIG. 5illustrates an embodiment of computer system100where files F are stored based on the memory blocks defined in storage resources114J of node108J. In this example, a specific file Fjis stored in a memory block160ion one of the disks of resources114J. In other words, the file units in this case are memory blocks160in storage resources114J. we note that file Fjis too large to be stored in single block160i, so its remainder is distributed over other blocks160. What is important, is that metadata162iassociated with selected file unit here embodied by memory block160i, also be stored in key database134. This is indicated by arrow164inFIG. 5.

Of course, a computer system that operates a shared file system that ensures high availability of cryptographic keys, such as the encrypted cryptographic keys derived in accordance with the invention, does not necessarily have to be deployed in a cluster environment. In other words, a computer system according to the invention may be deployed locally on just a few machines or in a small network. However, many benefits of high availability during failover events accrue when the invention is deployed in a cluster of nodes.

Specifically, cluster implementation is most well-adapted to the transfer of the common virtual address between different master key managers during failover. This process not only ensures uninterrupted availability of encrypted cryptographic keys during failover, but it is also similar to the handling of general failover between a master node in charge of the cluster and its backup master node candidates. Therefore, in some embodiments the set of machines including the master key manager and the master key manager candidates is assigned to nodes of the cluster that correspond to the cluster's master node and its backup nodes.

In view of the above teaching, a person skilled in the art will recognize that the invention can be embodied in many different ways in addition to those described without departing from the spirit of the invention. Therefore, the scope of the invention should be judged in view of the appended claims and their legal equivalents.