# Patent

## Title
Host based rekeying

## Abstract

A system and method for re-keying ciphertext on a storage system is resident on a host/client communicating with a storage system. The generation of encryption keys and tracking which storage system blocks are encrypted with what keys remain with the security appliance or storage system, but the policy governing re-keying and initiating actions in accordance with that policy reside with the client/host.

## Background
FIELD OF THE INVENTION
The present invention relates to secure storage in a storage system, and more specifically, to generation and control of encryption keys used by secure systems, and even more particularly to control of re-keying operations.
BACKGROUND OF THE INVENTION
A storage system is a computer that provides storage service relating to the organization of information on writable persistent storage devices, such as memories, tapes or disks. The storage system is commonly deployed within a storage area network (SAN) or a network attached storage (NAS) environment. When used within a NAS environment, the storage system may be embodied as a file server including an operating system that implements a file system to logically organize the information as a hierarchical structure of data containers, such as files on, e.g., the disks. Each “on-disk” file may be implemented as a set of data structures, e.g., disk blocks, configured to store information, such as the actual data (i.e., file data) for the file.
Herein the term “client” and “host” are used interchangeably.
Where the information is organized as files, the client requesting the information, typically maintains file mappings and manages file semantics, while its requests (and storage system responses) address the information in terms of block addressing on disk using, e.g., a logical unit number (lun).
A network environment may be provided wherein information (data) is stored in secure storage served by one or more storage systems coupled to one or more security appliances. Each security appliance is configured to transform unencrypted data (cleartext) generated by clients (or initiators) into encrypted data (ciphertext) destined for secure storage or “cryptainers” on the storage system (or target). As used herein, a cryptainer is a piece of storage on a storage device, such as a disk, in which the encrypted data is stored. In the context of a SAN environment, a cryptainer can be, e.g., a disk, a region on the disk or several regions on one or more disks that, in the context of a SAN protocol, is accessible as a lun (logical unit number).
Each cryptainer is associated with its own encryption key, e.g., a cryptainer key, which is used by the security appliance to encrypt and decrypt the data stored on the cryptainer. An encryption key is a code or number which, when taken together with an encryption algorithm, defines a unique transformation used to encrypt or decrypt data. Data remains encrypted while stored in a cryptainer until requested by an authorized client. At that time, the security appliance retrieves the encrypted data from the cryptainer, decrypts it and forwards the unencrypted data to the client.
In practical systems encrypted data stored on a disk block (or other cryptainer) is often re-keyed, that is re-encrypted, according to a policy. If for example, the policy may dictate that, if an encryption key is compromised, the data may not be secure and must be re-keyed or re-encrypted. The policy may also dictate that, as a matter of common practice, data might be regularly re-encrypted on a time basis. For example, for critical information the data may be re-encrypted every day or week or month, whereas other data may be re-encrypted over longer time periods, e.g., every year. Herein re-keying involves reading and decrypting ciphertext data with the old key to yield clear data, and then re-encrypting the clear data with a new encryption key and restoring the data to the storage medium.
In some systems the security appliance performs the encryption and decryption, and, in addition, the security appliance may control access to both the storage and the client or authentication, where only “secure known” clients are served. The security appliance is the proxy for both the client and the storage system. That is the security appliance appears as the storage system to the client and appears as the client to the storage system. Additionally, re-keying is also handled by the security appliance, where the security appliance manages the key(s) and the block(s) associated with each key. In known systems each file may be encrypted with a given key and the file and the key are stored and tracked by the security appliance using a file system to manage all the file keys.
The security appliance logs and tracks the activities for a large number of differs ent clients, a large number of different file blocks, a large number of different encryption keys that must be generated and implemented, and a large number of different storage systems. The security appliance store the policy governing such re-keying while tracking a very large number of new keys and corresponding very large number of blocks being re-keyed. The security appliance interacts with the storage system to reserve space and to initiate reads and writes to that space, and it ensures that there is no interference among the clients, files, keys, and storage systems.
The volume of tasks enumerated above may affect system performance. The operations may be slowed and become more inflexible due to the work load. Accordingly, it may be advantageous to offload or shift some of the tasks performed by the security appliance to improve performance.
SUMMARY OF THE INVENTION
The present invention addresses limitations in the known systems by shifting tasks from the security appliance to the clients or hosts. In particular the policy governing compromised keys and initiating re-writing (re-keying) of stored ciphertext may be transferred from the security appliance to the client/host. In such a system, the security appliance still generates encryption keys, performs the encryption and decryption and tracks which block or blocks of a storage system are encrypted with what keys.
System performance may be improved with the security appliance performing fewer tasks, and, with improved performance, the system becomes more flexible to accommodate other possible configurations and arrangements.
It will be appreciated by those skilled in the art that although the following Detailed Description will proceed with reference being made to illustrative embodiments, the drawings, and methods of use, the present invention is not intended to be limited to these embodiments and methods of use. Rather, the present invention is of broad scope and is intended to be defined as only set forth in the accompanying claims.



BRIEF DESCRIPTION OF THE DRAWINGS
The above and further advantages of the invention may be better understood by referring to the following description in conjunction with the accompanying drawings in which like reference numerals indicate identical or functionally similar elements:
FIGS. 1A and B are schematic block diagrams of an environment including a multi-protocol security appliance that may be advantageously used with the present invention;
FIG. 2 is a schematic block diagram of the multi-protocol security appliance that may be advantageously used with the present invention;
FIG. 3 is a schematic diagram illustrating an arrangement of software processes and modules executing on the security appliance in accordance with the present invention;
FIG. 4 is a schematic block diagram of a lifetime key management (LKM) server that may be advantageously used with the present invention; and
FIG. 5 is a flowchart illustrating a re-keying procedure in accordance with an illustrative embodiment of the present invention.



DETAILED DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT
The present invention includes a host initiating re-keying of stored ciphertext. The host may directly connect to a security appliance or the storage system having a security module 404 as illustrated in FIGS. 1A and 1B.
FIG. 1A is a schematic block diagram of an environment 100 including a multi-protocol security appliance 200 that may be advantageously used with the present invention. The security appliance 200 is coupled between one or more clients 102 and one or more storage systems 110, such as an application server or filer. The security appliance 200 intercepts a data access request issued by client 102, destined for the storage system 110, wherein the data access request may be a read request to retrieve certain data stored on storage devices, such as disks 120, coupled to the storage system 110 or a write request to store data on the disks. In the case of a write request, the security appliance 200 intercepts the request, encrypts the data associated with the request and forwards the encrypted data to the storage system 110 for storage at a specified location (address) on disk 120. In the case of a read request, the security appliance intercepts the request and forwards it onto the storage system, which returns the requested data to the appliance in encrypted form. The security appliance 200 then decrypts the encrypted data and returns the decrypted data to the client 102.
In the illustrative embodiment, the security appliance employs a conventional encryption algorithm, e.g., the Advanced Encryption Standard (AES) or other appropriate algorithms, to transform unencrypted data (cleartext) generated by the clients 102 into encrypted data (ciphertext) intended for secure storage, i.e., one or more cryptainers, on the storage system 110. To that end, the security appliance illustratively uses a high-quality, software or hardware-based pseudo random number generation technique to generate encryption keys. The encryption and decryption operations are performed using these encryptions keys, such as a cryptainer key associated with each cryptainer. As described herein, the security appliance 200 uses an appropriate cryptainer key to encrypt or decrypt portions of data stored in a particular cryptainer. The security appliance, in addition to encrypting and decrypting, addresses the storage system 110, provides authentication, secure-logging and “virtualization.” Virtualization refers to the security appliance appearing as if it were the storage system to the client, while appearing as if it were the client to the storage system.
A lifetime key management (LKM) server 400 is configured to manage all encryption keys used by the security appliance 200 to encrypt and decrypt data securely stored on the storage system 110, ensuring encryption key availability for the life of the secured data. For example, the LKM server 400 receives encrypted cryptainer keys from the security appliance 200 and sends encrypted cryptainer keys on demand to the appliance. The LKM server is further configured to support a plurality of security appliances 200 such that, when a particular appliance encounters a data access request directed to a cryptainer for which it does not have the appropriate key, that appliance accesses the LKM server 400 to receive the appropriate key.
In FIG. 1B, the security aspects are part of the storage system itself, and the LKM functions are handled by a device 402 connected to the storage system. In some illustrative embodiments, the LKM may be constructed as backup storage for the encryption keys used by the single storage system.
The actual implementations within the systems of FIGS. 1A and 1B may comprise hardware, firmware, software or combinations thereof.
The clients may be running conventional operating system software, for example Windows or Unix, but other operating systems may be used, and, further, the operations may be implemented in hardware or firmware/hardware, etc. The protocol governing the communications between the clients and the security appliance may be XML, but others may be used.
FIG. 2 is a schematic block diagram of the multi-protocol security appliance 200 that may be advantageously used with the present invention. As used herein, a security appliance denotes a computer having features such as simplicity of security service management for users (system administrators) and clients/hosts of network attached storage (NAS) and storage area network (SAN) deployments. The security appliance comprises one or more processors, e.g., central processing units (CPU 220a,b), a memory 210, one or more network adapters 220a,b, and a storage encryption processor (SEP 390) interconnected by a system bus 240, such as a conventional Peripheral Component Interconnect (PCI) bus. The SEP 390 is configured to perform all encryption and decryption operations for the security appliance in a secure manner; for example, the SEP is configured to protect plaintext encryption keys from system software executing on each CPU 202. Accordingly, the SEP is illustratively embodied as a FIPS 140-2 level-3 certified module that is epoxy-potted onto a dedicated interface card or other similar card.
The SEP 390 illustratively implements the AES-256 encryption algorithm. However, in alternative embodiments, the SEP 390 may implement additional and/or differing encryption algorithms including, e.g., HMAC-SHA-512, deterministic random number generator (DRNG) using SHA-1, elliptic curve cryptography, etc.
The network adapters 220a,b couple the security appliance 200 between one or more clients 102 and one or more storage systems 110 over point-to-point links, wide area networks, virtual private networks implemented over a public network (Internet) or shared local area networks. In a SAN environment configured to support various Small Computer Systems Interface (SCSI)-based data access protocols, including SCSI encapsulated over TCP (iSCSI) and SCSI encapsulated over FC (FCP), the network adapters 220 may comprise host bus adapters (HBAs) having the mechanical, electrical and signaling circuitry needed to connect the appliance 200 to, e.g., a FC network or a support computer system. In a NAS environment configured to support, e.g., the conventional Common Internet File System (CIFS) and the Network File System (NFS) data access protocols, the network adapters 220a,b may comprise network interface cards (NICs) having the mechanical, electrical and signaling circuitry needed to connect the appliance to, e.g., an Ethernet network.
The memory 210 illustratively comprises storage locations that are addressable by the processors and adapters for storing software programs and data structures associated with the present invention. The processor and adapters may, in turn, comprise processing elements and/or logic circuitry configured to execute the software programs and manipulate the data structures. An operating system 212, portions of which are typically resident in memory and executed by the processing elements, functionally organizes the appliance 200 by, inter alia, invoking security operations in support of software processes and/or modules implemented by the appliance. It will be apparent to those skilled in the art that other processing and memory means, including various computer readable media, may be used for storing and executing program instructions pertaining to the invention described herein.
Since the SEP 390 protects encryption keys from being “touched” (processed) by the system software executing on the CPU 202, a mechanism is needed, at least initially to load keys into and retrieve keys from the SEP. To that end, the card reader 230 provides an interface between a “smart” system card 250 and SEP 390 for purposes of exchanging encryption keys. Illustratively, the system card is a FIPS 140-2 certified card that is configured with a customized software code. As described further below, a portion of the cryptographic information, needed to initialize the security appliance, is stored on the system card 250, thereby preventing the initialization of the appliance 200 without the presence of the card 250. The security appliance (and card reader 230) are further configured to support additional smart cards 260.
Operationally, encryption keys are exchanged between the SEPs 390 and system card 250, where they are “secret shared” (cryptographically assigned) with the recovery cards 260 as recovery keys. These recovery keys can thereafter be applied (via the recovery cards) to the security appliance 200 to enable restoration of other encryption keys (such as cryptainer keys). A quorum setting for the recovery cards 260 may be provided such that the recovery keys stored on the recovery cards are backed up in a threshold scheme whereby, e.g., any 2 of the 5 default cards can recover the keys. The use of these recovery cards may be referred to herein as activities performed by recovery officers, since the recovery cards are controlled by humans.
The operating system 212 illustratively organizes the memory 210 into an address space arrangement available to the software processes and modules executing on the processors.
FIG. 3 is a schematic diagram illustrating an arrangement 300 of software processes and modules executing on the security appliance 200 in accordance with the present invention. In the illustrative embodiment, the operating system software is a customized version of a Unix type operating system, although other operating systems may be used.
For both NAS and SAN environments, data is received at a proxy 320 of the security appliance. The proxy 320 is a module embodied as, e.g., the network protocol stack configured to interpret the protocol over which data is received and to enforce certain access control rules based on one or more policies. Each policy is served by a box manager 360. The box manager 360 is illustratively embodied as a database application process configured to manage a configuration repository or database (ConfigDB 370) that stores system-wide settings and encryption keys. A socket server 380 provides interfaces to the box manager 360, including (i) an HTTP web interface 382 embodied as, e.g., a graphical user interface (GUI) adapted for web-based administration, (ii) an SSH interface 384 for command line interface (CLI) command administration, and (iii) and a SNMP interface 386 for remote management and monitoring.
Specifically, the box manager 360 supplies permissions and encryption keys to the proxy 320, which intercepts data access requests and identifies the source (clients 102) of the those requests, as well as the types of requests and the storage targets (cryptainer) of those requests. The proxy also queries the box manager for permissions associated with each client/host and, in response, the box manager 360 supplies the appropriate permissions and encryption key (e.g., a cryptainer key). The proxy 320 then bundles the data together with the encryption key and forwards that information to a crypto layer 330.
The crypto layer 330 interacts with the DCC 340 (Data Crypto Card) by accessing (reading and writing) registers on the DCC and, to that end, functions as a system bus interface to present the starting and ending points of data, as well as offsets into the data and the encrypted keys used to encrypt the data. The DCC 340 includes one or more previously loaded keys used to decrypt the supplied encrypted keys; upon decrypting an encrypted key, the DCC uses the decrypted key to encrypt the supplied data. Upon completion of encryption of the data, the DCC returns the encrypted data as ciphertext to the proxy 320, which forwards the encrypted data to the storage system 110.
FIG. 4 is a schematic block diagram of the LKM server 400 that may be advantageously used with the present invention. The LKM server may be illustratively embodied as software running on a machine, such as a computer executing, e.g., the conventional Windows operating system. However, it will be understood to those of skill in the art that the LKM server could also be embodied as hardware. The LKM 400 includes a key database 410, which is illustratively a relational database (e.g., a conventional SQL database) or other database configured to store keys. A network interface 420 provides connections to all security appliances 200 that utilize the LKM server. The LKM 400 illustratively communicates with the security appliance 200 via a proprietary protocol, preferably based on XML. A cryptographic module 430 is configured to implement algorithms needed to perform cryptographic operations for the LKM.
An overlapping, data decryption software (DDS) module 460 “shares” the cryptographic module (e.g., its algorithms) on the LKM server. Illustratively, the function of the DDS 460 is to decrypt a piece of storage (e.g., a file) with an appropriate key to thereby enable recovery of the data contained in the file. To that end, the DDS functions as a data recovery software module. The LKM server 400 also includes a GUI 470 that provides a user with an interface to the functionality of the LKM and DDS 460. Alternatively, an external user interface 480 may run on a stand alone machine, such as a Windows management station, to provide GUI functionality for the LKM.
In the illustrative embodiment, the LKM 400 is embodied as an independent server that is coupled to one or more security appliances 200. When initializing a security appliance, a system administrator specifies the LKM 400 with which the appliance is to communicate. In addition, a trust relationship is established between the security appliance and the LKM using, e.g., a shared secret or certificate to establish a secure communication channel. Note that storage of the key on the key database 410 is primarily for redundancy/backup of the key to thereby reduce the chances of data loss for information protected by the key.
In accordance with an aspect of the invention, the client/host may store and enforce one or more policies that may include re-writing ciphertext. When initializing the system of FIG. 1A or 1B, an administrator may communicate with the host via a GUI attached to the host or to the computing functions within the storage system 110. The administrator may enter re-writing policies within the host that are to be exercised by the host. Those policies reduce the chances of others obtaining information protected by a key. Among the more common policies is that if a key has been or thought to have been compromised re-writing may be ordered. Another common policy includes setting an expiration time for the keys, where the host tracks the expiration time with the key. When a key's expiration time is reached, the data encrypted with that key is re-written. In each instance of re-keying the old keys are deleted after their final use. The host may enforce other attributes input by the administrator, one such attribute may restrict the key to only reading and not writing data.
For example, if a key has been compromised, or if a time limit attribute of a key has expired, policy may instruct the client/host to re-write (re-key) group or all the blocks of a storage system. In response, the client host issues a re-write command to the security appliance, which generates new encryption keys. The stored ciphertext in the blocks is read and decrypted using the old keys and then re-encrypted using the new keys and written back into the same storage blocks. Thereafter, the old keys are deleted. The security appliance maintains a table or database that associates encryption keys with storage blocks.
In some systems, not shown, encryption keys are wrapped within another encrypted layer using a second encryption key. Such system may be used advantageously with the present invention. Notably, a file key may be used to encrypt a data file, and an cryptainer key may be used to encrypt part of a metadata portion of the file. That is, each file key is encrypted and signed with the cryptainer key.
FIG. 5 is a flowchart illustrating a re-keying procedure 800 in accordance with the present invention. The procedure 800 starts at Step 802 and proceeds to Step 804 where a client/host operating under a given policy initiates a re-writing of a specified group (or all) of the stored blocks of data. The security appliance, in response, initiates re-keying of the ciphertext data on the designated areas (blocks) of the storage system. At step 806 security appliance generates new encryption keys, and in step 808 reads ciphertext from the storage system. The ciphertext is decrypted in step 810 using the old key or keys, and then the data is re-encrypted using the new keys in step 812. In step 814 the security appliance writes the re-encrypted ciphertext into the same blocks in the storage system. The procedure ends in step 816. In other illustrative embodiments, the re-encrypted ciphertext may be written into other blocks within the storage system, as known to those skilled in the art.
The foregoing description has been directed to specific embodiments of this invention. It will be apparent, however, that other variations and modifications may be made to the described embodiments, with the attainment of some or all of their advantages. For instance, it is expressly contemplated that the procedures, processes, layers and/or modules described herein may be implemented in hardware, software, embodied as a computer-readable medium having executable program instructions, firmware, or a combination thereof. Accordingly this description is to be taken only by way of example and not to otherwise limit the scope of the invention. Therefore, it is the object of the appended claims to cover all such variations and modifications as come within the true spirit and scope of the invention.

## Claims

1. A method comprising:
receiving, by a computing device, a re-key command issued by a host computer for one or more stored ciphertexts each associated with one of a plurality of designated data blocks stored on one or more disk storage devices, wherein the issuing of the re-key command is initiated based on one or more re-writing policies that tracks thresholds associated with one or more old encryption key attributes stored on the host computer; and
generating, by the computing device, a plurality of new encryption keys, upon receipt of the re-key command issued from the host computer, wherein the plurality of new encryption keys are adapted to apply to the one or more stored ciphertexts;
reading, by the computing device, the one or more stored ciphertexts;
decrypting, by the computing device, the one or more stored ciphertexts using one or more old encryption keys to produce a decrypted data file comprising content;
re-encrypting, by the computing device, a portion of the decrypted data file using a first one of new encryption keys and re-encrypting another portion of the data file using a second one of new encryption keys to produce re-encrypted ciphertext; and
writing, by the computing device, the re-encrypted ciphertext, into the one or more disk storage devices.

2. The method of claim 1 wherein the issuing of the re-key command is initiated based on one or more re-writing policies that tracks temporal thresholds associated with one or more old encryption key attributes stored on the host computer.

3. The method of claim 1 wherein the a portion of the decrypted data file comprises a data portion of the file and the first one of new encryption keys comprises a file key.

4. The method of claim 3 wherein the another portion of the decrypted data file comprises a metadata portion of the file and the second one of new encryption keys comprises a cryptainer key.

5. The method of claim 4 wherein the file key is encrypted and signed with the cryptainer key.

6. The method of claim 1 wherein writing the re-encrypted ciphertext comprises writing into the same one of the plurality of designated data blocks from which the one or more stored ciphertexts were read.

7. The method of claim 1 to providing a logical unit number that contains a location of the one or more stored ciphertexts.

8. A computing device comprising:
a memory containing a machine readable medium comprising machine executable code having stored thereon instructions for performing a method of providing data sessions with clients that access data containers of a shared storage; and
a processor coupled to the memory, the processor configured to execute the machine executable code to cause the processor to:
receive a re-key command issued by a host computer for one or more stored ciphertexts each associated with one of a plurality of designated data blocks stored on one or more disk storage devices, wherein the issuing of the re-key command is initiated based on one or more re-writing policies that tracks thresholds associated with one or more old encryption key attributes stored on the host computer; and
generate a plurality of new encryption keys, upon receipt of the re-key command issued from the host computer, wherein the plurality of new encryption keys are adapted to apply to the one or more stored ciphertexts;
read the one or more stored ciphertexts;
decrypt the one or more stored ciphertexts using one or more old encryption keys to produce a decrypted data file comprising content;
re-encrypt a portion of the decrypted data file using a first one of new encryption keys and re-encrypting another portion of the data file using a second one of new encryption keys to produce re-encrypted ciphertext; and
write the re-encrypted ciphertext, into the one or more disk storage devices.

9. The device of claim 8 wherein the processor is further configured to execute the machine executable code to cause the processor to write the re-encrypted ciphertext into a same one or more blocks from which the stored ciphertext were read.

10. The device of claim 8 wherein the processor is further configured to execute the machine executable code to cause the processor to provide a logical unit number that contains a location of the ciphertext.

11. The device of claim 8 wherein the issuing of the re-key command is initiated based on one or more re-writing policies that tracks temporal thresholds associated with one or more old encryption key attributes stored on the host computer.

12. The device of claim 8 wherein the a portion of the decrypted data file comprises a data portion of the file and the first one of new encryption keys comprises a file key.

13. The device of claim 12 wherein the another portion of the decrypted data file comprises a metadata portion of the file and the second one of new encryption keys comprises a cryptainer key.

14. The device of claim 13 wherein the file key is encrypted and signed with the cryptainer key.

15. A non-transitory machine readable medium having stored thereon instructions for performing a method comprising executable code that, when executed by at least one machine, causes the machine to:
receive a re-key command issued by a host computer for one or more stored ciphertexts each associated with one of a plurality of designated data blocks stored on one or more disk storage devices, wherein the issuing of the re-key command is initiated based on one or more re-writing policies that tracks thresholds associated with one or more old encryption key attributes stored on the host computer; and
generate a plurality of new encryption keys, upon receipt of the re-key command issued from the host computer, wherein the plurality of new encryption keys are adapted to apply to the one or more stored ciphertexts;
read the one or more stored ciphertexts;
decrypt the one or more stored ciphertexts using one or more old encryption keys to produce a decrypted data file comprising content;
re-encrypt a portion of the decrypted data file using a first one of new encryption keys and re-encrypting another portion of the data file using a second one of new encryption keys to produce re-encrypted ciphertext; and
write the re-encrypted ciphertext, into the one or more disk storage devices.

16. The medium of claim 15 wherein the issuing of the re-key command is initiated based on one or more re-writing policies that tracks temporal thresholds associated with one or more old encryption key attributes stored on the host computer.

17. The medium of claim 15 wherein the a portion of the decrypted data file comprises a data portion of the file and the first one of new encryption keys comprises a file key;
wherein the another portion of the decrypted data file comprises a metadata portion of the file and the second one of new encryption keys comprises a cryptainer key.

18. The medium of claim 17 wherein the file key is encrypted and signed with the cryptainer key.

19. The medium of claim 15 wherein to write the re-encrypted ciphertext comprises writing into the same one of the plurality of designated data blocks from which the one or more stored ciphertexts were read.

20. The medium of claim 15 to provide a logical unit number that contains a location of the ciphertext.