Random access of a member file in a compressed tar archive

A cloud computing infrastructure hosts a web service with customer accounts. In a customer account, files of the customer account are listed in an index. Files indicated in the index are arranged in groups, with files in each group being scanned using scanning serverless functions in the customer account. The files in the customer account include a compressed tar archive of a software container. Member files of a compressed tar archive in a customer account are randomly-accessed by way of locators that indicate a tar offset, a logical offset, and a decompressor state for a corresponding member file. A member file is accessed by seeking to the tar offset in the compressed tar archive, restoring a decompressor to the decompressor state, decompressing the compressed tar archive using the decompressor, and moving to the logical offset in the decompressed data.

TECHNICAL FIELD

The present disclosure is directed to file processing and cloud service cybersecurity.

BACKGROUND

Third-party cloud services, such as the Amazon Web Services (AWS)™ cloud service, provide computing resources, such as storage and compute resources, to customers so that the customers can run application programs without having to purchase and maintain their own infrastructure. These cloud services are third-party relative to their customers in that they are not associated with the customers. A cloud service may charge customers based on central processing unit (CPU), memory, data storage, and/or network bandwidth consumption.

Third-party cybersecurity services are available to customers of cloud services. These cybersecurity services are third party relative to the customers and the cloud service. Providing third-party cybersecurity services to customers of cloud services is problematic because of performance and cost concerns. As a particular example, to perform an antimalware scan of customer data stored in an Amazon Elastic Block Store (EBS)™ storage volume, a cybersecurity service may have to take a snapshot of the customer data in the customer's account in the AWS™ cloud service, copy the snapshot to the cybersecurity service's account in the AWS™ cloud service, reform the snapshot back into a new EBS™ storage volume, and thereafter attach the new EBS™ storage volume to an Amazon Elastic Compute Cloud (EC2)™ instance for scanning. The just-mentioned scanning approach violates privacy laws because the cybersecurity service moves a complete copy of the customer data into its account and requires use of any encryption key outside of the customer's account. Furthermore, the scanning approach moves large amounts of data between accounts, thereby lengthening the scanning time and increasing the cost of the scanning.

BRIEF SUMMARY

In one embodiment, a cloud computing infrastructure hosts a web service with customer accounts. In a customer account, files of the customer account are listed in an index. Files indicated in the index are arranged in groups, with files in each group being scanned using scanning serverless functions in the customer account. The files in the customer account may include a compressed tar archive of a software container. Member files of a compressed tar archive in a customer account are randomly-accessed by way of locators that indicate a tar offset, a logical offset, and a decompressor state for a corresponding member file. A member file is accessed by seeking to the tar offset in the compressed tar archive, restoring a decompressor to the decompressor state, decompressing the compressed tar archive using the decompressor, and moving to the logical offset in the decompressed data.

DETAILED DESCRIPTION

Embodiments of the present invention are explained in the context of services available from the AWS™ cloud service. As can be appreciated, the embodiments are equally applicable to other cloud services.

FIG.1shows a logical diagram of a system for scanning cloud resources of customers of a cloud service in accordance with an embodiment of the present invention. The system ofFIG.1includes a cloud service100and a computer103of a customer of the cloud service100. In one embodiment, the cloud service100is the AWS™ cloud service. The cloud service100is hosted by a cloud computing infrastructure102, which comprises one or more computers and associated software for providing the cloud service100. The customer, using the computer103, accesses the cloud service100over a computer network, which in this example is over the Internet.

The cloud service100has a plurality of accounts101(i.e.,101-1,101-2, . . . ,101-n), one for each customer. The cloud service100allows an account101to subscribe to one or more cloud service-provided resources, which in the example ofFIG.1includes a block-level storage104, an object-level storage105, virtual machines106, serverless platform107, etc. In one embodiment, the block-level storage104is the Amazon EBS™ storage, the object-level storage105is the Amazon Simple Storage Service (S3)™ storage, the virtual machines106are Amazon EC2™ instances, and the serverless platform107is the AWS Lambda™ platform. The cloud service100provides an interface that allows the customer to log onto the account101and employ the cloud service-provided resources that the account101are subscribed.

An account101may be employed to maintain a server for ecommerce, run a business, stream content, or for other purpose that requires a computing infrastructure. The account101may have associated customer data stored in the block-level storage104or object-level storage105, for example. Customer data includes files of an application program, software container (“container”), logs, or other data in the account101. In one embodiment, as will be explained with reference toFIG.2, one or more accounts101are subscribed to a third-party cybersecurity service that scans the customer data for cybersecurity threats or other purpose.

FIG.2shows a logical diagram of the configuration of the web service100in accordance with an embodiment of the present invention. As previously noted, an account101may be configured to subscribe to a cybersecurity service to scan data of the account101. In the example ofFIG.1, a customer account101-1is that of a customer, whereas a cybersecurity account101-2is that of a cybersecurity service. In one embodiment, the cybersecurity service protects file-based cloud resources, such as customer data that is stored as files in the block-level storage104and/or object-level storage105of the customer account101-1. The cybersecurity service may scan customer data using serverless functions that are executed in the customer account, i.e., in the account101-1in the example ofFIG.1. The cybersecurity service may provide AWS CloudFormation™ templates to facilitate installation of the serverless functions in the customer account.

Generally, “serverless” is a cloud computing execution model in which the cloud service allocates machine resources on demand, to run application programs on behalf of its customers. A serverless function is a programmatic function for a particular purpose and executes in accordance with the serverless execution model. A serverless function is stateless and ephemeral. More particularly, a serverless function does not maintain data. The serverless function executes upon occurrence of a triggering event, and is destroyed after performing its function. A step function is a serverless orchestration service that lets a customer coordinate multiple serverless functions into one or more workflows. In one embodiment, the cybersecurity service is provided using serverless functions that are coordinated by a step function in the AWS Lambda™ platform.

In the example ofFIG.2, the snapshot provider function201is a serverless function that is configured to take a snapshot211of a volume215of the block-level storage104. The block-level storage104stores data in storage blocks. Unlike the file-based storage105, the block-level storage104cannot be directly accessed over the Internet.

The volume215is a group of storage blocks, which in this example store files of the customer. A snapshot211is a point-in-time copy of the volume215. In one embodiment, the snapshot211is an Amazon EBS™ snapshot. A scheduler210triggers the snapshot provider function201to take the snapshot211of the volume215. The scheduler210may trigger the snapshot provider function201(see arrow230) to take the snapshot211(see arrow231) in accordance with a predetermined schedule. In one embodiment, the scheduler210is implemented using the cron job feature of the AWS CloudWatch™ service.

The creation of the snapshot211generates an event (see arrow232) that starts the scanner233to scan the snapshot211. As will be further explained below, the scanner233comprises a step function and a plurality of serverless functions for scanning file-based cloud resources, such as files in the snapshot211. The results of the scanning are provided by the scanner233as a report213(see arrow234), which is stored as a file in the object-level storage105. The writing of the report213in the object-level storage105triggers the send report function203(see arrow235), which is a serverless function, to send the report213from the object-level storage105to the cybersecurity account101-2by way of an Application Programming Interface (API)214provided by the cloud service100for this purpose (see arrow236).

In the cybersecurity account101-2, the parse report function205is a serverless function that receives and parses the report213to detect anomalies. The parse report function205is configured to perform a response action, such as to raise an alert (e.g., send a message to an administrator; display an alert message on a screen), in response to finding an anomaly in the report213. Such anomalies include presence of malware, exploitable vulnerabilities, invalid configurations, etc. detected in or from files in the volume215.

The scanner233may employ a plurality of patterns to detect anomalies. More particularly, the scanner233may include a plurality of serverless functions that scan data against one or more patterns. For example, data that matches a malware pattern is detected to be infected by malware. The patterns may be updated from time to time. The updated patterns may be provided in an AWS Lambda™ layer that is published from the cybersecurity account101-2. In the cybersecurity account101-2, a forward pattern function204is a serverless function that is triggered by availability of the updated patterns (see arrow237) in the cybersecurity account101-2. Upon being triggered, the forward pattern function204triggers an update pattern function202in the customer account101-1(see arrow238), by way of an API212provided by the cloud service for this purpose. The update pattern function202is a serverless function that when triggered updates the scanning serverless functions of the scanner233(see arrow239) to use the updated patterns in the newly published AWS Lambda™ layer.

FIG.3shows a logical diagram of the scanner233in accordance with an embodiment of the present invention. In one embodiment, the scanner233comprises a coordination step function310, a volume parsing function311, a plurality of cybersecurity scan functions312(i.e.,312-1,312-2,312-3,312-4, . . . ), an integrity scan function313, and a final report function314that all operate in the customer account101-1. The coordination step function310is a step function, whereas the functions311-314are serverless functions.

The coordination step function310orchestrates the triggering of the functions311-314. The coordination step function310initiates the scanning of the snapshot211in response to creation of the snapshot211. The coordination step function310triggers the volume parsing function311to parse the volume215(see arrow321), as reflected in the snapshot211, to identify a plurality of files301(i.e.,301-1,301-2,301-3, . . . ,301-n) that are stored in the volume215. The volume parsing function311identifies the files301in the volume215and generates an index303of the files301. Because the volume parsing function311runs in the customer account101-1, the volume parsing function311may be configured to use the customer's encryption keys that may be needed to decrypt the volume215.

The index303is a listing that indicates the files301and the locations of the files301(e.g., starting sector, length, etc.) in the volume215. The files301in the index303may be arranged in groups, with the groups of files301being scanned in parallel. The volume parsing function311writes the index303as a file in the object-level storage105of the customer account101-1(see arrow322). As its name indicates, the object-level storage105stores data at the object level, which in this example is at the file level. Being a serverless function, the volume parsing function311is destroyed after parsing the volume215and writing the index303in the object-level storage105.

After the volume215has been parsed by the volume parsing function311, the coordination step function310triggers the scan functions312to scan the files301indicated in the index303(see arrow323) and triggers the integrity scan function313to perform an integrity check of the volume215(see arrow324).

In one embodiment, each file301is subjected to a scan chain341, which scans the file301in sequence using the scan functions312. In the example ofFIG.3, the scan function312-1scans a file301for malware, the scan function312-2scans the file301for vulnerability, the scan function312-3scans the file301for log monitoring, and the scan function312-4scans the file301for configuration monitoring. More particularly, for each file301assigned to a scan chain341, the coordination step function triggers the scan function312-1to scan the file301; after the scanning by the scan function312-1, the coordination step function310triggers the scan function312-2to scan the file301; after the scanning by the scan function312-2, the coordination step function310triggers the scan function312-3to scan the file301; and after the scanning by the scan function312-3, the coordination step function310triggers the scan function312-4to scan the file301. As can be appreciated, the number and type of scan functions in a scan chain341depend on the file and particular cybersecurity application. The scan functions312may scan the file301using conventional cybersecurity algorithms.

The number of files301assigned to each scan chain341depends on the number of files301in the index303. For load balancing, the files301indicated in the index303may be divided as equally as possible among the available scan chains341. For example, assuming ten files301in the index303and there are ten scan chains341, each file301is assigned to a single scan chain341. As another example, assuming 100 files301in the index303and there are ten scan chains341, ten files301are assigned to each scan chain341.

The coordination step function310may initiate scanning of the files301in groups, with each group of files301being assigned to a scan chain341. For example, assuming each group has ten files301, the coordination step function310may initiate ten scanning chains341to execute in parallel. Each of the ten files301that are grouped together in a scanning chain341is scanned in sequence.

The integrity scan function313scans the volume215for changes or differences in the volume215since the last scan, such as new files created, deleted, or modified. The integrity scan provides a system view that is useful in detecting a security threat. For example, files added or deleted when no file addition or deletion is expected indicates a possible security breach. Another example of a possible security breach is modification or deletion of special files, such as password files.

The scan functions312and the integrity scan function313generate sub-reports302indicating the result of their respective scans and write the sub-reports302in the object-level storage105of the customer account101-1(see arrow325). Being serverless functions, the scan functions312and the integrity scan function313are destroyed after completing their scans and writing the corresponding sub-reports302in the object-level storage105.

After completion of the cybersecurity and integrity scans, the coordination step function310triggers the final report function314to compile the sub-reports302(see arrow326) from the object-level storage105to generate the report213(see arrow327). As previously noted with reference toFIG.2, the report213is subsequently provided to the cybersecurity account101-2for analysis (seeFIG.2, arrow235).

FIG.4shows a flow diagram of a method of parsing the volume215of the block-level storage104in accordance with an embodiment of the present invention. The method ofFIG.4may be performed by the volume parsing function311(shown inFIG.3).

The method ofFIG.4is initiated when the coordination step function310triggers the volume parsing function311to parse the volume215as represented in the snapshot211(step401). The volume parsing function311initiates the parsing of the volume215(step402) by reading the partition table of the block-level storage104from the snapshot211(step403). The partition table provides block location information in the block-level storage104, which allows the volume parsing function311to get a listing of the files stored in the volume215(step404). More particularly, the volume parsing function311retrieves and parses the file system metadata of the block-level storage104(step405) to identify and find files in all directories and sub-directories (step406) in the volume215. The volume parsing function311sorts the identified files into the index303and writes out the index303to the object-level storage105(step407). The volume parsing function311is thereafter destroyed.

FIG.5shows a flow diagram of a method of scanning cloud resources of customers of a cloud service in accordance with an embodiment of the present invention. The method ofFIG.5is explained with reference toFIGS.1-4. As can be appreciated, the method ofFIG.5may also be performed using other components or steps without detracting from the merits of the present invention. In the example ofFIG.5, steps451-456are performed in a customer account (FIG.2, customer account101-1), whereas step457is performed in a cybersecurity account (FIG.2, cybersecurity account101-2).

In step451, a cloud resource (e.g., volume215of the block-level storage104) of a customer account (e.g., customer account101-1) is parsed using a volume parsing serverless function (e.g., volume parsing function311). In step452, the parsing results in identification of objects (e.g., files301) in the cloud resource. In step453, the volume parsing serverless function lists the identified objects in an index (e.g., index303). In step454, the objects in the index are arranged in groups (scan chains341). In step455, the groups are scanned in parallel, with objects in a group being scanned by a sequence of scanning serverless functions (e.g., scan functions312). In step456, a final report serverless function (e.g., final report function314) generates a report (e.g., report213) that includes sub-reports of the results of the scanning. In step457, the report is analyzed in the cybersecurity account (e.g., account101-2) by a parse report serverless function (e.g., parse report function205) to identify an anomaly, such as presence of malware in the objects.

The above-described embodiments scan customer data that are stored in block-level storage. As can be appreciated, the embodiments can also scan customer data that are stored in other types of storage, e.g., customer data stored as files in object-level storage. In that case, the volume parsing function would simply read the files from the object-level storage and sort the files into an index. The files in the index can then be scanned as explained.

Embodiments of the present invention are especially advantageous in cloud services because the embodiments allow efficient scanning of software containers (“containers”). More particularly, application programs of a customer account in a cloud service may be provided as containers. A container wraps an application program to include auxiliary program code, such as runtime, system tools, system libraries, etc., that the application program needs to run in a host machine. Unlike a virtual machine, a container does not have its own, separate operating system; the container shares the host operating system with other containers in the host machine. Containers are lightweight and relatively easy to deploy compared to virtual machines, hence the popularity of containers. In the cloud service100, containers may be run in an instance of the virtual machines106, for example.

A container image comprises one or more layers that pack an application program (or application programs) and an environment for running the application program, whereas a container is a running instance of a container image. The application program that is containerized in the image may provide a web server, software as a service (SaaS), or other service or function.

Tar is a software utility for collecting a plurality of files into one archive file for distribution or backup purposes. A container image may be stored as a plurality of tar archives, with each tar archive being a layer of the container image. A file in a tar archive is also referred to as a “member file.” To save storage space, a tar archive may be compressed using a compressor; the compressed tar archive can be decompressed using an associated decompressor. The compression and decompression may be performed using a variety of algorithms. For example the GNU Zip (GZIP) algorithm may be employed as a compressor for compressing a tar archive and as a decompressor for decompressing the compressed tar archive. A container image may be distributed as a “tar.gz” file, which is a tar archive that has been compressed using the GZIP algorithm.

Open source code for implementing the GZIP algorithm is available on the Internet. One can use the open source code to write an application program that uses the GZIP algorithm as a compressor/decompressor, and have access to internal data of the GZIP algorithm, such as data structures of the states maintained by the GZIP algorithm for compression and decompression. It is to be noted that embodiments of the present invention are not limited to the use of the GZIP algorithm. Other suitable algorithms for compression/decompression may also be used without detracting from the merits of the present invention.

FIG.6shows a logical diagram of a conventional tar archive. In a tar archive, each member file (i.e., a file archived in the tar archive) has a corresponding file entry, which consists of a file header followed by a corresponding file data. The file header describes the member file whose contents are in the immediately following file data. The file header has a standard format that is readily distinguishable from that of the file data, and indicates the length of the following file data. The file entries are written in sequence, one after another. That is, in a tar archive, a file header is immediately followed by its corresponding file data, which is immediately followed by another file header that is immediately followed by its corresponding file data, and so on. A null header at the end of the tar archive serves as an end of file indicator, i.e., indicates the end of the archive.

The distance between file headers varies with the size of the file data between them. In uncompressed tar archives, it is possible to read the first file header, skip the following file data, read the next file header, skip the next following file data, and so on until the file header of a target member file is found. The offset to the file entry of the target member file may be recorded to allow random access of the target member file by seeking directly to the offset without having to read the file entries of preceding member files. Under this approach, if the offset to each file entry is saved, any member file in the tar archive may be randomly accessed.

Compression of a tar archive is done in a streaming and continuous manner, beginning from the start of the tar archive towards the null header. The compressor starts in a known, standard startup state. The first data that the compressor compresses is a first file header, followed immediately by a first file data, followed immediately by a second file header, followed immediately by a second file data, and so on. The compression process makes it very difficult to determine the size of the compressed file data. As a result, it becomes very difficult to determine the distance from one file header to the next following file header in the compressed tar archive.

Decompression of a compressed tar archive is also performed in a streaming and continuous manner, beginning from the start of the compressed tar archive towards the null header. A compressed tar archive can be decompressed while reading through the compressed tar archive. The offset to each file entry in the compressed tar archive may be noted and saved during decompression. However, an offset to a target file entry in the compressed tar archive, by itself, is not enough to locate the target file entry in the decompressed data. This is because a small amount of compressed data could decompress to a large amount of data. A logical offset in the decompressed data is needed to locate and randomly access the file entry of the member file.

As a particular example, if a file is compressed from 1000 bytes to 100 bytes, the resulting compressed data written to disk is 100 bytes. When a first block of 10 bytes of the compressed data is decompressed, the first block of 10 bytes will expand in memory to somewhere around 100 bytes. Similarly, decompressing the following block of 10 bytes expands it in memory to around 100 bytes, which is appended to the prior 100 bytes. This decompression process continues until the full 1000 bytes of the file are re-assembled. In this particular example, an offset between blocks of data in the compressed data is 10 bytes. The logical offset is a pointer to a location in the corresponding 100 bytes in memory.

For any target file entry in a compressed tar archive, an offset to the target file entry in the compressed tar archive and a logical offset that identifies the target file entry in the corresponding decompressed data are not enough to return to the target file entry, in a random access manner, primarily because of how compressors/decompressors operate.

Generally, a compressor, such as that used by the GZIP algorithm, includes a sliding window of last seen data and an encoding table (e.g., Huffman encoding). The sliding window is moved through a file being compressed, in streaming fashion from the beginning of the file, and the last seen data of the file in the sliding window is encoded in accordance with the encoding table to compress the last seen data. The compressor keeps and updates a state as the compressor reads the file. The current state of the compressor is made of elements of what has been seen in the file up to the present point in the file. If this state is not saved, reconstruction of the file would require reading the file backwards (i.e., towards the start of the file), which becomes recursive until the start of the file is reached. The state of the compressor at the start of the file is a known constant set by the algorithm type (e.g., GZIP/BZIP2/etc.). In terms of complexity, to know the compressor state at byte N, byte N−1 needs to be read; to know the compressor state at byte N−1, byte N−2 needs to be read; and so on.

The decompressor works essentially the way as the compressor except the decompressor uses the encoding table in reverse, i.e., to decode compressed data back to its decompressed form. More particularly, the sliding window is moved through the compressed file, in streaming and continuous fashion from the beginning of the compressed file, with the last seen data of the compressed file in the sliding window being decompressed using the encoding table. The state of the decompressor at a particular point in the compressed file indicates portions of the compressed file that has already been decompressed up to that particular point. The state of the decompressor at that particular point, in conjunction with an offset to that particular point and a logical offset to the corresponding decompressed data, may be used to seek to a target file entry (and thus target member file) in the compressed file.

FIG.7shows a logical diagram that illustrates random access of a member file in a compressed tar archive in accordance with an embodiment of the present invention. By random access, it is meant that a target member file is accessed without having to decompress preceding member files in the compressed tar archive.

In the example ofFIG.7, a compressed tar archive600includes compressed data followed by a plurality of locators611(i.e.,611-1,611-2, . . . ,611-n) and a locator header610. The compressed data contains compressed file entries of member files. The locators611and the locator header610are appended to the end of the compressed data, so that they will not be processed by a program that is unaware of their format. In one embodiment, the locators611and the locator header610are not compressed.

Each file entry in the compressed tar archive600has a corresponding locator611. The locator header610lists the locators611and indicates the file entries that correspond to the locators611. This allows an application program, hereinafter “random access program”, to randomly access a member file in the compressed tar archive600by reading the locator header610to identify the locator611that corresponds to the file entry of the member file, and seek directly to the file entry using information from the locator611.

In one embodiment, a locator611includes a tar offset601, a logical offset602, and a decompressor state603. For any file entry in the compressed tar archive600, the tar offset601indicates a location in the compressed tar archive from which decompression is initiated to access the file entry. That is, the target file entry is downstream (i.e., going towards the end of the compressed tar archive600) of the tar offset601, at some point in the portion604of the compressed data.

The logical offset602is the location (e.g., start) of the target file entry in the decompressed data resulting from the decompression of compressed data that begins at the tar offset601. More particularly, in the example ofFIG.7, the sliding window of the decompressor contains the portion604, which is decoded using the Huffman coding to yield the decompressed data in memory. The logical offset602points to the beginning of the target file entry in the decompressed data that result from decompression of the portion604immediately following the tar offset601.

The decompressor state603is the state of the decompressor at the tar offset601. In other words, the decompressor state603is the state of the decompressor at a point just prior to decompressing compressed data that is downstream of the tar offset601, which in the example ofFIG.7is just prior to the portion604of the compressed data. The decompressor state603includes information on portions of the compressed data that have already been through the sliding window and decompressed using the Huffman encoding table that was used in the compression.

To seek (i.e., move without reading preceding data) directly to the target file entry, the random access program identifies the locator611of the target file entry from the locator header610. The random access program then seeks directly to the tar offset601indicated in the locator611. The random access program thereafter restores the decompressor to the decompressor state603indicated in the locator611. The decompressor then decompresses, from the tar offset601, the compressed data downstream of the tar offset601(i.e., the portion604), resulting in corresponding decompressed data in main memory. The random access program then accesses the target file entry at the logical offset602(indicated in the locator611) in the decompressed data in the main memory.

The locators611and the locator header610may be generated during compression of a tar archive or decompression of the compressed tar archive. This is because the compressor and decompressor states are the same data structures; data is added to the data structures during compression, and data is read out of the data structures during decompression. That is, the compressor state and the decompressor state are the same at the tar offset601. More particularly, in the case of GZIP, the Huffman table is built during compression, so the data goes from the original file into the Huffman table. During decompression, data is read out from the Huffman table. The Huffman table for compression and decompression need to be the same to end up with the same file.

The tar offset, compressor state at the tar offset, and logical offset in the uncompressed (decompressed) data may be noted for each member file during compression of a tar archive. Similarly, the tar offset, decompressor state at the tar offset, and logical offset in the decompressed data may be noted for each member file during decompression of a compressed tar archive. The tar offset, compressor/decompressor state, and logical offset of each member file may be indicated in a locator, with the locators of the member files being indicated in a locator header. The locators and the locator header may then be appended to the compressed tar archive. This process may be performed for all tar archives and compressed tar archives in a customer account of a cloud service.

FIG.8shows a flow a diagram of a method of random access of a member file in a compressed tar archive in accordance with an embodiment of the present invention. The method ofFIG.8is explained with reference toFIG.7. As can be appreciated, the method ofFIG.8may also be performed using other components or steps without detracting from the merits of the present invention.

In step801, a locator (e.g., locator611) is provided for each member file of a compressed tar archive (e.g., compressed tar archive600). Each locator indicates a tar offset in the compressed tar archive, a logical offset to a file entry of the member file in the corresponding decompressed data, and a decompressor state at the tar offset. In step802, the locators are appended at the end of the compressed tar archive.

In step803, to randomly access a member file in the compressed tar archive, a random access program identifies the locator for the file entry of the member file. In one embodiment, a locator header that identifies the locators of the member files are appended at the end of the locators in the compressed tar archive.

In step804, the random access program seeks directly to the tar offset in the compressed tar archive. In step805, the random access program restores the state of the decompressor to the decompressor state indicated in the locator. In step806, with the decompressor at the decompressor state, the random access program uses the decompressor to decompress the compressed data following the tar offset. Decompressing the compressed data results in corresponding decompressed data in memory. In step807, the random access program moves to the logical offset in the decompressed data in memory. From the logical offset, the random access program accesses the file entry of the member file.

Random access of compressed tar archives as described in the embodiments is especially advantageous with regards to container images. The embodiments allow random access of layers of a container image that is in compressed tar format. More particularly, the embodiments allow individual layers of the container image to be randomly accessed so that the layers can be scanned in parallel. Without the embodiments, the layers of the container image will have to be accessed conventionally in streaming fashion, and will have to be scanned serially one after another.