Verification of core file debugging resources

Systems and methods for implementing a verification of core file debugging resources are disclosed. A plurality of mappings are created from a core file comprising a plurality of filenames and a plurality of target build identifiers (IDs). The core file corresponds to a computer program crash event and each one of the plurality of mappings map one of the plurality of filenames to a corresponding one of the plurality of target build IDs. Responsive to creating the plurality of mappings, a first file is located that corresponds to a first filename included in a first mapping from the plurality of mappings. The first comprises a first target build ID. A processing device utilizes the first file to analyze the computer program crash event in response to determining that the first file build ID matches the first target build ID.

TECHNICAL FIELD

Aspects of the present disclosure relate to program analysis, and more particularly, to utilizing correct versions of files to analyze a computer program crash event.

BACKGROUND

Computer programs typically access other files when executing, such as accessing shared library files. When a computer program crashes, an operating system generates a core file that captures the state of the computer program at the time of the crash. Because computer programs bring in components of the shared files into their memory space when executing, the core file includes information that contain the components of the shared files, which includes build IDs of the shared files. Build IDs are specific identifiers for each file (e.g., shared library file, executable program, etc.), which are assigned at compilation time by the compilation pipeline.

A debugger is a computer program used by programmers to analyze and debug a computer program. When a computer program crashes and the operating system generates the core file, the debugger uses the core file to create a snapshot (e.g., debug environment) that includes the computer program and corresponding shared files.

DETAILED DESCRIPTION

For a debugger to accurately analyze a core file (e.g., a core dump), the debugger must utilize the correct versions of the shared libraries and computer program to generate an accurate snapshot of the program at the time of the crash. Prior debugger approaches may use incorrect versions of files and wait until an error is generated to inform the user of an issue.

The present disclosure addresses the above-noted and other deficiencies by using a processing device to create a plurality of mappings from a core file comprising a plurality of filenames and a plurality of target build identifiers (IDs). The the core file corresponds to a computer program crash event and each one of the plurality of mappings map one of the plurality of filenames to a corresponding one of the plurality of target build IDs. Responsive to creating the plurality of mappings, the processing device may locate a first file corresponding to a first filename included in a first mapping from the plurality of mappings, wherein the first mapping comprises a first target build ID. The processing device may utilize the first file to analyze the computer program crash event in response to determining that the first file build ID matches the first target build ID.

In some embodiments, the processing device may build a debug environment responsive to creating the plurality of mappings. The processing device may load the first file into the debug environment responsive to determining that the first file build ID matches the first target build ID. The processing device may debug the computer program crash event using the debug environment.

In some embodiments, the plurality of mappings is a plurality of first mappings and, prior to building the debug environment, the processing device may locate a plurality of files corresponding to the plurality of target build IDs, wherein each file in the plurality of files includes one of a plurality of sonames. The processing device may create a plurality of second mappings, wherein each one of the plurality of second mappings map one of the plurality of sonames to a corresponding one of the plurality of target build IDs.

In some embodiments, during the building of the debug environment, the processing device may locate a second file, from the plurality of files, that corresponds to a first soname in the plurality of sonames. The second file includes a second file build ID. The processing device may select one of the plurality of second mappings that correspond to the first soname, wherein the selected mapping includes a second target build ID. The processing device may load the second file into the debug environment in response to determining that the second file build ID matches the second target build ID.

In some embodiments, the first file is located on a first storage area, and the processing device may invoke a query to locate a second file on a second storage area responsive to the first file build ID not matching the first target build ID. The processing device may, responsive to receiving a query response that locates the second file on the second storage area, retrieve the second file from the second storage area, wherein the second file includes a second file build ID. The processing device may load the second file into the debug environment in response to determining that the second file build ID matches the first target build ID.

In some embodiments, the processing device may, responsive to the second file build ID not matching the first target build ID, transmit a notification message to a user, prior to debugging the computer program crash event, that indicates a correct version corresponding to the target build ID is unavailable.

In some embodiments, the processing device may identify a first section in the core file, wherein the first section comprises a start address and a second filename from the plurality of filenames. The processing device may locate a second section in the core file that begins at the start address, wherein the second section comprises a second target build ID. The processing device may create one of the plurality of mappings that include the second filename and the second target build ID.

FIG.1is a block diagram that illustrates an example system100. As illustrated inFIG.1, system100includes a computing device110, and a plurality of computing devices150. The computing devices110and150may be coupled to each other (e.g., may be operatively coupled, communicatively coupled, may communicate data/messages with each other) via network140. Network140may be a public network (e.g., the internet), a private network (e.g., a local area network (LAN) or wide area network (WAN)), or a combination thereof. In some embodiments, network140may include a wired or a wireless infrastructure, which may be provided by one or more wireless communications systems, such as a WiFi™ hotspot connected with the network140and/or a wireless carrier system that can be implemented using various data processing equipment, communication towers (e.g. cell towers), etc. In some embodiments, the network140may be an L3 network. The network140may carry communications (e.g., data, message, packets, frames, etc.) between computing device110and computing devices150. Each computing device110and150may include hardware such as processing device115(e.g., processors, central processing units (CPUs)), memory120(e.g., random access memory120(e.g., RAM)), storage devices (e.g., hard-disk drive (HDD), solid-state drive (SSD), etc.), and other hardware devices (e.g., sound card, video card, etc.). In some embodiments, memory120may be a persistent storage that is capable of storing data. A persistent storage may be a local storage unit or a remote storage unit. Persistent storage may be a magnetic storage unit, optical storage unit, solid state storage unit, electronic storage units (main memory), or similar storage unit. Persistent storage may also be a monolithic/single device or a distributed set of devices. Memory120may be configured for long-term storage of data and may retain data between power on/off cycles of the computing device110. Each computing device may comprise any suitable type of computing device or machine that has a programmable processor including, for example, server computers, desktop computers, laptop computers, tablet computers, smartphones, set-top boxes, etc. In some examples, each of the computing devices110and150may comprise a single machine or may include multiple interconnected machines (e.g., multiple servers configured in a cluster). The computing devices110and150may be implemented by a common entity/organization or may be implemented by different entities/organizations. For example, computing device110may be operated by a first company/corporation and one or more computing devices150may be operated by a second company/corporation. Each of computing device110and computing devices150may execute or include an operating system (OS) such as host OS125and host OS155respectively, as discussed in more detail below. The host OS of a computing device110and150may manage the execution of other components (e.g., software, applications, etc.) and/or may manage access to the hardware (e.g., processors, memory, storage devices etc.) of the computing device. In some embodiments, computing device110may implement a control plane (e.g., as part of a container orchestration engine) while computing devices150may each implement a compute node (e.g., as part of the container orchestration engine).

In some embodiments, a container orchestration engine130(referred to herein as container host130), such as the Redhat™ OpenShift™ module, may execute on the host OS125of computing device110and the host OS155of computing device150, as discussed in further detail herein. The container host module130may be a platform for developing and running containerized applications and may allow applications and the data centers that support them to expand from just a few machines and applications to thousands of machines that serve millions of clients. Container host130may provide an image-based deployment module for creating containers and may store one or more image files for creating container instances. Many application instances can be running in containers on a single host without visibility into each other's processes, files, network, and so on. Each container may provide a single function (often called a “micro-service”) or component of an application, such as a web server or a database, though containers can be used for arbitrary workloads. In this way, the container host130provides a function-based architecture of smaller, decoupled units that work together.

When developers develop computer program applications, the developers may be required to debug the computer program applications. To properly configure a debug environment to debug the computer program application, the debug environment must utilize correct versions of files. In one embodiment, computing device110employs a debugger to perform the steps discussed herein to verify that the debug environment includes correct versions of files.

Container host130may include a storage driver (not shown), such as OverlayFS, to manage the contents of an image file including the read only and writable layers of the image file. The storage driver may be a type of union file system which allows a developer to overlay one file system on top of another. Changes may be recorded in the upper file system, while the lower file system (base image) remains unmodified. In this way, multiple containers may share a file-system image where the base image is read-only media.

An image file may be stored by the container host130or a registry server. In some embodiments, the image file may include one or more base layers. An image file may be shared by multiple containers. When the container host130creates a new container, it may add a new writable (e.g., in-memory) layer on top of the underlying base layers. However, the underlying image file remains unchanged. Base layers may define the runtime environment as well as the packages and utilities necessary for a containerized application to run. Thus, the base layers of an image file may each comprise static snapshots of the container's configuration and may be read-only layers that are never modified. Any changes (e.g., data to be written by the application running on the container) may be implemented in subsequent (upper) layers such as in-memory layer. Changes made in the in-memory layer may be saved by creating a new layered image.

While the container image is the basic unit containers may be deployed from, the basic units that the container host130may work with are called pods. A pod may refer to one or more containers deployed together on a single host, and the smallest compute unit that can be defined, deployed, and managed. Each pod is allocated its own internal IP address, and therefore may own its entire port space. Containers within pods may share their local storage and networking. In some embodiments, pods have a lifecycle in which they are defined, they are assigned to run on a node, and they run until their container(s) exit or they are removed based on their policy and exit code. Although a pod may contain more than one container, the pod is the single unit that a user may deploy, scale, and manage. The control plane135of the container host130may include replication controllers (not shown) that indicate how many pod replicas are required to run at a time and may be used to automatically scale an application to adapt to its current demand.

By their nature, containerized applications are separated from the operating systems where they run and, by extension, their users. The control plane135may expose applications to internal and external networks by defining network policies that control communication with containerized applications (e.g., incoming HTTP or HTTPS requests for services inside the cluster165).

A typical deployment of the container host130may include a control plane135and a cluster of compute nodes165, including compute nodes165A and165B (also referred to as compute machines). The control plane135may include REST APIs which expose objects as well as controllers which read those APIs, apply changes to objects, and report status or write back to objects. The control plane135manages workloads on the compute nodes165and also executes services that are required to control the compute nodes165. For example, the control plane135may run an API server that validates and configures the data for pods, services, and replication controllers as well as provides a focal point for the cluster165's shared state. The control plane135may also manage the logical aspects of networking and virtual networks. The control plane135may further provide a clustered key-value store (not shown) that stores the cluster165's shared state. The control plane135may also monitor the clustered key-value store for changes to objects such as replication, namespace, and service account controller objects, and then enforce the specified state.

The cluster of compute nodes165are where the actual workloads requested by users run and are managed. The compute nodes165advertise their capacity and a scheduler (not shown), which is part of the control plane135, determines which compute nodes165containers and pods will be started on. Each compute node165includes functionality to accept and fulfill requests for running and stopping container workloads, and a service proxy, which manages communication for pods across compute nodes165. A compute node165may be implemented as a virtual server, logical container, or GPU, for example.

FIG.2is a block diagram that illustrates an example system for utilizing correct versions of files to analyze a computer program crash event. When a computer program crashes, the operating system generates core file200in memory120. Processing device115executes a debugger as discussed herein to read core file200and generate mappings215, which maps filenames to target build IDs based on information from core file200. Then, the debugger locates a file corresponding to one of the filenames (file230), and checks whether file230's file build ID matches the target build ID in the corresponding mapping in mappings215. If the build IDs match, the debugger determines that file230is the correct version of the file and, in one embodiment, loads file230into a debug environment for further analysis.

FIGS.3A and3Bare block diagrams illustrating examples of a debugger generating mappings for that are used to verify that correct versions of files are utilized to analyze a computer program crash event.

Referring toFIG.3A, debugger300uses core file200to build a filename to target build ID (FTBID) map314. Core file200includes line302, which shows that the executable /bin/exec (program that caused the core dump) begins at start address 1 in core file200. Start address 1 begins at segment308and includes target build ID 0x123abc, which is the build ID of the executable file. As such, debugger300adds mapping316to FTBID map314that maps /bin/exec to build ID 0x123abc. Debugger300then proceeds through a series of steps discussed herein to locate the correct version of the executable file in local store325or remote store328by matching the target build ID to a file build ID included in a located file. In one embodiment, when debugger does not locate the correct version of a file in local store325, debugger300uses a query server to automatically search for the file in remote store328, such as debuginfod. Debuginfod is a file server that serves debugging resources to debugger-like tools. The server periodically scans directory trees and RPM archives to extract the build IDs of any executable and debuginfo files found (seeFIG.5and corresponding text for further details).

Similarly, lines304and306show that start addresses 2 and 3 in core file200correspond to start address segments of libabc.so.123 and libxyz.so.789, respectively. Debugger300locates segment310at start address 2, identifies target build ID 0x456def, and adds mapping318to FTBID map314that maps libabc.so.123 to target build ID 0x456def. Debugger300then proceeds through a series of steps discussed herein to locate the correct version of the shared library file in local store325or remote store328by matching the target build ID to a file build ID included in a located file. In one embodiment, as discussed above, when debugger does not locate the correct version of a file in local store325, debugger300uses a query server to search for the file in remote store328, such as debuginfod.

Debugger300also locates segment312at start address 3, identifies target build ID 0x987fed, and adds mapping320to FTBID map314that maps libxyz.so.789 to target build ID 0x987fed. In turn, in one embodiment, as debugger300builds a debug environment, debugger uses FTBID314to verify that files loaded into the debug environment are the same versions as the files that were in use at the time of the executable program crash event that created core file200.

Referring toFIG.3B, in one embodiment, debugger300also builds a soname-to-target build ID (STBID) map360. A soname (shared object name) is a field of data in a shared object file. The soname is a string, which is used as a “logical name” describing the functionality of the object. Typically, the soname name is equal to the filename of the library, or to a prefix thereof, and the soname is often used to provide version backwards-compatibility information.

During program execution, files may be called by different aliases, such as being called by its filename in some areas and being called by its soname in other areas. As such, as debugger300constructs a snapshot, debugger300may need to download a file based on its soname and therefore needs to confirm that it is the correct version. To provide this feature, debugger300generates STBID map360to verify the files based on their soname.

Debugger300, in one embodiment, uses temporary storage330to store files downloaded or found locally. File335is the executable file with build ID 0x123abc. File340is shared library file libabc.so.123 (line342) with file build ID 0x456def (line344). Line346shows that the soname of the shared library is libabc.so. As such, debugger300adds mapping364to STBID map360that maps soname libabc.so to file build ID 0x456def. Likewise, file350is shared library file libxyz.so.789 (line352) with file build ID 0x987fed (line354). Line356shows that the soname of the shared library is libxyz.so.7. As such, debugger300adds mapping366to STBID map360that maps soname libxyz.so.7 to file build ID 0x987fed. In turn, when debugger300access or downloads files while building a snapshot in the debug environment, debugger300verifies each file using the STBID map360and/or FTBID map314.

FIG.4is a flow diagram of a method400for utilizing correct versions of files to analyze a computer program crash event. Method400may be performed by processing logic that may comprise hardware (e.g., circuitry, dedicated logic, programmable logic, a processor, a processing device, a central processing unit (CPU), a system-on-chip (SoC), etc.), software (e.g., instructions running/executing on a processing device), firmware (e.g., microcode), or a combination thereof. in some embodiments, the method400may be performed by a computing device (e.g., computing device110or150illustrated inFIGS.1and2).

At block410, computing device110may create a plurality of mappings from a core file comprising a plurality of filenames and a plurality of target build identifiers (IDs). The core file corresponds to a computer program crash event and each one of the plurality of mappings map one of the plurality of filenames to a corresponding one of the plurality of target build IDs.

At block420, responsive to creating the plurality of mappings, computing device110may locate a first file corresponding to a first filename included in a first mapping from the plurality of mappings. The first mapping comprises a first target build ID. At block430, computing device110may utilize the first file to analyze the computer program crash event in response to determining that the first file build ID matches the first target build ID.

In one embodiment, computing device110may perform additional steps to automatically locate a correct version of a file as well as verifying that files referenced by soname are the correct version (seeFIG.5and corresponding text for further details).

FIG.5is a flow diagram of a method500showing another embodiment to utilize correct versions of files to analyze a computer program crash event. Method500may be performed by processing logic that may comprise hardware (e.g., circuitry, dedicated logic, programmable logic, a processor, a processing device, a central processing unit (CPU), a system-on-chip (SoC), etc.), software (e.g., instructions running/executing on a processing device), firmware (e.g., microcode), or a combination thereof. in some embodiments, the method500may be performed by a computing device (e.g., computing device110illustrated inFIGS.1and2).

At block510, computing device110may read segment start addresses and corresponding file names in core file200(refer toFIG.3Aand corresponding text for further details). Computing device110may use the start addresses to locate the segments also included in core file200and identify their corresponding target build-IDs. At block512, computing device110may construct a filename to target build-ID (FTBID) mapping (FTBID map314). In one embodiment, computing device100performs steps in blocks512through540or550on an individual basis for each mapping in FTBID314.

Computing device110, for each mapping, may then check for a local file based on the filename (block512). If computing device110locates the file locally, decision515branches to the ‘yes’ branch whereupon, at block520, computing device110may read the file build-ID from the located file and compare the file build ID with the target build-ID in FTBID map314. If the file build ID matches the target build ID, then decision525branches to the ‘yes’ branch whereupon, at block540, computing device110may verify that the file is the same version that was in use at the time of core file creation.

On the other hand, if the file build ID in the local file does not match the target build ID, then decision525branches to the ‘no’ branch to block530. Similarly, referring back to decision515, if computing device110does not locate the file locally, then decision515branches to the ‘no’ branch. At block530, computing device110may use the target build-ID to query debuginfod servers for the correct version of the file being searched. In one embodiment, computing device110(e.g., debugger300) automatically performs the step shown in block530.

When the file is located on a server, computing device110may compare the located file build ID to the target build ID and determine as to whether they match (decision535). If the two build IDs match, then decision535branches to the ‘yes’ branch whereupon, at block540, computing device110may verify that the file is the same version that was in use at the time of core file creation. On the other hand, if the build IDs do not match, then decision535branches to the ‘no’ branch whereupon, at block545computing device110may inform the user that an incorrect version of the file may be used to build the snapshot in the debug environment.

At block550, computing device110may download the files and search the files for their corresponding sonames. Computing device110may then construct a soname-to-build ID (STBID) map360, which maps the sonames to their corresponding target build IDs. At block555, in one embodiment, during the debug environment build stage, computing device110may open files via filenames and sonames and compare the file build-IDs in the opened files with target build IDs in FTBID314or STBID360.

Computing device110may determine, on an individual file-by-file basis, as to whether the file build ID matches the corresponding target build ID (decision560). If the build IDs match, then decision560branches to the ‘yes’ branch whereupon, at block565, computing device110may verify that the same version of the file was in use at the time of the core file creation and, in one embodiment, load the file into the debug environment in response to determining that the build IDs match. On the other hand, if the build IDs do not match, then decision560branches to the ‘no’ branch whereupon, at block570, computing device110may inform the user that the file does not match the version referenced in the core file.

FIG.6illustrates a diagrammatic representation of a machine in the example form of a computer system600within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein for debug analysis.

The exemplary computer system600includes a processing device602, a main memory604(e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), a static memory606(e.g., flash memory, static random access memory (SRAM), etc.), and a data storage device618which communicate with each other via a bus630. Any of the signals provided over various buses described herein may be time multiplexed with other signals and provided over one or more common buses. Additionally, the interconnection between circuit components or blocks may be shown as buses or as single signal lines. Each of the buses may alternatively be one or more single signal lines and each of the single signal lines may alternatively be buses.

Computing device600may further include a network interface device608which may communicate with a network620. The computing device600also may include a video display unit610(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device612(e.g., a keyboard), a cursor control device614(e.g., a mouse) and an acoustic signal generation device616(e.g., a speaker). In some embodiments, video display unit610, alphanumeric input device612, and cursor control device614may be combined into a single component or device (e.g., an LCD touch screen).

The data storage device618may include a machine-readable storage medium628, on which is stored one or more sets of debugger instructions625(e.g., software) embodying any one or more of the methodologies of functions described herein. The debugger instructions625may also reside, completely or at least partially, within the main memory604or within the processing device602during execution thereof by the computer system600; the main memory604and the processing device602also constituting machine-readable storage media. The debugger instructions625may further be transmitted or received over a network620via the network interface device608.