System and method for virtualization of processor resources

A system and method for virtualization of processor resources is presented. A thread is created on a processor and the processor's local memory is mapped into an effective address space. In doing so, the processor's local memory is accessible by other processors, regardless of whether the processor is running. Additional threads create additional local memory mappings into the effective address space. The effective address space corresponds to either a physical local memory or a “soft” copy area. When the processor is running, a different processor may access data that is located in the first processor's local memory from the processor's local storage area. When the processor is not running, a softcopy of the processor's local memory is stored in a memory location (i.e. locked cache memory, pinned system memory, virtual memory, etc.) for other processors to continue accessing.

BACKGROUND OF THE INVENTION

1. Technical Field

The present invention relates in general to a system and method for virtualization of processor resources. More particularly, the present invention relates to a system and method for virtualizing processor memory such that one or more threads may access the processor memory, regardless of a processor's state.

2. Description of the Related Art

Computer systems are becoming more and more complex. The computer industry typically doubles the performance of a computer system every 18 months (e.g. personal computer, PDA, gaming console). In order for the computer industry to accomplish this task, the semiconductor industry produces integrated circuits that double in performance every 18 months. A computer system uses integrated circuits for particular functions based upon the integrated circuits' architecture. Two fundamental architectures are 1) microprocessor-based and 2) digital signal processor-based.

An integrated circuit with a microprocessor-based architecture is typically used to handle control operations whereas an integrated circuit with a digital signal processor-based architecture is typically designed to handle signal-processing manipulations (i.e. mathematical operations). As technology evolves, the computer industry and the semiconductor industry realize the importance of using both architectures, or processor types, in a computer system design.

Many computer systems use a multi-processor architecture in order to provide a substantial amount of processing power while attempting to support a wide range of software applications. A challenge found, however, is that each of these processors includes dedicated internal memory that is not accessible by other processors, such as a local storage area and register files. Another challenge found is that if a particular processor is unavailable, other processors may not access data from the unavailable processor.

Furthermore, a processor may run multiple threads that access the same memory space. A challenge found is that when a particular thread is accessing a particular memory space, the other threads must wait until the particular thread is complete before the other threads are able to access the memory space.

What is needed, therefore, is a system and method to virtualize processor memory resources such that the resources are concurrently accessible by one or more threads, regardless of a processor's state.

SUMMARY

It has been discovered that the aforementioned challenges are resolved by mapping a processor's local memory into an effective address space for each thread that an application creates, and using a page table entry for each thread in which an address translator accesses in order to determine whether to retrieve a “soft” data copy for the thread to process. For each thread that an application creates, an operating system stores a page table entry in a page table that includes an effective address and a real address. The real address corresponds to either a local storage location or a soft copy area. When an address translator receives a request from the thread to retrieve data, the address translator looks-up the page table entry and retrieves the data from the real address location.

A first processor includes an operating system whereby the operating system may be a real-time operating system (e.g. a gaming operating system) or a virtual operating system (e.g. a web-browsing operating system). The operating system invokes a thread for a particular application. The thread is responsible for performing particular tasks, such as terrain rendering for a gaming application. When the operating system invokes the thread, the operating system sends a page table entry to a page table. The page table entry includes an effective address and a real address that corresponds a location of data that the thread utilizes. The real address may correspond to a physical local memory location (e.g. local store), or the real address may correspond to a soft copy area. For example, the soft copy area may be in cache, pinned system memory, or disk.

The thread sends a request that includes an effective address to an address translator. The effective address indicates the effective memory location of data that the thread wishes to access. The address translator receives the request, and looks-up a page table entry in the page table whereby the address translator identifies the real address that corresponds to the effective address. If the real address corresponds to a processor's local storage area, the address translator retrieves the data from the processor's local storage area and passes the data to the thread. If the real address corresponds to a soft copy area, the address translator retrieves a copy of the data from the soft copy area and passes the data copy to the thread.

The operating system manages the thread's data by saving and restoring the data to and from the soft copy area based upon particular policies and the state of a corresponding processor. In particular instances, such as when a processor's memory is being utilized by multiple threads, the operating system may create soft copies of the local store for each thread whereby the threads utilize different soft copies. The operating system creates a new page table entry in the page table each time the operating system performs a save or restore operation.

DETAILED DESCRIPTION

FIG. 1is a diagram showing a main processor accessing a secondary processor's local memory. Processor A100is a main processor that includes operating system115. Operating system115may be a real-time operating system, such as a gaming operating system, or operating system115may be a virtual operating system, such as a web-browsing operating system.

Operating system115receives a thread request from application105, whereby operating system115invokes thread1110. Thread1110is responsible for performing particular tasks, such as terrain rendering in a gaming application. When operating system115invokes thread1110, operating system115sends page table entry150to page table140. Page table entry150includes an effective address and a real address that corresponds to data location that thread1110utilizes. The real address may correspond to a physical local memory location (e.g. local store), or the real address may correspond to a soft copy area. For example, a soft copy of the data may be stored in cache, pinned system memory, or disk (seeFIG. 4and corresponding text for further details regarding physical local memory and soft copy areas.

Thread1110sends request120to address translator130requesting access to a particular portion of memory. Request120includes an effective address that indicates the effective memory location of data that thread1110wishes to access. Address translator130receives request120, and looks-up page table entry150in page table140. Address translator130identifies that the real address location is a located on processor B160's local store B170. Address translator130retrieves data180from local store B170, and sends data180to thread1110. In one embodiment, address translator130incrementally sends data160to thread1110, such as on a page-by-page basis. In one embodiment, a thread may reside on processor B160whereby the thread retrieves data from processor A100's local store (seeFIG. 2and corresponding text for further details regarding threads residing on processor B160).

FIG. 2is a diagram showing a thread that is included on a secondary processor accessing a main processor's local memory (e.g. local store).FIG. 2is similar toFIG. 1with the exception that processor B160includes a thread that requests data from processor A100's local memory, such as local store A260.

Processor A100includes operating system115, whereby, when operating system115invokes thread200, operating system115sends page table entry240to page table140. Page table entry240includes an effective address and a real address that corresponds to a data location that thread200utilizes.

Thread2200sends request220to address translator130. Address translator130looks-up page table entry240, and determines that the data is located in local store A260. Address translator130retrieves data280from local store260, and provides data280to thread2200. Address translator130, page table140processor B160, processor A100, and operating system115are the same as that shown inFIG. 1.

FIG. 3is a diagram of a thread accessing a soft copy of a processor's local memory that is located in a soft copy area. During operation, an operating system saves and restores physical memory to soft copy areas based upon policy information. For example, an operating system may save and restore data at particular time intervals (seeFIG. 6and corresponding text for further details). When data is swapped from a local store to a soft copy area, the operating system changes a page table entry in page table140such that address translator130retrieves the data from the correct soft copy area.

Processor A100includes thread3300, which sends request310to address translator130. In turn, address translator130identifies a page table entry in page table140, and determines that the data is located in a soft copy area, such as soft copy areas320. Soft copy areas320include cache330, kernel340, and disk store350. Cache330may be an L1 or L2 cache, kernel340may be pinned system memory, and disk store350may be an external hard drive.

Address translator130retrieves data360from soft copy areas320, and provides data360to thread3300for further processing. Address translator130, page table140, and processor A100are the same as that shown inFIG. 1.

FIG. 4is a high-level flow chart showing steps taken in an operating system managing processor memory based upon one or more threads. Processing commences at400, whereupon the operating system receives a thread request from application105(step410). For example, application105may wish to invoke a thread that performs complex computations, such as terrain rendering. The operating system initializes a thread at step420whereby the operating system identifies resources to support the thread, such as processor B160. Application105and processor B160are the same as that shown inFIG. 1.

A step430, processing identifies processor B160's task state. For example, if a thread requires data from a particular processor's local memory, the operating system identifies whether the same processor is the processor that is assigned to support the thread.

A determination is made as to whether the thread should access a processor's physical local memory (local store) to retrieve data or whether the thread should access a soft copy of the data in a soft copy area (decision440). For example, if the thread executes on the same processor that includes the data, the thread may access the processor's local store. In another example, a thread may execute on a different processor than the location of the data, in which case the operating system copies the data to a soft copy area such that the thread may access the soft copy, regardless of whether the corresponding processor is inactive.

If the thread should use the physical local memory, decision440branches to “Physical” branch442whereupon the operating system creates a page table entry in page table140(step450), which includes a real address that is used by an address translator to access data from a processor's local memory, or local store. Page table140is the same as that shown inFIG. 1.

On the other hand, if the thread should use a soft copy of the local store, decision440branches to “Soft Copy” branch448whereupon processing copies the data in the local store to a soft copy area (pre-defined process block460, seeFIG. 5and corresponding text for further details). The operating system creates a page table entry in page table140(step470), which includes a real address that is used by an address translator to access a soft copy of data in a soft copy area. Therefore, when the thread requests data, the address translator accesses a soft copy area in order to provide the data to the thread, regardless of the corresponding processor's state (seeFIG. 3and corresponding text for further details regarding soft copy areas).

The operating system performs memory management by restoring and saving the data into and out of the local store based upon particular policy management. During the memory management process, the operating system may restore data from a soft copy area to a processor's local store, or the operating system may save the data in the local store to a soft copy area (pre-defined process block480, seeFIG. 6and corresponding text for further details).

A determination is made as to whether the operating system should continue processing thread requests and managing memory (decision490). If the operating system should continue, decision490branches to “Yes” branch492which loops back to process more thread requests. This looping continues until the operating system should stop, at which point decision490branches to “No” branch498whereupon the operating system ends at499.

FIG. 5is a flowchart showing steps taken in copying data that is included in a secondary processor's local memory (i.e. local store) into a soft copy area. Processing commences at500, whereupon processing retrieves data from local store B170at step510. Local store B170is the same as that shown inFIG. 1.

A determination is made as to whether to save the data in cache, such as locked L1 cache or locked L2 cache (decision520). If the operating system should save the data in cache, decision520branches to “Yes” branch522whereupon processing stores a copy of the data in cache330at step530, and returns at540. Cache330is the same as that shown inFIG. 3. On the other hand, if the operating system should not store the data in cache, decision520branches to “No” branch528bypassing data storing steps in cache.

A determination is made as to whether to save the data in pinned system memory, such as memory that is reserved within a kernel (decision550). If the operating system should save the data in pinned system memory, decision550branches to “Yes” branch552whereupon processing stores a copy of the data in kernel340at step560, and returns at570. Kernel340is the same as that shown inFIG. 3. On the other hand, if the operating system should not store the data in pinned system memory, decision550branches to “No” branch558bypassing data storing steps in pinned system memory.

A determination is made as to whether to save the data on disk, such as an external hard drive (decision580). If the operating system should save the data on disk, decision580branches to “Yes” branch582whereupon processing stores the data in disk store350at step590, and returns at595. Disk store350is the same as that shown inFIG. 3. On the other hand, if the operating system should not store the data on disk, decision580branches to “No” branch588bypassing data storing steps on disk, and returns at595.

FIG. 6is a flowchart showing steps taken in restoring and saving data to and from a processor's local store (i.e. local memory). An operating system manages a thread's corresponding data by saving and restoring the data based upon particular policies and the state of the corresponding processors. An operating system attempts to provide a thread with the ability to access data in the physical local memory as opposed to accessing a soft copy of the data in a soft copy area. In particular instances, such as when a processor's memory is being utilized by multiple threads, the operating system creates soft copies of the local store whereby the threads utilize the soft copies.

Processing commences at600, whereupon processing retrieves policies from policies store610at step605. Policies store610may be stored on a nonvolatile storage area, such as a computer hard drive. Processing selects a first task at step615. In one embodiment, processing may select a first thread instead of a first task.

A determination is made as to whether to restore data from a soft copy area to a local store (decision620). For example, a thread may have been on a low priority and, in the meantime, utilized a soft copy of data in a soft copy area. To continue with this example, the operating system may move the thread to a high priority and the operating system restores the local store with the soft copy and, in turn instructs an address translator to retrieve data from the local store to provide to the thread.

If the operating system wishes to restore data from a soft copy area to a local store, decision620branches to “Yes” branch622whereupon processing retrieves the soft copy from soft copy area320at step625, and stores the soft copy in local store B170(step630). Soft copy areas320and local store B170are the same as that shown inFIGS. 3 and 1, respectively. Processing changes a page table entry in page table140at step635that includes a new real address corresponding to the local store location in which an address translator retrieves data to provide to the thread. During steps625,630, and635, the address translator is “locked” such that local memory B170is inaccessible by other threads until the restore operation is complete. On the other hand, if the operating system should not perform a restore operation, decision620branches to “No” branch628bypassing data restoring steps.

A determination is made as to whether to save data from a local store to a soft copy area (decision640). For example, a thread may be placed on low priority and, therefore, the operating system instructs an address translator, through a page table entry, to utilize a soft copy of the data.

If the operating system wishes to save data to a soft copy area, decision640branches to “Yes” branch642whereupon processing retrieves the data from local store B170at step645, and copies the data to soft copy areas320(step650). Soft copy areas320may include cache, pinned system memory, or disk (seeFIGS. 3,5, and corresponding text for further details regarding soft copy areas).

Processing changes a page table entry in page table140at step655that includes a new real address of corresponding to the soft copy area in which an address translator uses to retrieve data for the corresponding thread. During steps645,650, and655, the address translator is “locked” such that local memory B170is inaccessible by other threads until the save operation is complete.

A determination is made as to whether there are more tasks to process (decision660). If there are more tasks to process, decision660branches to “Yes” branch662which loops back to select (step670) and process the next task. This looping continues until there are no more tasks to process, at which point decision660branches to “No” branch668whereupon processing returns at680.

FIG. 7is a flowchart showing steps taken in translating an address using a page table entry, and providing data to a thread based upon the translated address. Processing commences at700, whereupon the address translator receives a request from thread720at step710. Thread720may be a particular thread, such as thread1110, thread2200, or thread3300that are shown inFIGS. 1,2, and3respectively. Thread720includes an effective address that corresponds to the location of data that thread720wishes to access.

At step730, the address translator accesses page table140and translates the effective address to a real address using one of the page table entries. An operating system manages the page table entries such that each page table entry includes a real address that corresponds to either a physical local memory or a soft copy area (seeFIG. 6and corresponding text for further details regarding page table entry management). Page table140is the same as that shown inFIG. 1.

The address translator retrieves data from either soft copy areas320or local memory B170based upon the translated real address (step740). Soft copy area320and local memory B170are the same as that shown inFIGS. 3 and 1, respectively. At step750, processing passes the retrieved data to thread720. In one embodiment, the address translator incrementally passes data to thread720, such as one page of data at a time.

A determination is made as to whether the address translator should continue to process thread requests (decision760). If the address translator should continue to process thread requests, decision760branches to “Yes” branch762which loops back to process more thread requests. This looping continues until there are no more thread requests to process, at which point decision760branches to “No” branch768whereupon address translation ends at770.

FIG. 8is a diagram showing a plurality of processors executing a plurality of threads whose addresses are mapped into a common address space. The present invention allows multiple threads to operate on multiple processors, all the while each thread's respective memory access is managed by an operating system and mapped into an address space, such as address space850.

Processor B1800include thread E805and thread F810. Thread E805's and thread F810's effective address space are effective address space E855and effective address space F860, respectively, both of which are included in address space850. Each thread performs independent operations and, in addition, some threads may be controlled by a first operating system and other threads may be controlled by a second operating system (seeFIG. 9and corresponding text for further details regarding multiple operating systems).

Processor B2815includes three threads, which are thread G820, thread H825, and thread I830. As can be seen inFIG. 8, thread G820's, thread H825's, and thread I830's effective address are effective address G865, effective address H870, and effective address I875, respectively, which are located in address space850.

Lastly, processor B3860includes two threads, which are thread J840and thread K845. Thread J840's and thread K845's effective address space are effective address space J880and effective address space K885, respectively, which are located in address space850.

The threads shown inFIG. 8may be managed such their corresponding processors are “virtual” to an operating system or an application. For example, two operating systems may exist such that one operating system thinks that it is controlling three processors, while the other operating system thinks that it is controlling two processors, all the while there are only three processors in the computer system (seeFIG. 9and corresponding text for further details regarding resource sharing between two operating systems.

FIG. 9is a diagram showing two operating systems sharing processor resources in a heterogeneous processor environment.FIG. 9shows the use of virtualizing processor local memory in order to share resources between separate operating systems. For example, a computer system may run two operating systems and include eight processors. In this example, the first operating system requires six processors and the second operating system requires all eight processors. In this example, processor resources are virtualized and shared between the two operating systems in order to meet the requirements of both operating systems.

Processor A100includes operating system1900and operating system2950. Each operating system may be responsible for particular functions. For example, operating system1900may be a real-time operating system for a gaming application and operating system2950may be a virtual operating system that manages web browsing.

FIG. 9shows processor B1800, processor B2815, and processor B3835executing multiple threads. Operating system1900utilizes threads E805, H825,1830, J840, and K845. As such, operating system900thinks that it has control of all three processors (B1800, B2815, and B3835). Processors B1800, B2815, and B3835are the same as those shown inFIG. 8.

In addition,FIG. 9shows that operating system2950utilizes threads F810and G820. As such, operating system950thinks that it has control of two processors (B1800and B2815). Combined, the operating systems think that there are five processors in the computer system, when in reality there are only three. Threads E805, F810, G820, H825,1830, J840, and K845are the same as those shown inFIG. 8.

Each of the threads may access local memory corresponding to one of the other processors using the invention described herein. For example, thread E805may access processor B3835's local memory by sending a request to an address translator to access processor B3835's local memory. In this example, the address translator identifies a real address corresponding to processor B3835's local memory using a page table entry that is located in a page table. The real address may correspond to a physical local memory, or the real address may correspond to a soft copy area which includes a soft copy of processor B3835's data (seeFIGS. 1 through 7and corresponding text for further details regarding address translation).

FIG. 10is a diagram showing a processor element architecture that includes a plurality of heterogeneous processors. The heterogeneous processors share a common memory and a common bus. Processor element architecture (PEA)1000sends and receives information to/from external devices through input output1070, and distributes the information to control plane1010and data plane1040using processor element bus1060. Control plane1010manages PEA1000and distributes work to data plane1040.

Control plane1010includes processing unit1020which runs operating system (OS)1025. For example, processing unit1020may be a Power PC core that is embedded in PEA1000and OS1025may be a Linux operating system. Processing unit1020manages a common memory map table for PEA1000. The memory map table corresponds to memory locations included in PEA1000, such as L2 memory1030as well as non-private memory included in data plane1040(seeFIG. 11A,11B, and corresponding text for further details regarding memory mapping).

Data plane1040includes Synergistic Processing Complex's (SPC)1045,1050, and1055. Each SPC is used to process data information and each SPC may have different instruction sets. For example, PEA1000may be used in a wireless communications system and each SPC may be responsible for separate processing tasks, such as modulation, chip rate processing, encoding, and network interfacing. In another example, each SPC may have identical instruction sets and may be used in parallel to perform operations benefiting from parallel processes. Each SPC includes a synergistic processing unit (SPU) which is a processing core, such as a digital signal processor, a microcontroller, a microprocessor, or a combination of these cores.

SPC1045,1050, and1055are connected to processor element bus1060which passes information between control plane1010, data plane1040, and input/output1070. Bus1060is an on-chip coherent multi-processor bus that passes information between I/O1070, control plane1010, and data plane1040. Input/output1070includes flexible input-output logic which dynamically assigns interface pins to input output controllers based upon peripheral devices that are connected to PEA1000. For example, PEA1000may be connected to two peripheral devices, such as peripheral A and peripheral B, whereby each peripheral connects to a particular number of input and output pins on PEA1000. In this example, the flexible input-output logic is configured to route PEA1000's external input and output pins that are connected to peripheral A to a first input output controller (i.e. IOC A) and route PEA1000's external input and output pins that are connected to peripheral B to a second input output controller (i.e. IOC B).

FIG. 11Aillustrates an information handling system which is a simplified example of a computer system capable of performing the computing operations described herein. The example inFIG. 11Ashows a plurality of heterogeneous processors using a common memory map in order to share memory between the heterogeneous processors. Device1100includes processing unit1130which executes an operating system for device1100. Processing unit1130is similar to processing unit1020shown inFIG. 10. Processing unit1130uses system memory map1120to allocate memory space throughout device1100. For example, processing unit1130uses system memory map1120to identify and allocate memory areas when processing unit1130receives a memory request. Processing unit1130access L2 memory1125for retrieving application and data information. L2 memory1125is similar to L2 memory1030shown inFIG. 10.

System memory map1120separates memory mapping areas into regions which are regions1135,1145,1150,1155, and1160. Region1135is a mapping region for external system memory which may be controlled by a separate input output device. Region1145is a mapping region for non-private storage locations corresponding to one or more synergistic processing complexes, such as SPC1102. SPC1102is similar to the SPC's shown inFIG. 10, such as SPC A1045. SPC1102includes local memory, such as local store1110, whereby portions of the local memory may be allocated to the overall system memory for other processors to access. For example, 1 MB of local store1110may be allocated to non-private storage whereby it becomes accessible by other heterogeneous processors. In this example, local storage aliases1145manages the 1 MB of nonprivate storage located in local store1110.

Region1150is a mapping region for translation lookaside buffer's (TLB's) and memory flow control (MFC registers. A translation lookaside buffer includes cross-references between virtual address and real addresses of recently referenced pages of memory. The memory flow control provides interface functions between the processor and the bus such as DMA control and synchronization.

Region1155is a mapping region for the operating system and is pinned system memory with bandwidth and latency guarantees. Region1160is a mapping region for input output devices that are external to device1100and are defined by system and input output architectures.

Synergistic processing complex (SPC)1102includes synergistic processing unit (SPU)1105, local store1110, and memory management unit (MMU)1115. Processing unit1130manages SPU1105and processes data in response to processing unit1130's direction. For example SPU1105may be a digital signaling processing core, a microprocessor core, a micro controller core, or a combination of these cores. Local store1110is a storage area that SPU1105configures for a private storage area and a non-private storage area. For example, if SPU1105requires a substantial amount of local memory, SPU1105may allocate 100% of local store1110to private memory. In another example, if SPU1105requires a minimal amount of local memory, SPU1105may allocate 10% of local store1110to private memory and allocate the remaining 90% of local store1110to non-private memory (seeFIG. 11Band corresponding text for further details regarding local store configuration).

The portions of local store1110that are allocated to non-private memory are managed by system memory map1120in region1145. These non-private memory regions may be accessed by other SPU's or by processing unit1130. MMU1115includes a direct memory access (DMA) function and passes information from local store1110to other memory locations within device1100.

FIG. 11Bis a diagram showing a local storage area divided into private memory and non-private memory. During system boot, synergistic processing unit (SPU)1160partitions local store1170into two regions which are private store1175and non-private store1180. SPU1160is similar to SPU1105and local store1170is similar to local store1110that are shown inFIG. 11A. Private store1175is accessible by SPU1160whereas non-private store1180is accessible by SPU1160as well as other processing units within a particular device. SPU1160uses private store1175for fast access to data. For example, SPU1160may be responsible for complex computations that require SPU1160to quickly access extensive amounts of data that is stored in memory. In this example, SPU1160may allocate 100% of local store1170to private store1175in order to ensure that SPU1160has enough local memory to access. In another example, SPU1160may not require a large amount of local memory and therefore, may allocate 10% of local store1170to private store1175and allocate the remaining 90% of local store1170to non-private store1180.

A system memory mapping region, such as local storage aliases1190, manages portions of local store1170that are allocated to non-private storage. Local storage aliases1190is similar to local storage aliases1145that is shown inFIG. 11A. Local storage aliases1190manages non-private storage for each SPU and allows other SPU's to access the non-private storage as well as a device's control processing unit.

While the computer system described inFIGS. 10,11A, and11B are capable of executing the processes described herein, this computer system is simply one example of a computer system. Those skilled in the art will appreciate that many other computer system designs are capable of performing the processes described herein.