Runahead execution for graphics processing units

A method, in accordance with an embodiment of the invention, includes detecting a memory page miss associated with a thread operating on a Graphics Processing Unit (GPU). A request can be issued to receive the memory page associated with the memory page miss. There can be a switch into a runahead mode. During the runahead mode, a future memory page miss can be detected. During the runahead mode, a request can be issued to receive the future memory page associated with the future memory page miss.

BACKGROUND

It is understood that future operating systems will require Graphic Processing Units (GPUs) to be able to support virtual memory. Therefore, GPUs will no longer be limited to the physical amount of directly connected video-memory or accessible system memory that can be allocated to them. As such, video memory becomes a virtualized resource that the operating system may page in on demand from one or more external storage devices such as disk drives.

When supporting virtual memory, it is understood that the GPU can request data that is not currently within video-memory, thereby causing the GPU to experience a page miss. In response, a request is sent to a Central Processing Unit (CPU) to initiate the fetching of the desired page from disk memory, which has such a significant latency that it can adversely affect the performance of the GPU. Therefore, servicing virtual page misses becomes a potentially severe bottleneck. While traditionally GPUs are able to hide the memory latency of an internal cache miss, e.g., by building long graphics pipes and having many pixels in flight at the same time, it is unrealistic to expect GPUs to build up to the point where it becomes possible to hide the latency of a virtual memory page miss because page-in times from disk are too long for that to be practically feasible.

Regardless, applications involving graphics are going to take advantage of the virtual memory model. As such, the GPU is expected to incur the occasional virtual page miss with potentially disastrous performance breakdowns to the GPU as a consequence.

SUMMARY

Accordingly, embodiments of the invention are directed toward enabling GPUs to operate within a runahead mode. A method, in accordance with an embodiment of the invention, includes detecting a memory page miss associated with a thread operating on a Graphics Processing Unit (GPU). A request can be issued to receive the memory page associated with the memory page miss. There can be a switch into a runahead mode. During the runahead mode, a future memory page miss can be detected. During the runahead mode, a request can be issued to receive the future memory page associated with the future memory page miss.

Another embodiment of the invention includes a computer-readable medium containing a plurality of instructions which when executed cause a GPU to implement a method. The method includes detecting a memory page miss associated with a thread operating on the GPU. The method also includes requesting to receive the memory page associated with the memory page miss. Additionally, the method includes switching the GPU into a runahead mode. Moreover, the method includes detecting during the runahead mode a future memory page miss. The method includes requesting during the runahead mode to receive the future memory page associated with the future memory page miss.

Yet another embodiment of the invention includes a computing system including a disk drive and a Central Processing Unit (CPU) coupled to the disk drive. The computing system also includes a video memory and a GPU coupled to the video memory and the CPU. The GPU can detect a memory page miss associated with a thread operating on the GPU. Also, the GPU can issue a request to the CPU to receive from the disk drive the memory page associated with the memory page miss. Additionally, the GPU can switch into a runahead mode. Furthermore, the GPU can detect during the runahead mode a future memory page miss. Moreover, the GPU can issue a request to the CPU during the runahead mode to receive from the disk drive the future memory page associated with the future memory page miss.

While particular embodiments of the present invention have been specifically described within this summary, it is noted that the invention is not limited to these embodiments. The invention is intended to cover alternatives, modifications and equivalents, which may be included within the scope of the invention as construed according to the Claims.

DETAILED DESCRIPTION

Reference will now be made in detail to various embodiments in accordance with the invention, examples of which are illustrated in the accompanying drawings. While the invention will be described in conjunction with various embodiments, it will be understood that these various embodiments are not intended to limit the invention. On the contrary, the invention is intended to cover alternatives, modifications and equivalents, which may be included within the scope of the invention as construed according to the Claims. Furthermore, in the following detailed description of various embodiments in accordance with the invention, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be evident to one of ordinary skill in the art that the invention may be practiced without these specific details. In other instances, well known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the invention.

In accordance with one embodiment of the invention, when a Graphics Processing Unit (GPU) encounters a virtual memory page miss that involves fetching a page of data from disk memory, the GPU can be switched from its normal execution mode into a runahead mode. While the page of data is being fetched from disk memory, the runahead mode can enable the GPU to detect and prefetch other virtual memory page misses that would have otherwise occurred sequentially, to now be done in parallel. In this manner, the runahead mode can enable the overlapping of virtual memory page misses, which can result in hiding some of the latency of subsequent virtual memory page misses underneath the latency of the first virtual memory page miss.

FIG. 1is a block diagram of an exemplary computing system100wherein various embodiments in accordance with the invention can operate. Computing system100depicts the components of a basic computer system in accordance with embodiments of the invention that can provide the execution platform for certain hardware-based and software-based functionality. For example, as part of enabling a Graphics Processing Unit (GPU)110of computing system100to support virtual memory, runahead execution mode (or computation) in accordance with various embodiments can be implemented with GPU110as a method for reducing the occurrence of virtual memory page misses to disk memory108and thereby improve the performance of the GPU110.

Runahead computation for GPU110can operate as follows, but is not limited to such. When the GPU110encounters a virtual memory page miss that involves fetching a page of data from disk memory108, GPU110does not sit idle and wait for the requested data to arrive within its video (or graphics) memory112. Instead, GPU110can be switched from the original thread it was executing in normal mode into a runahead mode and can continue executing program instructions from the original thread with non-existing data. At the same time the original requested data is being fetched from disk memory108, GPU110can generate additional valid virtual memory page miss requests that would have subsequently occurred within the original normal mode thread. As such, the runahead mode enables GPU110to perform intelligent prefetching of pages of data into video memory112that would have otherwise resulted in subsequent virtual page misses to disk memory108if not prefetched.

WithinFIG. 1, it is noted that since GPU110continues execution without data from the original virtual memory page miss, the non-existing or invalid data can be tracked to avoid fetching data from erroneous memory locations. Once the original page miss data is fetched from disk memory108and stored within video memory112via a Northbridge106and a Southbridge116of computing system100, GPU110can be switched out of the runahead mode and into the normal mode to resume execution of the original thread. Going forward, GPU110should experience less virtual memory page misses since the prefetching of those data pages has occurred and/or are in flight to video memory112.

Within computing system100, a Central Processing Unit (CPU)102can include a GPU software driver114operating thereon that enables the proper operation of GPU110. Furthermore, in various embodiments, GPU driver114can be implemented to include functionality that enables GPU110to operate in runahead mode or primary execution mode. The primary execution mode can also be referred to as a normal execution mode or a default execution mode or a base execution mode, but is not limited to such. It is noted that the runahead mode functionalities that can be associated with GPU driver114are described in detail herein.

WithinFIG. 1, computing system100can include CPU102, system memory104, GPU110, and Southbridge116that are each coupled to Northbridge106. It is noted that instead of being coupled to Northbridge106, system memory104can alternatively be coupled to CPU102(not shown). Additionally, disk memory108can be coupled to Southbridge116while video memory112can be coupled to GPU110. The couplings between the elements of computing system100can be implemented in a wide variety of ways. For example, the couplings between elements of computing system100can be implemented by using, but not limited to, one or more hardware communication buses, one or more communication protocols, wired and/or wireless communication, and the like. Note that computing system100can include more or less elements than those shown withinFIG. 1. Moreover, the elements of computing system100can be coupled in different ways than those shown withinFIG. 1in accordance with various embodiments.

It is pointed out that GPU110can be implemented as a discrete component, a discrete graphics card designed to couple to computer system100via a connector (e.g., AGP slot, PCI-Express slot, etc.), a discrete integrated circuit die (e.g., mounted directly on a motherboard), or as an integrated GPU included within the integrated circuit die of a computer system chipset component (not shown). Additionally, CPU102can be implemented as, but not limited to, one or more microprocessors, one or more processors, and the like.

WithinFIG. 1, system100can be implemented as, for example, a desktop computer system, server computer system, laptop computer system, portable computing device, and the like. It is noted that components can be included with system100that add peripheral buses, specialized graphics memory, input/output (IO) devices, and the like. Similarly, system100can be implemented as a system-on-a-chip, or the like, suited for low power handheld devices (e.g., cell phone, mobile phone, etc.), or can be implemented as a set-top video game console device such as, for example, the Xbox®, available from Microsoft Corporation of Redmond, Wash., or the PlayStation3®, available from Sony Computer Entertainment Corporation of Tokyo, Japan, and the like.

It is noted that GPU110can be implemented as a discrete component, a discrete graphics card designed to couple to computer system100via a connector (e.g., AGP slot, PCI-Express slot, etc.), a discrete integrated circuit die (e.g., mounted directly on a motherboard), or as an integrated GPU included within the integrated circuit die of a computer system chipset component (not shown). Additionally, a local graphics memory can be included for the GPU110for high bandwidth graphics data storage.

WithinFIG. 1, computing system100can include one or more CPUs similar to CPU102, and at least one GPU110. Note that CPU102can alternatively be directly coupled to the system memory104via a memory controller (not shown) internal to CPU102. Furthermore, GPU110can be coupled to a display device (not shown). One or more additional GPUs can optionally be coupled to system100to further increase its computational power.

FIG. 2Ais a diagram illustrating an exemplary software based GPU (e.g.,110) runahead approach200in accordance with various embodiments of the invention. Note thatFIG. 2Awill be described in conjunction with computing system100ofFIG. 1. It is pointed out that the driver software (e.g.,114) of GPU110can implement some of the operations shown and described with reference toFIG. 2A. WithinFIG. 2A, a main thread202can be operating on GPU110in normal execution mode201. During the operation of main thread202, the GPU driver114can detect a virtual memory page miss, it can cause GPU110to issue a virtual page miss request to CPU102to receive the memory page of data, and then it can switch the GPU110into runahead mode203(as indicated by arrow210). It is noted that the memory page miss can involve any kind of memory slower than the video or graphics memory (e.g.,112). For example in an embodiment, the memory page miss can involve requesting virtual memory from a disk drive (e.g.,108), or requesting a page of main memory to be transferred to graphics memory (e.g.,112). The switching of GPU110can include the GPU driver114performing a context save, which can involve storing within disk memory108the current operating state of GPU110with reference to main thread202. The switching of GPU110can also include the GPU driver114changing GPU110's state to disable all external memory writes. Note that disabling external writes can include, but is not limited to, disabling any color writes, disabling any alpha writes, disabling any z-writes, and/or any combination thereof. Next, the GPU driver114can switch GPU110into runahead mode203such that GPU110is performing runahead thread212. Note that runahead thread212can allow GPU110to runahead with data (valid or invalid) within the execution instructions associated with main thread202, thereby enabling the possibility of generating additional virtual memory page miss requests along the way. As such, during the operation of runahead thread212, the GPU110in combination with GPU driver114try to detect and prefetch as many of the subsequent virtual memory page misses that may occur during the normal mode201operations of main thread202.

For example, during the operation of runahead thread212, the GPU110in combination with GPU driver114is able to determine or detect a future virtual memory page miss to disk memory108that would have subsequently occurred during the normal mode201operations of main thread202. As such, GPU driver114can cause GPU110to issue a virtual page miss request to CPU102in order to receive the memory page associated with that future virtual page miss. Furthermore, GPU110in combination with GPU driver114can determine or detect another future virtual page miss that would have subsequently occurred during the normal mode201operations of main thread202. Therefore, GPU driver114can cause GPU110to issue a virtual page miss request to CPU102in order to receive the memory page associated with that future virtual page miss. It is understood that when CPU102receives each of the virtual page miss requests, the CPU102can initiate the prefetching of each of the corresponding memory pages, which will eventually be received and stored by video memory112. Note that the corresponding memory pages can each be received and stored by video memory112during and/or after runahead mode203. It is noted that more or less future virtual page misses can be detected or determined within runahead thread212.

Once the originally requested memory page is returned to video memory112, the GPU driver114can switch GPU110out of the runahead mode203and into the main thread202of the normal execution mode201, as indicated by arrow224. It is understood that during normal execution mode201, GPU driver114and GPU110may subsequently encounter one or more additional virtual memory page misses to disk memory108that would be similar to the original virtual page miss as described above.

WithinFIG. 2A, in one embodiment, during the operation of runahead thread212, the GPU110in combination with GPU driver114can determine to switch from runahead mode203into a suspend or idle mode205(as indicated by arrow227) because there may be little or no advantage for continuing execution of runahead thread212. For example in one embodiment, a point may be reached where there is not enough useful data to continue executing runahead thread212without its future results being substantially useless. Within the suspend mode205, the GPU driver114can basically suspend the operations of GPU110and can cause it to wait until the originally requested memory page is returned to video memory112. At that point, the GPU driver114can switch GPU110out of the suspend mode205and into the main thread202of normal execution mode201, as indicated by arrow225.

It is noted that in an embodiment, one or more processors and/or software can operate in runahead mode203while the GPU110enters a suspend mode205. Specifically, during the operation of main thread202, the GPU driver114can detect a virtual memory page miss, and it can cause GPU110to issue a virtual page miss request to CPU102to receive the memory page of data. The GPU driver114can then switch the GPU110from normal mode201into a suspend mode205(as indicated by arrow229), and then the GPU driver114operating on the CPU102can switch into runahead mode203(as indicated by arrow211) in order to emulate the GPU110in runahead mode203as described herein, but is not limited to such. Note that the switching of GPU110into the suspend mode205can include the GPU driver114performing a context save, which can involve storing within disk memory108the current operating state of GPU110with reference to main thread202. Additionally, the switching of GPU110into suspend mode205can include the GPU driver114changing GPU110's state to disable all external memory writes as described herein, but is not limited to such. Once the originally requested memory page is returned to video memory112, the GPU driver114can switch out of the runahead mode203(as indicated by arrow224). Furthermore, the GPU driver114can also switch GPU110out of the suspend mode205and into the main thread202of normal execution mode201(as indicated by arrow225).

FIG. 2Bis a diagram of an exemplary table250pertaining to various embodiments in accordance with the invention. Specifically, table250illustrates different types of semantics254that can be utilized by a GPU driver (e.g.,114) in combination with a GPU (e.g.,110) when operating in accordance with various embodiments of the invention. For example, the GPU driver114can operate the GPU110using strong semantics258during runahead mode, which means everything is precisely executed by the GPU110as originally defined in an API (application program interface). For instance in an embodiment, the GPU driver114can operate the GPU110using strong semantics258during runahead mode by executing the original thread (or command stream) in the same manner as it did during normal mode. Additionally, the GPU driver114can operate the GPU110using weak semantics256during runahead mode, which means GPU110just executes what is needed to find more virtual memory page misses. If something does not contribute to that purpose, GPU110can safely skip it. For instance, typically all addressing is done with integer calculations while a lot of graphics use floating point calculations. In certain situations it may be basically pointless for the GPU110to perform floating point calculations while in runahead mode because typically a floating point number is not used to produce an address. In an embodiment, weak semantics256can include any execution mode designed to be a more efficient mode to predict or detect one or more future page misses. In an embodiment, weak semantics256can include a more efficient execution mode to approximately predict future page misses (e.g., not all future page misses may be identified or at least one future memory page miss may be misidentified). It is pointed out that a GPU (e.g.,110) operating using weak semantics256can thus be more efficient in executing a run-ahead task and thus identify and request more pages from a disk memory (e.g.,108), as compared to a GPU operating using strong semantics258.

It is pointed out that table250includes different modes252that the GPU110can operate in, which can include normal execution mode201and runahead mode203. Additionally, as mentioned previously, table250includes different semantics254that the GPU110can operate in, which can include weak semantics256and strong semantics258. Specifically for normal execution mode201, strong semantics258is typically required and weak semantics256is not allowed. Furthermore, for the runahead mode203, weak semantics256can be desirable and strong semantics258is acceptable. It is pointed out that there is no restriction for using strong semantics258for runahead mode203, but it may reduce the amount of virtual memory page misses detected and it may also result in more power consumption by GPU110. However, in one embodiment, it may be simpler to implement runahead mode203using strong semantics258.

FIG. 3is a block diagram illustrating an exemplary hardware based GPU (e.g.,110′) runahead approach in accordance with various embodiments of the invention. Within the exemplary hardware based GPU runahead approach ofFIG. 3, the hardware that can be implemented as part of GPU110′ can include, but is not limited to, the addition of one or more transistor gates, which can be included as part of runahead mode hardware310. Alternatively or in addition, invalid bits can be included with all registers (e.g.,302and304) that are utilized within runahead mode and during runahead computations. The GPU runahead hardware can also include the use of strong and weak semantics for operations involving GPU110′. Note that running a thread (e.g.,202) with strong semantics can be equivalent to the normal, non-runahead operation of the GPU110′. However, weak semantics can implement the operations of the runahead mode to properly propagate the invalid register bits, and implement optimizations based on this invalid-bit propagation, as well as the fact that invalid data should not get written to memory.

The result of implementing weak semantics within the runahead hardware approach should be a GPU (e.g.,110) that executes faster in runahead mode. Note that weak semantics can allow GPU110to runahead further in the same amount of time as compared to executing the same instructions with strong semantics.

WithinFIG. 3, it is understood that in order to implement GPU runahead, restoring the state to the point when runahead computations started can be efficiently emulated by fine-grained switching, but is not limited to such. It is pointed out that the hardware based GPU runahead approach enables one or more future virtual page misses to be prefetched (or at least initiated) from disk memory108during the runahead mode.

Note that both the GPU software runahead technique ofFIG. 2and the GPU hardware runahead technique ofFIG. 3benefit from the fact that GPU applications are specialized. Typically, the most critical applications operating on GPUs are single thread, full-screen problems, such as but not limited to, personal computer (PC) video games. Thus, when such an application experiences a virtual page miss, there are usually no other graphics threads available to run.

In addition, the programs executing on a GPU (e.g.,110) are largely sequential stream operations since they are mostly branch free, rarely branch based on input data, and are generally data-coherent. Thus, it is noted that runahead computation (or runahead mode) is more efficient the less branches there are, the less data-dependent the program execution is, and the more coherent the data.

Moreover, GPUs (e.g.,110) are generally complex architectures that encode a long pipeline of disparate specialized functions. Each one of these functional units has their own set of caches and thus potential cache misses. GPU runahead mode advantageously provides the ability to prefetch data for all these different caches within the system with little overhead.

FIG. 4is a block diagram illustrating an exemplary queue GPU runahead system400in accordance with various embodiments of the invention. WithinFIG. 4, it is understood that GPUs (e.g.,110) can execute instructions out of a queue402that can store one or more frames of instructions received from CPU102. As such, the queue402can be viewed as an extremely deep instruction queue.

WithinFIG. 4, the queue GPU runahead system400includes CPU102coupled to the queue402thereby enabling the transmission (or transfer) of frame instructions406from CPU102into queue402. The queue402is coupled to GPU110thereby enabling the transmission (or transfer) of frame instructions406from queue402into GPU110. It is noted that as part of a runahead mode, the GPU software driver114′ operating on the CPU102can be constantly examining the frame instructions406stored within queue402, as indicated by line404.

FIG. 5is a diagram of an exemplary translation look-aside buffer500in accordance with various embodiments of the invention. Specifically, the translation look-aside buffer500is a virtual memory map that translates a virtual memory page address with its corresponding physical page address within disk memory108. The translation look-aside buffer500can be stored in video memory112and can be utilized by GPU110. Note that each entry (e.g.,502or504) within the translation look-aside buffer500has a runahead mode invalid bit (e.g.,510or516) associated with it.

Within the translation look-aside buffer500, a runahead mode invalid bit (e.g.,510) can be utilized during runahead mode to indicate that a particular memory page of data is not really in video memory (e.g.,112). But in order to have the GPU110operate during runahead mode when a virtual page miss to disk108occurs, the GPU110can be instructed that the address translation is valid now. However, any loads or stores from hardware that uses that page of memory can propagate that invalid bit information, if appropriate. Specifically, the invalid bit (e.g.,510or516) would get propagated through any runahead computations that could result in a further miss within the translation look-aside buffer500. In this manner, the GPU110or its driver114can determine whether or not to produce a memory request cycle during runahead mode based on whether specific data produced an invalid address. So the propagation that you want to protect the GPU110from is indicating invalid or bogus memory page prefetches. It is understood that in runahead mode, one of its main purposes is to generate virtual memory page miss addresses that can subsequently be prefetched.

WithinFIG. 5, the invalid bit (e.g.,510or516) avoids computation that are undesirable during runahead mode (e.g., computation that will not result in addresses), and determines whether a computed address in runahead mode is invalid or valid. However, it is noted that there are choices that can be made once an address is known to be invalid. For example, the first option is not to use it and just skip over it. Alternatively, another option is to potentially fetch the data from the bogus address with the understanding that the bogus address is known to be close enough to where the desired data is located. For example, if the desired data is within a small enough range that is known, such as a byte, it may be desirable to fetch a whole memory page in that vicinity knowing that the desired data is located within that page.

It is understood that the translation look-aside buffer500can include any number of entries that are similar to entries502and504. Entry502can include a virtual address506along with its corresponding physical address508and runahead invalid bit510. Additionally, entry504can include a virtual address512along with its corresponding physical address514and runahead invalid bit516.

FIG. 6is a flow diagram of a method600in accordance with various embodiments of the invention for executing a runahead mode on a GPU. Method600includes exemplary processes of various embodiments of the invention which can be carried out by a processor(s) and electrical components under the control of computing device readable and executable instructions (or code), e.g., software. The computing device readable and executable instructions (or code) may reside, for example, in data storage features such as volatile memory, non-volatile memory and/or mass data storage that are usable by a computing device. However, the computing device readable and executable instructions (or code) may reside in any type of computing device readable medium. Although specific operations are disclosed in method600, such operations are exemplary. Method600may not include all of the operations illustrated byFIG. 6. Also, method600may include various other operations and/or variations of the operations shown byFIG. 6. Likewise, the sequence of the operations of method600can be modified. It is noted that the operations of method600can be performed by software, by firmware, by electronic hardware, or by any combination thereof.

Specifically, a determination can be made as to whether a GPU encounters a virtual memory page miss while operating on a main thread. If not, the determination can be repeated until a virtual memory page miss is encountered by the GPU. If the GPU does encounter a virtual memory page miss, the GPU can perform a switch out of the main thread. It is understood that the switch out of the main thread can include saving to memory the GPU's current state within the main thread. The state of the GPU can be changed in order to disable it from performing any external memory write operations. Additionally, the GPU can be switched into a runahead mode in order to try and generate any additional virtual memory page miss requests. In this manner, any additional virtual memory page miss requests can be initiated and prefetched into video memory of the GPU to be used by the GPU when it returns to performing the main thread. A determination can be made as to whether the original page miss has been fetched. If not, process600can continue the runahead mode to generate any additional virtual memory page miss requests. However, if the original page miss has been fetched, the GPU can be switched out of the runahead mode and back into the saved state of the main thread. Note that the saved state can be retrieved from memory in order to return the GPU to that specific state of the main thread.

At operation602ofFIG. 6, a determination can be made as to whether a GPU (e.g.,110) encounters a virtual memory page miss while operating on a main thread. If it is determined that the GPU is not encountering a virtual memory page miss at operation602, process600can return to the beginning of operation602in order to repeat its determination. However, if the GPU does encounter a virtual memory page miss at operation602, process600can proceeds to operation604. Operation602can be implemented in a wide variety of ways. For example in one embodiment, driver software (e.g.,114) for the GPU can be implemented to perform operation602. It is understood that operation602can be implemented in any manner similar to that described herein, but is not limited to such.

At operation604, the GPU can perform a switch out of the main thread. Understand that operation604can be implemented in a wide variety of ways. For example in one embodiment, it is understood that the switch out of the main thread by the GPU can include saving to memory the GPU's current state within the main thread. It is noted that operation604can be implemented in any manner similar to that described herein, but is not limited to such.

At operation606ofFIG. 6, the state of the GPU can be changed or modified in order to disable it from performing any external memory write operations while in a runahead mode. Operation606can be implemented in a wide variety of ways. For example in one embodiment, driver software (e.g.,114) of the GPU can perform operation606. Note that operation606can be implemented in any manner similar to that described herein, but is not limited to such.

At operation608, the GPU can be switched into a runahead execution mode in order to try and generate any additional subsequent virtual memory page miss requests. In this manner, any additional virtual memory page miss requests can be initiated and prefetched into video memory of the GPU to be eventually used by the GPU when it returns to performing the main thread. It is noted that operation608can be implemented in a wide variety of ways. For example in one embodiment, driver software of the GPU can enable the GPU to operate in runahead mode and to generate any additional virtual memory page miss requests at operation608. Operation608can be implemented in any manner similar to that described herein, but is not limited to such.

At operation610ofFIG. 6, a determination can be made as to whether the original page miss has been fetched or retrieved. If it is determined at operation610that the original page miss has not been fetched, process600can proceed to operation614. However, if it is determined at operation610that the original page miss has been fetched, process600can proceed to operation612. Understand that operation610can be implemented in a wide variety of ways. For example, operation610can be implemented in any manner similar to that described herein, but is not limited to such.

At operation614, the GPU can continue the runahead mode to generate any additional virtual memory page miss requests. Operation614can be implemented in a wide variety of ways. For example in one embodiment, driver software (e.g.,114) of the GPU can enable the GPU to continue within the runahead mode to generate any additional virtual memory page miss requests at operation614. Understand that operation614can be implemented in any manner similar to that described herein, but is not limited to such.

At operation612ofFIG. 6, the GPU can be switched out of the runahead mode and back into the saved state of the main thread where the original virtual memory page miss occurred or was encountered. It is noted that operation612can be implemented in a wide variety of ways. For example in one embodiment, the saved state can be retrieved from memory at operation612in order to return the GPU to that specific state within the main thread where the original virtual memory page miss occurred or was encountered. It is understood that operation612can be implemented in any manner similar to that described herein, but is not limited to such. Once operation612is completed, process600can proceed to operation602.

FIG. 7is a flow diagram of a method700in accordance with various embodiments of the invention for executing a runahead mode on a GPU. Method700includes exemplary processes of various embodiments of the invention which can be carried out by a processor(s) and electrical components under the control of computing device readable and executable instructions (or code), e.g., software. The computing device readable and executable instructions (or code) may reside, for example, in data storage features such as volatile memory, non-volatile memory and/or mass data storage that are usable by a computing device. However, the computing device readable and executable instructions (or code) may reside in any type of computing device readable medium. Although specific operations are disclosed in method700, such operations are exemplary. Method700may not include all of the operations illustrated byFIG. 7. Also, method700may include various other operations and/or variations of the operations shown byFIG. 7. Likewise, the sequence of the operations of method700can be modified. It is noted that the operations of method700can be performed by software, by firmware, by electronic hardware, or by any combination thereof.

Specifically, a memory page miss can be detected that is associated with a thread operating on a Graphics Processing Unit (GPU) in normal execution mode. A request can be issued to receive the memory page associated with the memory page miss. The GPU can be disabled from performing one or more external memory writes. The GPU can be switched from normal execution mode into a runahead mode. It is noted that the switching can include saving the current state of the GPU, after the detecting of the memory page miss. During the runahead mode, one or more future memory page misses can be detected. A request can be issued during the runahead mode to receive the future memory page associated the future memory page miss. A determination can be made as to whether the memory page has been received by video memory. The GPU can be switched out of the runahead mode and into the thread of its normal execution mode, in response to the memory page being received. The future memory page can be received and stored for the GPU during the runahead mode.

At operation702ofFIG. 7, a memory page miss can be detected that is associated with a thread operating on a GPU (e.g.,110) in normal execution mode. Operation702can be implemented in a wide variety of ways. For example in one embodiment, driver software (e.g.,114) for the GPU can be implemented to perform operation702. It is understood that operation702can be implemented in any manner similar to that described herein, but is not limited to such.

At operation704, a request can be issued to receive the memory page associated with the memory page miss. Understand that operation704can be implemented in a wide variety of ways. For example, operation704can be implemented in any manner similar to that described herein, but is not limited to such.

At operation706ofFIG. 7, the GPU can be disabled from performing one or more external memory writes. Operation706can be implemented in a wide variety of ways. For example, operation706can be implemented in any manner similar to that described herein, but is not limited to such.

At operation708, the GPU can be switched from normal execution mode into a runahead mode. Operation708can be implemented in a wide variety of ways. For example in one embodiment, the switching can include saving the current state of the GPU, after the detecting of the memory page miss. It is noted that operation708can be implemented in any manner similar to that described herein, but is not limited to such.

At operation710ofFIG. 7, during the runahead mode, one or more future memory page misses can be detected. It is understood that operation710can be implemented in a wide variety of ways. For example, operation710can be implemented in any manner similar to that described herein, but is not limited to such.

At operation712, a request can be issued during the runahead mode to receive the future memory page associated with each future memory page miss. It is noted that operation712can be implemented in a wide variety of ways. For example, operation712can be implemented in any manner similar to that described herein, but is not limited to such.

At operation714ofFIG. 7, a determination can be made as to whether the memory page has been received by video memory. Operation714can be implemented in a wide variety of ways. For example, operation714can be implemented in any manner similar to that described herein, but is not limited to such.

At operation716, the GPU can be switched out of the runahead mode and into the thread of its normal execution mode, in response to the memory page being received. It is understood that operation716can be implemented in a wide variety of ways. For example, operation716can be implemented in any manner similar to that described herein, but is not limited to such.

At operation718ofFIG. 7, the one or more future memory pages can be received and stored for the GPU during the runahead mode. It is noted that operation718can be implemented in a wide variety of ways. For example, operation718can be implemented in any manner similar to that described herein, but is not limited to such.

With regard to embodiments of the invention, it is noted that whenever a GPU (e.g.,110or110′) is ideal for whatever reason, the GPU can be switched into a runahead thread (or mode) in order to prefetch memory pages of data (or any other amount of data from memory) in any manner similar to that described herein, but is not limited to such. With regard to embodiments of the invention, it is pointed out that a GPU can be switched into a runahead mode during any type of virtual memory miss. As such, the GPU can detect and prefetch any future virtual memory miss of any size.

The foregoing descriptions of various specific embodiments in accordance with the invention have been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed, and it is evident that many modifications and variations are possible in light of the above teaching. The invention can be construed according to the Claims and their equivalents.