Abstract:
A “virtual on-chip memory” that provides advantages as compared to an on-chip memory that utilizes a cache. In accordance with the invention, when a CPU attempts to access a memory address that is not on-chip, the access is aborted and the abort is handled at a page level. A single page table is utilized in which each entry constitutes an address in the virtual address space that will be mapped to a page of on-chip memory. The CPU obtains the missing data, updates the page table, and continues execution from the aborted point. Because aborts are handled at the page level rather than the line level, the virtual on-chip memory is less expensive to implement than a cache. Furthermore, critical real-time applications can be stored within a non-virtual portion of the memory space to ensure that they are not stalled.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    The present application claims the benefit of U.S. Provisional Patent Application No. 60/758,537 filed on Jan. 13, 2006, which is incorporated herein by reference in its entirety. 
     
    
     BACKGROUND OF THE INVENTION 
       [0002]    1. Field of the Invention 
         [0003]    The present invention is generally directed to microprocessor devices. In particular, the present invention relates to microprocessor devices which constitute a System-on-a-Chip (SoC), including but not limited to SoC implementations of a multiprocessor system. 
         [0004]    2. Background 
         [0005]    Conventional System-on-a-Chip (SoC) implementations utilize large static on-chip memories to meet their memory needs. This memory may potentially be shared among several processors running multiple processes. Given the complexity of combining very large memories on an integrated circuit (IC) with one or more processors, the amount of storage space in these memories may be limited. The conventional solution is to provide the SoC with an even larger off-chip memory in order to meet its memory demands, and treating the on-chip memory as a cache. The downside of this approach is that when a process requests data from the cache which it does not have a copy of, the entire processor must stall while the cache memory goes to the off-chip memory to retrieve a copy of the information. The delays associated with such off-chip memory retrieval are substantial, and not conducive to real-time processing. 
         [0006]    Accordingly, what is desired is a system and method that resolves the problem of delays associated with off-chip memory retrieval. 
       BRIEF SUMMARY OF THE INVENTION 
       [0007]    The present invention is directed to using an on-chip memory to support the addressing of a Virtual Address Space (VAS) in a configuration called Virtual On-Chip Memory (VOM). VOM is targeted primarily to System-on-a-Chip (SoC) implementations which typically consist of one or more processors on the same integrated circuit (IC) as a memory. In particular, and as will be described in more detail herein, embodiments of the invention provide a method, system, and a memory for running real-time processes concurrent with non-real-time processes without stalling the real-time process on access to off-chip memory. The solution is also applicable to systems without the demands of real-time processing. 
         [0008]    An on-chip memory in accordance with an embodiment of the present invention allows one or more processors located on the same chip as the on-chip memory to retrieve information from a VAS that is much larger than the on-chip memory. Information stored in the on-chip memory represents a subset of the VAS, and pieces of the VAS can be substituted into the on-chip memory as needed by a processor. 
         [0009]    In accordance with another embodiment of the present invention, the information stored in memory that is a subset of the information in the VAS is written into the memory by a processor whenever the information is needed by first retrieving the information from the VAS and storing the information into the memory. A notation is made in a page table to indicate which subset of information from the VAS is presently stored in the memory. 
         [0010]    In accordance with yet another embodiment of the present invention, a memory will have a separate section which is not a subset of the information from the VAS. This section of memory is used by processes that have a critical need for information, such as real-time processes. 
         [0011]    Further features and advantages of the invention, as well as the structure and operation of various embodiments of the invention, are described in detail below with reference to the accompanying drawings. It is noted that the invention is not limited to the specific embodiments described herein. Such embodiments are presented herein for illustrative purposes only. Additional embodiments will be apparent to persons skilled in the relevant art(s) based on the teachings contained herein. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS/FIGURES 
         [0012]    The accompanying drawings, which are incorporated herein and form part of the specification, illustrate the present invention and, together with the description, further serve to explain the principles of the invention and to enable a person skilled in the relevant art(s) to make and use the invention. 
           [0013]      FIG. 1  depicts basic elements of an example System-on-a-Chip using Virtual On-Chip Memory in accordance with an embodiment of the present invention. 
           [0014]      FIG. 2  depicts the basic elements of an example System-on-a-Chip using Virtual On-Chip Memory and introduces further complexities in accordance with an embodiment of the present invention. 
           [0015]      FIG. 3  illustrates a flowchart of a method for reading information from a Virtual On-Chip Memory and responding to a situation where the information is not presently located in the paged memory in accordance with an embodiment of the present invention. 
           [0016]      FIG. 4  depicts a virtual memory addressing scheme where an address into the virtual address space is used to consult the page table and find the corresponding paged area of memory in which the requested information is kept in accordance with an embodiment of the present invention. 
           [0017]      FIG. 5  depicts the basic elements of an example System-on-a-Chip using Virtual On-Chip Memory and further shows a data bus which is used by a processor in order to retrieve information from a data stream that corresponds to a particular location within a virtual address space in accordance with an embodiment of the present invention. 
       
    
    
       [0018]    The features and advantages of the present invention will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, in which like reference characters identify corresponding elements throughout. In the drawings, like reference numbers generally indicate identical, functionally similar, and/or structurally similar elements. The drawing in which an element first appears is indicated by the leftmost digit(s) in the corresponding reference number. 
       DETAILED DESCRIPTION OF THE INVENTION 
     A. Introduction to Virtual On-Chip Memory 
       [0019]    Common to many integrated circuit (IC) implementations of a processor is the usage of memory that is tightly coupled to the processor for providing rapid input/output (I/O) rates of information. Such a memory is commonly referred to as a cache. Caches are generally very responsive in many applications, but the responsiveness can suffer greatly when information is requested of the cache that is not present, causing a cache miss which necessitates retrieving the information from off-chip memory. An application which has requested the information will then be required to stall while waiting for the information to be loaded from off-chip memory, a process with a significant delay. 
         [0020]      FIG. 1  depicts basic elements of an example System-on-a-Chip (SoC) using Virtual On-Chip Memory (VOM) that avoids the aforementioned shortcomings of conventional implementations. A SoC consists of all of the necessary parts of a computer system on a single IC  100 . In example IC  100 , there are one or more processors  102   a - 102   n  coupled to a memory  106  over a memory bus  104 . Memory  106  is responsible for storing a paged area  108  of information and a corresponding page table  110  that holds mappings into the page table. One skilled in the relevant arts will appreciate that paged area  108  and page table  110  can be stored on different memory implementations, and need not share a common memory device such as memory  106 . 
         [0021]    Any one of the processors  102   a - 102   n  reads the paged area  108  of memory  106  by sending an address to read from over the memory bus  104 . The processor  102   a - 102   n  first reads the page table  110  to determine where in the paged area  108  of memory the information the processor  102   a - 102   n  seeks is located. The page table  110  consists of one entry per addressable memory location in the paged area  108  of memory  106 . Each entry in the page table  110  identifies the source address within the context of a larger virtual memory, thereby labeling the origin of the information in the addressable memory locations in the paged area  108  of memory  106 . As will be appreciated by persons skilled in the relevant art(s) based on the teachings provided herein, there are a variety of methods of using the page table  110  to address the paged area  108  of memory  106 , and as such the present invention is not limited to any particular method of populating and reading the page table. Persons skilled in the relevant art(s) based on the teachings provided herein will further appreciate that the page table  110  will generally be at a fixed location within memory  106 , but can be located anywhere, even outside of memory  106 , where a processor  102   a - 102   n  will know to look for it. 
         [0022]      FIG. 2  depicts a similar example SoC on IC  200  to the one in  FIG. 1 , and introduces further complexities, in accordance with an embodiment of the present invention. Processors  202   a - 202   n  are each connected to a data memory  216   a - 216   n  which allows the processor  202   a - 202   n  direct access to data which only it needs. This is trivial in the situation where there is only a single processor  202   a - 202   n , but in a situation where there are multiple processors  202   a - 202   n , each would only be able to access its own data memory  216   a - 216   n . Each processor  202   a - 202   n  is also coupled to a cache  218   a - 218   n  which functions like a traditional cache memory for any information passed over the memory bus  204 . This cache  218   a - 218   n  enjoys the benefit of being able to refer to on-chip memory  206  for rapid retrieval of information in the event of a cache miss. Persons skilled in the relevant art(s) based on the teachings provided herein will appreciate that such a cache is optional, but its usage provides for faster information access under certain constraints. Persons skilled in the relevant art(s) based on the teachings provided herein will further appreciate that when such a cache is present, the cache will be required to handle interactions with the next level of memory, and when the cache is not present, the processor will be responsible for the interactions itself. For the purposes of this specification, when a reference is made to the processor storing or requesting information to or from the memory, it should be assumed that this same procedure is handled by the optional cache if the cache is present. Memory  206  is similar to the memory  106  in  FIG. 1  except that it further shows the Virtual Address Space (VAS)  214 . Memory  206  also contains an unpaged area  212  which is read and written without consulting the page table  210 . 
         [0023]    The VAS  214  is a virtual memory construct which is composed of any number of physical I/O data streams. For the purposes of this description, it is to be assumed that the VAS  214  behaves much like a single large external memory. 
         [0024]    When the processor  202   a - 202   n  stores or requests information to or from the memory  206 , it sends a message over the memory bus  204  containing the address on which the operation is to be performed. This address refers directly to a location in the unpaged area  212  of memory  206 , or alternatively to a location within the VAS  214 . For addresses within the unpaged area  212 , the information is stored directly or retrieved and returned over the memory bus  204  immediately. On the other hand, for addresses referring to a location within the VAS  214 , it is necessary to consult the page table  210 , as previously mentioned, in order to determine whether the information presently resides within the paged area  208  of memory  206 . 
         [0025]    In accordance with an embodiment of the present invention, VAS  214  is stored in a compressed form in an on-chip memory. When processor  202   a - 202   n  presents a reference address within the VAS  214  for a read operation, if the information is not available within page table  210 , then the requested information is decompressed from VAS  214  and entered into page table  210 . Similarly, information in pages flushed from page table  210  is compressed and stored in VAS  214  if modified by a write operation. In accordance with an additional embodiment of the present invention, memory used by a real-time process is stored in on-chip memory. In accordance with a further embodiment of the present invention, memory  206  is stored in its entirety in on-chip memory. One skilled in the relevant arts will appreciate that VAS  214  can similarly be substituted by any desired software algorithm used to provide information to the paged area  208  of memory  206 , and need not be embodied in a physical memory structure. 
       B. Using Virtual On-Chip Memory to Facilitate Real-Time Processing 
       [0026]    One of the challenges of real-time processing is reducing the amount of time the a processor spends in a stalled state, waiting for further information before being able to proceed. VOM helps address these challenges by providing a paging technique on an on-chip memory, eliminating processor stalls associated with waiting on a cache miss in a conventional system. While a cache can still be utilized between the processor and the on-chip memory to provide even further speed benefits, a cache miss in this situation isn&#39;t critical since the next level of memory is still on-chip. 
         [0027]    As shown in  FIG. 2 , memory  206  is divided into an unpaged area  212  and a paged area  208  of memory, with some space allocated for a page table  210 . As will be appreciated by persons skilled in the relevant art(s) based on the teachings provided herein, the unpaged area of memory is not necessary in the implementation of VOM, but providing an unpaged area of memory may be helpful in accelerating certain memory access schemes, such as those needed for real-time processing. Each processor  202   a - 202   n  is capable of running both real-time and non-real-time processes simultaneously. Information needed by real-time processes is preloaded in the unpaged area  212  of memory, and information needed by non-real-time processes is kept somewhere within the VAS  214 . Generally, the real-time processes are each different instances of the same code base when there are multiple processors  202   a - 202   n , making it possible for the processes to share the same information within the unpaged area  212  of memory, minimizing the system&#39;s overall memory footprint. As will be appreciated by persons skilled in the relevant art(s) based on the teachings provided herein, any combination of real-time and non-real time processes can be run on any or all of the processors  202   a - 202   n , and as such the present invention is not limited to the method of configuring a single or multiple processors  202   a - 202   n  to necessarily run a real-time process in combination with a non-real-time process. 
         [0028]    By placing real-time process information in the unpaged area  212  of memory  206 , a developer can be assured that the real-time process will never need to wait for a period of time longer than the delay associated with the information retrieval mechanism of the memory  206  and the propagation delays associated with the memory bus  204 . A real-time process in which all of its information is kept in the unpaged area  212  will therefore never suffer the performance hit of a cache miss that requires fetching information from off-chip memory. 
         [0029]    In contrast, in this embodiment, non-real-time processes request their information from the VAS  214 , since their execution time is less critical than real-time processes. When a non-real-time process requests information from the VAS  214 , after the processor  202   a - 202   n  consults the cache  218   a - 218   n  for the information, the address is checked against the page table  210  to determine if the information requested is presently available in the paged area  208  of memory  206 . If it is, then the information is read from the paged area  208  and returned over the memory bus  204 , affording the non-real-time process no more of a delay than the real-time process, except perhaps the time associated with consulting the page table  210 . However, if the information is not available in the paged area  208  of memory  206  according to the page table  210 , then a page fault message is issued and the processor  202   a - 202   n  will then abort the non-real-time process that requested the information until the requested information becomes available. The processor  202   a - 202   n  can then continue execution of any non-real-time or real-time processes it is responsible for without stalling. In accordance with an embodiment of the present invention, real-time processes may request information from the VAS  214  efficiently if the pages associated with the real-time process are maintained permanently on-chip and are never used to page in other information. 
       C. Utilizing the Virtual Address Space 
       [0030]      FIG. 3  illustrates a flowchart  300  of a method for reading information from a Virtual On-Chip Memory and responding to the situation where the information is not presently located in the paged memory in accordance with an embodiment of the present invention. The invention, however, is not limited to the description provided by the flowchart  300 . Rather, it will be apparent to persons skilled in the relevant art(s) from the teachings provided herein that other functional flows are within the scope and spirit of the present invention. 
         [0031]    Flowchart  300  will be described with continued reference to SoC IC  200  described above in reference to  FIG. 2 . The invention, however, is not limited to that embodiment. 
         [0032]    The method of flowchart  300  begins at step  302 , in which a processor  202   a - 202   n  has requested information from the memory  206  by transmitting an associated address on which to perform an operation. At step  304 , a determination is made as to whether this information is already present in the processor&#39;s  202   a - 202   n  cache  218   a - 218   n , and if so, is immediately read and processed  320 . Otherwise, some analysis of the address needs to be done  306  to determine if the reference is to the unpaged area  212  of memory  206 . If the requested information is within the unpaged area  212 , then it can be immediately provided by the memory  206  over the memory bus  204  to the requesting processor  202   a - 202   n . On the way, the information may be cached  314  by the processor&#39;s  202   a - 202   n  cache  218   a - 218   n . The information is once again ready to be processed  320 . 
         [0033]    If the address of the requested information is not in the unpaged area  212 , then it must lie somewhere within the VAS  214 . The next step is to consult  310  the page table  210  to make a determination  312  of whether the requested information is already stored in the paged area  208  of memory  206 . If the information is already present, then it is simply read, cached  314 , and processed  320  in a similar manner to the operation of accessing the unpaged area  212 . 
         [0034]    If the address of the requested information is further not in the paged area  208 , then it becomes necessary to fetch the information from the VAS  214 . The process requesting the information is alerted to this situation when it is sent a page fault  316 , causing the process to respond by aborting. It is then necessary to locate the information within the VAS  214  and store a copy of the information  318  within the paged area  208  of memory  206 . With the information  318  stored in the paged area  208  of memory  206 , the process is resumed and completes its request for information. In accordance with an embodiment of the present invention, while the process is aborted, real-time processes may continue to execute. 
         [0035]      FIG. 4  depicts a virtual memory addressing scheme where an address into the virtual address space is used to consult the page table and find the corresponding paged area of memory in which the requested information is kept in accordance with an embodiment of the present invention. 
         [0036]    The memory  406  is similar to the memory  206  shown in  FIG. 2  except that it further depicts a memory controller  402  within the memory  406 . This memory controller is used for managing the requests sent by a processor over the memory bus  404 , which is similar to the memory bus  204  shown in  FIG. 2 . The memory  406  further includes a paged area  408 , an unpaged area  412 , and a page table  410 , which are also similar to the paged area  208 , the unpaged area  212 , and the page table  210  shown in  FIG. 2 . The VAS  414  is also similar to the VAS  214  in  FIG. 2 . 
         [0037]    The controller  402  can be configured to perform a number of the steps that a processor requesting information could handle. For example, the controller  402  can be configured to issue a page table miss message to a processor whenever information requested from the VAS  414  is not found in the page table  410 . As will be appreciated by persons skilled in the relevant art(s) based on the teachings provided herein, the controller  402  is capable of handling the responsibilities associated with memory reads or writes to which it has been assigned, such as issuing a page table miss message to a requesting processor. 
         [0038]    When an address to be operated upon, such as with a read operation, is placed on the memory bus  404 , it is received by the memory  406  and is handled by the controller  402 . The address, as shown in  FIG. 4 , is broken up into a page P[i] and a page address A[j]. The full address can therefore be represented by P[i]:A[j]. This example is not limiting, however, and one of sufficient skill in the relevant art(s) will appreciate that there are numerous ways to represent a memory address to the same effect. 
         [0039]    The entire VAS  414  is broken down into n pages, from P[ 0 ] to P[n]. The granularity of the page breakdown determines how many directly addressable VAS  414  entries are within each page, a quantity called m. Each page P[i] therefore has m addresses within it ranging from A[ 0 ] to A[m]. Thus any one smallest unit of directly addressable VAS  414  memory is addressable as P[i]:A[j]. 
         [0040]    The paged area  408  is broken down into p entries, where each entry consists of m individually addressable locations. This has the effect of making each entry in the paged area  408  the same size as each entry in the VAS  414 , and an entire page P[i] can be stored within each entry. However, the size of the paged area  408  is much smaller than the entire VAS  414 . The controller  402  can manage a request for paged memory by checking the page table  410  to see if a particular page P[i] has had its contents inserted within the paged area  408 , which means the particular address A[j] within that page is also in the paged area  408 . If the page P[i] is not in the paged area  408 , then the controller issues a page fault message, notifying the requester that the information needs to be retrieved from the VAS  414  and the new page inserted into the paged area  408 . 
         [0041]    The paged area  408  has, as noted above, a total of p page entries, a subset of the n page entries available in the VAS  414 .  FIG. 4  refers to some of these entries as P[i′], P[i″], through P[i p ]. This convention indicates that the pages available in the page table  408  do not necessarily appear sequentially in the VAS  414 , and actually consist of any p pages from the VAS  414 . 
       D. Populating the Virtual Address Space 
       [0042]      FIG. 5  depicts the basic elements of an example System-on-a-Chip using Virtual On-Chip Memory and further shows a data bus which is used by a processor in order to retrieve information from a data stream that corresponds to a particular location within the virtual address space in accordance with an embodiment of the present invention. 
         [0043]    The IC  500  is similar to the IC  200  in  FIG. 2 , except that the processor  502   a - 502   n  has been simplified to remove the optional data memory  216   a - 216   n  and cache  218   a - 218   n  shown in  FIG. 2 .  FIG. 5  further shows an I/O Data Bus (IODB)  520  which connects processors  502   a - 502   n  to a number of stream sources. One of sufficient skill in the relevant art(s) will appreciate, however, that the IODB  520  can actually consist of a number of coupling mechanisms used to provide an interface to an information source with which a processor  502   a - 502   n  can communicate, and is not limited to the single bus implementation, nor is it limited to the particular information streams in  FIG. 5 . 
         [0044]    The IODB  520  is connected to a number of streams such as a cable/DSL modem  522  for providing TCP/IP data streams, a video recorder card  524  for providing a video stream, a hard disk drive  526  for providing a stream of data stored on it, or an off-chip memory  528  for providing a stream of data stored on it. 
         [0045]    As previously described, the VAS  414  shown in  FIG. 4  can be broken down into n pages with m addressable memory locations each. However, the VAS  414  is not a physical entity, but is instead comprised of a number of information sources, where each page P[i] refers to a specific block of m addresses within a stream. When a page miss occurs, a processor  502   a - 502   n  must look at P[i] to determine which stream of data contains the needed information, and will then retrieve the page of information and place it within the paged area  508  of memory  506 . The processor will then have to update the page table  510  to reflect the fact that P[i] is now occupying a particular location within the paged area  508 . One of sufficient skill in the relevant art(s) will appreciate that the process of receiving the page fault message, retrieving the needed information from a data stream, storing the information in the paged area  508 , and updating the page table  510  can be accomplished by any device that has the ability to interface with both the data streams and the memory  506 , such as any of the processors  502   a - 502   n  or even, in certain circumstances, a device such as the memory controller  402  shown in  FIG. 4 . 
       E. Multiple Processor Architectures 
       [0046]    As previously described, the VOM consists of one or more processors  202   a - 202   n  as shown in  FIG. 2 . Each of these processors  202   a - 202   n  has access to read and write data to the memory bus  204  either directly or through its own cache  218   a - 218   n . This section deals specifically with the situation where there are a plurality of processors  202   a - 202   n . Further, for the purposes of this discussion, the processors  202   a - 202   n  will be indicated as receiving or sending messages over the memory bus  204 , when in fact there may be an intermediate cache  218   a - 218   n  coupled between the processor  202   a - 202   n  and the memory bus  204 . As one skilled in the relevant art(s) will appreciate, this is due to the usage of the cache  218   a - 218   n  being optional in the implementation of VOM. 
         [0047]    When a page table miss occurs, a page table miss message is generated and sent to one of the processors  202   a - 202   n  which is then responsible for retrieving the information from the VAS  214  and storing it in the paged area  208  of memory  206 . The processor  202   a - 202   n  that receives this message need not be the same one requesting information from the memory  206 . In an implementation, one processor  202   a - 202   n  is dedicated to receiving page miss messages and handling the retrieval of the requested information from the VAS  214  and the storage of the information in the paged area  208  of memory  206 . In another implementation, the responsibility of receiving and handling page miss messages as described above is rotated between each of the processors  202   a - 202   n . In yet another implementation, the responsibility of receiving and handling page miss messages as described above is held by the processor  202   a - 202   n  that placed the request for information from the paged area  208  of memory  206 . These examples are not limiting, however, and one of sufficient skill in the relevant art(s) will appreciate that any number of schemes can be implemented to decide which processor  202   a - 202   n  will handle the page miss message processing. 
         [0048]    When a processor  202   a - 202   n  requests information from the paged area  208  of memory  206  that is not available, the requesting processor  202   a - 202   n  aborts the process requesting the information and returns control to another process running on the same processor  202   a - 202   n . Generally, the aborted process is a non-real-time process, since as described above real-time processes do not access the paged area of memory in the preferred implementation, or the pages associated with real-time processes are maintained permanently on-chip. In this situation, control is restored to the real-time process, facilitating the processing demands required by a real-time system. 
         [0049]    As mentioned above, there are any number of schemes that can be implemented to decide which processor  202   a - 202   n  will handle a page table miss message. In at least one of these schemes, such as the case where each processor  202   a - 202   n  is responsible for updating the paged area  208  of memory  206  after a page table miss, there are situations in which a conflict will arise between two or more processors  202   a - 202   n  attempting to write to the same location in the page table  210 . To resolve this, a semaphore system is used whereby a processor  202   a - 202   n  attempting to write to the page table  210  (and subsequently to the paged area  208  of memory  206 ) first checks if the semaphore is set to a locked state. If the semaphore has been locked, then the processor  202   a - 202   n  will have to wait until the semaphore is unlocked. Once the semaphore is unlocked, the processor  202   a - 202   n  then locks the semaphore and proceeds to write the new status of the paged area  208  of memory  206  to the page table  210 . After writing this information, the processor  202   a - 202   n  then unlocks the semaphore, allowing other processors  202   a - 202   n  to write to the page table  210 . One of sufficient skill in the relevant art(s) will appreciate that this is a general description of a memory protection scheme, and that there are other schemes that can be implemented to protect the page table  210  and paged area  208  of memory  206  from being written by multiple processors  202   a - 202   n  simultaneously. 
       F. Conclusion 
       [0050]    While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example only, and not limitation. It will be understood by those skilled in the relevant art(s) that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined in the appended claims. Accordingly, the breadth and scope of the present invention should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.