Patent Description:
Computer-based applications such as servers, cloud infrastructure and database controllers make use of virtual machines, such as the Java Virtual Machine, to provide the ability to execute programs in a platform-independent environment. Virtual machines allow for programs to implemented in a variety of computing platforms while requiring only relatively minor code changes. Managed programming languages such as JAVA, PYTHON and JAVASCRIPT provide programmers with tools for creating flexible applications that may be executed in a virtual machine environment. A managed runtime is used in such managed languages, providing memory management, garbage collection, and hot method profiling. In a managed runtime, an application is typically rendered as a machine independent set of instructions (byte-code), which will then be interpreted during execution. A hot method profiler collects runtime information to identify which methods are hot spots. Identified hot methods thus may be compiled into native codes via a Just-in-Time (JIT) compiler for faster execution. A garbage collector typically runs in an independent thread to manage and recycle data heap.

Managing runtime allocation of objects for memory storage, particularly in a heterogeneous memory architecture using two or more tiers of memory having different performance, capacity and cost, provides challenges when attempting to optimize for higher performance or lower cost.

Non-patent literature <NPL> discloses a collaborative framework that employs object placement, cross-layer communication, and page-level management to effectively distribute application objects in the DRAM hardware to achieve desired power/performance goals. It describes the design and implementation of a framework, which integrates automatic object profiling and analysis at the application layer with fine-grained management of memory hardware resources in the operating system. It demonstrates the utility of the framework by employing it to more effectively control memory power consumption. It discloses a custom memory-intensive workload to show the potential of the approach. It discloses a sampling and profiling-based analyses and modifies the code generator in the HotSpot VM to understand object usage patterns and automatically determine and control the placement of hot and cold objects in a partitioned VM heap. This information is communicated to the operating system, which uses it to map the logical application pages to the appropriate DRAM ranks according to user-defined provisioning goals.

It is provided a semiconductor apparatus, a non-transitory computer readable storage medium and a method. Further aspects are set out in the appended set of claims.

<FIG> shows a block diagram of a conventional managed runtime environment <NUM>, which may operate in a virtual machine environment such as a Java virtual machine. In conventional managed runtime environment <NUM>, a hot method profiler <NUM> identifies hot methods, based on the frequency of method calls, to be compiled by JIT compiler <NUM> into native code methods <NUM>. Methods not identified as hot methods or methods otherwise not to be compiled via JIT compiler <NUM> are interpreted as byte-code methods <NUM>. Execution of byte-code methods <NUM> and native code methods <NUM> results in placement of objects in object heap <NUM>. A garbage collector <NUM>, running in an independent thread, operates to manage and recycle objects in object heap <NUM>. Hot method profiler <NUM>, JIT compiler <NUM> and garbage collector <NUM> may form a runtime engine. Byte-code methods <NUM>, native code methods <NUM> and object heap <NUM> may be included in a runtime data area.

<FIG> shows a block diagram of an example enhanced managed runtime environment <NUM> with tiered object memory placement according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. In addition to several components illustrated in <FIG> (hot method profiler <NUM>, JIT compiler <NUM>, byte-code methods <NUM> and native code methods <NUM>), enhanced managed runtime environment <NUM> may include hot object profiler <NUM>, object allocation redirector <NUM>, enhanced garbage collector <NUM>, hot object heap <NUM>, and cold object heap <NUM>. Hot object heap <NUM> and cold object heap <NUM> may store objects designated as hot or cold, respectively. Objects designated as hot objects are those objects that are likely to be accessed more often than other objects, which are designated as cold objects. Enhanced garbage collector <NUM> includes object migrator <NUM>, which may migrate objects between hot object heap <NUM> and cold object heap <NUM>. Hot object profiler <NUM>, object allocation redirector <NUM>, enhanced garbage collector <NUM>, hot object heap <NUM>, and cold object heap <NUM> may each be implemented in code as an extension to the runtime environment (e.g., as an extension to a Java virtual machine). Enhanced managed runtime environment <NUM> may also include memory interface <NUM>, which operates to provides a convenient way to allocate, query and move objects within the tiered memory architecture. As an example, memory interface <NUM> may be provided via the Intel® OneAPI memory interface. Hot method profiler <NUM>, JIT compiler <NUM>, hot object profiler <NUM>, object allocation redirector <NUM>, and enhanced garbage collector <NUM> may form an enhanced runtime engine. Hot and cold object heaps <NUM>-<NUM> may be included in a runtime data area, apportioned among a first memory tier (tier <NUM>, hot object heap), and a second memory tier (tier <NUM>, cold object heap). Byte-code methods <NUM> and native code methods <NUM> may be included in the runtime data area in the first memory tier, second memory tier, and/or other runtime memory (not specified).

Hot object profiler <NUM> receives hot methods (e.g., as a list of hot methods) from hot method profiler <NUM> and may identify hot objects based on the hot methods. Hot object profiler <NUM> may also monitor objects in hot object heap <NUM> and/or objects in cold object heap <NUM> and update a hotness score for one or more monitored objects. Additional details and features of hot object profiler <NUM> are further described below with reference to <FIG> and 4A-4B.

Object allocation redirector <NUM> manages the allocation of newly-created objects (i.e., objects as they are created) into hot object heap <NUM> and cold object heap <NUM>. Object allocation redirector <NUM> may be incorporated within JIT compiler <NUM> (e.g., forming an enhanced JIT compiler) or may otherwise be closely associated with JIT compiler <NUM>. Object allocation redirector <NUM> may employ extensions to byte-code methods <NUM> and native code methods <NUM> (e.g., see <FIG>). Additional details and features of object allocation redirector <NUM> are further described below with reference to <FIG> and <FIG>.

Enhanced garbage collector <NUM> operates to manage and recycle objects in hot object heap <NUM> and cold object heap <NUM>. Object migrator <NUM> operates as part of the garbage collection process to migrate objects between hot object heap <NUM> and cold object heap <NUM>, based on a hotness score for each object. Enhanced garbage collector <NUM> may run in an execution thread independent of an execution thread for JIT compiler <NUM> and object allocation redirector <NUM>. Additional details and features of enhanced garbage collector <NUM> (including object migrator <NUM>) are further described below with reference to <FIG> and <FIG>.

Hot object heap <NUM> may store hot objects in a first memory tier, labeled as Memory Tier <NUM>. Cold object heap <NUM> may store cold objects in a second memory tier, labeled as Memory Tier <NUM>. Memory tier <NUM> may include memory components of a different type or quality as compared to memory components of Memory tier <NUM>. For example, on the one hand memory tier <NUM> may include memory components having higher performance, higher cost and/or lower capacity, while on the other hand memory tier <NUM> may include memory components having lower cost, lower performance and/or higher capacity. For example, memory tier <NUM> may have a tight constraint on capacity (e.g., due to higher cost), while memory tier <NUM> may be more abundant (e.g., due to lower cost). As an illustration, memory tier <NUM> may include DRAM (dynamic random access memory), while memory tier <NUM> may include Intel® Optane™ memory. As another illustration, memory tier <NUM> may include eDRAM (embedded DRAM), while memory tier <NUM> may include traditional DRAM. Other memory tier combinations are possible. Additional details and features of hot object heap <NUM> and cold object heap <NUM> are further described below with reference to <FIG> and <FIG>.

<FIG> shows a block diagram illustrating example tiered object memory placement within a hybrid memory architecture <NUM> according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Hybrid memory architecture <NUM> may include a hot object heap <NUM> maintained in a first memory tier (labeled as memory tier <NUM>) and a cold object heap <NUM> maintained in a second memory tier (labeled as memory tier <NUM>). It will be understood that, in some embodiments, the hybrid memory architecture may have more than <NUM> memory tiers; for example, <NUM> memory tiers are shown in <FIG> as discussed below. While memory tier <NUM> and memory tier <NUM> are illustrated as blocks of equal size, it will be understood that memory tier <NUM> and memory tier <NUM> may each have the same memory size/capacity or may have different memory size/capacity. The size/capacity for each memory tier may be chosen based on performance and/or cost considerations. For example, if high performance (e.g., bandwidth, access latency, etc.) is desired, a relatively higher ratio of tier <NUM> memory (higher-performing memory) to tier <NUM> memory (lower-performing memory) may be chosen. As another example, if lower overall cost is desired, a relatively lower ratio of tier <NUM> memory (higher cost) to tier <NUM> memory (lower cost) may be chosen.

Hot object heap <NUM> may store objects designated as hot objects. Cold object heap <NUM> may store objects designated as cold objects. Hot object heap <NUM> may be hot object heap <NUM>, and cold object heap <NUM> may be cold object heap <NUM>, each as described above with reference to <FIG>. As described above, objects designated as hot objects are those objects that are likely to be accessed more often than other objects. Objects designated as cold objects are those objects that are likely to be accessed less often than other objects. Each object may include a hotness score which may be used in determining whether an object is designated as hot or cold. An object's hotness score may be based, e.g., on whether the object was created or allocated by a hot method, and/or the frequency of object invocation or access.

Object classes are organized to contain most related methods and their related data, and different classes have least data-interdependency. That is, a method in one class doesn't access a data field in another class directly; instead, it accesses through well-defined interfaces/methods such as getField() and setField(). If the class instance a hot method belongs to is considered a hot object, the memory placement of hot objects (i.e., accessed by hot methods) are most likely to impact overall performance. Of note, second layer object access (i.e., a reference of another object's data from a hot object). may be naturally captured when that object's getField()/setField() become hot methods.

Accordingly, apportioning objects between memory tiers based on an object's hotness promotes the ability to optimize or enhance performance and/or cost for a hybrid memory architecture. For example, allocating hot objects to a higher-performing memory tier will promote improved overall performance, since those objects accessed (or likely to be accessed) frequently will have a greater impact on performance compared to objects accessed less frequently. Similarly, allocating cold objects to a lower cost (but lower-performing) memory tier will have a lesser impact on overall performance, since any performance difference for infrequently-accessed objects would not be likely to add up to a significant difference in overall performance; yet allocating cold objects to a lower cost memory tier will be more cost effective than placing those cold objects in a more costly memory tier. Overall, the tiered object memory placement techniques described herein may provide for reduced cost of memory while minimizing any degradation of performance caused through use of lower-performing memory in a memory tier.

In one or more embodiments, objects may additionally have a designation as young or old, based on the relative age of the object (e.g., the length of time since the object was created or allocated). Objects may include a parameter or field which may be marked (or flagged) to indicate whether the object is young or old. There may be many ways to handle objects flagged as young or old, respectively, including but not limited to organizing them into separate spaces or storage areas. As shown in <FIG>, hot object heap <NUM> may include young objects <NUM> (i.e., objects designated as young) and old objects <NUM> (i.e., objects designated as old). Similarly, cold object heap <NUM> may include young objects <NUM> (i.e., objects designated as young) and old objects <NUM> (i.e., objects designated as old). While objects may be conceptually organized in terms of a young object heap (containing young objects) and an old object heap (containing old objects), the relative age of objects may have little (or nothing) to do with an object's frequency of access or hotness. For example, some young objects may have been in existence only a short time but may be accessed frequently (or infrequently). In contrast, some old objects may have been in existence a relatively long period of time but may be accessed only infrequently (or in some examples, frequently). Thus, if a goal is to optimize or enhance memory performance and/or low cost in a hybrid memory architecture, an object's age (young vs. old) does not present a good basis for apportioning objects between different memory tiers. Accordingly, memory tier <NUM> and memory tier <NUM> may each contain both young and old objects. It will be understood that a particular memory tier may only contain young objects, or may only contain old objects.

<FIG> shows a block diagram illustrating example tiered object memory placement within a hybrid memory architecture <NUM> according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Hybrid memory architecture <NUM> may include a first object heap <NUM> maintained in a first memory tier (labeled as memory tier <NUM>), a second object heap <NUM> maintained in a second memory tier (labeled as memory tier <NUM>), and a third object heap <NUM> maintained in a third memory tier (labeled as memory tier <NUM>). The first object heap <NUM> may contain the hottest objects, the second object heap <NUM> may contain objects of an intermediate level of hotness, and the third object heap <NUM> may contain the coldest objects - in each case based on object hotness scores. While memory tier <NUM>, memory tier <NUM> and memory tier <NUM> are illustrated as blocks of equal size, it will be understood that the memory tiers may each have the same memory size/capacity or may have different memory size/capacity. The size/capacity for each memory tier may be chosen based on performance and/or cost considerations.

In one or more embodiments, objects may additionally have a designation as young or old, based on the relative age of the object (e.g., the length of time since the object was created or allocated) as discussed above. As shown in <FIG>, the first object heap <NUM> may contain young objects <NUM> (i.e., objects designated as young) and old objects <NUM> (i.e., objects designated as old); similarly, the second object heap <NUM> may contain young objects <NUM> and old objects <NUM>, and the third object heap <NUM> may contain young objects <NUM> and old objects <NUM>. As discussed above, objects may be conceptually organized in terms of a young object heap (containing young objects) and an old object heap (containing old objects); but because the relative age of objects may have little (or nothing) to do with an object's frequency of access or hotness, an object's age (young vs. old) may not present a good basis for apportioning objects between different memory tiers. Accordingly, each memory tier may contain both young and old objects. It will be understood that some memory tier(s) may only contain young objects, or may only contain old objects.

<FIG> shows a block diagram illustrating functionality of an example hot object profiler <NUM> according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. An object may have a hotness score based, e.g., on whether the object was created or allocated by a hot method, and/or the frequency of object invocation or access. Hot object profiler <NUM> receives from hot method profiler <NUM> methods identified as hot methods based on the frequency of method calls. The hot methods from hot method profiler <NUM> may be received as a list of methods, such as, for example:
hot_method_list:
{
method_1,
method_2,. }
These hot methods are used by hot profiler <NUM> to efficiently identify hot objects. For each received hot method, hot object profiler <NUM> may identify internal objects <NUM>, which are objects that are allocated (created) internally within the method, and external objects <NUM>, which are objects allocated (created) externally to the method and passed to the method as arguments in the method call. For each internal object <NUM>, hot object profiler <NUM> may identify the internal object type (or class) as hot (label <NUM>). This identification may be used by object allocation redirector <NUM> in allocating objects newly-created (by the hot method) of that object type to the hot object heap, as described further below.

For each external object <NUM>, hot object profiler <NUM> may count the invocation of each object referenced as an argument (label <NUM>). In some embodiments, hot object profiler <NUM> may generate a weighted list of objects based on invocation frequency, such as, e.g.:
external_obj_list(i) [with method_i from hot_method_list]
{method_i, eobject_1, weight_1(%)}
{method_i, eobject_2, weight_2(%)}
{method_i, eobject_3, weight_3(%)}. Hot object profiler <NUM> may then apply a filter (label <NUM>) to cull out objects having lower invocation frequency (or lower weight in the weighted list), resulting in objects having higher invocation frequency (or higher weight in the weighted list) which may then be assigned a hotness score attributable to hot objects (label <NUM>). Object filtering may be based on a threshold (e.g., a predetermined threshold) of weight (%), or may be performed dynamically based on memory tier budgets or relative memory tier availability. Hot object profiler <NUM> may assign a hotness score to each such hot object. As an example, the hotness score assigned to the hot objects may be a value of <NUM>. The lower invocation frequency (or lower weight) objects culled out by filter <NUM> may be assigned a hotness score attributable to cold objects (label <NUM>) by hot object profiler <NUM>, and each such cold object may be assigned a hotness score; as an example, the hotness score assigned to the cold objects may be a value of <NUM>. Objects identified as hot objects or cold objects by hot object profiler <NUM> may remain in their current memory location (current object heap), subject to migration (relocation) to the appropriate hot/cold object heap in a subsequent garbage collection cycle by enhanced garbage collector <NUM>.

Hot object profiler <NUM> may also monitor heap objects (label <NUM>) and update a hotness score for one or more monitored objects. Monitored heap objects may include objects in hot object heap <NUM> and /or objects in cold object heap <NUM>. For each monitored heap object, hot object profiler <NUM> may count the invocation of each object. Hot object profiler <NUM> may then modify a hotness score based on counting invocation by hot methods. In some embodiments, the hotness score for objects may be increased for objects invoked by a hot method (e.g., increasing hotness score by <NUM>), and the hotness score for objects may be decreased for objects not currently invoked by a hot method (e.g., decreasing hotness score by <NUM>). In some embodiments, the hotness score for objects may be reset to a hot value for objects invoked by a hot method (e.g., reset hotness to a value of <NUM>), and the hotness score for objects may be reset to a cold value for objects not currently invoked by a hot method (e.g., reset hotness to a value of <NUM>).

<FIG> is a flowchart illustrating a method <NUM> carried out by an example hot object profiler <NUM> according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. At block <NUM>, hot object profiler <NUM> may receive a list of hot methods. The list of hot methods may be obtained from hot method profiler <NUM>. Each hot method_i in the list of received hot methods may be evaluated as now described.

At block <NUM>, each internal object (e.g., of internal objects <NUM>) for received hot method_i may be identified as a hot object. Hot object profiler <NUM> may assign a hotness score to each such internal object identified as a hot object (e.g., the hotness score assigned to the hot internal objects may be a value of <NUM>).

At block <NUM>, hot object profiler <NUM> may count the invocation of each external argument referenced as an argument in hot method_i. In some embodiments, hot object profiler <NUM> may generate a weighted list of objects based on invocation frequency.

At block <NUM>, hot object profiler <NUM> may determine if it has reached the end of the list of hot methods. If no, the process may return to block <NUM> and evaluate the next hot method in the hot method list. If yes, the process may continue with block <NUM>.

At block <NUM>, hot object profiler <NUM> may identify external objects as hot (or cold) based on invocation frequency. Hot object profiler <NUM> may filter out objects having lower invocation frequency (or lower weight in the weighted list), resulting in objects having higher invocation frequency (or higher weight in the weighted list) which may then be identified as hot objects. Hot object profiler <NUM> may assign a hotness score to each such hot object for indicating the object as hot (e.g., the hotness score assigned to the hot external objects may be a value of <NUM>). External objects that are filtered out -- i.e., having a lower invocation frequency or lower weight may be identified as cold objects, and hot object profiler <NUM> may assign a hotness score (e.g., a value of <NUM>) to each of the objects identified as cold objects.

<FIG> illustrates an example extended object structure <NUM> according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Each object may have an original object structure <NUM> as may be designed or otherwise determined for a particular program or application. Two fields may be appended to original object structure <NUM> for each object: object allocation address <NUM> and object hotness <NUM>. Object allocation address <NUM> provides an address for the object's allocation - i.e., the address for the instruction that allocates (creates) the object. Object hotness <NUM> provides for a value that indicates the hotness of an object such as, e.g., the hotness score for an object as described above. Object hotness <NUM> may have an initial value assigned upon allocation (creation) of the object. Different object allocators may assign different values to object hotness <NUM> upon allocation of an object. In some embodiments, as a default position, upon initial allocation an object may be considered cold and a default value for object hotness may be set to <NUM>, unless the object is otherwise identified as hot. Object hotness <NUM> may be accessed or modified by hot object profiler <NUM>, object allocation redirector <NUM>, and enhanced garbage collector <NUM>.

<FIG> illustrates example extensions <NUM> to native code functions according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Native code functions <NUM> may include any functions written or compiled in native code (i.e., code for a particular computing platform or environment), and may include functions as compiled at runtime by JIT compiler <NUM>. Two allocation functions may be added to provide an extended set of native code functions: hot_heap_allocator (size, hotness) (label <NUM>) and cold_heap_allocator (size, hotness) (label <NUM>). The size parameter represents the amount of memory to be allocated for that object. The hotness parameter represents an initial hotness score that may be assigned to the object (e.g., via object hotness <NUM>) upon allocation. The function hot_heap_allocator () may be called to allocate hot objects in a hot object heap (e.g., hot object heap <NUM>). The function cold_heap_allocator () may be called to allocate cold objects in a cold object heap (e.g., cold object heap <NUM>). Object allocation redirector <NUM> may redirect allocation for hot external objects <NUM> (such as hot external objects identified as described above) to allocation function hot_heap_allocator (size, hotness=<NUM>), and for hot internal objects <NUM> (such as hot internal objects identified as described above) to allocation function hot_heap_allocator (size, hotness=<NUM>). Allocation for other objects <NUM> not identified as hot may be directed to cold_heap_allocator (size, hotness=<NUM>), which may be a default native code allocator.

<FIG> illustrates example extensions <NUM> to byte-code functions according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Byte-code functions <NUM> may include any functions written or compiled into a set of machine-independent instructions to be interpreted by the runtime environment of a virtual machine (e.g., Java Virtual Machine) during execution. Two allocation functions may be added to provide an extended set of byte-code functions: new_instance_hot () (label <NUM>) and new_instance_cold () (label <NUM>). Similar to the native code function extensions described above, parameters for memory size and object hotness may be included with each of the extended byte-code functions. The function new_instance_hot () may be called to allocate hot objects in a hot object heap (e.g., hot object heap <NUM>). The function new_instance_cold () may be called to allocate cold objects in a cold object heap (e.g., cold object heap <NUM>). Object allocation redirector <NUM> may redirect allocation for hot external objects <NUM> (such as external objects identified as hot but where the method is not a hot method) to allocation function new_instance_hot (). Allocation for other objects <NUM> not identified as hot may be directed to new_instance_cold (), which may be a default byte-code allocator.

<FIG> is a flowchart illustrating an example method <NUM> carried out by object allocation redirector <NUM> according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Object allocation redirection according to method <NUM> may handle object allocation for hot objects identified by hot object profiler <NUM>. Default allocation for cold objects (or any objects not identified as hot objects) may be handled by calls to cold_heap_allocator () (for native code calls) or new_instance_cold () (for byte-code calls), as described above.

At block <NUM>, a method_i to which an allocation address belongs is examined.

At block <NUM>, a query is made to determine whether the method_i has already been compiled to native code via JIT compilation (e.g., method_i is a hot method). If yes, the process may continue with block <NUM>; if no, the process may continue with block <NUM>.

At block <NUM>, native code functions are executed via JIT compilation. A call to cold_heap_allocator () may be replaced with (i.e., redirected to) a call to hot_heap_allocator (), which will allocate the object to hot object heap <NUM>.

At block <NUM>, it is determined whether the method_i is going to be compiled to native code via JIT compilation (e.g., method_i is newly-identified as a hot method). If no, the process may continue with block <NUM>; if yes, the process may continue with block <NUM>.

At block <NUM>, byte-code functions are executed via interpretation by the runtime environment of the virtual machine. A call to new_instance_cold () may be replaced with (i.e., redirected to) a call to new_instance_hot (), which will allocate the object to hot object heap <NUM>.

At block <NUM>, byte-code functions are executed via interpretation by the runtime environment of the virtual machine, but the respective method is to be compiled to native code by JIT compiler <NUM> for execution. A call to new_instance_cold () may be compiled to a native code call to hot_heap_allocator (), which will allocate the object to hot object heap <NUM>.

<FIG> illustrate example object migration according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Object migration may be carried out by object migrator <NUM> (part of enhanced garbage collector <NUM>) during a garbage collection process when objects may be scanned or reviewed. Migrating objects from cold object heap to hot object heap may be considered a process of object promotion. Migrating objects from hot object heap to cold object heap may be considered a process of object demotion.

In <FIG>, object migration <NUM> illustrates migrating objects between hot object heap <NUM> and cold object heap <NUM>, without consideration of any generational designation of objects (i.e., young objects or old objects). During the garbage collection process, if an object identified as a hot object (e.g., object hotness exceeds a threshold value, such a <NUM>) is resident in cold object heap <NUM>, object migrator <NUM> moves that object to hot object heap <NUM> (i.e., object promotion) (label <NUM>). Similarly, during the garbage collection process if an object identified as a cold object (e.g., object hotness equal to or less than a threshold value, such as <NUM>) is resident in hot object heap <NUM>, object migrator <NUM> moves that object to cold object heap <NUM> (i.e., object demotion) (label <NUM>).

In <FIG>, object migration <NUM> illustrates promoting objects to hot object heap <NUM> from cold object heap <NUM>, where hot object heap <NUM> may have young objects <NUM> and old objects <NUM>, and cold object heap <NUM> may have young objects <NUM> and old objects <NUM>. During the garbage collection process, if a young object <NUM> resident in cold object heap <NUM> is identified as a hot object, object migrator <NUM> will move that object to hot object heap <NUM>. If the hot object has aged, the object will be stored as a hot old object <NUM> (label <NUM>). However, if the hot object has not aged, the object will be stored as a hot young object <NUM> (label <NUM>). In some embodiments, if a cold old object <NUM> is identified as a hot object, the object may be promoted to hot old objects <NUM> during the garbage collection process (label <NUM>).

In <FIG>, object migration <NUM> illustrates demoting objects to cold object heap <NUM> from hot object heap <NUM>, where hot object heap <NUM> may have young objects <NUM> and old objects <NUM>, and cold object heap <NUM> may have young objects <NUM> and old objects <NUM>. During the garbage collection process, if a young object <NUM> resident in hot object heap <NUM> is identified as a cold object, object migrator <NUM> will move that object to cold object heap <NUM>. If the cold object has aged, the object will be stored as a cold old object <NUM> (label <NUM>). However, if the cold object has not aged, the object will be stored as a cold young object <NUM> (label <NUM>). Also during the garbage collection process, if an old object <NUM> resident in hot object heap <NUM> is identified as a cold object, object migrator <NUM> will move that object to cold object heap <NUM> as an old object <NUM>.

In some embodiments, migrating objects between hot object heap <NUM> and cold object heap <NUM> may include moving a first object from the cold object heap to the hot object heap if the hotness score associated with the first object is greater than a threshold value, and may include moving a second object from the hot object heap to the cold object heap if the hotness score associated with the second object is less than or equal to the threshold value. In some embodiments, the threshold value may be determined based on a size of the first tier memory. For example, for a larger first tier memory the hot object heap can hold a greater number of hot objects, so the hotness threshold may be set lower which would result in more hot objects in the hot object heap. As another example, for a smaller first tier memory the hot object heap can only hold a lower number of hot objects, so the hotness threshold may be set higher which would result in fewer hot objects in the hot object heap. In some embodiments, the threshold value may be determined based on a relative capacity of the first tier memory and the second tier memory. In some embodiments, the threshold value may be determined based on workload characteristics, such as to capture as hot objects specific object types used frequently.

<FIG> provide flowcharts illustrating operation of an example object migrator according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description, with reference to components and features described above including but not limited to the figures and associated description. Object migration may be carried out by object migrator <NUM> (part of enhanced garbage collector <NUM>) during a garbage collection process.

<FIG> is a flowchart illustrating a method <NUM> of object migration, which involves object promotion without regard to object age. At block <NUM>, objects in the cold object heap (such as cold object heap <NUM>) are scanned.

At block <NUM>, a query is made to determine whether a scanned object (k) is a hot object. If yes (i.e., the object is now hot), the process may continue with block <NUM>; if no (i.e., the object remains cold), the process may continue with block <NUM>.

At block <NUM>, the hot object (k) is moved (i.e., promoted) from the cold object heap and stored in the hot object heap (such as hot object heap <NUM>).

At block <NUM>, enhanced garbage collector <NUM> may determine if it has reached the end of the objects in the cold object heap. If no, the process may return to block <NUM> and evaluate the next object in the cold object heap. If yes, the process may continue with the procedures as described below with reference to <FIG>. It will be understood that objects in the heap may be scanned as a group and then evaluated one object at a time, or each object in the heap may be scanned and evaluated one object at a time.

<FIG> is a flowchart illustrating a method <NUM> of object migration, which involves object demotion without regard to object age. At block <NUM>, objects in the hot object heap (such as hot object heap <NUM>) are scanned.

At block <NUM>, a query is made to determine whether a scanned object (k) is a hot object. If no (i.e., the object is now cold), the process may continue with block <NUM>; if yes (i.e., the object remains hot), the process may continue with block <NUM>.

At block <NUM>, the hot object (k) is moved (i.e., demoted) from the hot object heap and stored in the cold object heap (such as cold object heap <NUM>).

At block <NUM>, enhanced garbage collector <NUM> may determine if it has reached the end of the objects in the hot object heap. If no, the process may return to block <NUM> and evaluate the next object in the hot object heap. If yes, the process may end. It will be understood that objects in the heap may be scanned as a group and then evaluated one object at a time, or each object in the heap may be scanned and evaluated one object at a time. It will be further understood that the processes outlined in method <NUM> and in method <NUM> may be combined into a single process for object promotion and demotion as part of the garbage collection process.

<FIG> is a flowchart illustrating a method <NUM> of object migration, which involves object promotion with consideration of object age. At block <NUM>, objects in the cold object heap <NUM> (which may include young objects <NUM> and/or old objects <NUM>) are scanned.

At block <NUM>, a query is made to determine whether a scanned object (k) has aged. If yes (i.e., the object is now old), the process may continue with block <NUM>; if no (i.e., the object remains young), the process may continue with block <NUM>. In some embodiments, if the object is already marked as old the procedures regarding object age at blocks <NUM>-<NUM> may be skipped. In some embodiments, the procedures regarding object age at blocks <NUM>-<NUM> may be carried out instead as a separate process regarding monitoring object age.

At block <NUM>, object (k) may be marked (or flagged) as an old object. For example, a parameter or field of object (k) may be changed from an indication as a young object to an indication of an old object.

At block <NUM>, a query is made to determine whether the scanned object (k) is a hot object. If yes (i.e., the object is now hot), the process may continue with block <NUM>; if no (i.e., the object remains cold), the process may continue with block <NUM>.

At block <NUM>, the hot object (k) is moved (i.e., promoted) from the cold object heap and stored in the hot object heap (such as hot object heap <NUM>). If the object (k) is marked as an old object, it will become one of the hot old objects <NUM>; if the object (k) is marked as a young object, it will become one of the hot young objects <NUM>.

At block <NUM>, enhanced garbage collector <NUM> may determine if it has reached the end of the objects in the cold object heap. If no, the process may return to block <NUM> and evaluate the next object in the cold object heap. If yes, the process may continue with the procedures described below with reference to <FIG>. It will be understood that objects in the heap may be scanned as a group and then evaluated one object at a time, or each object in the heap may be scanned and evaluated one object at a time. It will be further understood that the order of considering object age and object hotness may be reversed. In some embodiments, if a cold old object <NUM> is identified as a hot object, the object may be promoted to hot old objects <NUM> using a similar process as described with reference to <FIG>.

<FIG> is a flowchart illustrating a method <NUM> of object migration, which involves object demotion with consideration of object age. At block <NUM>, objects in the hot object heap <NUM> (which may include young objects <NUM> and/or old objects <NUM>) are scanned.

At block <NUM>, a query is made to determine whether the scanned object (k) is a hot object. If no (i.e., the object is now cold), the process may continue with block <NUM>; if yes (i.e., the object remains hot), the process may continue with block <NUM>.

At block <NUM>, the cold object (k) is moved (i.e., demoted) from the hot object heap and stored in the cold object heap (such as cold object heap <NUM>). If the object (k) is marked as an old object, it will become one of the cold old objects <NUM>; if the object (k) is marked as a young object, it will become one of the cold young objects <NUM>.

At block <NUM>, enhanced garbage collector <NUM> may determine if it has reached the end of the objects in the hot object heap. If no, the process may return to block <NUM> and evaluate the next object in the hot object heap. If yes, the process may end. It will be understood that objects in the heap may be scanned as a group and then evaluated one object at a time, or each object in the heap may be scanned and evaluated one object at a time. It will be further understood that the order of considering object age and object hotness may be reversed. It will be further understood that the processes outlined in method <NUM> and in method <NUM> may be combined into a single process for object promotion and demotion as part of the garbage collection process.

<FIG> is a flowchart illustrating a method <NUM> of operating an example managed runtime computing apparatus with tiered object memory placement according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description.

At block <NUM>, the apparatus assigns a hotness score to an object having an object type based on an invocation count of objects referenced by a hot method.

At block <NUM>, the apparatus allocates a newly-created object to one of a hot object heap, said hot object heap assigned to store hot objects in a first memory tier, or a cold object heap, said cold object heap assigned to store cold objects in a second memory tier, based on the hotness score associated with the object type for the newly-created object.

At block <NUM>, the apparatus migrates a plurality of objects between the hot object heap and the cold object heap based on a hotness score associated with each object, said object migration operating in an execution thread independent of an execution thread for said object allocation.

<FIG> is a flowchart illustrating a method <NUM> regarding monitoring heap objects and updating a hotness score for monitored objects, according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Method <NUM> may be carried out by, e.g., hot object profiler <NUM>.

At block <NUM>, the apparatus may monitor the invocation frequency of objects in the hot object heap and the invocation frequency of objects in the cold object heap.

At block <NUM>, the apparatus may modify the hotness score of at least one object based on the monitored invocation frequency of that object. In some embodiments, the hotness score for objects may be increased for objects invoked by a hot method (e.g., increasing hotness score by <NUM>), and the hotness score for objects may be decreased for objects not currently invoked by a hot method (e.g., decreasing hotness score by <NUM>). In some embodiments, the hotness score for objects may be reset to a hot value for objects invoked by a hot method (e.g., reset hotness to a value of <NUM>), and the hotness score for objects may be reset to a cold value for objects not currently invoked by a hot method (e.g., reset hotness to a value of <NUM>).

<FIG> is a flowchart illustrating a method <NUM> regarding migrating a plurality of objects in operating an example managed runtime computing apparatus with tiered object memory placement according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Method <NUM> may be carried out by, e.g., enhanced garbage collector <NUM> (including object migrator <NUM>).

At block <NUM>, the apparatus may move a first object from the cold object heap to the hot object heap if the hotness score associated with the first object is greater than a threshold value.

At block <NUM>, the apparatus may move a second object from the hot object heap to the cold object heap if the hotness score associated with the second object is less than or equal to the threshold value.

In some embodiments, the threshold value may be determined based on a relative capacity of the first tier memory and the second tier memory. In some embodiments, allocating a newly-created object to a hot object heap may include replacing a cold object allocator function with a hot object allocator function. In some embodiments, the apparatus may additionally assign a predetermined hotness score for a second newly-created object, the second newly-created object created by the hot method, and allocate the second newly-created object to the hot object heap.

<FIG> each show a block diagram illustrating an example enhanced managed runtime computing system <NUM> (labeled 1010a and 1010b respectively as illustrated) with tiered object memory placement according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. System <NUM> may generally be part of an electronic device/platform having computing functionality (e.g., server, cloud infrastructure controller, database controller, notebook computer, desktop computer, personal digital assistant/PDA, tablet computer, convertible tablet, smart phone, etc.), imaging functionality (e.g., camera, camcorder), media playing functionality (e.g., smart television/TV), wearable functionality (e.g., watch, eyewear, headwear, footwear, jewelry), vehicular functionality (e.g., car, truck, motorcycle), robotic functionality (e.g., autonomous robot), etc., or any combination thereof. In the illustrated example, system <NUM> may include a host processor <NUM> (e.g., central processing unit/CPU) having an integrated memory controller (IMC) <NUM> that may be coupled to system memory <NUM>. Host processor <NUM> may include any type of processing device, such as, e.g., microcontroller, microprocessor, RISC processor, ASIC, etc., along with associated processing modules or circuitry. System memory <NUM> may include any non-transitory machine- or computer-readable storage medium such as RAM, ROM, PROM, EEPROM, firmware, flash memory, etc., configurable logic such as, for example, PLAs, FPGAs, CPLDs, fixed-functionality hardware logic using circuit technology such as, for example, ASIC, CMOS or TTL technology, or any combination thereof suitable for storing instructions <NUM>.

System <NUM> may also include an input/output (I/O) subsystem <NUM>. I/O subsystem <NUM> may communicate with for example, one or more input/output (I/O) devices <NUM>, a network controller <NUM> (e.g., wired and/or wireless NIC), and storage <NUM>. Storage <NUM> may be comprised of any appropriate non-transitory machine- or computer-readable memory type (e.g., flash memory, DRAM, SRAM (static random access memory), solid state drive (SSD), hard disk drive (HDD), optical disk, etc.). Storage <NUM> may include mass storage. In some embodiments, host processor <NUM> and/ or I/O subsystem <NUM> may communicate with storage <NUM> (all or portions thereof) via network controller <NUM>. In some embodiments, system <NUM> may also include a graphics processor <NUM>.

System <NUM> may include two or more memory tiers, such as a first memory tier <NUM> (labeled as Memory Tier <NUM>) and a second memory tier <NUM> (labeled as Memory Tier <NUM>). First memory tier <NUM> may be comprised of a different memory type than second memory tier <NUM>. As illustrated in <FIG>, system <NUM> (labeled as 1010a) may include both first memory tier <NUM> and second memory tier <NUM> as part of system memory <NUM>. In the configuration illustrated in <FIG>, for example, first memory tier <NUM> may include embedded DRAM, while second memory tier <NUM> may include traditional DRAM. As illustrated in <FIG>, system <NUM> (labeled as 1010b) may include first memory tier <NUM> as part of system memory <NUM> and second memory tier <NUM> as part of storage <NUM>. In the configuration illustrated in <FIG>, for example, first memory tier <NUM> may include high-performing DRAM, while second memory tier <NUM> may include Intel® Optane™ memory. Other memory tier configurations are possible. In some embodiments, first memory tier <NUM> and second memory tier <NUM> may be organized as a database.

Host processor <NUM> and/or I/O subsystem <NUM> may execute program instructions <NUM> retrieved from system memory <NUM> and/or storage <NUM> to perform one or more aspects of the processes described above, including processes for hot object profiling described with reference to <FIG>, processes for object allocation redirection described with reference to <FIG> and <FIG>, processes for object migration described with reference to <FIG> and <FIG>. Host processor <NUM> and/or I/O subsystem <NUM> may execute program instructions <NUM> retrieved from system memory <NUM> and/or storage <NUM> to perform one or more aspects of the processes for operating a managed runtime computing apparatus described above with reference to <FIG>. In some embodiments, graphics processor <NUM> may execute program instructions <NUM> retrieved from the system memory <NUM> and/or storage <NUM>.

Computer program code to carry out the processes described above may be written in any combination of one or more programming languages, including an object-oriented programming language such as JAVA, JAVASCRIPT, PYTHON, SMALLTALK, C++ or the like and/or conventional procedural programming languages, such as the "C" programming language or similar programming languages, and implemented as program instructions <NUM>. Additionally, program instructions <NUM> may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, state-setting data, configuration data for integrated circuitry, state information that personalizes electronic circuitry and/or other structural components that are native to hardware (e.g., host processor, central processing unit/CPU, microcontroller, microprocessor, etc.).

Host processor <NUM> and I/O subsystem <NUM> may be implemented together on a semiconductor die as a system on chip (SoC) <NUM>, shown encased in a solid line. SoC <NUM> may therefore operate as a managed runtime computing apparatus. In some embodiments, SoC <NUM> may also include one or more of system memory <NUM>, network controller <NUM>, and/or graphics processor <NUM> (shown encased in dotted lines).

I/O devices <NUM> may include one or more of input devices, such as a touch-screen, keyboard, mouse, cursor-control device, touch-screen, microphone, digital camera, video recorder, camcorder, biometric scanners and/or sensors; input devices may be used to enter information and interact with system <NUM> and/or with other devices. I/O devices <NUM> may also include one or more of output devices, such as a display (e.g., touch screen, liquid crystal display/LCD, light emitting diode/LED display, plasma panels, etc.), speakers and/or other visual or audio output devices. Input and/or output devices may be used, e.g., to provide a user interface.

<FIG> shows a block diagram illustrating an example semiconductor apparatus <NUM> for managing a runtime computing environment having tiered object memory placement according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Semiconductor apparatus <NUM> may be implemented, e.g., as a chip, die, or other semiconductor package. Semiconductor apparatus <NUM> may include one or more substrates <NUM> comprised of, e.g., silicon, sapphire, gallium arsenide, etc. Semiconductor apparatus <NUM> may also include logic <NUM> comprised of, e.g., transistor array(s) and other integrated circuit (IC) components) coupled to the substrate(s) <NUM>. Logic <NUM> may implement system on chip (SoC) <NUM> described above with reference to <FIG>. Logic <NUM> may implement one or more aspects of the processes described above, including processes for hot object profiling described with reference to <FIG>, processes for object allocation redirection described with reference to <FIG> and <FIG>, processes for object migration described with reference to <FIG> and <FIG>. Logic <NUM> may implement one or more aspects of the processes for operating a managed runtime computing apparatus described above with reference to <FIG>.

Semiconductor apparatus <NUM> may be constructed using any appropriate semiconductor manufacturing processes or techniques. Logic <NUM> may be implemented at least partly in configurable logic or fixed-functionality hardware logic. For example, logic <NUM> may include transistor channel regions that are positioned (e.g., embedded) within substrate(s) <NUM>. Thus, the interface between logic <NUM> and substrate(s) <NUM> may not be an abrupt junction. Logic <NUM> may also be considered to include an epitaxial layer that is grown on an initial wafer of substrate(s) <NUM>.

<FIG> is a block diagram illustrating an example processor core <NUM> according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Processor core <NUM> may be the core for any type of processor, such as a microprocessor, an embedded processor, a digital signal processor (DSP), a network processor, or other device to execute code. Although only one processor core <NUM> is illustrated in <FIG>, a processing element may alternatively include more than one of the processor core <NUM> illustrated in <FIG>. Processor core <NUM> may be a single-threaded core or, for at least one embodiment, processor core <NUM> may be multithreaded in that it may include more than one hardware thread context (or "logical processor") per core.

<FIG> also illustrates a memory <NUM> coupled to processor core <NUM>. Memory <NUM> may be any of a wide variety of memories (including various layers of memory hierarchy) as are known or otherwise available to those of skill in the art. Memory <NUM> may include one or more code <NUM> instruction(s) to be executed by processor core <NUM>. Code <NUM> may implement one or more aspects of the processes described above, including processes for hot object profiling described with reference to <FIG>, processes for object allocation redirection described with reference to <FIG> and <FIG>, processes for object migration described with reference to <FIG> and <FIG>. Code <NUM> may implement one or more aspects of the processes for operating a managed runtime computing apparatus described above with reference to <FIG>. Processor core <NUM> follows a program sequence of instructions indicated by code <NUM>. Each instruction may enter a front end portion <NUM> and be processed by one or more decoders <NUM>. Decoder <NUM> may generate as its output a micro operation such as a fixed width micro operation in a predefined format, or may generate other instructions, microinstructions, or control signals which reflect the original code instruction. The illustrated front end portion <NUM> also includes register renaming logic <NUM> and scheduling logic <NUM>, which generally allocate resources and queue the operation corresponding to the convert instruction for execution.

Processor core <NUM> is shown including execution logic <NUM> having a set of execution units <NUM>-<NUM> through <NUM>-N. Some embodiments may include a number of execution units dedicated to specific functions or sets of functions. Other embodiments may include only one execution unit or one execution unit that can perform a particular function. The illustrated execution logic <NUM> performs the operations specified by code instructions.

After completion of execution of the operations specified by the code instructions, back end logic <NUM> retires the instructions of code <NUM>. In one embodiment, the processor core <NUM> allows out of order execution but requires in order retirement of instructions. Retirement logic <NUM> may take a variety of forms as known to those of skill in the art (e.g., re-order buffers or the like). In this manner, processor core <NUM> is transformed during execution of code <NUM>, at least in terms of the output generated by the decoder, the hardware registers and tables utilized by the register renaming logic <NUM>, and any registers (not shown) modified by the execution logic <NUM>.

Although not illustrated in <FIG>, a processing element may include other elements on chip with processor core <NUM>. For example, a processing element may include memory control logic along with processor core <NUM>. The processing element may include I/O control logic and/or may include I/O control logic integrated with memory control logic. The processing element may also include one or more caches.

<FIG> is a block diagram illustrating an example of a multi-processor based computing system <NUM> according to one or more embodiments, with reference to components and features described above including but not limited to the figures and associated description. Multiprocessor system <NUM> includes a first processing element <NUM> and a second processing element <NUM>. While two processing elements <NUM> and <NUM> are shown, it is to be understood that an embodiment of the system <NUM> may also include only one such processing element.

The system <NUM> is illustrated as a point-to-point interconnect system, wherein the first processing element <NUM> and the second processing element <NUM> are coupled via a point-to-point interconnect <NUM>. It should be understood that any or all of the interconnects illustrated in <FIG> may be implemented as a multi-drop bus rather than point-to-point interconnect.

As shown in <FIG>, each of processing elements <NUM> and <NUM> may be multicore processors, including first and second processor cores (i.e., processor cores 1374a and 1374b and processor cores 1384a and 1384b). Such cores 1374a, 1374b, 1384a, 1384b may be configured to execute instruction code in a manner similar to that discussed above in connection with <FIG>.

Each processing element <NUM>, <NUM> may include at least one shared cache 1896a, 1896b. The shared cache 1896a, 1896b may store data (e.g., instructions) that are utilized by one or more components of the processor, such as the cores 1374a, 1374b and 1384a, 1384b, respectively. For example, the shared cache 1896a, 1896b may locally cache data stored in a memory <NUM>, <NUM> for faster access by components of the processor. In one or more embodiments, the shared cache 1896a, 1896b may include one or more mid-level caches, such as level <NUM> (L2), level <NUM> (L3), level <NUM> (L4), or other levels of cache, a last level cache (LLC), and/or combinations thereof.

While shown with only two processing elements <NUM>, <NUM>, it is to be understood that the scope of the embodiments are not so limited. In other embodiments, one or more additional processing elements may be present in a given processor. Alternatively, one or more of processing elements <NUM>, <NUM> may be an element other than a processor, such as an accelerator or a field programmable gate array. For example, additional processing element(s) may include additional processors(s) that are the same as a first processor <NUM>, additional processor(s) that are heterogeneous or asymmetric to processor a first processor <NUM>, accelerators (such as, e.g., graphics accelerators or digital signal processing (DSP) units), field programmable gate arrays, or any other processing element. There can be a variety of differences between the processing elements <NUM>, <NUM> in terms of a spectrum of metrics of merit including architectural, micro architectural, thermal, power consumption characteristics, and the like. These differences may effectively manifest themselves as asymmetry and heterogeneity amongst the processing elements <NUM>, <NUM>. For at least one embodiment, the various processing elements <NUM>, <NUM> may reside in the same die package.

The first processing element <NUM> may further include memory controller logic (MC) <NUM> and point-to-point (P-P) interfaces <NUM> and <NUM>. Similarly, the second processing element <NUM> may include a MC <NUM> and P-P interfaces <NUM> and <NUM>. As shown in <FIG>, MC's <NUM> and <NUM> couple the processors to respective memories, namely a memory <NUM> and a memory <NUM>, which may be portions of main memory locally attached to the respective processors. While the MC <NUM> and <NUM> is illustrated as integrated into the processing elements <NUM>, <NUM>, for alternative embodiments the MC logic may be discrete logic outside the processing elements <NUM>, <NUM> rather than integrated therein.

The first processing element <NUM> and the second processing element <NUM> may be coupled to an I/O subsystem <NUM> via P-P interconnects <NUM> and <NUM>, respectively. As shown in <FIG>, the I/O subsystem <NUM> includes P-P interfaces <NUM> and <NUM>. Furthermore, I/O subsystem <NUM> includes an interface <NUM> to couple I/O subsystem <NUM> with a high performance graphics engine <NUM>. In one embodiment, bus <NUM> may be used to couple the graphics engine <NUM> to the I/O subsystem <NUM>. Alternately, a point-to-point interconnect may couple these components.

In turn, I/O subsystem <NUM> may be coupled to a first bus <NUM> via an interface <NUM>. In one embodiment, the first bus <NUM> may be a Peripheral Component Interconnect (PCI) bus, or a bus such as a PCI Express bus or another third generation I/O interconnect bus, although the scope of the embodiments are not so limited.

As shown in <FIG>, various I/O devices <NUM> (e.g., biometric scanners, speakers, cameras, sensors) may be coupled to the first bus <NUM>, along with a bus bridge <NUM> which may couple the first bus <NUM> to a second bus <NUM>. In one embodiment, the second bus <NUM> may be a low pin count (LPC) bus. Various devices may be coupled to the second bus <NUM> including, for example, a keyboard/mouse <NUM>, communication device(s) <NUM>, and a data storage unit <NUM> such as a disk drive or other mass storage device which may include code <NUM>, in one embodiment. The illustrated code <NUM> may implement one or more aspects of the processes described above, including processes for hot object profiling described with reference to <FIG>, processes for object allocation redirection described with reference to <FIG> and <FIG>, processes for object migration described with reference to <FIG> and <FIG>. Code <NUM> may implement one or more aspects of the processes for operating a managed runtime computing apparatus described above with reference to <FIG>. The illustrated code <NUM> may be similar to the code <NUM> (<FIG>), already discussed. Further, an audio I/O <NUM> may be coupled to second bus <NUM> and a battery <NUM> may supply power to the computing system <NUM>.

Note that other embodiments are contemplated. For example, instead of the point-to-point architecture of <FIG>, a system may implement a multi-drop bus or another such communication topology. Also, the elements of <FIG> may alternatively be partitioned using more or fewer integrated chips than shown in <FIG>.

Each of the systems and methods described above, and each of the embodiments (including implementations) thereof, for managing a runtime computer apparatus for tiered object memory placement is considered to be performance-enhanced at least to the extent that it includes identifying hot objects, allocating hot objects to a hot object heap in a first memory tier and allocating cold objects to a cold object heap in a second memory tier, and migrating objects between the hot object heap and the cold object heap based on an object hotness score. Apportioning objects between memory tiers based on an object's hotness promotes the ability to optimize or enhance performance and/or cost for a hybrid memory architecture. For example, allocating hot objects to a higher-performing memory tier will promote improved overall performance, since those objects accessed (or likely to be accessed) frequently will have a greater impact on performance compared to objects accessed less frequently. Similarly, allocating cold objects to a lower cost (but lower-performing) memory tier will have a lesser impact on overall performance, since any performance difference for infrequently-accessed objects would not be likely to add up to a significant difference in overall performance; yet allocating cold objects to a lower cost memory tier will be more cost effective than placing those cold objects in a more costly memory tier. Overall, the tiered object memory placement techniques described herein may provide for reduced cost of memory while minimizing any degradation of performance caused through use of lower-performing memory in a memory tier. Indeed, the enhanced performance may translate into better responsiveness and/or smoothness of applications and an improved user experience.

Example <NUM> includes an enhanced computing system for managing a runtime computing environment, comprising a processor, and a memory coupled to the processor, the memory including a set of instructions which, when executed by the processor, cause the computing system to assign a hotness score to an object having an object type based on an invocation count of objects referenced by a hot method, allocate a newly-created object to one of a hot object heap, said hot object heap assigned to store hot objects in a first memory tier, or a cold object heap, said cold object heap assigned to store cold objects in a second memory tier, based on the hotness score associated with the object type for the newly-created object, and migrate a plurality of objects between the hot object heap and the cold object heap based on a hotness score associated with each object, said object migration operating in an execution thread independent of an execution thread for said object allocation.

Example <NUM> includes the computing system of Example <NUM>, wherein the instructions, when executed, further cause the computing system to monitor the invocation frequency of objects in the hot object heap and the invocation frequency of objects in the cold object heap, and modify the hotness score of at least one object based on the monitored invocation frequency of that object.

Example <NUM> includes the computing system of Example <NUM>, wherein to migrate a plurality of objects between the hot object heap and the cold object heap, the instructions, when executed, cause the computing system to move a first object from the cold object heap to the hot object heap if the hotness score associated with the first object is greater than a threshold value, and move a second object from the hot object heap to the cold object heap if the hotness score associated with the second object is less than or equal to the threshold value.

Example <NUM> includes the computing system of Example <NUM>, wherein the threshold value is determined based on a relative capacity of the first tier memory and the second tier memory.

Example <NUM> includes the computing system of Example <NUM>, wherein to allocate a newly-created object to a hot object heap, the instructions, when executed, cause the computing system to replace a cold object allocator function with a hot object allocator function.

Example <NUM> includes the computing system of any one of Examples <NUM> to <NUM>, wherein the instructions, when executed, further cause the computing system to assign a predetermined hotness score for a second newly-created object, the second newly-created object created by the hot method, and allocate the second newly-created object to the hot object heap.

Example <NUM> includes a semiconductor apparatus for managing a runtime computing environment, comprising one or more substrates, and logic coupled to the one or more substrates, wherein the logic is implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, the logic coupled to the one or more substrates to assign a hotness score to an object having an object type based on an invocation count of objects referenced by a hot method, allocate a newly-created object to one of a hot object heap, said hot object heap assigned to store hot objects in a first memory tier, or a cold object heap, said cold object heap assigned to store cold objects in a second memory tier, based on the hotness score associated with the object type for the newly-created object, and migrate a plurality of objects between the hot object heap and the cold object heap based on a hotness score associated with each object, said object migration operating in an execution thread independent of an execution thread for said object allocation.

Example <NUM> includes the semiconductor apparatus of Example <NUM>, wherein the logic coupled to the one or more substrates is further to monitor the invocation frequency of objects in the hot object heap and the invocation frequency of objects in the cold object heap, and modify the hotness score of at least one object based on the monitored invocation frequency of that object.

Example <NUM> includes the semiconductor apparatus of Example <NUM>, wherein to migrate a plurality of objects between the hot object heap and the cold object heap, the logic coupled to the one or more substrates is to move a first object from the cold object heap to the hot object heap if the hotness score associated with the first object is greater than a threshold value, and move a second object from the hot object heap to the cold object heap if the hotness score associated with the second object is less than or equal to the threshold value.

Example <NUM> includes the semiconductor apparatus of Example <NUM>, wherein the threshold value is determined based on a relative capacity of the first tier memory and the second tier memory.

Example <NUM> includes the semiconductor apparatus of Example <NUM>, wherein to allocate a newly-created object to a hot object heap, the logic coupled to the one or more substrates is to replace a cold object allocator function with a hot object allocator function.

Example <NUM> includes the semiconductor apparatus of any one of Examples <NUM> to <NUM>, wherein the logic coupled to the one or more substrates is further to assign a predetermined hotness score for a second newly-created object, the second newly-created object created by the hot method, and allocate the second newly-created object to the hot object heap.

Example <NUM> includes at least one non-transitory computer readable storage medium comprising a set of instructions for managing a runtime computing environment which, when executed by a computing system, cause the computing system to assign a hotness score to an object having an object type based on an invocation count of objects referenced by a hot method, allocate a newly-created object to one of a hot object heap, said hot object heap assigned to store hot objects in a first memory tier, or a cold object heap, said cold object heap assigned to store cold objects in a second memory tier, based on the hotness score associated with the object type for the newly-created object, and migrate a plurality of objects between the hot object heap and the cold object heap based on a hotness score associated with each object, said object migration operating in an execution thread independent of an execution thread for said object allocation.

Example <NUM> includes the at least one non-transitory computer readable storage medium of Example <NUM>, wherein the instructions, when executed, further cause the computing system to monitor the invocation frequency of objects in the hot object heap and the invocation frequency of objects in the cold object heap, and modify the hotness score of at least one object based on the monitored invocation frequency of that object.

Example <NUM> includes the at least one non-transitory computer readable storage medium of Example <NUM>, wherein to migrate a plurality of objects between the hot object heap and the cold object heap, the instructions, when executed, cause the computing system to move a first object from the cold object heap to the hot object heap if the hotness score associated with the first object is greater than a threshold value, and move a second object from the hot object heap to the cold object heap if the hotness score associated with the second object is less than or equal to the threshold value.

Example <NUM> includes the at least one non-transitory computer readable storage medium of Example <NUM>, wherein the threshold value is determined based on a relative capacity of the first tier memory and the second tier memory.

Example <NUM> includes the at least one non-transitory computer readable storage medium of Example <NUM>, wherein to allocate a newly-created object to a hot object heap, the instructions, when executed, cause the computing system to replace a cold object allocator function with a hot object allocator function.

Example <NUM> includes the at least one non-transitory computer readable storage medium of any one of Examples <NUM> to <NUM>, wherein the instructions, when executed, further cause the computing system to assign a predetermined hotness score for a second newly-created object, the second newly-created object created by the hot method, and allocate the second newly-created object to the hot object heap.

Example <NUM> includes a method of operating a computing apparatus for managing a runtime computing environment, comprising assigning a hotness score to an object having an object type based on an invocation count of objects referenced by a hot method, allocating a newly-created object to one of a hot object heap, said hot object heap assigned to store hot objects in a first memory tier, or a cold object heap, said cold object heap assigned to store cold objects in a second memory tier, based on the hotness score associated with the object type for the newly-created object, and migrating a plurality of objects between the hot object heap and the cold object heap based on a hotness score associated with each object, said object migration operating in an execution thread independent of an execution thread for said object allocation.

Example <NUM> includes the method of Example <NUM>, further comprising monitoring the invocation frequency of objects in the hot object heap and the invocation frequency of objects in the cold object heap, and modifying the hotness score of at least one object based on the monitored invocation frequency of that object.

Example <NUM> includes the method of Example <NUM>, wherein migrating a plurality of objects between the hot object heap and the cold object heap comprises moving a first object from the cold object heap to the hot object heap if the hotness score associated with the first object is greater than a threshold value, and moving a second object from the hot object heap to the cold object heap if the hotness score associated with the second object is less than or equal to the threshold value.

Example <NUM> includes the method of Example <NUM>, wherein the threshold value is determined based on a relative capacity of the first tier memory and the second tier memory.

Example <NUM> includes the method of any one of Examples <NUM> to <NUM>, further comprising assigning a predetermined hotness score for a second newly-created object, the second newly-created object created by the hot method, and allocating the second newly-created object to the hot object heap.

Example <NUM> includes an apparatus for managing a runtime computing environment, comprising means for assigning a hotness score to an object having an object type based on an invocation count of objects referenced by a hot method, means for allocating a newly-created object to one of a hot object heap, said hot object heap assigned to store hot objects in a first memory tier, or a cold object heap, said cold object heap assigned to store cold objects in a second memory tier, based on the hotness score associated with the object type for the newly-created object, and means for migrating a plurality of objects between the hot object heap and the cold object heap based on a hotness score associated with each object, said object migration operating in an execution thread independent of an execution thread for said object allocation.

Example <NUM> includes the apparatus of Example <NUM>, further comprising means for monitoring heap objects and updating a hotness score for monitored objects.

Example <NUM> includes the method of Example <NUM>, wherein allocating a newly-created object to a hot object heap comprises replacing a cold object allocator function with a hot object allocator function.

Thus, technology described herein improves the performance of managed runtime computer environments through implementing tiered object memory placement by identifying hot objects, allocating hot objects to a hot object heap in a first memory tier and allocating cold objects to a cold object heap in a second memory tier, and migrating objects between the hot object heap and the cold object heap based on an object hotness score. Apportioning objects between memory tiers based on an object's hotness promotes the ability to optimize or enhance performance and/or cost for a hybrid memory architecture. For example, allocating hot objects to a higher-performing memory tier will promote improved overall performance, since those objects accessed (or likely to be accessed) frequently will have a greater impact on performance compared to objects accessed less frequently. Similarly, allocating cold objects to a lower cost (but lower-performing) memory tier will have a lesser impact on overall performance, since any performance difference for infrequently-accessed objects would not be likely to add up to a significant difference in overall performance; yet allocating cold objects to a lower cost memory tier will be more cost effective than placing those cold objects in a more costly memory tier. Overall, the tiered object memory placement techniques described herein may provide for reduced cost of memory while minimizing any degradation of performance caused through use of lower-performing memory in a memory tier. Indeed, the enhanced performance may translate into better responsiveness and/or smoothness of applications and an improved user experience. The technology described herein may be applicable in any number of managed runtime environments, including servers, cloud computing, browsers, and/or environments using JIT or AOT (ahead of time) compilation/processing.

Embodiments are applicable for use with all types of semiconductor integrated circuit ("IC") chips. Examples of these IC chips include but are not limited to processors, controllers, chipset components, programmable logic arrays (PLAs), memory chips, network chips, systems on chip (SoCs), SSD/NAND controller ASICs, and the like. In addition, in some of the drawings, signal conductor lines are represented with lines. Some may be different, to indicate more constituent signal paths, have a number label, to indicate a number of constituent signal paths, and/or have arrows at one or more ends, to indicate primary information flow direction. This, however, should not be construed in a limiting manner. Rather, such added detail may be used in connection with one or more exemplary embodiments to facilitate easier understanding of a circuit. Any represented signal lines, whether or not having additional information, may actually comprise one or more signals that may travel in multiple directions and may be implemented with any suitable type of signal scheme, e.g., digital or analog lines implemented with differential pairs, optical fiber lines, and/or single-ended lines.

Example sizes/models/values/ranges may have been given, although embodiments are not limited to the same. As manufacturing techniques (e.g., photolithography) mature over time, it is expected that devices of smaller size could be manufactured. In addition, well known power/ground connections to IC chips and other components may or may not be shown within the figures, for simplicity of illustration and discussion, and so as not to obscure certain aspects of the embodiments. Further, arrangements may be shown in block diagram form in order to avoid obscuring embodiments, and also in view of the fact that specifics with respect to implementation of such block diagram arrangements are highly dependent upon the platform within which the embodiment is to be implemented, i.e., such specifics should be well within purview of one skilled in the art. Where specific details (e.g., circuits) are set forth in order to describe example embodiments, it should be apparent to one skilled in the art that embodiments can be practiced without, or with variation of, these specific details. The description is thus to be regarded as illustrative instead of limiting.

The term "coupled" may be used herein to refer to any type of relationship, direct or indirect, between the components in question, and may apply to electrical, mechanical, fluid, optical, electromagnetic, electromechanical or other connections. In addition, the terms "first", "second", etc. may be used herein only to facilitate discussion, and carry no particular temporal or chronological significance unless otherwise indicated.

As used in this application and in the claims, a list of items joined by the term "one or more of" may mean any combination of the listed terms. For example, the phrases "one or more of A, B or C" may mean A, B, C; A and B; A and C; B and C; or A, B and C.

Claim 1:
A semiconductor apparatus (<NUM>) for managing a runtime computing environment, comprising:
one or more substrates (<NUM>); and
logic (<NUM>) coupled to the one or more substrates (<NUM>), wherein the logic is adapted to be implemented at least partly in one or more of configurable logic or fixed-functionality hardware logic, the logic (<NUM>) coupled to the one or more substrates (<NUM>) to:
assign a hotness score to an object having an object type based on an invocation count of objects referenced by a hot method;
wherein a method is identified as hot method based on a frequency of method calls; and
allocate a newly-created object to one of a hot object heap (<NUM>; <NUM>), said hot object heap (<NUM>; <NUM>) assigned to store hot objects in a first memory tier (<NUM>), or a cold object heap (<NUM>; <NUM>), said cold object heap (<NUM>; <NUM>) assigned to store cold objects in a second memory tier (<NUM>), based on the hotness score associated with the object type for the newly-created object; and
migrate a plurality of objects between the hot object heap (<NUM>; <NUM>) and the cold object heap (<NUM>; <NUM>) based on a hotness score associated with each object, said object migration operating in an execution thread independent of an execution thread for said object allocation.