Source: http://www.google.co.uk/patents/US9262333
Timestamp: 2018-01-16 13:35:55
Document Index: 121511182

Matched Legal Cases: ['Application No. 60', 'art 1100', 'art 1200', 'art 1100', 'art 1200', 'art 1100']

Patent US9262333 - Asymmetric memory migration in hybrid main memory - Google Patents
Main memory is managed by receiving a command from an application to read data associated with a virtual address that is mapped to the main memory. A memory controller determines that the virtual address is mapped to one of the symmetric memory components of the main memory, and accesses memory use characteristics...http://www.google.co.uk/patents/US9262333?utm_source=gb-gplus-sharePatent US9262333 - Asymmetric memory migration in hybrid main memory
Publication number US9262333 B2
Application number US 14/047,603
Priority date 4 Nov 2006
Also published as US7774556, US7818489, US7913055, US7930513, US8156288, US8156302, US8205061, US8266407, US8555002, US8555024, US8639910, US8782373, US9223719, US9262334, US9672158, US20080109592, US20080109593, US20080109629, US20080177978, US20100325383, US20110022788, US20110167205, US20110173371, US20120198140, US20120198141, US20120260030, US20130007338, US20140258603, US20140258653, US20150012721, US20160117131, US20160117258, WO2008055269A2, WO2008055269A3, WO2008055270A2, WO2008055270A3, WO2008055271A2, WO2008055271A3, WO2008055272A2, WO2008055272A3
Publication number 047603, 14047603, US 9262333 B2, US 9262333B2, US-B2-9262333, US9262333 B2, US9262333B2
Patent Citations (54), Non-Patent Citations (15), Referenced by (3), Classifications (32), Legal Events (2)
US 9262333 B2
determining, based on memory use characteristics of the data associated with the virtual address, that the data associated with the virtual address is suitable for access on an asymmetric memory component of the main memory;
2. The method of claim 1 wherein the symmetric memory component includes dynamic random access memory (DRAM) and the asymmetric memory component includes NOR flash, where the DRAM is configured to enable random access write operations in which an address within a block of the DRAM is written without affecting the availability of other addresses within the block of the DRAM, and where the NOR flash is configured to enable block write operations in which an address within a block of one of the NOR flash affects the availability of other addresses within the block of the NOR flash, and further comprising:
determining, using the memory management unit, that the virtual address is mapped to one of the DRAM blocks;
determining, based on the memory use characteristics of the data associated with the virtual address, that the data associated with the virtual address is suitable for access on NOR flash of the main memory;
loading, using the CPU, the data associated with the virtual address to the NOR flash of the main memory;
retrieving, responsive to the received command, the data associated with the virtual address from the NOR flash of the main memory.
3. The method of claim 1 wherein determining that the data associated with the virtual address is suitable for access on the asymmetric memory component of the main memory includes determining that the access characteristics are related to a profile that indicates whether the data should be loaded to the asymmetric memory component based on a number of times the data has been read and the number of times that the data has been updated using a write instruction.
the asymmetric memory component has asymmetric access characteristics including:
(1) the asymmetric memory component performs read operations at a first rate and performs write operations at a second rate that is more than an order of magnitude different than the first rate,
(2) the asymmetric memory component uses an asymmetric memory access protocol, and
(3) the asymmetric memory component includes non-volatile storage capability;
and the symmetric memory component has symmetric access characteristics including:
(1) the symmetric memory component performs read operations at a rate that that differs by less than an order of magnitude from a rate at which write operations are performed,
(2) the symmetric memory component uses a symmetric memory access protocol, and
(3) the symmetric memory component includes a volatile storage capability.
5. The method of claim 1 wherein loading the data associated with the virtual address to the asymmetric memory component of main memory includes loading the data as part of a block write that includes additional content that is different than the data.
identifying, from a first read instruction, the data to be loaded to the asymmetric memory component as a first portion;
storing the first portion in the symmetric memory component of the main memory;
identifying, from a second read instruction, other content to be loaded to the symmetric memory component as a second portion;
storing the second portion in the symmetric memory component of the main memory;
structuring the first portion and the second portion as a block to be written to the asymmetric memory component; and
writing the block that includes the first portion and the second portion to the asymmetric memory component.
7. The method of claim 1 wherein determining that the data associated with the virtual address is suitable for access on the asymmetric memory component includes:
determining that an application is operating in a constrained environment for memory resources; and
adjusting constrained metrics used in determining whether to load the data to the asymmetric memory component to increase a likelihood that the data is loaded the asymmetric memory component.
8. The method of claim 1 wherein determining that the data associated with the virtual address is suitable for access on the asymmetric memory component includes:
determining that an application is unconstrained for memory resources; and
using unconstrained metrics to determine whether to load the data to the asymmetric memory component.
9. The method of claim 1 further comprising setting a write protection bit for the data that has been loaded to asymmetric memory that regulates subsequent write access to the virtual address for the data.
10. The method of claim 1 wherein determining, based on the memory use characteristics, that the data associated with the virtual address is suitable for access on the asymmetric memory component of the main memory includes determining that a first threshold number of reads of the data has occurred in an observation window.
11. The method of claim 1 further comprising determining that the data associated with the virtual address is not suitable for access on the asymmetric memory component of the main memory in response to determining that a second threshold number of updates to the data has been requested in an observation window.
12. A system that manages a main memory that includes symmetric memory components associated with a first collection of memory operations and asymmetric memory components associated with a second collection of memory operations, where the symmetric memory components are each configured to enable random access write operations in which an address within a block of the symmetric memory component is written without affecting the availability of other addresses within the block of the symmetric memory component, and where the asymmetric memory components are each configured to enable block write operations in which an address within a block of one of the asymmetric memory components affects the availability of other addresses within the block of the asymmetric memory component, the system instructions that when executed on a processor perform operations that include:
13. The system of claim 12 wherein the symmetric memory component includes dynamic random access memory (DRAM) and the asymmetric memory component includes NOR flash, where the DRAM is configured to enable random access write operations in which an address within a block of the DRAM is written without affecting the availability of other addresses within the block of the DRAM, and where the NOR flash is configured to enable block write operations in which an address within a block of one of the NOR flash affects the availability of other addresses within the block of the NOR flash, and further comprising:
14. The system of claim 12 wherein determining that the data associated with the virtual address is suitable for access on the asymmetric memory component of the main memory includes determining that the access characteristics are related to a profile that indicates whether the data should be loaded to the asymmetric memory component based on a number of times the data has been read and the number of times that the data has been updated using a write instruction.
(3) the asymmetric memory component includes non-volatile storage capability; and
the symmetric memory component has symmetric access characteristics including:
16. The system of claim 12 wherein loading the data associated with the
virtual address to the asymmetric memory component of main memory includes loading the data as part of a block write that includes additional content that is different than the data.
18. The system of claim 12 wherein determining that the data associated with the virtual address is suitable for access on the asymmetric memory component includes:
19. The system of claim 12 wherein determining that the data associated with the virtual address is suitable for access on the asymmetric memory component includes:
20. A system that manages a main memory that includes symmetric memory components associated with a first collection of memory operations and asymmetric memory components associated with a second collection of memory operations, where the symmetric memory components are each configured to enable random access write operations in which an address within a block of the symmetric memory component is written without affecting the availability of other addresses within the block of the symmetric memory component, and where the asymmetric memory components are each configured to enable block write operations in which an address within a block of one of the asymmetric memory components affects the availability of other addresses within the block of the asymmetric memory component, the system instructions that when executed on a processor perform operations that include:
means for receiving, on a central processing unit (CPU), a command from an application to read data associated with a virtual address that is mapped to the main memory;
means for determining, using the memory management unit, that the virtual address is mapped to one of the symmetric memory components of the main memory;
means for determining, based on memory use characteristics of the data associated with the virtual address, that the data associated with the virtual address is suitable for access on an asymmetric memory component of the main memory;
means for loading, using the CPU, the data associated with the virtual address to the asymmetric memory component of the main memory;
means for receiving, after the loading and using the memory management unit, a command from the application to read the data associated with the virtual address; and
means for retrieving, responsive to the received command, the data associated with the virtual address from the asymmetric memory component of the main memory.
This application is a continuation of U.S. patent application Ser. No. 13/443,086, filed Apr. 10, 2012, now allowed, and entitled “ASYMMETRIC MEMORY MIGRATION IN HYBRID MAIN MEMORY,” which is a continuation of U.S. patent application Ser. No. 12/853,135, filed Aug. 9, 2010, and entitled “ASYMMETRIC MEMORY MIGRATION IN HYBRID MAIN MEMORY,” now U.S. Pat. No. 8,156,288, which is a continuation of U.S. patent application Ser. No. 11/935,224, filed Nov. 5, 2007, and entitled “ASYMMETRIC MEMORY MIGRATION IN HYBRID MAIN MEMORY,” now U.S. Pat. No. 7,774,556, which claims priority to U.S. Provisional Application No. 60/864,399, entitled, “MEMORY MANAGEMENT OF HYBRID MAIN MEMORY INCLUDING SYMMETRIC AND ASYMMETRIC MEMORY,” and filed on Nov. 4, 2006. The contents of the prior applications are incorporated herein by reference in their entirety.
Referring now to FIG. 1, an example of a first layer model 100 of a computing system is illustrated. The first layer model 100 includes an application (AP) layer 101, an operating system (OS) layer 102, and a hardware (HW) layer 103. The AP layer 100 includes one or more application processes AP 1 110A through APN 110N.
Although other application processes may be employed, each application process (AP 1 110A to APN 110N) loaded into memory may be allocated space in the main memory including a text region 111, a data region 112, and a stack region 113. The text region 111 may include executable instructions associated with the application program. The data region 112 may include data associated with the application. The data may be dynamic and/or static. The stack region 113 may include a state of a function or procedure of the application program and its activation frame.
Flow chart 1100 illustrates the logic used in deciding whether to load data to asymmetric memory. FIG. 12 is a flow chart 1200 of a process by which data is selectively and intelligently loaded to an asymmetric memory component using a hypervisor . . . . The operations shown in flow chart 1100 and 1200 should be seen as complementary operations in that the operations shown in flow chart 1200 are designed to benefit from operations previously performed in flow chart 1100 in deciding whether to load the data to asymmetric memory.
The computer system then uses the unconstrained profile and the access characteristics for the data to determine whether to load the data to asymmetric memory (1450). Thus, where the unconstrained profile requires that a perceived physical address be read more than five times in an observation widow and not been written to during the same observation window, a first perceived physical address that has been read twice is not loaded to asymmetric memory. A second perceived physical address that has been read ten times and written once also is not loaded to asymmetric memory. A third perceived physical address that has been read seven times and never been written to since data has been loaded to the perceived physical address meets the criteria required in the unconstrained profile. As a result, the computer system loads the data associated with the third perceived physical address to asymmetric memory.
US5012408 15 Mar 1990 30 Apr 1991 Digital Equipment Corporation Memory array addressing system for computer systems with multiple memory arrays
US5404485 8 Mar 1993 4 Apr 1995 M-Systems Flash Disk Pioneers Ltd. Flash file system
US6134619 * 3 Jun 1999 17 Oct 2000 Intel Corporation Method and apparatus for transporting messages between processors in a multiple processor system
US6226734 * 10 Jun 1998 1 May 2001 Compaq Computer Corporation Method and apparatus for processor migration from different processor states in a multi-processor computer system
US6393545 5 Jun 2001 21 May 2002 Canon Kabushiki Kaisha Method apparatus and system for managing virtual memory with virtual-physical mapping
US6549959 4 Nov 1999 15 Apr 2003 Ati International Srl Detecting modification to computer memory by a DMA device
US6564326 24 Oct 2001 13 May 2003 Walter A. Helbig, Sr. Method and apparatus for enhancing computer system security
US7761623 30 Aug 2007 20 Jul 2010 Virident Systems, Inc. Main memory in a system with a memory controller configured to control access to non-volatile memory, and related technologies
US7761624 30 Aug 2007 20 Jul 2010 Virident Systems, Inc. Systems and apparatus for main memory with non-volatile type memory modules, and related technologies
US7761625 30 Aug 2007 20 Jul 2010 Virident Systems, Inc. Methods for main memory with non-volatile type memory modules, and related technologies
US7761626 30 Aug 2007 20 Jul 2010 Virident Systems, Inc. Methods for main memory in a system with a memory controller configured to control access to non-volatile memory, and related technologies
US7774556 5 Nov 2007 10 Aug 2010 Virident Systems Inc. Asymmetric memory migration in hybrid main memory
US7818489 5 Nov 2007 19 Oct 2010 Virident Systems Inc. Integrating data from symmetric and asymmetric memory
US7913055 5 Nov 2007 22 Mar 2011 Virident Systems Inc. Seamless application access to hybrid main memory
US7930513 5 Nov 2007 19 Apr 2011 Virident Systems Inc. Writing to asymmetric memory
US8156288 9 Aug 2010 10 Apr 2012 Virident Systems Inc. Asymmetric memory migration in hybrid main memory
US8555002 10 Apr 2012 8 Oct 2013 Virident Systems Inc. Asymmetric memory migration in hybrid main memory
US8638805 * 30 Sep 2011 28 Jan 2014 Lsi Corporation Packet draining from a scheduling hierarchy in a traffic manager of a network processor
US20020051350 3 Apr 2001 2 May 2002 Mitsubishi Denki Kabushiki Kaisha Load adjustment board and data processing apparatus
US20020118593 29 Jan 2002 29 Aug 2002 Fujitsu Limited Memory system
US20020133684 17 Jan 2002 19 Sep 2002 Honeywell International, Inc. Enhanced memory module architecture
US20020138600 26 Mar 2001 26 Sep 2002 International Business Machines Corporation Method, apparatus and program for multi-machine network install using writeable media
US20030137862 13 Jan 2003 24 Jul 2003 Brunelle Steven J. Reversed memory module socket, motherboard and test system including same, and method of modifying motherboard
US20030188082 29 Mar 2002 2 Oct 2003 D-Link Corporation Method for implementing PC Card bus compatible element on application PCI bus based apparatus and circuits therefor
US20030188083 27 Mar 2002 2 Oct 2003 Kumar Mohan J. Method and apparatus for setting timing parameters
US20040117581 9 Jul 2003 17 Jun 2004 Samsung Electronics Co., Ltd. Computer system and control method thereof
US20040160835 19 Feb 2004 19 Aug 2004 Altman Erik R. Symmetric multi-processing system
US20040186948 24 Mar 2004 23 Sep 2004 Lofgren Karl M. J. Device and method for controlling solid-state memory system
US20040193783 26 Mar 2003 30 Sep 2004 Manish Sharma Computer system with operating system permitting dynamic reallocation of main memory
US20040240745 27 Feb 2004 2 Dec 2004 Ntt Docomo, Inc. Image encoding apparatus, image encoding method, image encoding program, image decoding apparatus, image decoding method and image decoding program
US20050240745 17 Dec 2004 27 Oct 2005 Sundar Iyer High speed memory control and I/O processor system
US20050243570 25 Apr 2005 3 Nov 2005 Chaves Julio C Optical manifold for light-emitting diodes
US20050251617 7 May 2004 10 Nov 2005 Sinclair Alan W Hybrid non-volatile memory system
US20050273570 3 Jun 2004 8 Dec 2005 Desouter Marc A Virtual space manager for computer having a physical address extension feature
US20060106984 18 Nov 2004 18 May 2006 International Business Machines Corporation Methods and apparatus for efficient memory usage
US20060230250 12 Apr 2005 12 Oct 2006 Nokia Corporation Memory interface for volatile and non-volatile memory devices
US20080109626 * 31 Oct 2007 8 May 2008 Kai-Li Lee Method of accessing data in a deinterleaving device
US20080109629 5 Nov 2007 8 May 2008 Virident Systems, Inc. Asymmetric memory migration in hybrid main memory
US20080177978 5 Nov 2007 24 Jul 2008 Virident Systems Inc. Integrating data from symmetric and asymmetric memory
US20080263348 9 Apr 2008 23 Oct 2008 Texas Instruments Incorporated Dynamic asymmetric partitioning of program code memory in network connected devices
US20090106478 20 Oct 2008 23 Apr 2009 Virident Systems Inc. Managing Memory Systems Containing Components with Asymmetric Characteristics
US20090157989 15 Dec 2008 18 Jun 2009 Virident Systems Inc. Distributing Metadata Across Multiple Different Disruption Regions Within an Asymmetric Memory System
US20100325383 * 9 Aug 2010 23 Dec 2010 Virident Systems Inc. Asymmetric memory migration in hybrid main memory
US20110022788 7 Oct 2010 27 Jan 2011 Virident Systems Inc. Integrating data from symmetric and asymmetric memory
US20110173371 22 Mar 2011 14 Jul 2011 Virident Systems Inc. Writing to asymmetric memory
US20120198140 10 Apr 2012 2 Aug 2012 Virident Systems Inc. Asymmetric memory migration in hybrid main memory
US20120198141 10 Apr 2012 2 Aug 2012 Virident Systems Inc. Integrating data from symmetric and asymmetric memory
1 International Search Report and Written Opinion for International Application No. PCT/US07/79622, dated Apr. 9, 2008, 12 pages.
2 International Search Report and Written Opinion for International Application No. PCT/US07/80015, dated Aug. 13, 2008, 13 pages.
3 International Search Report and Written Opinion for International Application No. PCT/US07/83666, dated Apr. 21, 2008, 9 pages.
4 International Search Report and Written Opinion for International Application No. PCT/US07/83668, dated May 15, 2008, 8 pages.
5 International Search Report and Written Opinion for International Application No. PCT/US07/83669, dated Apr. 30, 2008, 9 pages.
6 International Search Report and Written Opinion for International Application No. PCT/US07/83671, dated May 22, 2008, 14 pages.
7 Lee, H.G., and Chang, N., "Energy-Aware Memory Allocation in Heterogeneous Non-Volatile Memory Systems," Low Power Electronics and Design, Aug. 2003, retrieved from http://ieeexplore.ieee.org/ie15/8720/27594/01231941.pdf?tp=&isnumber=&arnumber=12394> on May 1, 2008.
8 Office Action for U.S. Appl. No. 11/864,763, mailed Jun. 16, 2008, 11 pages.
9 Tseng, H.W., et al., "An Energy-Efficient Virtual Memory System with Flash Memory as the Secondary Storage," Low Power Electronics and Design, Oct. 4-6, 2006, retrieved from http://ieeexplore.ieee.org/xpls/abs-all.jsp?arnumber=4271879> on May 1, 2008.
10 U.S. Final Office Action for U.S. Appl. No. 12/853,135 dated Jun. 22, 2011, 10 pages.
11 U.S. Non-Final Office Action for U.S. Appl. No. 12/853,135 dated Oct. 8, 2010, 11 pages.
12 U.S. Non-Final Office Action for U.S. Appl. No. 13/443,086 dated Dec. 26, 2012, 10 pages.
13 U.S. Notice of Allowance for U.S. Appl. No. 11/935,224 dated Apr. 5, 2010, 10 pages.
14 U.S. Notice of Allowance for U.S. Appl. No. 12/853,135 dated Dec. 5, 2011, 5 pages.
15 U.S. Notice of Allowance for U.S. Appl. No. 13/443,086 dated Jun. 5, 2013, 6 pages.
US9672158 * 7 Jan 2016 6 Jun 2017 Virident Systems Inc. Asymmetric memory migration in hybrid main memory
US9686191 * 21 Aug 2015 20 Jun 2017 KnuEdge Incorporation Performing read operations in network on a chip architecture
US20170054635 * 21 Aug 2015 23 Feb 2017 Knu Edge Incorporated Performing Read Operations in Network on a Chip Architecture
International Classification G06F12/10, G06F13/16, G06F12/00, G06F12/12, G06F12/14, G06F9/50, G06F12/08, G06F3/06
Cooperative Classification Y02B60/1228, G06F3/0647, G06F3/0604, G06F12/0207, G06F12/1009, G06F12/10, G06F3/064, G06F13/1657, G06F3/0638, G06F2212/205, G06F3/0646, G06F3/068, G06F12/0223, G06F12/0292, G06F3/0628, G06F3/0685, G06F12/121, Y02B60/1225, G06F2212/657, G06F12/08, Y02B60/142, G06F13/1694, G06F9/5016, G06F12/1475
22 Sep 2014 AS Assignment
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KARAMCHETI, VIJAY;OKIN, KENNETH A.;GANAPATHY, KUMAR;AND OTHERS;SIGNING DATES FROM 20140426 TO 20140725;REEL/FRAME:033788/0562
Free format text: CHANGE OF NAME;ASSIGNOR:VIRIDENT SYSTEMS, INC.;REEL/FRAME:042456/0710