Patent Description:
A System-on-a-Chip (SoC) embodies technology integrating complex systems with various functions into a single semiconductor chip. There is a converging trend of integrating computers, communications, and broadcasting. Uses for an application specific integrated circuit (ASIC) and an application specific standard product (ASSP) are each shifting to SoC technology. In addition, miniaturization and weight reduction of information technology (IT) devices are driving SoC-related businesses.

As mobile applications develop, the use of processors and memory is increasing. Thus, a new SoC may be desired for supporting the usage of service software for users while minimizing the increase in processor and memory usage within limited power consumption design specifications.

<CIT> discloses forwarding a memory transaction to a device based on a shadow address mapping table.

<CIT> discloses accessing a device based on a shadow address matching a device address.

Embodiments will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings in which:.

Hereinafter, the inventive concept is described in detail by way of non-limiting embodiments with reference to the accompanying drawings. The components described by referring to terms such as parts or units, modules, blocks, or the like used in the detailed description and the functional blocks illustrated in the drawings may be implemented in the form of software, hardware, or a combination thereof. For example, the software may be machine code, firmware, embedded code, and/or application software. For example, the hardware may include electrical circuits, electronic circuits, processors, computers, integrated circuits, integrated circuit cores, pressure sensors, inertial sensors, microelectromechanical systems (MEMS), passive devices, and/or a combination thereof.

<FIG> illustrates a System-on-a-Chip (SoC) according to an embodiment.

A SoC <NUM> may be mounted on an electronic device, and for example, the electronic device may include a mobile device such as a smartphone, a tablet, a personal computer (PC), a personal digital assistant (PDA), a portable multimedia player (PMP), a laptop computer, a wearable device, a global positioning system (GPS) device, an electronic book terminal, a digital broadcast terminal, an MP3 player, a digital camera, a wearable computer, a navigation system, a drone, or the like. For example, the electronic device may also include an internet of things (IoT) device, a home appliance, and/or an advanced driver assistance system (ADAS). The SoC <NUM> may include a controller and a processor that controls an operation of the electronic device. The SoC <NUM> may refer to an application processor (AP), a mobile AP, and/or a control chip.

The SoC <NUM> may be mounted in an electronic device such as a camera, a smartphone, a wearable device, an internet of things (IoT) device, a home appliance, a tablet PC, a PDA, a PMP, a navigation system, a drone, an ADAS, or the like.

In addition, the SoC <NUM> may be mounted on electronic devices equipped as components in vehicles, furniture, manufacturing facilities, doors, and various measurement devices.

Referring to <FIG>, the SoC <NUM> may include a processor <NUM>, a sub-processing circuit <NUM>, a memory <NUM>, intellectual property cores (IPs) such as IP1 <NUM> and/or IP2 <NUM>, and a system bus <NUM>. The SoC <NUM> may further include a communications function module, an image sensor module, or the like. The components of the SoC <NUM>, such as the sub-processing circuit <NUM>, the memory <NUM>, and the IPs <NUM> and <NUM> may transmit and receive data via the system bus <NUM>.

The processor <NUM> may be a main processor of the SoC <NUM>, and may control an overall operation of the SoC <NUM>. The processor <NUM> may run an operating system (OS) and execute various applications (application software) of the electronic device on which the SoC <NUM> is mounted. The processor <NUM> may process various types of arithmetic operations and/or logical operations, for example. The processor <NUM> may include a single processor core (single core) or multiple processor cores (multi-core). The processor <NUM> may include cache memories used for each of one or more processor cores to perform various operations. Cache memories may temporarily store instructions and/or parameter values used for the processor <NUM> to execute an application.

The processor <NUM> may store data in the memory <NUM> or read data from the memory <NUM> in a process of executing an operating system and applications. For example, the application may read data from the memory <NUM>, process the read data, and store the processed data back into the memory <NUM>. The processor <NUM> may send an access request for reading or writing to the memory <NUM> to read data from the memory <NUM> and write the processed data to the memory <NUM>, and the access request may include a physical address indicating an area in which data is stored or to be written among storage areas of the memory <NUM>.

The application may read data from a virtual address space, which is provided in a virtual space, or write data into the virtual address space, and the processor <NUM> may convert, such as by using a page table (e.g., PGTB in <FIG>), a virtual address (which may be referred to as a logical address) indicating a virtual address space into a physical address indicating one of multiple address areas of an effective physical address space of the memory <NUM> in which data is stored or to be stored.

In an embodiment, the page table may be used to map the virtual address to either an effective physical address space of the memory <NUM>, that is, a physical address representing one of address areas included in an actual physical address space, or a physical address indicating one of address areas deviating from the effective physical address space (e.g., shadow physical address space). Hereinafter, the physical address corresponding to the effective physical address space of the memory <NUM> is referred to as an 'effective physical address', and the physical address corresponding to a shadow physical address space is referred to as a 'shadow physical address'.

<FIG> illustrates a page table according to an embodiment, and <FIG> illustrates address areas according to an embodiment.

Referring to <FIG>, a page table PGTB may include a virtual address VA and a physical address PA. The virtual address VA may indicate address areas of the virtual address space VAS, and the physical address PA may indicate corresponding address areas of an effective physical address space EPAS and corresponding address areas of a shadow physical address space SPAS. The virtual address space VAS may include an address space provided by a virtual memory technology and may be recognized by the operating system and the applications executing in the processor (<NUM> in <FIG>). The effective physical address space EPAS may have the same size as a system memory, such as, for example, the memory <NUM>, without limitation thereto. The shadow physical address space SPAS may include an area outside of the system memory.

Referring to <FIG>, when the memory <NUM> has a capacity of <NUM> gigabytes (GB) and the SoC <NUM> is a <NUM>-bit system, the address area may include a first address area AR1 with <NUM> GB and an address of 0x0000_0000 to 0x7fff_ffff, a second address area AR2 with <NUM> GB and an address of 0x8000_0000 to 0xbfff_ffff, and a third address area AR3 with <NUM> GB and an address of 0xc000_0000 to 0xffff_ffff. The first address area AR1 may be set as the effective physical address space EPAS, and one of the address areas outside the first address area AR1, such as, for example, the third address area AR3, may be set as the shadow physical address space SPAS.

In an embodiment, the virtual address space VAS may be divided into a pages (e.g., PN0 to PNn, where n is an integer of <NUM> or more), and each of the pages PN0 to PNn may be an address area indicated by the virtual address VA. A size of each page PN0 to PNn may be <NUM> KB, but is not limited thereto.

The effective physical address space EPAS and the shadow physical address space SPAS may be divided into frames FN0 to FNn, and the physical address PA corresponding to the virtual address VA may indicate the frames FN0 to FNn. Some of the frames FN0 to FNn may be provided as address areas of the effective physical address space EPAS (hereinafter referred to as an effective physical address area), and others may be provided as address areas of the shadow physical address space SPAS (hereinafter referred to as a shadow physical address area). For example, frames FN1, FN7, and FN8 in <FIG> may be effective physical address areas, and frames FNn-<NUM> and FNn may be shadow physical address areas. Among the physical addresses PA, an effective physical address may indicate one of the effective physical address areas, such as frames FN1, FN7, or FN8, and a shadow physical address may indicate one of the shadow physical address areas, such as frames FNn-<NUM> or FNn.

The virtual addresses VA of the page table PGTB may sequentially correspond to the pages PN0 to PNn, and the physical addresses PA mapped to the virtual addresses VA may sequentially or non-sequentially correspond to the frames FN0 to FNn. The connection between the virtual addresses VA and the physical addresses PA may not be permanent and may be disconnected or adjusted according to various events.

Referring back to <FIG>, when accessing the memory <NUM>, the processor <NUM> may generate, as an access address, an effective physical address or a shadow physical address mapped to the virtual address, based on the page table PGTB in <FIG>, and may output an access request having the access address, to the system bus <NUM>. The processor <NUM> may have direct access to the memory <NUM> based on the effective physical address. In addition, the processor <NUM> may have indirect access to the memory <NUM> via the sub-processing circuit <NUM> based on the shadow physical address. The indirect access to the memory <NUM> may be described below in greater detail with reference to <FIG>.

The sub-processing circuit <NUM> may support a function that the processor <NUM> does not provide, such as, for example, data processing, for data to be read by the application from the memory <NUM> or data to be written into the memory <NUM>. The sub-processing circuit <NUM> may convert the shadow physical address into the effective physical address indicating the physical address area of the memory <NUM>, and read data from the memory <NUM> based on the effective physical address. The sub-processing circuit <NUM> may process data and transmit the processed data to the processor <NUM>. In addition, the sub-processing circuit <NUM> may process data requested by the processor <NUM> for writing together with the shadow physical address, convert the shadow physical address into the effective physical address, and write (store) the processed data into the memory <NUM> based on the effective physical address.

In an embodiment, the sub-processing circuit <NUM> may include a compressor and decompressor and may compress or decompress received data. When the data that is stored in the memory <NUM> that is to be read by the application is compressed data, even though the data is read from the memory <NUM>, the application might not be able to use the data unless the processor <NUM> provides a decompression function. The sub-processing circuit <NUM> may read data from the memory <NUM> based on the shadow physical address and decompress the read data, and transmit the decompressed data to the processor <NUM>. Moreover, the sub-processing circuit <NUM> may compress the data provided from the processor <NUM> together with the shadow physical address and write (store) the compressed data into the memory <NUM>.

In an embodiment, the sub-processing circuit <NUM> may include an encoder and decoder and may encrypt or decrypt the received data. For example, the sub-processing circuit <NUM> may encrypt data provided from the processor <NUM>, such as data that is requested to be written to the memory <NUM> by the application, before storing the data in the memory <NUM>. By encrypting the data before storing the data into the memory <NUM>, the data may be protected from attacks such as cold-boot attacks. Since the encryption and decryption processing of the sub-processing circuit <NUM> is a process separated from the operation of the processor <NUM>, performance degradation of the application caused by the security function need not occur in executing the application, and the power efficiency may be optimized by using dedicated hardware versus when the security function is provided by the processor <NUM>.

In an embodiment, the sub-processing circuit <NUM> may pre-fetch data that is expected to be accessed by the processor <NUM>, such as through a separate channel to which a cache coherence protocol is applied between the system bus <NUM> and the sub-processing circuit <NUM>.

The memory <NUM> may temporarily store data that is processed or to be processed by the processor <NUM>, the sub-processing circuit <NUM>, and the IPs <NUM> and <NUM>. The memory <NUM> may include a volatile memory such as a dynamic random-access memory (DRAM), a static RAM (SRAM), a synchronous RAM (SDRAM), and/or a nonvolatile memory such as a phase-change RAM (PRAM), a magneto-resistive RAM (MRAM), a ferroelectric RAM (FRAM), or the like. However, for ease of description, it may be assumed here that DRAM is used as the memory <NUM>, without limitation thereto. In <FIG>, the memory <NUM> is illustrated to be mounted in the SoC <NUM>, but it is not limited thereto. For example, the memory <NUM> may be implemented as a separate chip from the SoC <NUM>, and may transmit and receive data to and from other components of the SoC <NUM>.

The memory <NUM> may be a system memory. An operating system (OS), applications, and/or firmware may be loaded in the memory <NUM> in booting. For example, when an electronic device equipped with the SoC <NUM> is booted, an OS image stored in a storage space may be loaded into the memory <NUM> according to a boot sequence. Overall input/output operations of the SoC <NUM> may be supported by the operating system OS. Moreover, applications and/or firmware (e.g., related to graphics processing) may be loaded into the memory <NUM> according to a user's selection and/or basic services.

Each of the IPs <NUM> and <NUM> may include a unit module or a combination of unit modules designed to perform a specific function in the SoC <NUM>. An IP may be referred to as a functional module or a processing circuit. The IPs <NUM> and <NUM>, such as, for example, a first IP <NUM> and a second IP <NUM>, may include a graphics processing unit (GPU), an image signal processor (ISP), a digital signal processor (DSP), a power management unit (PMU), a clock management unit (CMU), a universal serial bus (USB) controller, a peripheral component interconnect (PCI) controller, a wireless interface, a generic controller, embedded software, a codec, a video module such as a camera interface, a joint photographic experts group (JPEG) processor, a video processor, a mixer, or the like, a <NUM>-dimentional graphics core, an audio system, and/or a driver. The IPs <NUM> and <NUM> may be implemented in hardware, software or firmware, or any combination thereof. In <FIG>, the processor <NUM> and the sub-processing circuit <NUM> are illustrated in a separate configuration from the IPs <NUM> and <NUM>, but the processor <NUM> and/or the sub-processing circuit <NUM> may also be referred to as IP.

The system bus <NUM> may connect the components of the SoC <NUM> to one another, such as the processor <NUM>, the sub-processing circuit <NUM>, the memory <NUM>, and the IPs <NUM> and <NUM>, and may provide a transmission path for data or signals between the components.

In an embodiment, the system bus <NUM> may be implemented in a network-on-a-chip (NoC) method. The NoC method is a method of connecting processing circuits in a semiconductor chip by applying packet or circuit network technology between general computers or communications devices, to a semiconductor chip. The system bus <NUM> may include a router and a switching circuit to provide a transmission path for data and signals between the processing circuits in the SoC, such as between the processor <NUM>, the sub-processing circuit <NUM>, the memory <NUM>, and the IPs <NUM> and <NUM>.

In an embodiment, the system bus <NUM> may be implemented in the form of an NoC to which a protocol having a preset standard bus specification is applied. For example, an advanced microcontroller bus architecture (AMBA) protocol of an advanced Reduced Instruction Set Computer (RISC) machine (ARM) protocol may be applied as the standard bus specification. The bus types of the AMBA protocol may include one or more of an advanced high-performance bus (AHB), an advanced peripheral bus (APB), an advanced extensible interface (AXI), an AXI4, AXI coherency extensions (ACE), or the like. Among the bus types described above, the AXI is an interface protocol between functional blocks that provides a multiple outstanding address function and a data interleaving function. In addition, other types of protocols, such as Sonics Inc. 's uNetwork, IBM's CoreConnect, and/or the OCP-IP's Open Core Protocol may also be applied to the system bus <NUM>.

The system bus <NUM> may receive an access request from at least one component of the SoC <NUM>, such as, for example, the processor <NUM>, the sub-processing circuit <NUM>, the first IP <NUM>, and the second IP <NUM>, and may transmit, based on a physical address included in the access request such as an access address, the access request to a component having a corresponding physical address, such as, for example, the memory <NUM>. In addition, the system bus <NUM> may transmit a response to the access request to the component that provided the access request.

In the SoC <NUM> according to an embodiment, when the physical address in the access request is received from the processor <NUM>, such as when the access address is an effective physical address, the system bus <NUM> may transmit the access request to the memory <NUM>. Accordingly, the processor <NUM> may have direct access to the memory <NUM> based on the effective physical address. As used herein, 'direct access' to the memory <NUM> means that the memory access is performed without any processing circuits and includes access through the system bus <NUM>.

When the physical address is a shadow physical address, the system bus <NUM> may transmit the access request to the sub-processing circuit <NUM>. As described above, the sub-processing circuit <NUM> may convert the shadow physical address into the effective physical address. The system bus <NUM> may receive an access request including an effective physical address from the sub-processing circuit <NUM> and transmit the access request to the memory <NUM>. Accordingly, the processor <NUM> may have indirect access to the memory <NUM> via the sub-processing circuit <NUM> based on the shadow physical address. As used herein, 'indirect access' to the memory <NUM> means the memory access is performed through a processing circuit, such as via the sub-processing circuit <NUM>.

As described above, in the SoC <NUM> according to an embodiment, the page table PGTB in <FIG> may include an effective physical address corresponding to the physical address area and a shadow physical address corresponding to the shadow physical address area, which is an address space other than the physical address area, and the processor <NUM> may have direct access to the memory <NUM> through the system bus <NUM> or may have indirect access to the memory <NUM> through the system bus <NUM> and the sub-processing circuit <NUM>. In executing the application, the sub-processing circuit <NUM> may provide functions that are not provided by the processor <NUM>, such as, for example, a compression/decompression function, and/or an encryption/decryption function, without limitation thereto. Accordingly, the SoC <NUM> may support various applications without any modifications to the processor <NUM> and/or any load increase to the processor <NUM>. In addition, the usage bandwidth of the memory <NUM> may be reduced by the compression/decompression function, to thereby optimize performance of the SoC <NUM> and/or minimize power consumption of the SoC <NUM>.

<FIG> shows a SoC according to an embodiment.

Referring to <FIG>, a SoC 100a according to an embodiment may include the processor <NUM>, the sub-processing circuit <NUM>, IPs <NUM> and <NUM>, the system bus <NUM>, and a memory controller <NUM>. The SoC 100a shown in <FIG> is a modified example of the SoC <NUM> shown in <FIG>. Accordingly, substantially duplicate description may be omitted.

In an embodiment, the memory <NUM> may be implemented as a separate chip outside of the SoC 100a. The memory <NUM> may include a system memory. Moreover, various types of memories that may be applied to the memory <NUM> in <FIG> may be applied to the memory <NUM>.

The memory controller <NUM> may receive an access request including an access address from the system bus <NUM>, and transmit the access request to the memory <NUM>. In addition, the memory controller <NUM> may transmit a processing result and/or a response to the access request to the system bus <NUM>.

<FIG> and <FIG> illustrate a path through which the processor has access to the memory in a SoC 100b according to an embodiment, and <FIG> shows a method of providing access to the memory from the processor in the SoC 100b according to an embodiment.

Referring to <FIG> and <FIG>, the SoC 100b may include the processor <NUM>, the sub-processing circuit <NUM>, the memory <NUM>, and a router <NUM>. The router <NUM> may be arranged in the system bus <NUM> of <FIG>. The SoC 100b may further include other components described with reference to <FIG>. For example, the memory <NUM> is shown to be provided in the SoC 100b, but is not limited thereto. As described with reference to <FIG>, the memory <NUM> may be implemented as a separate chip outside of the SoC 100b. In this case, the SoC 100b may further include the memory controller <NUM> of <FIG> that provides a communications path with the memory <NUM>.

Referring to <FIG>, the processor <NUM> may generate a first access request signal including a first access address at step S110. The processor <NUM> may execute an application and operate a page table (e.g., PGTB of <FIG>) corresponding to the application according to an embodiment. The processor <NUM> may convert the virtual address VA of <FIG>, which is issued by the application for an access of the memory <NUM>, into the physical address PA of <FIG> by using the page table PGTB of <FIG>. In this case, the physical address PA may include the effective physical address and the shadow physical address. The processor <NUM> may generate a first access request signal including the effective physical address or the shadow physical address as a first access address AA1.

The processor <NUM> may transmit the first access request signal (e.g., a write request command or a read request command) for the memory <NUM> including the first access address AA1 to the system bus <NUM> of <FIG> at step S120. The first access request signal may be transmitted to the router <NUM> on the system bus <NUM>.

The router <NUM> may determine whether the first access address AA1 corresponds to the shadow physical address area at step S130. The router <NUM> may include information about the shadow physical address corresponding to the shadow physical address area and determine whether the first access address is the shadow physical address based on the information about the shadow physical address.

When determining that the first access address AA1 does not correspond to the shadow physical address area EPAS, the router <NUM> may transmit the first access request signal via the system bus to the memory at step S140. The router <NUM> may transmit the received access address to the memory <NUM> or to the sub-processing circuit <NUM>.

When determining that the first access address AA1 corresponds to the physical address area PAS of <FIG>, the router <NUM> may transmit the first access address AA1 to the memory <NUM>, such as shown in <FIG>. For example, when the first access address AA1 corresponds to the effective physical address, the router <NUM> may transmit the first access address AA1 to the memory <NUM>.

When determining that the first access address AA1 corresponds to the shadow physical address area SPAS of <FIG>, the router <NUM> may transmit the first access request signal to the sub-processing circuit <NUM> at step S150. When the first access address AA1 is determined to correspond to the shadow physical address area, such as when the first access address AA1 is determined to correspond to the shadow address area, the router <NUM> may transmit the first access address AA1 to the sub-processing circuit <NUM> rather than the memory <NUM> as shown in <FIG>. For example, when the first access address AA1 corresponds to the shadow physical address, the router <NUM> may transmit the first access address AA1 to the sub-processing circuit <NUM>. The operations below are described with reference to <FIG> and <FIG>.

The sub-processing circuit <NUM> converts the first access address AA1 into a second access address AA2 corresponding to the physical address area at step S160. For example, the sub-processing circuit <NUM> may convert the first access address AA1, which is the shadow physical address, into the second access address AA2, which is the effective physical address.

The sub-processing circuit <NUM> may transmit a second access request signal including the second access address AA2 to the system bus <NUM> at step S170. For example, the sub-processing circuit <NUM> may output the second access address AA2 to the router <NUM>.

The system bus <NUM> may transmit the second access request signal to the memory <NUM> at step S180. As described above for operation S130, the router <NUM> may determine whether the received access address corresponds to the shadow physical address area. Here, since the second access address AA2 is the effective physical address, it does not correspond to the shadow physical address area. Accordingly, the router <NUM> may transmit the second access address AA2 to the memory <NUM>.

In the SoC 100b according to this embodiment, the processor <NUM> may have direct access to the memory <NUM> based on the effective physical address, and have indirect access to the memory <NUM> via the sub-processing circuit <NUM> based on the shadow physical address.

<FIG> illustrates an operation of a SoC 100c according to an embodiment.

Referring to <FIG>, the SoC 100c according to this embodiment may include the processor <NUM>, the sub-processing circuit <NUM>, the memory <NUM>, the router <NUM>, and a memory management unit (MMU) <NUM>. The SoC 100c of <FIG> is a modified example of the SoC 100b in <FIG> and <FIG>, and thus, any further detailed descriptions on the same elements are omitted and the descriptions on the SoC 100c are focused on the differences.

As described above, when the first access address AA1 included in the first access request signal generated by the processor <NUM> is included in the shadow physical address area, such as when the first access address AA1 is a shadow physical address SPA, the router <NUM> may transmit the first access request signal to the sub-processing circuit <NUM>.

The sub-processing circuit <NUM> may convert the first access address AA1 from the shadow physical address SPA into a virtual address VA. In an embodiment, the sub-processing circuit <NUM> may include an address matching table having mapping information for the virtual address VA corresponding to the shadow physical address SPA. Thus, the sub-processing circuit <NUM> may convert the shadow physical address SPA into the virtual address VA by using the address matching table. Accordingly, the sub-processing circuit <NUM> may convert the first access address AA1, such as the shadow physical address SPA, into the virtual address VA.

The MMU <NUM> may convert the virtual address VA, received from the sub-processing circuit <NUM>, into an effective physical address EPA. In an embodiment, the MMU <NUM> may include a page table having mapping information for the effective physical address EPA corresponding to the virtual address VA. The page table used by the MMU <NUM> may be different from the page table used by the processor <NUM>. For example, the MMU <NUM> may include a system MMU that supports one or more processing circuits of the SoC 100c, and the page table may be the same as another page table used by at least one other processing circuit of the SoC 100c, such as by the graphics processing unit (GPU), the image signal processor (ISP), or the like, without limitation thereto.

The MMU <NUM> may generate a second access request including the effective physical address EPA as the second access address AA2, and transmit the second access request to the router <NUM>. The router <NUM>, in turn, may transmit the second access request to the memory <NUM>.

<FIG> illustrates an operation of the SoC 100d according to an embodiment.

Referring to <FIG>, the SoC 100d according to an embodiment may include the processor <NUM>, the sub-processing circuit <NUM>, the memory <NUM>, and the router <NUM>. The SoC 100d of <FIG> is a modified example of the SoC 100b of <FIG>. Substantially duplicate description may be omitted.

In an embodiment, the router <NUM> may include a cache CC. An additional channel may be provided for managing the cache CC between the sub-processing circuit <NUM> and the router <NUM>, and the sub-processing circuit <NUM> may transmit a cache management request signal QRC, such as a read command, a stash command, or the like destined for the cache CC to the router <NUM> in response to a cache coherence protocol.

Here, the router <NUM> may operate as a cache coherency controller. The router <NUM> may store data in the cache CC or read out stored data from the cache CC in response to the cache management request signal QRC from the sub-processing circuit <NUM>. The router <NUM> may provide a path through which the data stored in the memory <NUM> is read from and/or written into the cache CC. The router <NUM> may be configured to maintain consistency between the cache CC and at least one cache of the processor <NUM>, such as but not limited to local caches and/or shared caches in the processor <NUM>, or between the cache CC and at least one cache of the sub-processing circuit <NUM>, such as but not limited to local caches and/or shared caches in the sub-processing circuit <NUM>.

In an embodiment, the cache CC is shown to be inside the router <NUM>, but is not limited thereto. For example, the cache CC may be separately placed outside the router <NUM>, such as in the system bus <NUM> of <FIG>, or may be a shared cache between at least one processor.

The sub-processing circuit <NUM> may pre-fetch data, such as data which is expected to be used by the processor <NUM>, in the cache CC in response to the cache management request signal QRC. The sub-processing circuit <NUM> may support a data pre-fetch in a form that the processor <NUM> need not support. Accordingly, performance may be optimized when the processor <NUM> executes the application.

In an embodiment, with reference to <FIG> and <FIG>, the SoC 100d may further include the MMU <NUM> of <FIG>. In this case, the sub-processing circuit <NUM> may directly transmit the cache management request signal QRC, such as a read command, a stash command, or the like, to the router <NUM>. The router <NUM>, in turn, may transmit a response to the cache management request signal QRC, such as cache hit or miss status and relevant data in response to the read request, to the sub-processing circuit <NUM>.

<FIG> and <FIG> illustrate a read operation of a SoC 100e according to an embodiment. Compressed image data may be stored in the memory <NUM> in <FIG> and <FIG>. In an embodiment the compressed image data may be read out by the sub-processing circuit <NUM> performing the indirect access to the memory <NUM> in response to an image data read request from the processor <NUM>.

Referring to <FIG>, the processor <NUM> may execute an application for performing an image processing operation, such as but not limited to face recognition and correction, image quality improvement, or the like. The application may request to read out image data from the memory <NUM>, and may generate a first virtual address indicating an area in which the image data is stored.

The processor <NUM> may convert the first virtual address into a first physical address PA1 based on a first page table PGTB1 set for the application. The first page table PGTB1 may be used to map the virtual address to the effective physical address or the shadow physical address, such as described above with reference to <FIG> and <FIG>. The first physical address PA1 may be a shadow physical address.

The memory <NUM> may include a compression buffer CBUF, and compressed image data CDT such as in Figure 7B may be stored in the compression buffer CBUF.

<FIG> illustrates compressed image data CDT stored in the compression buffer CBUF, according to an embodiment.

An image processing circuit, such as but not limited to an Image Signal Processor (ISP), may compress image data by the sub-block SBL. For example, the sub-block SBL may include <NUM> pixels arranged in a <NUM>×<NUM> matrix. A packet of the compressed image data CDT may include a header HD and a payload PL. The payload PL may include sub-blocks SBL that are compressed therein. The header HD may include information about the storage order and storage size of the compressed sub-blocks SBL, which may be compressed individually or as a group and arranged in the payload PL. The header information may further include a start address of the payload PL, such as but not limited to a start address of a first sub-block.

The compression buffer CBUF may include a payload area PLA with n payloads PL stored therein, where n is a positive integer of two (<NUM>) or more, and a header area HDA, with n headers HD stored therein. A footprint PF of substantially the same size may be set for each payload PL. Accordingly, the start address and/or end address of the payload PL may be recognized according to the order of the payloads PL.

For example, the ISP may compress the original image data received from the image sensor per sub-block and store the compressed image data CDT in the compression buffer CBUF. The ISP may store the compressed image data CDT in the effective physical address generated based on a second page table PGTB2 set for the ISP, where the effective physical address corresponds to an area of the compression buffer CBUF to which the compressed image data CDT is to be stored.

Referring to <FIG> and <FIG>, the processor <NUM> need not provide a compression and/or decompression function for an application. In such a case, when the processor <NUM> itself reads out the compressed image data CDT from the memory <NUM>, the application might not be able to use the compressed image data CDT. Accordingly, the processor <NUM> may control the sub-processing circuit <NUM> such as to decompress the compressed image data CDT, and receive the decompressed image data from the sub-processing circuit <NUM>.

Therefore, the processor <NUM> may convert the first virtual address into the shadow physical address, rather than the effective physical address, by using the first page table PGTB1, and generate a first read request signal RD1, including the first physical address PA1 corresponding to the shadow physical address, as the access address.

In such a case where the first physical address PA1 received from the processor <NUM> corresponds to the shadow physical address area rather than the physical address area of the memory <NUM>, the router <NUM> may transmit the first read request signal RD1 of <FIG> to the sub-processing circuit <NUM> instead of to the memory <NUM>.

The sub-processing circuit <NUM> may include an address conversion circuit ACC that converts a physical address into a virtual address. In an embodiment, the address conversion circuit ACC may convert a physical address into a virtual address by using an address matching table. The address matching table may include the virtual address corresponding to the physical address of the payload PL, and the address conversion circuit ACC may include information on the image data, such as but without limitation to height, width, format of image data, or the like. The sub-processing circuit <NUM> may convert the first physical address PA1, which is included in the first read request signal RD1 received from the router <NUM>, into a second virtual address VA2. In an embodiment, the address conversion circuit ACC may calculate the virtual address for each header HD and payload PL of the compressed image data CDT corresponding to the first physical address PA1, such as by using an address matching table.

The sub-processing circuit <NUM> may also include a compressor COMP. The compressor COMP may compress and/or decompress the received data, such as described below with reference to <FIG>, without limitation thereto.

The MMU <NUM> may convert the second virtual address VA2, received from the sub-processing circuit <NUM>, into the second physical address PA2 by using the second page table PGTB2. The MMU <NUM> may transmit a second read request signal RD2 including the second physical address PA2 to the router <NUM>. In an embodiment, the second page table PGTB2 may include a page table set for another processing circuit, such as but not limited to an ISP, for compressing the image data and storing the compressed image data CDT into the memory <NUM>. Moreover, the second page table PGTB2 may map the effective physical address to the virtual address. The second physical address PA2 may be the effective physical address.

Here, since the second physical address PA2 corresponds to the physical address area of the memory <NUM>, the router <NUM> may transmit the second read request signal RD2 to the memory <NUM>.

Referring now to <FIG> and <FIG>, the compressed image data CDT may be read out from the second physical address PA2 in the compression buffer CBUF of the memory <NUM>. The router <NUM> may transmit the compressed image data CDT to the sub-processing circuit <NUM>. The compressor COMP of the sub-processing circuit <NUM> may decompress the compressed image data CDT. The sub-processing circuit <NUM> may transmit the decompressed image data DCDT to the processor <NUM> through the router <NUM>.

The decompressed image data DCDT may include more than just the read image data requested by the processor <NUM>, such as but not limited to pixel values for some pixels included in the image data of a single frame. Moreover, the sub-processing circuit <NUM> may transmit image data corresponding to the first physical address PA1, which is received from the processor <NUM>, to the processor <NUM> among the decompressed image data DCDT.

In an embodiment, , the sub-processing circuit <NUM> may include a cache CC such as but not limited to that described in detail with reference to <FIG>. In response to the cache management request signal QRC received from the sub-processing circuit <NUM>, the sub-processing circuit <NUM> may store remaining data of the decompressed image data DCDT in the cache, but need not store some of the decompressed image data DCDT transmitted to the processor <NUM>. The processor <NUM> may be relatively likely to request reading of continuous image data in a single frame, without limitation thereto. Therefore, when the read image data requested by the processor <NUM> is stored in the cache, the sub-processing circuit <NUM> may transmit the image data stored in the cache CC to the processor <NUM> without further access to the memory <NUM>.

<FIG> illustrates a software layer of an SoC, such as but not limited to the SoC <NUM>, according to an embodiment. For conveniences of description, hardware connected to the SoC <NUM> may be illustrated together as hardware (H/W) <NUM> hereinafter.

An application <NUM> and an OS <NUM> may be performed by a processor, such as but not limited to the processor <NUM> of <FIG>. The application <NUM> refers to software (S/W) and/or a service for implementing a specific function. The user <NUM> refers to an entity such as but not limited to a person, Artificial Intelligence (AI), or other system using the application <NUM>. The user <NUM> may communicate with the application <NUM> via the user interface UI. The application <NUM> may be manufactured and/or configured based on each desired service, and may communicate with the user <NUM> via the user interface suitable for each such service. The application <NUM> may perform the operation requested by the user <NUM>, and may call the contents of an application protocol interface (API) <NUM> and/or a library <NUM> as desired.

The API <NUM> and/or the library <NUM> may perform macro operations responsible for specific functions, or provide interfaces when communications with the lower layers is desired. When the application <NUM> requests the lower layer to operate through the API <NUM> and/or the library <NUM>, the API <NUM> and/or the library <NUM> may classify the received requests into fields for security <NUM>, network <NUM>, and/or management <NUM>. The API <NUM> and/or the library <NUM> may operate a particular layer suitable for the requested field. For example, when the application <NUM> requests a function related to the network <NUM>, the API <NUM> may transmit a parameter for a layer of the network <NUM> and call a related function. In this case, the network <NUM> may communicate with a lower layer to perform the requested operation. When there is no corresponding lower layer, the API <NUM> and/or the library <NUM> itself may perform the corresponding operation, without limitation thereto.

A driver <NUM> may manage the hardware <NUM> of the SoC <NUM>, for example, and check the operation states of the hardware <NUM>,. When receiving a request classified by the upper layers, the driver <NUM> may deliver the classified request to a corresponding layer of the hardware <NUM>.

When the driver <NUM> delivers the request to the layer of the hardware <NUM>, firmware <NUM> may convert the request into a form that is acceptable to the hardware <NUM>. The firmware <NUM>, for converting the request and transmitting the converted request to the hardware <NUM>, may be provided in the driver <NUM> and/or in the hardware <NUM>.

For example, the SoC <NUM> of <FIG> may include an operating system (OS) <NUM> for managing components of the SoC <NUM> including the API <NUM>, the driver <NUM>, and the firmware <NUM>. The OS <NUM> may be stored in a non-volatile memory in the form of control command code and/or data.

The hardware <NUM> may include a processor <NUM>, a sub-processing circuit <NUM>, a memory <NUM>, a system memory management unit (MMU) <NUM>, an image signal processor (ISP) <NUM>, a graphics processing unit (GPU) <NUM>, an input/output (I/O) display <NUM>, and/or the like. The hardware <NUM> may execute the request or command delivered by the driver <NUM> and the firmware <NUM>, in order and/or out of order, and store the execution results in a memory <NUM>, in a register inside the hardware <NUM>, or in a memory such as a dynamic random-access memory (DRAM) connected to the hardware <NUM>. The stored execution results may be returned to the driver <NUM> and/or the firmware <NUM>.

The hardware <NUM> may generate an interrupt to request a desired operation for the upper layer. When the interrupt is generated, the hardware <NUM> may check the interrupt in the management field <NUM> of the OS <NUM>, and may process the interrupt by communicating with the core of the hardware <NUM>.

In an embodiment, the API <NUM> may set an environment in which the processor <NUM>, such as but not limited to the processor <NUM> of <FIG>, may have indirect access to the memory <NUM>, such as but not limited to the memory <NUM> of <FIG>, by using the sub-processing circuit <NUM>, such as but not limited to the sub-processing circuit <NUM> of <FIG>. The API <NUM> may generate a page table, such as but not limited to the page table PGTB of <FIG> and/or the first page table PGTB1 of <FIG>, corresponding to the application <NUM> that uses the function of sub-processing circuit <NUM>, and set information for the operation of the sub-processing circuit <NUM>, such as the address matching table and/or the second page table PGTB2 of <FIG>.

Accordingly, when the application <NUM> is executed by the processor <NUM>, the page table corresponding to the application <NUM>, the address matching table for the sub-processing circuit <NUM>, and the page table may be provided, and the processor <NUM> may have access to the memory <NUM> by using the sub-processing circuit <NUM> based on the shadow physical address, such as described with reference to <FIG>, supra.

In an embodiment, the API <NUM> is shown to be provided in the OS or control software <NUM>, but is not limited thereto. For example, the API <NUM> may be provided by the application <NUM> depending on different design choice criteria of various embodiments.

<FIG> illustrates matching between a virtual address space VAS and an effective physical address space EPAS according to a comparative example. <FIG> shows a result of memory access according to the comparative example.

Referring to <FIG>, the virtual address space VAS includes compressed buffers comp_buf_o0 and comp_buf_o1 as storage areas for applications. The compressed buffers comp_buf_o0 and comp_buf_o1 may match effective physical address areas valid_p0 and valid_p1 of the effective physical address space EPAS. The effective physical address areas valid_p0 and valid_p1 may be areas in the compression buffer of a memory, such as but not limited to the compression buffer CBUF of <FIG> and the memory130 of <FIG>.

Referring to <FIG>, code CD1 shows that the compressed buffers comp_buf_o0 and comp_buf_o1 are allocated to the virtual address space VAS. For example, the compressed buffers comp_buf_o0 and comp_buf_o1 may be allocated to the virtual address space VAS for a camera process. The first header of the virtual address corresponding to the compressed buffers comp_buf_o0 and comp_buf_o1 may indicate an image size, the first payload may be <NUM>, and the second payload may indicate a size of the payload, without limitation thereto.

When an application requests data from the compressed buffers comp_buf_o0 and comp_buf_o1 based on a solution function foo_sol according to code CD2, a processor, such as the processor <NUM> of <FIG>. may access the effective physical address areas valid_p0 and valid_p1 of a memory, such as the memory <NUM> as described with respect to any of the <FIG>, to read out the compressed data stored in the effective physical address areas valid_p0 and valid_p1, such as described with reference to <FIG>. Moreover, even when data is read out from empty address areas such as caused by compression of the image data, or the compressed data is read out, the application might not be able to interpret the compressed data because the decompression function need not be provided by the processor <NUM>.

<FIG> illustrates matching between virtual and physical address spaces according to an embodiment. <FIG> illustrates matching between a virtual address space and a physical address space based on an API according to an embodiment.

Referring to <FIG>, the virtual address space VAS may include the compressed buffers comp_buf_o0 and comp_buf_o1 and uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1 corresponding thereto. The compressed buffers comp_buf_o0 and comp_buf_o1 may match the effective physical address areas valid_p0 and valid_p1 of the effective physical address space EPAS, and the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1 may match the shadow physical address areas shadow_p0 and shadow_p1 of the shadow physical address space SPAS. The matching between the virtual address space VAS and the effective physical address space EPAS and the matching between the virtual address space and the shadow physical address space SPAS may be set by the API <NUM> of <FIG>, and the matching information may be generated as a page table (e.g., PGTB of <FIG> and PGTB1 of <FIG>).

Referring to <FIG>, the compressed buffers comp_buf_o0 and comp_buf_o1 may be allocated to the virtual address space VAS according to code CD1 such as described above for <FIG>. Substantially duplicate description may be omitted.

Code CD3 shows that a compression API is applied to allocate the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1 to the virtual address space VAS. Accordingly, the compression API may provide an uncompressed view to the processor <NUM> of <FIG>. A function c_api of the compression API may generate the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1 based on the compressed buffers comp_buf_o0 and comp_buf_o1, image information, and/or the like. For example, the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1 may be set as virtual address space of the operating system (OS), or as temporary virtual address space of the application for the processor <NUM> to have access to the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1.

When the data in the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1 is requested based on the solution function 'foo_sol' according to code CD4, a memory function of the compression API, such as '*memory c_api', may map the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1 to the shadow physical address areas shadow_p0 and shadow_p1, and generate the page table such as described in code CD5. In addition, the memory function of the compression API may generate a matching table that is used in the sub-processing circuit <NUM> of <FIG>. Here, the shadow physical address areas shadow_p0 and shadow_p1, corresponding to the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1, are to be continuously allocated in the shadow physical address space SPAS. For example, a value obtained by adding a buffer size, which is a size of each shadow physical address area, to a start address indicating a start area 'shadow_p0' among the shadow physical address areas, may be calculated as an end address indicating a last area such as 'shadow_p1' among the shadow physical address areas.

For example, the processor <NUM> may indirectly read out the decompressed data from the compression buffer CBUF in the memory <NUM> of <FIG> corresponding to the compressed buffers comp_buf_o0 and comp_buf_o1 through the sub-processing circuit <NUM> of <FIG>. The application may process the decompressed data.

When the use of the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1 is completed, the uncompressed buffers uncomp_buf_v0 and uncomp_buf_v1 may be returned to the operating system (OS) according to the free function c_free of code CD6.

<FIG> illustrates an application processor (AP) <NUM>, according to an embodiment.

Referring to <FIG>, the AP <NUM> according to an embodiment may include a central processing unit (CPU) <NUM>, a sub-processing circuit <NUM>, a system interconnection circuit (SICC) <NUM>, a network-on-a-chip (NoC) <NUM>, and a memory management unit (MMU) <NUM>. The AP <NUM> may further include other components, for example, an ISP, a GPU, a communications module, a random-access memory (RAM), or a read-only memory (ROM). The AP <NUM> may be implemented as a SoC, and data and signals between the components in the chip, such as the CPU <NUM>, the sub-processing circuit <NUM>, the MMU <NUM>, and other processing function blocks, may be transmitted and received through the SICC <NUM> and the NoC <NUM>. The SICC <NUM> and the NoC <NUM> may correspond to the router <NUM> in the embodiment described above, without limitation thereto. A dynamic RAM (DRAM) may be connected to the AP <NUM> as a memory of the AP <NUM>. The AP <NUM> may include a DRAM controller, and the AP <NUM> may transmit and receive data to and from DRAM under the control of the DRAM controller.

The CPU <NUM> may be a main processor of the AP <NUM> to control overall operations of the AP <NUM>, without limitation thereto, and may include one processor core such as a single core processor, or multiple processor cores such as a multi-core processor. The CPU <NUM> may process or execute programs and/or data stored in the RAM, DRAM and/or ROM.

The CPU <NUM> may correspond to the processor <NUM> described with reference to <FIG>. The CPU <NUM> may execute an operating system and an application, and may convert a virtual address generated by the application into a physical address. Based on the first page table PGTB set for the application, the CPU <NUM> may convert the virtual address corresponding to the virtual address area, into an effective physical address corresponding to the physical address area of the DRAM <NUM>, or into a shadow physical address corresponding to the shadow physical address area outside of the physical address area of the DRAM <NUM>. The CPU <NUM> may transmit, to the SICC <NUM>, an access request signal for accessing the DRAM <NUM>, the access request signal may include a physical address, such as an effective physical address or a shadow physical address. The CPU <NUM> need not support the compression of data that is to be stored in the DRAM <NUM>, nor the decompression of data that is to be read from the DRAM <NUM>. Therefore, when the application requests access to the compression buffer CBUF of the DRAM <NUM>, the CPU <NUM> may convert the virtual address into the effective physical address based on the first page table PGTB, and the application may have indirect access to the compression buffer CBUF of the DRAM <NUM> through the sub-processing circuit <NUM> that supports the data compression and/or decompression. Here, the compression buffer CBUF may store compressed data, such as but not limited to compressed image data.

When the physical address included in the received access request signal is an effective physical address, the SICC <NUM> may send the access request signal to the DRAM <NUM>. Here, when the physical address is a shadow physical address, the SICC <NUM> may send the access request signal to the sub-processing circuit <NUM>. When the physical address included in the access request signal received from the CPU <NUM> is the effective physical address, the SICC <NUM> may transmit the access request signal to the DRAM <NUM>, and the CPU <NUM> may have relatively direct access to the DRAM <NUM> directly through the SICC <NUM>. On the other hand, when the physical address included in the access request signal is the shadow physical address, the SICC <NUM> may transmit the access request signal to the sub-processing circuit <NUM>, and the CPU <NUM> may have indirect access to the DRAM <NUM> via the sub-processing circuit <NUM>.

The SICC <NUM> may transmit data read from the DRAM <NUM> to the corresponding processing circuit, such as the CPU <NUM> and/or the sub-processing circuit <NUM> that requested access to the data, in response to the access request signal. The SICC <NUM> may include the cache CC, such as but not limited to a system cache, and may store data read from the memory <NUM> into the cache CC, or read data from the cache CC in response to a cache management request signal. In addition, the SICC <NUM> may function as a cache coherency controller that maintains data consistency between local caches and/or shared caches of processing circuits, such as the CPU <NUM>, the sub-processing circuit <NUM>, and other processing circuits without limitation thereto.

The sub-processing circuit <NUM> may compress or decompress the received data, and may, for example, temporarily store it in the memory <NUM>. The sub-processing circuit <NUM> may convert the shadow physical address included in the received access request signal into a virtual address matching the effective physical address, such as by using the address matching table (AMT). As described with reference to <FIG>, the address matching table AMT may be generated by the compressed API, without limitation thereto. Moreover, the sub-processing circuit <NUM> may output the cache management request signal to the SICC <NUM>, such as via the NoC <NUM>.

The MMU <NUM> may convert the virtual address received from the SICC <NUM>, such as via the sub-processing circuit <NUM>, into the effective physical address. The MMU <NUM> may convert the virtual address into the effective physical address with reference to the second page table PGTB2. In an embodiment, the second page table PGTB2 may be the same as another page table used by other processing circuits, such as by the ISP, the GSP, or the like, that have access to the compression buffer CBUF of the DRAM <NUM>.

The NoC <NUM> may transmit a request signal that is output from the sub-processing circuit <NUM> and the MMU <NUM>, such as the access request signal and/or the cache management request signal, to the SICC <NUM>, and may also function as a data path through which the data is transmitted and received between the sub-processing circuit <NUM> and the SICC <NUM>. The NoC <NUM> may receive the access request signal including the effective physical address from the MMU <NUM> and transmit the access request signal to the SICC <NUM>. The NoC <NUM> may transmit the compressed data, which is read from the compression buffer CBUF of the DRAM <NUM> based on the access request signal received by the SICC <NUM> from the MMU <NUM>, to the sub-processing circuit <NUM>. In addition, the NoC <NUM> may transmit the decompressed data, which is decompressed by the sub-processing circuit <NUM>, to the SICC <NUM>, and the SICC <NUM>, in turn, may transmit the decompressed data to the CPU <NUM>.

<FIG> and <FIG> illustrate a read operation and a write operation of the AP <NUM>, according to an embodiment. <FIG> shows a process of reading the compressed image data from the compression buffer CBUF of the DRAM <NUM> by the CPU <NUM> of the AP <NUM> in the indirect memory access mode, and <FIG> shows a process of writing the compressed image data to the compression buffer CBUF of the DRAM <NUM> by the CPU <NUM> of the AP <NUM> in the indirect memory access mode.

Referring to <FIG>, the CPU <NUM> may output a read request signal including the shadow physical address A to the SICC <NUM> at operation ①. The read request signal may make a request to read pixel values corresponding to a portion of the image data, such as, for example, at least one area of the image data in a single frame, from the DRAM <NUM>.

The SICC <NUM> may determine whether the physical address included in the received access request signal is a shadow physical address or an effective physical address, and transmit a read request signal including the shadow physical address A to the sub-processing circuit <NUM> at operation ②.

The sub-processing circuit <NUM>, in turn, may convert the shadow physical address A into the virtual address VA at operation ③. Moreover, the sub-processing circuit <NUM> may convert the shadow physical address A into a virtual address B of a header and a virtual address C of a payload of a packet having pixels requested to be read among packets of the compressed image data. The sub-processing circuit <NUM> may convert the shadow physical address A into the virtual address B of the header and the virtual address C of the payload with reference to the address matching table, such as, for example, AMT of <FIG>.

<FIG> shows an example of the address matching table, according to an embodiment. It may be assumed that the address matching table AMT of <FIG> is generated based on the mapping relationship between the virtual address space VAS and the shadow physical address space SPAS in <FIG> and <FIG>, without limitation thereto.

Referring to <FIG>, an address matching table AMT according to an embodiment may include the physical address for a decompressed view, that is, a shadow physical address SPAS, a virtual address VACB corresponding to the shadow physical address SPAS, such as a virtual address of the compression buffer, a header virtual address VAHD and information about image data IF_IMG, such as, for example, a width, a height, a format, or the like of the image data. In an embodiment, the shadow physical address SPAS indicates a start address of the shadow physical address area and may be referred to as the start address SPAS hereinafter. For example, a first compressed buffer comp_buf_o0 and a first header address comp_buf_o0 + img0_size may be mapped to a first shadow physical address shadow_p0 as the virtual address VACB and the header virtual address VAHD, respectively, and a second compressed buffer comp_buf_o1 and a second header address comp_buf_o1 + img1_size may be mapped to a second shadow physical address shadow_p1 as the virtual address VACB and the header virtual address VAHD, respectively. Herein, the mark img0_size of <FIG> denotes a first image size of the compressed image data stored in the first compressed buffer comp_buf_o0 that is a type of the virtual address VACB, and the mark img1_size of <FIG> denotes a second image size of the compressed image data stored in the second compressed buffer comp_buf_o1 that is a type of the virtual address VACB. In an embodiment shown in <FIG>, the width, height, and format of first image data and second image data, such as uncompressed or decompressed image data, which is stored in the first shadow physical address shadow_p0 and the second shadow physical address shadow_p1, respectively, may be <NUM>, <NUM>, and NV12 in the order listed, without limitation thereto.

Referring back to <FIG>, the sub-processing circuit <NUM> may recognize the start address SPAS and an offset α from the shadow physical address A, and discover the virtual address VACB corresponding to the start address SPAS of the shadow physical address A, the size of the image data, and the information about the image data in the address matching table AMT. For example, when the shadow physical address A indicates a portion of the first shadow physical address area, such as, for example, a sub-block of the image data, the shadow physical address A may be represented as a value obtained by adding the offset α to the first shadow physical address shadow_p0. In this case, the offset α may be smaller than the size of the image data, without limitation thereto. In the address matching table AMT, the sub-processing circuit <NUM> may find the first compressed buffer comp_buf_o0 as the virtual address VACB corresponding to the first shadow physical address shadow_p0, the first image size img0_size as the size of image data, the width <NUM>, the height <NUM>, and the format NV12 as the information about the image data. The sub-processing circuit <NUM> may calculate coordinates (e.g., an x coordinate and a y coordinate) of the image data based on the offset α, the format, and the width of the image data in response to the read request of the CPU <NUM>. The sub-processing circuit <NUM> may calculate the virtual address B of the payload and the virtual address C of the header of the sub-block including pixels designated by the coordinates (x, y) of the image data, based on the first compressed buffer comp_buf_o0 of the virtual address VACB and the coordinates (x, y) of the image data.

The sub-processing circuit <NUM> may transmit the virtual address B of the payload and the virtual address C of the header to the MMU <NUM> at operation ④.

The MMU <NUM>, in turn, may convert the virtual address B of the payload and the virtual address C of the header into the physical address D of the payload and the physical address E of the header with reference to the second page table PGTB2. The physical address D of the payload and the physical address E of the header may be effective physical addresses. In an embodiment, the second page table PGTB2 may include another page table that is referenced by other processing circuits, such as the ISP, the GPU, or the like, for processing the image data in the AP <NUM>. The MMU <NUM> may transmit read request signals including the physical address D of the payload and the physical address E of the header to the NoC <NUM>, and the NoC <NUM> may transmit the read request signals to the SICC <NUM> at operation ⑤. Since the read request signals transmitted from the NoC <NUM> contain the effective physical address, the SICC <NUM> may transmit the read request signals to the DRAM <NUM>.

In the compression buffer CBUF of the DRAM <NUM>, the header HD and the payload PL of the compressed image data, which includes pixel values of at least one area of the image data requested by the CPU <NUM>, may be read from the DRAM <NUM>, and may be provided to the sub-processing circuit <NUM> through the SICC <NUM> and the NoC <NUM> at operation ⑥.

The sub-processing circuit <NUM> may decompress the compressed image data at operation ⑦. The sub-processing circuit <NUM> may decompress the compressed image data by decoding the payload PL based on compression information in the header HD. The sub-processing circuit <NUM> may transmit the decompressed image data DCDT to the CPU <NUM> through the NoC <NUM> and the SICC <NUM> at operation ⑧. The decompressed image data DCDT may be used in an application executed by the CPU, without limitation thereto.

In an embodiment, the sub-processing circuit <NUM> may transmit the cache management request signal QRC to the SICC <NUM> in relation to the decompressed image data at operation ⑨. For example, the decompressed image data DCDT may include pixel values of a single sub-block having pixel values of at least one area requested by the CPU <NUM>, and the sub-processing circuit <NUM> may transmit, to the SICC <NUM>, the cache management request signal QRC for requesting to store the remaining pixel values, except for the pixel values provided to the CPU <NUM>, into the cache CC. When the CPU <NUM> requests reading of continuous pixel values in a single sub-block, a cache hit may occur, and the pixel values already stored in the cache CC may be transmitted to the CPU <NUM> without any further access to the DRAM <NUM>. Accordingly, the hit ratio of the cache may increase.

In an embodiment, the sub-processing circuit <NUM> may analyze the read request signals from the CPU <NUM>, check a particular pattern of the image data that is requested by the read request signals, such as a stripe pattern in the image data or the like, and transmit a cache management request signal QRC to the SICC <NUM> to thereby pre-fetch, from the DRAM <NUM>, image data for which reading is expected to be requested by the CPU <NUM>.

Referring now to <FIG>, the CPU <NUM> may output a write request signal including the shadow physical address A to the SICC <NUM> at operation ①. The write request signal may indicate a request to write or store pixel values corresponding to a portion of the image data, such as, for example, at least one area of the image data in a single frame, into the DRAM <NUM>.

The SICC <NUM> may transmit a write request signal including the shadow physical address A to the sub-processing circuit <NUM> at operation ②. The sub-processing circuit <NUM> may convert the shadow physical address A into the virtual address VA at operation ③. The sub-processing circuit <NUM> may convert the shadow physical address A into the virtual address B of the payload and the virtual address C of the header with reference to the address matching table, such as, for example, AMT of <FIG>. The sub-processing circuit <NUM> may transmit the cache management request signal QRC to the SICC <NUM> in relation to a sub-block including the pixel values requested for writing at operation ④. In an embodiment, the sub-processing circuit <NUM> may transmit a 'clean invalid' request signal to the SICC <NUM> for the sub-block including the pixel values requested for writing. The SICC <NUM> may flush dirty lines from the caches for the CPU <NUM>, such as the L1 cache, the L2 cache, the L3 cache, and the last level cache (LLC), and a flushed line may be provided in the cache for the sub-processing circuit <NUM>, such as, for example, the cache CC.

In an embodiment, when no pixel values of the sub-block including the pixel values requested to be written are present in the cache CC, the sub-processing circuit <NUM> may read the header HD and payload PL corresponding to the sub-block from the compression buffer CBUF of the DRAM <NUM> at operation ⑥, and decompress the payload PL based on the compression information of the header HD at operation ⑦, such as according to operations ④ through ⑦ of <FIG>. The sub-processing circuit <NUM> may update the sub-block based on the pixel values that are requested to be written by the CPU <NUM>, compress the updated sub-block at operation ⑧, and write the compressed image data CDT, such as the header and the payload of the sub-block on the compression buffer CBUF of the DRAM <NUM> at operation ⑨.

Referring to <FIG>, a SoC <NUM> according to an embodiment may include a CPU <NUM>, a random-access memory (RAM) <NUM>, a multimedia IP core <NUM>, a memory controller <NUM>, a sub-processing circuit <NUM>, a sensor interface <NUM>, and a display controller <NUM>. The SoC <NUM> may further include other commonly used components such as a communications module, a read-only memory (ROM), and/or the like. The components of the SoC <NUM>, such as the CPU <NUM>, the RAM <NUM>, the multimedia IP core <NUM>, the memory controller <NUM>, the sub-processing circuit <NUM>, the sensor interface <NUM>, and the display controller <NUM>, may transmit and receive data via a bus <NUM>. The advanced microcontroller bus architecture (AMBA) protocols may be adopted as a standard specification of the bus <NUM>, without limitation thereto. For example, any other suitable protocols, such as uNetwork, CoreConnect, and the open core protocol of OCP-IP may similarly or additionally be adopted into the standard specification. In an embodiment, the bus <NUM> may be implemented in the form of a network-on-a-chip.

In an embodiment, the bus <NUM> is further configured to receive another access address from at least one intellectual property (IP) core, and transmit the other access address to the memory if the other access address corresponds to a physical address area of the memory, and to transmit the other access address to other processing circuits other than the memory if the other access address corresponds to a shadow physical address area other than the physical address area of the memory.

The CPU <NUM> may control overall operations of the SoC <NUM> and may correspond to the processor <NUM> of <FIG> and/or the CPU <NUM> of <FIG>, each as described above. The CPU <NUM> may execute an operating system and/or an application, and may convert a virtual address generated by the application into a physical address. In this case, based on a first page table set for the application, the CPU <NUM> may convert the virtual address into one of an effective physical address, which corresponds to the physical address area of a memory <NUM>, and a shadow physical address, which corresponds to the shadow address area outside the physical address area of the memory <NUM>. The CPU <NUM> may generate an access request signal, such as a read request signal or a write request signal, for accessing the memory <NUM> that includes the effective physical address or the shadow physical address. The access request signal may be transmitted to the memory controller <NUM> or to the sub-processing circuit <NUM> via the bus <NUM>.

The RAM <NUM> may be implemented as a volatile memory such as dynamic RAM (DRAM) and/or a static RAM (SRAM), and more particularly, as a resistive memory such as PRAM, MRAM, ReRAM, FRAM, or the like. The RAM <NUM> may temporarily store programs, data, and/or instructions.

The multimedia IP core <NUM> may perform image processing on the image data, such as, for example, still images or videos. For example, the multimedia IP core <NUM> may include at least one of an ISP, a GPU, a video processing unit (VPU), a display processing unit (DPU), and/or a neural network processing unit (NPU).

The ISP may change the format of the received image data or correct the image quality of the image data. For example, the ISP may receive RGB image data as input data, and convert the RGB image data into YUV image data. Moreover, the ISP may correct the image quality of the image data by performing image processing such as adjusting a gamma value and/or luminance of the received image data, widening a dynamic range (DR) of the received image data, and/or removing noise from the received image data.

The GPU may calculate and generate two-dimensional or three-dimensional graphics. The GPU may be specialized in processing graphics data and may process graphics data in parallel. Furthermore, the GPU may be used for performing complex operations, such as geometry calculations, scalar and vector floating point calculations, and the like. The GPU may execute various commands that are encoded by using an API, such as but not limited to OpenCL, OpenGL, and/or WebGL.

The VPU may correct the quality of the received video image or record and play images such as recording and playback of audio and video including the video image.

The DPU may perform image processing for displaying the received image data on a display device <NUM>. For example, the DPU may change the format of the received image data to a suitable format for displaying on the display and/or correct the image data based on a gamma value corresponding to the display.

The NPU may perform image processing on the received image data based on the learned neural network, derive features from image data, and recognize objects, backgrounds, or the like in image data based on the features. The NPU may be specialized for computation of one or more neural networks and may process image data in parallel.

The memory controller <NUM> may interface data or commands between the SoC <NUM> and the memory <NUM>. The memory controller <NUM> may receive an access request signal from the bus <NUM> and transmit the access request signal to the memory <NUM>. As described above with reference to <FIG>, the memory <NUM> may be implemented as a volatile memory, such as a DRAM, an SRAM, or an SDRAM, or a nonvolatile memory, such as a PRAM, an MRAM, a ReRAM, a FeRAM, or a NAND flash memory, without limitation thereto. The memory <NUM> may be implemented as a memory card, such as a multi-media card (MMC), an embedded MMC (eMMC), a secure digital (SD) card, a micro SD card, or the like. The memory <NUM> may include a compression buffer and may store the compressed image data, without limitation.

The multimedia IP core <NUM> may compress data processed by the image processing and store the compressed data in the memory <NUM>. The multimedia IP core <NUM> may include a memory management unit (MMU), and the MMU may convert a virtual address in which the compressed data is stored into an effective physical address, corresponding to the physical address area of the memory <NUM>, based on a second page table that is different from a first page table used by the CPU. Moreover, the multimedia IP core <NUM> may transmit the access request signal for accessing the memory <NUM> having the effective physical address to the memory controller <NUM> via the bus <NUM>.

The sub-processing circuit <NUM> may support a function that the CPU <NUM> need not provide, such as, for example, data processing for data read from the memory <NUM> or data to be written on the memory <NUM>. The sub-processing circuit <NUM> may convert the shadow physical address in the access request signal transmitted from the CPU <NUM> into the effective physical address indicating the physical address area of the memory <NUM>, and read data from the memory <NUM> based on the effective physical address. Accordingly, the CPU <NUM> may have indirect access to the memory <NUM> by using the sub-processing circuit <NUM>.

In an embodiment, the sub-processing circuit <NUM> may convert the shadow physical address into a virtual address by using an address matching table generated together with the first page table when the first page table is created, and may convert the virtual address into the effective physical address by using the second page table set for the multimedia IP core <NUM>. In an embodiment, a system MMU, for converting the virtual address into the effective physical address by using the second page table, may be implemented as a separate circuit from the sub-processing circuit <NUM>. In an embodiment, the MMU of the multimedia IP core <NUM> may be used as the system MMU.

The sub-processing circuit <NUM> may correspond to the sub-processing circuit <NUM> of <FIG> or the sub-processing circuit <NUM> of <FIG> as described above, without limitation thereto. Substantially duplicate description of the sub-processing circuit <NUM> may be omitted.

The sensor interface <NUM> may interface data or commands between the SoC <NUM> and the image sensor <NUM>, and may receive the image data from the image sensor <NUM>. The image data received from the image sensor <NUM> may be processed by at least one processing circuit of the multimedia IP core <NUM> for image processing, and may otherwise be processed by an application for such image processing running on the CPU <NUM>. The image data received from the image sensor <NUM>, and/or the image data undergoing the image processing, may be stored in the memory <NUM>.

The display controller <NUM> may interface display data, such as image data, for output to the display device <NUM>. The display device <NUM> may interpret the display data having images or videos on a display panel, such as a liquid crystal display (LCD) or an active-matrix organic light emitting diode (AMOLED) display, or the like.

<FIG> shows an electronic device on which an AP is mounted, according to an embodiment.

The electronic device <NUM> may include a mobile device, such as a smartphone, a tablet PC, a laptop computer, a wearable device, a GPS device, an e-book terminal, an MP3 player, a digital camera, a navigation device, a drone, an IoT device, a home appliance, an advanced driver assistance system (ADAS), and/or the like. In addition, the electronic device <NUM> may be provided as components in assemblies such as vehicles, furniture, manufacturing facilities, doors, and various measurement devices, without limitation thereto.

Referring to <FIG>, the electronic device <NUM> may include an application processor (AP) <NUM>, a camera module <NUM>, a working memory <NUM>, a storage <NUM>, a display device <NUM>, a communications module <NUM>, and a user interface <NUM>. The electronic device <NUM> may further include other commonly used components, without limitation.

The AP <NUM> may be implemented as SoC that controls overall operations of the electronic device <NUM> and drives an application program, an operating system, or the like. The AP <NUM> may perform image processing on the image data provided from the camera module <NUM>, and may store the image data in the storage <NUM> and/or provide the image data to the display device <NUM>. The SoC <NUM> of <FIG> and the SoC 100a of <FIG> and the AP <NUM> of <FIG> and the SoC <NUM> of <FIG>, each as described above, may be used as the AP <NUM>, without limitation thereto.

In an embodiment, the AP <NUM> may include a CPU <NUM>, a system bus <NUM>, and a sub-processing circuit <NUM>. When receiving a first access request signal having a physical address for accessing the working memory from the CPU <NUM>, the system bus <NUM> may determine whether the physical address is an effective physical address or a shadow physical address. The system bus <NUM> may transmit the first access request signal to the working memory <NUM> when the physical address is the effective physical address, and/or to the sub-processing circuit <NUM> when the physical address is the shadow physical address. The sub-processing circuit <NUM> may convert the physical address in the first access request signal into the effective physical address and transmit a second access request signal having the effective physical address to the system bus <NUM>. The system bus <NUM> may transmit the second access request signal to the working memory <NUM> from the sub-processing circuit <NUM>. In an embodiment, the sub-processing circuit <NUM> may perform compression and decompression or encryption and decryption on data received from the CPU <NUM> together with the first access request signal, or on data received from the working memory <NUM> in response to the second access request signal transmitted to the system bus <NUM>, and may pre-fetch data from the working memory <NUM> that is expected to be accessed by the CPU <NUM>. The CPU <NUM> may have indirect access to the working memory <NUM> by using the sub-processing circuit <NUM>, and the sub-processing circuit <NUM> may perform functions that the CPU <NUM> need not support, and thus support efficiency of an application running on the CPU <NUM>.

The camera module <NUM>, and may generate image data and transmit the image data to the AP <NUM>. The camera module <NUM> may include at least one camera, and the camera may include an image sensor and a lens. The image sensor may convert optical signals received through the lens into the image data. In an embodiment, the camera module <NUM> may include multiple cameras having different viewing angles. In an embodiment, the camera module <NUM> may generate image data with different exposures and transmit the image data to the AP <NUM>. The AP <NUM> may merge different image data to generate a high dynamic range (HDR) image.

The working memory <NUM> may be implemented as a volatile memory such as a DRAM, an SRAM or the like, or a nonvolatile memory such as a FeRAM, a RRAM, a PRAM, a NAND flash memory or the like, without limitation thereto. An operation program or an application program stored in the storage <NUM> may be loaded into the working memory <NUM> and executed in the CPU <NUM>. In addition, operating data generated in the operation of the electronic device <NUM> may be temporarily stored in the working memory <NUM>. The working memory <NUM> may store programs and/or data processed and/or executed by the AP <NUM>. For example, the AP <NUM> may perform image processing on the image data provided from the camera module <NUM>, compress the image data processed by the image processing, and temporarily store the compressed image data in the working memory <NUM>.

The storage <NUM> may be implemented as a nonvolatile memory such as a NAND flash and/or a resistive memory, and be provided as a memory card, such as an MMC, an eMMC, a SD, a microSD, or the like. The storage <NUM> may store data provided from the AP <NUM>. For example, the AP <NUM> may store the image data processed by the image processing into the storage <NUM>. In addition, the storage <NUM> may store an operation program, an application program, or the like of the electronic device <NUM>.

The wireless transceiver <NUM> may include a transceiver <NUM>, a modem <NUM>, and an antenna <NUM>. The wireless transceiver <NUM> may perform wireless communications with one or more external devices, and may receive data from one or more external devices and/or transmit data to one or more external devices.

The user interface <NUM> may be implemented with various devices capable of receiving a user input, such as a keyboard, curtain key panel, touch panel, fingerprint sensor, microphone, and/or the like. The user interface <NUM> may receive a user input, and provide a signal corresponding to the received user input to the AP <NUM>.

Claim 1:
A System-on-a-Chip, SoC, (<NUM>) comprising:
a first processor (<NUM>) configured to output a first access address;
a system bus (<NUM>) configured to transmit the first access address to a memory (<NUM>) if the first access address corresponds to a physical address area of the memory, and to transmit the first access address to other processing circuits other than the memory if the first access address corresponds to a shadow physical address area other than the physical address area of the memory; and
a sub-processing circuit (<NUM>) configured to receive the first access address from the first processor via the system bus, convert the first access address into a second access address corresponding to the physical address area, and transmit the second access address to the system bus to access the memory.