Patent Description:
According to one aspect of the present disclosure, a computing device is provided, including a plurality of memory devices, a plurality of direct memory access (DMA) controllers, and an on-chip interconnect. The on-chip interconnect is configured to implement control logic to convey a read request from a primary DMA controller of the plurality of DMA controllers to a source memory device of the plurality of memory devices. The on-chip interconnect is further configured to implement the control logic to convey a read response from the source memory device to the primary DMA controller and one or more secondary DMA controllers of the plurality of DMA controllers.

<FIG> shows an example computing system <NUM> in which a computing device in the form of an SOC <NUM> may be included. In addition to the SOC <NUM>, the computing system may include memory <NUM>, one or more processors <NUM>, one or more input devices <NUM>, one or more output devices <NUM>, and/or one or more other components. The components of the computing system <NUM> may, for example, be electrically coupled over a motherboard. In addition, the computing system <NUM> may be communicatively coupled to one or more other computing systems via the one or more input devices <NUM> and/or the one or more output devices <NUM>. For example, the computing system <NUM> may be included in a data center in which data processing and storage are performed at a plurality of interconnected computing systems.

The SOC <NUM> may include a plurality of memory devices <NUM>. For example, the SOC <NUM> may include one or more memory devices <NUM> that function as dynamic random-access memory (DRAM) and one or more memory devices <NUM> that function as static random-access memory (SRAM). Memory devices <NUM> may include other types of random access memory, as well. In addition, the SOC <NUM> may include one or more processing devices <NUM>. Each processing device <NUM> of the one or more processing devices <NUM> may, for example, be a central processing unit (CPU), a core of a CPU, a graphics processing unit (GPU), a core of a GPU, a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).

The plurality of memory devices <NUM> may be coupled to the one or more processing devices <NUM> by an on-chip interconnect <NUM> via which data may be transferred. The on-chip interconnect <NUM> may, for example, be a network-on-chip (NoC), a crossbar, or a ring network. In other examples, other network topologies may be used to couple the plurality of memory devices <NUM> to the plurality of processing devices <NUM>. In examples in which the on-chip interconnect <NUM> is an NoC, the on-chip interconnect <NUM> may include one or more routers configured to implement packet switching. In examples in which the on-chip interconnect <NUM> is a crossbar, the on-chip interconnect <NUM> may include, for each of the one or more processing devices <NUM>, a plurality of electrical traces coupling that processing device <NUM> to each of the plurality of memory devices <NUM>.

The SOC <NUM> may further include a plurality of direct memory access (DMA) controllers <NUM>, which may be coupled to the plurality of memory devices <NUM>, the one or more processing devices <NUM>, and the on-chip interconnect <NUM>. Turning now to <FIG>, in which the SOC <NUM> is shown in additional detail, the SOC <NUM> includes a primary DMA controller 20A and one or more secondary DMA controllers 20B. The plurality of DMA controllers <NUM> may each be configured to transfer data to and from the memory devices <NUM> of the SOC <NUM>. The DMA controllers <NUM> may each be configured to transfer data from a source memory device 12A to a destination memory device 12B, as discussed in further detail below. In addition, as shown in the example of <FIG>, each DMA controller <NUM> of the plurality of DMA controllers <NUM> may be communicatively coupled to a respective processing device <NUM> of a plurality of processing devices <NUM> included in the SOC <NUM>. Thus, the plurality of DMA controllers <NUM> may be configured to transfer data between the plurality of memory devices <NUM> and the plurality of processing devices <NUM> as well as between one or more source memory devices 12A and one or more destination memory devices 12B.

In existing SOCs, a coherent fabric or a chip-level cache is typically used when transferring data between components of the SOC. A coherent fabric is an on-chip interconnect that is configured to implement snooping logic for the plurality of DMA controllers. The snooping logic may be implemented when a secondary DMA controller requests data from the source memory. In response to receiving a read request for data from the source memory, the coherent fabric may implement the snooping logic by querying the destination memory of the primary DMA controller to determine whether the destination memory of the primary DMA controller stores the requested data. When the destination memory of the primary DMA controller includes the requested data, the requested data may be copied into the destination memory of the secondary DMA controller via the coherent fabric. Thus, cache coherency between the respective destination memory devices of the primary DMA controller and the secondary DMA controller may be maintained. However, implementing snooping logic may be slow and computationally expensive.

Existing SOCs may alternatively use a chip-level cache (e.g. a level <NUM> or level <NUM> cache) coupled to the on-chip interconnect. Whenever a DMA controller receives data from the source memory, the chip-level cache may be updated to include the received data. When another DMA requests the same data from the source memory, that data may instead be read from the chip-level cache. However, updating the chip-level cache increases the computational cost of operating the SOC and requires the SOC to include an additional hardware component.

In order to address the above issues with existing methods of transferring data to DMA controllers in an SOC, the on-chip interconnect <NUM> of the SOC <NUM> may be configured to implement control logic <NUM> as discussed below for the primary DMA controller 20A and the one or more secondary DMA controllers 20B to implement a read broadcast or a read multi-cast via one of several possible techniques. The control logic <NUM> is program logic encoded in software instructions (e.g. firmware instructions) executed by interconnect processing hardware <NUM> included in the on-chip interconnect <NUM> to control the exchange of messages through the on-chip interconnect <NUM>. For example, when the on-chip interconnect <NUM> implements the control logic <NUM>, the on-chip interconnect <NUM> is configured to convey a read request <NUM> from the primary DMA controller 20A to the source memory device 12A. The read request <NUM> may be generated at the primary DMA controller 20A and may be a request to copy specific data stored at the source memory device 12A to one or more destination memory devices 12B included in the SOC <NUM>.

In some examples, a secondary DMA controller 20B of the one or more secondary DMA controllers 20B may be configured to transmit a synchronization request <NUM> to the primary DMA controller 20A. The synchronization request <NUM> may be transmitted to the primary DMA controller 20A via the on-chip interconnect <NUM>. In such examples, the on-chip interconnect <NUM> may be configured to implement the control logic <NUM> to convey the read request <NUM> from the primary DMA controller 20A to the source memory device 12A in response to the primary DMA controller 20A receiving the synchronization request <NUM>. Thus, the secondary DMA controller 20B may be configured to request synchronization between data stored in its destination memory device 12B and the destination memory device 12B associated with the primary DMA controller 20A.

At the source memory device 12A, the SOC <NUM> may be configured to generate a read response <NUM> based at least in part on the read request <NUM>. When the on-chip interconnect <NUM> implements the control logic <NUM>, the on-chip interconnect <NUM> is further configured to convey the read response <NUM> from the source memory device 12A to the primary DMA controller 20A and one or more secondary DMA controllers 20B of the plurality of DMA controllers <NUM>. In some examples, the on-chip interconnect <NUM> may be configured to convey the read response <NUM> to each DMA controller <NUM> of the plurality of DMA controllers <NUM> included in the SOC <NUM>. Alternatively, as shown in <FIG>, the on-chip interconnect <NUM> may be configured to convey the read response <NUM> to a first subset <NUM> of the plurality of DMA controllers <NUM> and not convey the read response <NUM> to a second subset <NUM> of the plurality of DMA controllers <NUM>.

As discussed above, the plurality of memory devices <NUM> may further include a plurality of destination memory devices 12B respectively coupled to the plurality of DMA controllers. Subsequently to receiving the read response <NUM>, the primary DMA controller 20A and the one or more secondary DMA controllers 20B may each be configured to write data included in the read response <NUM> to their corresponding destination memory devices 12B. In this way, the data that has been read from the source memory device 12A is not only communicated to the primary DMA controller 20A that requested the data, but also to other DMA controllers <NUM> using a read broadcast or multi-cast approach. Therefore, each of the destination memory devices 12B may keep a coordinated and up-to-date memory cache without requiring a separate chip level cache or snooping logic.

<FIG> schematically shows information that may be included in the read request <NUM> and the read response <NUM>. In the example of <FIG>, the on-chip interconnect <NUM> is shown when a read request <NUM> is transmitted from the primary DMA controller 20A to the source memory device 12A and when a read response <NUM> is transmitted from the source memory device 12A to the primary DMA controller 20A. As shown in <FIG>, the read request <NUM> may include read request header <NUM>, which may include a sequential request indicator <NUM> and a response recipient indicator <NUM>. The sequential request indicator <NUM>, may, for example, be a timestamp or a sequentially assigned identification number for the read request <NUM>. The response recipient indicator <NUM> may indicate the first subset <NUM> of the plurality of DMA controllers <NUM> to which the read response <NUM> is configured to be conveyed. For example, the response recipient indicator <NUM> may include a corresponding bit for each DMA controller <NUM> included in the SOC <NUM> that indicates whether that DMA controller <NUM> is configured to receive the read response <NUM>. Thus, the sequential request indicator <NUM> and the response recipient indicator <NUM> may form response multicast metadata that identifies both the read request <NUM> and the plurality of DMA controllers <NUM> that are configured to receive a read response <NUM>. In addition to the read request header <NUM>, the read request <NUM> may further include a source memory access location <NUM>, which may be a pointer to a location in the source memory device 12A.

The example read response <NUM> shown in <FIG> may include a read response header <NUM> and a read response payload <NUM>. The read response header <NUM> may include metadata for the read response <NUM>, and the read response payload <NUM> may include the data received from the source memory device 12A in response to the read request <NUM>. The read response header <NUM> may include a sequential response indicator <NUM>, which may, for example, be a timestamp or a sequentially assigned identification number. In addition, the read response header <NUM> may include a pointer address <NUM> indicating a location in the destination memory device 12B to which the DMA controller <NUM> that receives the read response <NUM> may be configured to write the read response payload <NUM>. In some examples, the read response header <NUM> of the read response <NUM> may further include a primary DMA identifier <NUM> that indicates which DMA controller <NUM> of the plurality of DMA controllers <NUM> is the primary DMA controller 20A that initiated the read request <NUM> with which the read response <NUM> is associated. The read response <NUM> may be identified as being associated with the read request <NUM> by, in some examples, including the sequential request indicator <NUM> as well as the sequential response indicator <NUM> in the read response header <NUM>. Alternatively, the state of the read request <NUM> and the read response <NUM> may be monitored at an agent executed at the processing device <NUM> of the primary DMA controller 20A.

As shown in the example of <FIG>, at least one DMA controller <NUM> of the plurality of DMA controllers <NUM> may be configured to receive a plurality of read responses <NUM>. In the example of <FIG>, a DMA controller <NUM> receives a first read response 34A at a time t<NUM> and a second read response 34B at a time t<NUM>. The first read response 34A has a first sequential response indicator 62A included in a first read response header 60A, and further includes a first primary DMA identifier 65A and a first read response payload 66A. The second read response 34B has a second sequential response indicator 62B included in a second read response header 60B, and further includes a second primary DMA identifier 65B and a second read response payload 66B. In some examples, the on-chip interconnect <NUM> may be configured to transmit a plurality of read responses to the primary DMA controller 20A and the one or more secondary DMA controllers 20B in some order other than the order in which the corresponding plurality of read requests <NUM> were received. In order for the respective read response payloads <NUM> of the plurality of read responses <NUM> to be written to the destination memory device 12B in the correct order, the on-chip interconnect <NUM> may be configured to determine a write order <NUM> indicated by the respective sequential response indicators <NUM> of the plurality of read responses <NUM>. The DMA controller <NUM> may then write the respective read response payloads <NUM> of the plurality of read responses <NUM> to the corresponding destination memory device 12B of the DMA controller <NUM> as specified by the write order <NUM>. In the example of <FIG>, although the first read response 34A is received prior to the second read response 34B, the on-chip interconnect <NUM> is configured to compute a write order <NUM> in which the second read response payload 66B is written to the destination memory device 12B at time t<NUM> before the first read response payload 66A is written to the destination memory device 12B at time t<NUM>. For example, when the sequential response indicators <NUM> of the plurality of read responses <NUM> are sequentially ordered numbers, the on-chip interconnect <NUM> may compute the write order <NUM> such that the plurality of read responses <NUM> are written to the destination memory device 12B in ascending sequential response indicator order.

In some examples, the plurality of DMA controllers <NUM> may be configured to write the data included in the read response <NUM> to a plurality of different respective pointer addresses <NUM> at their corresponding destination memory devices 12B. As shown in the example of <FIG>, the primary DMA controller 20A is configured to receive a first read response 134A including a first pointer address 164A. Similarly, the secondary DMA controller 20B is configured to receive a second read response 134B including a second pointer address 164B. Subsequently to receiving the first read response 134A, the primary DMA controller 20A may be further configured to write the read response payload <NUM> included in the first read response 134A to the first pointer address 164A of a first destination memory device 112A coupled to the primary DMA controller 20A. In addition, subsequently to receiving the second read response 134B, the secondary DMA controller 20B may be further configured to write the read response payload <NUM> included in the second read response 134B to the second pointer address 164B of a second destination memory device 112B coupled to the secondary DMA controller 20B. As a result of writing the read response payload <NUM> to different pointer addresses in the first destination memory device 112A and the second destination memory device 112B, different memory allocation schemes may be used at the first destination memory device 112A and the second destination memory device 112B. Thus, more efficient memory allocation may be achieved.

As shown in <FIG>, the primary DMA controller 20A may be coupled to a first processing device 114A, and the secondary DMA controller 20B may be coupled to a second processing device 114B. In some examples, when the primary DMA controller 20A and the secondary DMA controller 20B respectively receive the first read response 134A and the second read response 134B, the primary DMA controller 20A and the secondary DMA controller 20B may be further configured to load the read response payload <NUM> into the first processing device 114A and the second processing device 114B respectively. In such examples, the first read response 134A and the second read response 134B may respectively include first processing setting metadata 138A and second processing setting metadata 138B. The first processing setting metadata 138A and the second processing setting metadata 138B may include settings with which the first processing device 114A and the second processing device 114B may be configured to perform one or more computations on read response payload <NUM>. In some examples, the first processing setting metadata 138A and the second processing setting metadata 138B may indicate one or more respective preprocessing steps that may be performed on the read response payload prior to writing the read response payload to the first destination memory device 112A and the second destination memory device 112B. The one or more preprocessing steps may, for example, be one or more steps of a compression operation or an encryption operation.

<FIG> shows a flowchart of an example method <NUM> for use with a computing device. The computing device at which the method <NUM> is performed may be an SOC, such as the SOC <NUM> of <FIG>. At step <NUM>, the method <NUM> may include implementing control logic at an on-chip interconnect for a plurality of DMA controllers. The on-chip interconnect may, for example, be an NoC, a crossbar, or a ring network. The plurality of DMA controllers may be configured to perform direct memory access for a plurality of memory devices included in the SOC. The plurality of memory devices may include a source memory device, which may be an SRAM device, and a plurality of destination memory devices, which may be a plurality of DRAM devices respectively coupled to the plurality of DMA controllers. In addition, the plurality of DMA controllers may be respectively coupled to a plurality of processing devices. Each processing device of the plurality of processing devices may, for example, be a CPU, a core of a CPU, a GPU, a core of a GPU, an FPGA, or an ASIC.

Step <NUM> may include, at step <NUM>, conveying a read request from a primary DMA controller of the plurality of DMA controllers to the source memory device of the plurality of memory devices. The read request may be generated at the primary DMA and may be a request to transmit data stored at the source memory device to a plurality of destination memory devices via the on-chip interconnect and two or more of the plurality of DMAs.

At step <NUM>, step <NUM> further includes conveying a read response from the source memory device to the primary DMA controller and one or more secondary DMA controllers of the plurality of DMA controllers. <FIG> shows additional steps of the method <NUM> that may be performed when performing step <NUM>. As shown in <FIG> at step <NUM>, conveying the read response to the primary DMA controller and the one or more secondary DMA controllers at step <NUM> may, in some examples, include conveying the read response to each DMA controller of the plurality of DMA controllers. Alternatively, as shown at step <NUM> of <FIG>, step <NUM> may include conveying the read response to a first subset of the plurality of DMA controllers. When step <NUM> is performed, step <NUM> may further include, at step <NUM>, not conveying the read response to a second subset of the plurality of DMA controllers. In some examples, the read request may include an indication of which DMA controllers of the plurality of DMA controllers are configured to receive copies of the read response.

<FIG> shows additional steps of the method <NUM> that may be performed in some examples. At step <NUM>, the method <NUM> may further include transmitting a synchronization request from a secondary DMA controller of the one or more secondary DMA controllers to the primary DMA controller. The synchronization request may be transmitted via the on-chip interconnect when implementing the control logic at step <NUM>. Step <NUM> may be performed prior to transmitting the read request to the source memory at step <NUM>. In examples in which step <NUM> is performed, the method <NUM> may further include, at step <NUM>, conveying the read request from the primary DMA controller to the source memory device in response to the primary DMA controller receiving the synchronization request. Thus, when the primary DMA controller receives the synchronization request, the primary DMA controller may convey a read request to the source memory to synchronize data stored in the destination memory of the primary DMA controller and the destination memory of the secondary DMA controller.

<FIG> also shows additional steps of the method <NUM> that may be performed in some examples. At step <NUM>, the method <NUM> may include receiving a plurality of read responses at a DMA controller of the plurality of DMA controllers. For example, the DMA controller may be a secondary DMA controller that receives a plurality of read responses associated with read requests it did not originate.

At step <NUM>, the method <NUM> may further include, at each of the primary DMA controller and the one or more secondary DMA controllers, writing data included in the read response to their corresponding destination memory devices. In some examples, the read response may include a sequential response indicator. The sequential response indicator may be included in a read response header, and may, for example, be a timestamp or a sequentially assigned number.

In examples in which step <NUM> is performed and the read response includes a sequential response indicator, step <NUM> may include, at step <NUM>, writing the respective data included in the plurality of read responses to the corresponding destination memory device of the at least one DMA controller in a write order indicated by the respective sequential response indicators of the plurality of read responses. For example, the write order may be set as a temporal order of respective timestamps included in the plurality of read responses. If the read responses are received at the DMA controller out of temporal order, the read responses may still be written to the destination memory device of the DMA controller in the temporal order in which the read responses were generated.

In some examples, step <NUM> may further include step <NUM>. Step <NUM> may include, at the plurality of DMA controllers, writing the data included in the read response to a plurality of different respective pointer addresses at their corresponding destination memory devices. The pointer address for each read response may, for example, be indicated in a header of the read response.

Using the systems and methods discussed above, data stored in the source memory may be shared with the plurality of DMA controllers without using large amounts of source memory read bandwidth. The above systems and methods may also allow the additional hardware complexity and computational costs associated with a coherent fabric or a chip-level cache to be avoided. In addition, since different processing settings or pointer addresses may be used for read responses received at different DMA controllers, the systems and methods discussed above may further allow for increased flexibility in the processing and storage of data included in the read responses received at the DMA controllers.

Computing system <NUM> may embody the computing system <NUM> described above and illustrated in <FIG>. Computing system <NUM> may take the form of one or more personal computers, server computers, tablet computers, home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices, and wearable computing devices such as smart wristwatches and head mounted augmented reality devices.

A computing device is provided, including a plurality of memory devices, a plurality of direct memory access (DMA) controllers, and an on-chip interconnect. The on-chip interconnect is configured to implement control logic to convey a read request from a primary DMA controller of the plurality of DMA controllers to a source memory device of the plurality of memory devices. The on-chip interconnect is further configured to implement the control logic to convey a read response from the source memory device to the primary DMA controller and one or more secondary DMA controllers of the plurality of DMA controllers.

According to an example, a secondary DMA controller of the one or more secondary DMA controllers may be configured to transmit a synchronization request to the primary DMA controller. The on-chip interconnect may be configured to implement the control logic to convey the read request from the primary DMA controller to the source memory device in response to the primary DMA controller receiving the synchronization request.

According to this example, the plurality of memory devices may further include a plurality of destination memory devices respectively coupled to the plurality of DMA controllers. The primary DMA controller and the one or more secondary DMA controllers may each be configured to write data included in the read response to their corresponding destination memory devices.

According to this example, at least one DMA controller of the plurality of DMA controllers may be configured to receive a plurality of read responses. The plurality of read responses may include a respective plurality of sequential response indicators.

According to this example, the at least one DMA controller of the plurality of DMA controllers may be configured to write the respective data included in the plurality of read responses to the corresponding destination memory device of the at least one DMA controller in a write order indicated by the respective sequential response indicators of the plurality of read responses.

According to this example, the plurality of DMA controllers may be configured to write the data included in the read response to a plurality of different respective pointer addresses at their corresponding destination memory devices.

According to this example, the on-chip interconnect may be configured to implement the control logic to convey the read response to each DMA controller of the plurality of DMA controllers.

According to this example, the on-chip interconnect may be configured to implement the control logic to convey the read response to a first subset of the plurality of DMA controllers and not convey the read response to a second subset of the plurality of DMA controllers.

According to this example, the read request may indicate the first subset of the plurality of DMA controllers to which the read response is configured to be conveyed.

According to this example, the computing device may further include a respective plurality of processing devices communicatively coupled to the plurality of DMA controllers. Each processing device of the plurality of processing devices may be a central processing unit (CPU), a core of a CPU, a graphics processing unit (GPU), a core of a GPU, a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC).

According to this example, the on-chip interconnect may be a network-on-chip (NoC), a crossbar, or a ring network.

A method for use with a computing device is provided. The method includes, at an on-chip interconnect, implementing control logic for a plurality of direct memory access (DMA) controllers. The control logic is implemented at least in part by conveying a read request from a primary DMA controller of the plurality of DMA controllers to a source memory device of a plurality of memory devices. The control logic is further implemented by conveying a read response from the source memory device to the primary DMA controller and one or more secondary DMA controllers of the plurality of DMA controllers.

According to an example, the method may further include transmitting a synchronization request from a secondary DMA controller of the one or more secondary DMA controllers to the primary DMA controller. The method may further include conveying the read request from the primary DMA controller to the source memory device in response to the primary DMA controller receiving the synchronization request.

According to this example, the plurality of memory devices may further include a plurality of destination memory devices respectively coupled to the plurality of DMA controllers. The method may further include, at each of the primary DMA controller and the one or more secondary DMA controllers, writing data included in the read response to their corresponding destination memory devices.

According to this example, the method may further include receiving a plurality of read responses at a DMA controller of the plurality of DMA controllers. The plurality of read responses may include a respective plurality of sequential response indicators.

According to this example, the method may further include, at the DMA controller of the plurality of DMA controllers, writing the respective data included in the plurality of read responses to the corresponding destination memory device of the at least one DMA controller in a write order indicated by the respective sequential response indicators of the plurality of read responses.

According to this example, the method may further include, at the plurality of DMA controllers, writing the data included in the read response to a plurality of different respective pointer addresses at their corresponding destination memory devices.

According to this example, implementing the control logic may further include conveying the read response to a first subset of the plurality of DMA controllers and not conveying the read response to a second subset of the plurality of DMA controllers.

Claim 1:
A computing device (<NUM>) comprising:
a plurality of memory devices (<NUM>);
a plurality of direct memory access, DMA, controllers (<NUM>); and
an on-chip interconnect (<NUM>) configured to implement control logic (<NUM>) to:
convey a read request (<NUM>) from a primary DMA controller (20A) of the plurality of DMA controllers to a source memory device (12A) of the plurality of memory devices; and
convey a read response (<NUM>) from the source memory device to the primary DMA controller and one or more secondary DMA controllers (20B) of the plurality of DMA controllers.