A data processing system, comprising: a central processing unit 'CPU'; and a plurality of accelerator cards (304, 305, 308, 335,501, 602, 604, 701) coupled to the CPU over a bus, each of the accelerator cards having a plurality of data processing 'DP' accelerators (303, or 117, 119, 121, 501, 503, 505,..., 515) to receive DP tasks from the CPU and to perform the received DP tasks, wherein at least two of the accelerator cards (304, 305, 308, 335) are coupled to each other via an inter-card connection, wherein at least two of the DP accelerators (303, or 117, 119, 121, 501, 503, 505,..., 515) within each of the accelerator cards are coupled to each other via an inter-chip connection, wherein each of the inter-card connection and the inter-chip connection is capable of being dynamically activated or deactivated, such that in response to a request received from the CPU, any one of the accelerator cards (304, 305, 308, 335,501, 602, 604, 701) or any one of the DP accelerators (303, or 117, 119, 121, 501, 503, 505,..., 515) within any one of the accelerator cards (304, 305, 308, 335,501, 602, 604, 701) is capable of being enabled or disabled to process any one of the DP tasks received from the CPU, wherein the DP accelerators (501, 503, 505,..., 515) in each of the accelerator cards are arranged in a plurality of rows and columns coupled to each other via one or more inter-chip connections, wherein each row of the DP accelerators (303, 117, 119, 121, 501, 503, 505,..., 515) of a first of the accelerator cards (304, 305, 308, 335,501, 602, 604, 701) are coupled in series via respective horizontal inter-chip connections, forming a horizontal physical accelerator ring (601, 603, 605, 607), and each column of the first accelerator card is coupled to a corresponding column of a second of the accelerator cards via one or more respective inter-card connections, forming a vertical physical accelerator ring (609, 611, 613, 615) or each column of the DP accelerators (303, 117, 119, 121, 501, 503, 505,..., 515) of a first accelerator card are coupled in series via respective vertical inter-chip connections, forming a vertical physical accelerator ring, and each row of the first accelerator card is coupled to a corresponding row of a second of the accelerator cards via one or more respective inter-card connections, forming a horizontal physical accelerator ring (701, 703, 705, 707); wherein for each physical ring, DP accelerators on both ends of each row or each column are connected to each other using a cable connection. a central processing unit 'CPU'; and a plurality of accelerator cards (304, 305, 308, 335,501, 602, 604, 701) coupled to the CPU over a bus, each of the accelerator cards having a plurality of data processing 'DP' accelerators (303, or 117, 119, 121, 501, 503, 505,..., 515) to receive DP tasks from the CPU and to perform the received DP tasks, wherein at least two of the accelerator cards (304, 305, 308, 335) are coupled to each other via an inter-card connection, wherein at least two of the DP accelerators (303, or 117, 119, 121, 501, 503, 505,..., 515) within each of the accelerator cards are coupled to each other via an inter-chip connection, wherein each of the inter-card connection and the inter-chip connection is capable of being dynamically activated or deactivated, such that in response to a request received from the CPU, any one of the accelerator cards (304, 305, 308, 335,501, 602, 604, 701) or any one of the DP accelerators (303, or 117, 119, 121, 501, 503, 505,..., 515) within any one of the accelerator cards (304, 305, 308, 335,501, 602, 604, 701) is capable of being enabled or disabled to process any one of the DP tasks received from the CPU, wherein the DP accelerators (501, 503, 505,..., 515) in each of the accelerator cards are arranged in a plurality of rows and columns coupled to each other via one or more inter-chip connections, wherein each row of the DP accelerators (303, 117, 119, 121, 501, 503, 505,..., 515) of a first of the accelerator cards (304, 305, 308, 335,501, 602, 604, 701) are coupled in series via respective horizontal inter-chip connections, forming a horizontal physical accelerator ring (601, 603, 605, 607), and each column of the first accelerator card is coupled to a corresponding column of a second of the accelerator cards via one or more respective inter-card connections, forming a vertical physical accelerator ring (609, 611, 613, 615) or each column of the DP accelerators (303, 117, 119, 121, 501, 503, 505,..., 515) of a first accelerator card are coupled in series via respective vertical inter-chip connections, forming a vertical physical accelerator ring, and each row of the first accelerator card is coupled to a corresponding row of a second of the accelerator cards via one or more respective inter-card connections, forming a horizontal physical accelerator ring (701, 703, 705, 707); wherein for each physical ring, DP accelerators on both ends of each row or each column are connected to each other using a cable connection.