Patent ID: 11887353
Assignee: ZHEJIANG LAB
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 0:
1. A deep learning image classification method oriented to heterogeneous computing devices, comprising:
modeling a deep learning model as a directed acyclic graph, wherein nodes of the directed acyclic graph represent operators of the deep learning model, directed edges of the directed acyclic graph represent data transmission between the operators, and acquiring computing time of the operators on each of computing devices and data transmission time of data between operators to be transferred between two of the computing devices;
wherein an original directed acyclic graph G is represented as:

G=(V,E)

wherein, V represents nodes of the original directed acyclic graph G, and E represents directed edges of the original directed acyclic graph G;
a new directed acyclic graph G is represented as:

G=(V,Ē)

wherein, a new node V comprises the node V of the original directed acyclic graph G and new nodes composed of the directed edges E of the original directed acyclic graph G; Ē represents new directed edges;
setting a serial number of each of the computing devices to k, and a set of the serial numbers of all of computing devices is K, processing time of a computing task i∈V on a computing device k is pik, average transmission latency of a communication task q∈V−V is pqk′k″comm, a memory overhead of the computing task i is mi, a maximum memory limit of the computing device k is Memk, in the original directed acyclic graph G, a set of one or more direct or indirect immediately following tasks for the computing task i is Succ(i); in the new directed acyclic graph G, a set of one or more direct or indirect immediately following tasks for computing task i or communication task i is Succ(i);
an assignment decision parameter xik∈{0,1}, xik=1 represents assigning the computing task i to a kth device for execution; a communication decision parameter zq=1 represents communication time of a communication task q existing between the computing task i and a computing task j, where (i,q),(q,j)∈Ē; a time decision parameter Si∈+ represents start time of a computing task i or communication task i; and
minimizing the reasoning completion time of the deep learning model is represented as:, min
   (
   
    
     
      max
         
     
     
      i
      ∈
      V
     
    
    ⁢
    
     C
     i
    
   
   )
  
  ,, wherein, Ci represents completion time of an ith operator,, max
   
    i
    ∈
    V
   
  
     
  
   C
   i, represents completion time of a last operator that ends a computation in the deep learning model;
generating a new directed acyclic graph by replacing each of the directed edges in the directed acyclic graph with a new node, and adding new directed edges between the new nodes and original nodes;
constructing processing time and a memory overhead for each of the computing devices to run each of computing tasks, transmission latency for each of communication tasks, one or more immediately following task for each of computing tasks based on the directed acyclic graph, and one or more immediately following task for each of the computing tasks or each of the communication tasks based on the new directed acyclic graph;
setting assignment decisions, communication decisions and time decisions, wherein the assignment decisions represent assigning the computing tasks to corresponding computing devices, the communication decisions represent communication time of the communication tasks, and the time decisions represent start time of the computing tasks;
dividing the deep learning model into a plurality of operators, and assigning the plurality of operators to a plurality of computing devices for execution, constructing one or more constraint condition based on the processing time and the memory overhead for each of the computing devices to run each of the computing tasks, the transmission latency for each of the communication tasks, the one or more immediately following task for each of the computing tasks based on the directed acyclic graph, and the one or more immediately following task for each of the computing tasks or each of the communication tasks based on the new directed acyclic graph, and the assignment decisions, the communication decisions and the time decisions, to minimize reasoning completion time of the deep learning model; and
inputting an image into a divided computing device, and classifying the image based on the deep learning model that minimizes the reasoning completion time.