Patent ID: 11868730
Assignee: JINGDONG DIGITS TECHNOLOGY HOLDING CO., LTD.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 5:
6. The system of claim 5, wherein the l-th GDT layer is configured to obtain the embedding of the dependency tree graph by:
concatenating the graph attention diffusions H(l+1) of a plurality of heads to obtain concatenated attention diffusion Ĥ(l+1) by: Ĥ(l+1)=Concat(h1(l+1), . . . , hT(l+1))W0, wherein each of h1(l+1), . . . , hT(l+1) corresponds to one of the plurality of graph attention diffusions H(l+1), W0=Tdh×Tdh, T is a number of heads, dh is hidden dimensions of each head, and dh=d/T;
performing {tilde over (H)}l+1=Ĥ(l+1)+Norm(H(l)); and
performing H(l+1)=W2 (σ(W1Norm({tilde over (H)}(l+1))))+Ĥ(l+1), wherein W1=∈d×d and W2=∈d×d are trainable matrix, σ represents ReLU activation function, and H(l+1) is the embedding of the dependency tree graph.