apply_query_key_layer_scaling 的用途是什么呢

#78

by yywind - opened Aug 25, 2023

yywind

Aug 25, 2023

apply_query_key_layer_scaling 这个参数是是什么意思呢？为什么要对 query_key_layer 做缩放呢？不做缩放可能会导致哪些后果呢？而且为什么缩放的值是layer_id

Sep 7, 2023

同问

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment