Cho đoạn script:
so we'll move into the stuff that I'm most familiar with and some of my work so looking at
1:01:43
activations um so this is this is work I've been doing for a while uh this idea of steering vectors so I mentioned I
1:01:50
poke models so it's thick steering Vector is that thick so it's basically a fix length vector that steers a language
1:01:57
model to generate a specific sequence exactly when added to the hidden sites of a model at a specific
1:02:02
location um and I'll I'll read this again it's there's a very like specific
1:02:09
form that I wrote this in so uh it's it's a fix length Vector that steers a language model to generate a specific
1:02:15
sequence exactly when added to the hidden states of a model at a specific point so this is different than um a
1:02:22
soft prompt or different than a model editing sort of approach um in this case there is a vector that
1:02:29
corresponds to a sequence and that Vector doesn't correspond to any other sequence there could be multiple vectors
1:02:35
and it turns out there are multiple vectors that correspond to that sequence it'll be a little bit clearer um based
1:02:41
on how we extract these things um so this is the stick that we're talking the language
1:02:48
model um so how do we extract them so this is gpt2 um basically this Z steer thing on
1:02:57
the left this is the steering Vector this gets initialized randomly um with
1:03:03
small like in a in a reasonable way uniformly and small um and for any
1:03:09
sequence a specific sequence that we want the model to generate we optimize this steering Vector Z steer uh
1:03:17
to generate that sequence keeping the rest of the model entirely fixed so think about it as we're nudging an a
1:03:23
frozen model to be able to generate a specific sequence at a specific time um
1:03:29
and we have a lot of different options on where to inject the steering intive can put it basically anywhere in the
1:03:35
model we can put it at any time step any number of these things in practice um
1:03:41
providing it just at the first time step and somewhere in the middle of the model basically not the first layer and not
1:03:48
the last layer works pretty well um and so more formally um forget the kind of
1:03:56
notation um but right here we initialize um this Z steer and for a few iterations
1:04:03
um we do forward passes first this starts as random and then this gets closer and closer and closer to being
1:04:09
able to generate this sequence and eventually we get to a point uh and this n is pretty small it's eight or 10 or
1:04:16
something like that um for most sequences we get to a point where we have found this stick that is allowed to
1:04:22
poke this model to generate that sequence exactly now when we greedy decode from the model we pass in just a
1:04:29
beginning of sequence token and this Z steer the steering vector and it's able
1:04:34
to uncover a whole sequence that whole sequence that we had at the beginning entirely
1:04:41
um this is weird and interesting because in a lot of cases um in like the
1:04:46
prompting world in the soft prompt World usually you need a pretty large uh width
1:04:52
of a prompt to be able to do things with um and this this generally in in that
1:04:57
structure you're doing a specific task and you're providing kind of a large a large prompt to do this with large soft
1:05:04
prompt to do this with this is often H this often has a width of 50 and a and a
1:05:10
length of the the hidden size or the embedding size of the model in our cases all of our steering vectors are withth
1:05:17
one and they're of just the hidden size of the model um so what ends
1:05:24
up happening um actually before I go to results any
1:05:29
questions this is a this is a weird setup and weird relative to what other
1:05:34
people do so happy to take any questions [Music]
1:05:42
yeah similarly if your prompt was um of a specific type so the prompt here is a
1:05:50
continuous Vector passed in it's a single length width hidden size
1:05:55
continuous Vector so um it's kind of like maybe collapsing your prompt into
1:06:02
this compressing it into this tiny VOR you can think of that way
1:06:08
yeah any other questions yeah this would be like
1:06:18
I'm things potentially um this is something
1:06:23
that I want to work on it uh like a year ago and didn't get didn't get this
1:06:30
sufficient Buy in and then had to apply to grad school and all these things so it went by the wayside but but
1:06:36
definitely something to something to pursue um there's a lot of scope there
1:06:43
any other questions all right so move over to
1:06:51
results so we can find steering vectors and that's and that's interesting thing um and we can find them pretty easily
1:06:58
and for most sequences even sequences that the model hasn't seen before the underlying language model hasn't seen
1:07:04
before um it also works for and this is kind of
1:07:09
a negative but it also works for random sequences of very small length but it's
1:07:16
harder to find so you can imagine if your uh steering Vector is basically a
1:07:22
giant bulldozer it doesn't matter what your model is learning learned similar to the probe situation if you can
1:07:28
compress all that information of that sequence into the vector you don't really need the language model um so
1:07:35
there are cases when you're looking at sequences of length like five seven eight something like this you can
1:07:40
uniformly sample from the vocabulary at random with replacement generate utter garbage and find steering vectors for
1:07:47
them takes a little while but your model is complex enough that you can basically
1:07:53
bulldo your model to be able to do this even if that sequence is incredibly low likelihood under the model but it works
1:08:00
better for things that are higher likelihood under the model um
1:08:05
predictably the I think the thing that surprised me the most was these steering vectors themselves have interpretable
1:08:12
properties U so distances in steering Vector space reflect semantic similarity
1:08:17
so if you have two sentences that are close um they're also close in steering
1:08:23
Vector space that's kind of nice um it does better than for example the
1:08:29
representations one would use for for probing so mean pooling Bert hidden States like we looked at before those do
1:08:37
actually worse than steering vectors um just a bit surprising um style transfer is possible
1:08:45
with simple Vector arithmetic so it' be nice to say that I have a sequence I want to subtract you know negativity and
1:08:52
add positivity for for sentiment or other sorts of Styles um we can do this
1:08:58
and we can do this reasonably well in steering VOR space um we can also decode from
1:09:05
interpolations in the Laten space so you take two steering vectors for two
1:09:10
sequences you look in the middle of them you linearly interpolate between them and you decode um if the space is kind
1:09:17
of weirdly peaky then you would have issues and what you would generate is garbage and there's no guarantee that
1:09:23
the space should be reasonable in between but it turns out it is um here's an example of one of these
1:09:30
style transfer cases so very very simple easy easy sentence we found steering
1:09:36
vectors for The Taste is excellent and and we took a sample of 100 positive
1:09:41
sentences and 100 negative sentences found their steering vectors took the mean and thought that you know that
1:09:47
looks like the positive concept steering Vector negative concept steering Vector we just did Vector arithmetic just did
1:09:54
uh current steering Vector uh plus negative minus positive
1:09:59
and we got the taste is unpleasant um and similarly um in the reverse
1:10:06
directions it turns out that the magnitude matters because um for every
1:10:12
single sequence there's kind of an end dimensional ball around that steering
1:10:17
Vector that we found that also decodes that specific sequence and so that shows
1:10:23
that the space is kind of reasonably well formed there's there's of course uh
1:10:28
a lot of weird sort of areas um and so if you go poke around in steering Vector
1:10:34
space and sort of try to sample from it eventually you'll find some weird edge cases and some garbage and repeated text
1:10:41
and little things like this any questions here before I kind of Rapid Fire through the the last few
1:10:50
things yeah like here
1:10:57
yeah so we went uh Beyond this um there was so in in these specific experiments we
1:11:04
looked at the middle of gpt2 um so this was like layer six layer seven and at
1:11:09
the first time step we didn't do any um like magnitude scaling and so you can
1:11:15
imagine if you put a giant Vector in there the models never the rest of the model has never seen something of that
1:11:21
magnitude so it's now in a weird State and it's just going to break so if you
1:11:26
put this to like I don't know 500 or something like this it break it just has no idea it's like basically like telling
1:11:32
the rest your model hey it's like a completely untrained model be it looks similar to like random performance you
1:11:38
get repeats and things like this smaller you end up staying in this ball for the
1:11:43
sequence two two seemed pretty reasonable but we didn't spend a lot of time just like the day before the paper
1:11:50
was do we were two seems reasonable we went to three we went to five 10 broke
1:11:56
five somewhat broke two seems reasonable um decent signings
1:12:03
hopefully

Viết lại đoạn script trên thành đoạn text nhỏ trong blog chuyên ngành bằng Tiếng Việt đảm bảo chính xác, rõ ràng, đầy đủ, dễ hiểu. Không bỏ công thức hay số liệu. Những ví dụ với từ hay câu được trích xuất nên giữ nguyên là Tiếng Anh.