microsoft

/

deberta-v3-large

like 136

Running fill mask on a sequence like 'Paris is the [MASK] of France.' yields the following response:

[{'score': 0.0035346089862287045,
  'token': 75647,
  'token_str': 'ferring',
  'sequence': 'Paris is theferring of France.'},
 {'score': 0.0016323102172464132,
  'token': 19566,
  'token_str': 'nicer',
  'sequence': 'Paris is the nicer of France.'},
 {'score': 0.0015437329420819879,
  'token': 21338,
  'token_str': 'fair',
  'sequence': 'Paris is thefair of France.'},
 {'score': 0.0011532771168276668,
  'token': 58335,
  'token_str': 'devolution',
  'sequence': 'Paris is the devolution of France.'},
 {'score': 0.0008077208767645061,
  'token': 21001,
  'token_str': 'substituted',
  'sequence': 'Paris is the substituted of France.'}]

Even something as simple as 'I am [MASK] happy.' yields this response:

[{'score': 0.004372601397335529,
  'token': 21338,
  'token_str': 'fair',
  'sequence': 'I amfair happy.'},
 {'score': 0.0012398710241541266,
  'token': 58335,
  'token_str': 'devolution',
  'sequence': 'I am devolution happy.'},
 {'score': 0.0009605953237041831,
  'token': 75647,
  'token_str': 'ferring',
  'sequence': 'I amferring happy.'},
 {'score': 0.0008473526104353368,
  'token': 124501,
  'token_str': '覚',
  'sequence': 'I am覚 happy.'},
 {'score': 0.0008009913726709783,
  'token': 63940,
  'token_str': 'inversely',
  'sequence': 'I am inversely happy.'}]

The code I used to generate these responses is provided below:

from transformers import pipeline
model = pipeline('fill-mask', model='microsoft/deberta-v3-large')
response_a = model('Paris is the [MASK] of France.')
response_b = model('I am [MASK] happy.')