microsoft

/

deberta-v3-base

like 176

Running fill mask on a sequence like 'Paris is the [MASK] of France.' yields the following response:

[{'score': 0.01858634687960148,
  'token': 14050,
  'token_str': 'worksheets',
  'sequence': 'Paris is the worksheets of France.'},
 {'score': 0.011057845316827297,
  'token': 108086,
  'token_str': 'polymorph',
  'sequence': 'Paris is the polymorph of France.'},
 {'score': 0.009536932222545147,
  'token': 30590,
  'token_str': 'thunderstorms',
  'sequence': 'Paris is the thunderstorms of France.'},
 {'score': 0.009330430999398232,
  'token': 28005,
  'token_str': 'Toll',
  'sequence': 'Paris is the Toll of France.'},
 {'score': 0.008391059935092926,
  'token': 72037,
  'token_str': 'RUSH',
  'sequence': 'Paris is the RUSH of France.'}]

Even something as simple as 'I am [MASK] happy.' yields this response:

[{'score': 0.015126903541386127,
  'token': 105157,
  'token_str': 'Pinning',
  'sequence': 'I am Pinning happy.'},
 {'score': 0.014554088935256004,
  'token': 86868,
  'token_str': 'Turpin',
  'sequence': 'I am Turpin happy.'},
 {'score': 0.013621648773550987,
  'token': 11519,
  'token_str': 'potent',
  'sequence': 'I am potent happy.'},
 {'score': 0.013600126840174198,
  'token': 121282,
  'token_str': 'unobtainable',
  'sequence': 'I am unobtainable happy.'},
 {'score': 0.009650844149291515,
  'token': 106716,
  'token_str': 'brimstone',
  'sequence': 'I am brimstone happy.'}]

The code I used to generate these responses is provided below:

from transformers import pipeline
model = pipeline('fill-mask', model='microsoft/deberta-v3-base')
response_a = model('Paris is the [MASK] of France.')
response_b = model('I am [MASK] happy.')