Issue 1:
❓ Questions & Help

The question about roberta.

I konw that Bert use the embedding of token 'cls' to do predict, but when it comes to roberta, I dont know it clearly. Can you tell me which token's embedding is used to do predict in this project? Is it '' ?

Issue 2:
Information

I am trying to build the documentation in docs/ using sphinx-build so that I can add it to Dash as one of the docsets. However, I get an assertion error with sphinx-build when I run make html in the docs/ folder only when tensorflow is installed. The build works without tensorflow installed, but the tensorflow methods and classes are emtpy in the generated documentation - only the pytorch ones have documentation in the resulting HTML.

To reproduce

Steps to reproduce the behavior:

Create a conda environment, and install pytorch, tensorflow and transformers using the official methods.

Run pip install -e ".[docs] in source directory, before running make html in docs folder. You will get the following error (full stacktrace given later):

Exception occurred:
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/util/docfields.py", line 260, in transform
assert len(field) == 2
AssertionError

Full stacktrace:

# Sphinx version: 2.4.4
# Python version: 3.6.10 (CPython)
# Docutils version: 0.16 release
# Jinja2 version: 2.11.1
# Last messages:
# reading sources... [ 13%] glossary
#
# reading sources... [ 16%] index
#
# reading sources... [ 18%] installation
#
# reading sources... [ 21%] main_classes/configuration
#
# reading sources... [ 24%] main_classes/model
#
# Loaded extensions:
# sphinx.ext.mathjax (2.4.4) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/ext/mathjax.py
# sphinxcontrib.applehelp (1.0.2) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinxcontrib/applehelp/__init__.py
# sphinxcontrib.devhelp (1.0.2) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinxcontrib/devhelp/__init__.py
# sphinxcontrib.htmlhelp (1.0.3) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinxcontrib/htmlhelp/__init__.py
# sphinxcontrib.serializinghtml (1.1.4) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinxcontrib/serializinghtml/__init__.py
# sphinxcontrib.qthelp (1.0.3) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinxcontrib/qthelp/__init__.py
# alabaster (0.7.12) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/alabaster/__init__.py
# sphinx.ext.autodoc.type_comment (2.4.4) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/ext/autodoc/type_comment.py
# sphinx.ext.autodoc (2.4.4) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/ext/autodoc/__init__.py
# sphinx.ext.coverage (2.4.4) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/ext/coverage.py
# sphinx.ext.napoleon (2.4.4) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/ext/napoleon/__init__.py
# recommonmark (0.6.0) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/recommonmark/__init__.py
# sphinx.ext.viewcode (2.4.4) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/ext/viewcode.py
# sphinx_markdown_tables (<module 'sphinx_markdown_tables.__version__' from '/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx_markdown_tables/__version__.py'>) from /Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx_markdown_tables/__init__.py
Traceback (most recent call last):
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/cmd/build.py", line 276, in build_main
app.build(args.force_all, filenames)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/application.py", line 349, in build
self.builder.build_update()
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 299, in build_update
len(to_build))
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 311, in build
updated_docnames = set(self.read())
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 418, in read
self._read_serial(docnames)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 439, in _read_serial
self.read_doc(docname)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/builders/__init__.py", line 479, in read_doc
doctree = read_doc(self.app, self.env, self.env.doc2path(docname))
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/io.py", line 316, in read_doc
pub.publish()
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/core.py", line 218, in publish
self.settings)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/io.py", line 130, in read
self.parse()
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/readers/__init__.py", line 77, in parse
self.parser.parse(self.input, document)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/parsers.py", line 93, in parse
self.statemachine.run(inputlines, document, inliner=self.inliner)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 171, in run
input_source=document['source'])
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 242, in run
context, state, transitions)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 459, in check_line
return method(match, context, next_state)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
self.section(title, source, style, lineno - 1, messages)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 242, in run
context, state, transitions)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 459, in check_line
return method(match, context, next_state)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2769, in underline
self.section(title, source, style, lineno - 1, messages)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 327, in section
self.new_subsection(title, lineno, messages)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 395, in new_subsection
node=section_node, match_titles=True)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 242, in run
context, state, transitions)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 459, in check_line
return method(match, context, next_state)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2342, in explicit_markup
nodelist, blank_finish = self.explicit_construct(match)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
return method(self, expmatch)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2097, in directive
directive_class, match, type_name, option_presets)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
result = directive_instance.run()
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/ext/autodoc/directive.py", line 157, in run
result = parse_generated_content(self.state, params.result, documenter)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/ext/autodoc/directive.py", line 104, in parse_generated_content
state.nested_parse(content, 0, node)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 242, in run
context, state, transitions)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 459, in check_line
return method(match, context, next_state)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2342, in explicit_markup
nodelist, blank_finish = self.explicit_construct(match)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
return method(self, expmatch)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2097, in directive
directive_class, match, type_name, option_presets)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
result = directive_instance.run()
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/domains/__init__.py", line 265, in run
return super().run()
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/directives/__init__.py", line 195, in run
self.state.nested_parse(self.content, self.content_offset, contentnode)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 282, in nested_parse
node=node, match_titles=match_titles)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 242, in run
context, state, transitions)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 459, in check_line
return method(match, context, next_state)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2344, in explicit_markup
self.explicit_list(blank_finish)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2374, in explicit_list
match_titles=self.state_machine.match_titles)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 319, in nested_list_parse
node=node, match_titles=match_titles)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 196, in run
results = StateMachineWS.run(self, input_lines, input_offset)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 242, in run
context, state, transitions)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/statemachine.py", line 459, in check_line
return method(match, context, next_state)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2647, in explicit_markup
nodelist, blank_finish = self.explicit_construct(match)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2354, in explicit_construct
return method(self, expmatch)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2097, in directive
directive_class, match, type_name, option_presets)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/docutils/parsers/rst/states.py", line 2146, in run_directive
result = directive_instance.run()
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/domains/__init__.py", line 265, in run
return super().run()
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/directives/__init__.py", line 198, in run
DocFieldTransformer(self).transform_all(contentnode)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/util/docfields.py", line 248, in transform_all
self.transform(child)
File "/Users/venkat/opt/miniconda3/envs/tf-pt/lib/python3.6/site-packages/sphinx/util/docfields.py", line 260, in transform
assert len(field) == 2
AssertionError

Uninstall tensorflow. Now when running make html, it does fininsh building, albeit with a bunch of warnings of the following form: AttributeError: module 'transformers' has no attribute 'TFCamembertForMaskedLM' -- for every TFmethod.

Expected behavior

make html should build with no errors.

Environment info

transformers version: 2.6.0

Platform: macOS 10.15.4

Python version: 3.6.10

PyTorch version (GPU?): 1.4.0 (No)

Tensorflow version (GPU?): 2.1.0 (No)

Using GPU in script?: No

Using distributed or parallel set-up in script?: No

sphinx version: 2.4.4