Repository: python/mypy
Pull request: Update third party modules
Fixes #6767 🔧

Updated the third party module list with the top 200 packages of the past 365 days from [here](https://hugovk.github.io/top-pypi-packages/). Not entirely sure how reliable the data is, however, the [old source](http://python3wos.appspot.com/) has been deprecated and has not been updated in a while.

Problem statement:
Update set of recognized third party modules
`mypy/moduleinfo.py` contains a hard-coded list of third party modules that mypy uses to generate better error messages in some cases. This list hasn't been updated in a while. Update the list based on up-to-date package popularity information.

Hints from issue discussion:
👍 
Taking a look at this

Commit message: Update third party modules

Fixes #6767. Updated the third party module list with the top 200
packages of the past 365 days from: https://hugovk.github.io/top-pypi-packages/

Patch to review:
diff --git a/mypy/moduleinfo.py b/mypy/moduleinfo.py
index 09df39dad227..8545e0a67c2f 100644
--- a/mypy/moduleinfo.py
+++ b/mypy/moduleinfo.py
@@ -21,198 +21,205 @@
 
 third_party_modules = {
     # From Python 3 Wall of Superpowers (https://python3wos.appspot.com/)
-    'simplejson',
-    'requests',
-    'six',
     'pip',
-    'virtualenv',
-    'boto',
-    'setuptools',
-    'dateutil',
-    'pytz',
+    'urllib3',
+    'six',
+    'botocore',
+    'python-dateutil',
+    's3transfer',
+    'pyyaml',
+    'requests',
     'pyasn1',
-    'nose',
-    'yaml',
-    'lxml',
-    'jinja2',
     'docutils',
-    'markupsafe',
-    'paramiko',
-    'Crypto',
+    'jmespath',
+    'certifi',
     'rsa',
-    'django',
-    'coverage',
-    'pika',
+    'setuptools',
+    'idna',
+    'awscli',
+    'futures',
     'colorama',
-    'ecdsa',
-    'psycopg2',
-    'httplib2',
-    'mock',
+    'chardet',
+    'wheel',
+    'simplejson',
+    'boto3',
+    'pytz',
+    'numpy',
+    'markupsafe',
+    'jinja2',
+    'cffi',
+    'cryptography',
+    'protobuf',
+    'awscli-cwlogs',
+    'enum34',
     'pycparser',
-    'pep8',
-    'redis',
-    'pymongo',
-    'sqlalchemy',
-    'pygments',
+    'asn1crypto',
+    'attrs',
+    'click',
+    'ipaddress',
+    'pytest',
+    'pyasn1-modules',
+    'future',
+    'decorator',
+    'pbr',
+    'google-api-core',
+    'pandas',
     'werkzeug',
-    'selenium',
-    'supervisor',
-    'zc.buildout',
-    'meld3',
-    'PIL',
-    'paste',
+    'pyparsing',
     'flask',
-    '_mysql',
-    'MySQLdb',
-    'greenlet',
-    'carbon',
     'psutil',
-    'paste.deploy',
-    'kombu',
-    'babel',
-    'anyjson',
+    'itsdangerous',
+    'google-cloud-core',
+    'grpcio',
+    'cachetools',
+    'virtualenv',
+    'google-auth',
     'py',
-    'OpenSSL',
-    'numpy',
-    'sphinx',
+    'pluggy',
+    'scipy',
+    'boto',
+    'coverage',
+    'mock',
+    'pyopenssl',
+    'scikit-learn',
+    'jsonschema',
+    'argparse',
+    'more-itertools',
+    'pygments',
+    'psycopg2',
+    'websocket-client',
+    'pillow',
+    'google-api-python-client',
+    'httplib2',
+    'googleapis-common-protos',
+    'matplotlib',
+    'oauth2client',
+    'docopt',
     'tornado',
-    'zope.interface',
-    'itsdangerous',
-    'fabric',
-    'amqp',
-    'mako',
-    'pyflakes',
-    'concurrent.futures',
-    'pyparsing',
-    'celery',
-    'unittest2',
-    'setuptools_git',
-    'decorator',
-    'ordereddict',
-    'urllib3',
-    'iso8601',
-    'gunicorn',
-    'prettytable',
-    'webob',
-    'flake8',
-    'pytest',
-    'billiard',
-    'backports.ssl_match_hostname',
-    'south',
+    'funcsigs',
+    'lxml',
+    'prompt-toolkit',
+    'paramiko',
+    'pyjwt',
+    'ipython',
+    'docker',
+    'docker-pycreds',
+    'oauthlib',
+    'mccabe',
+    'google-resumable-media',
+    'sqlalchemy',
+    'wrapt',
+    'bcrypt',
+    'ptyprocess',
+    'google-cloud-storage',
+    'requests-oauthlib',
+    'multidict',
+    'markdown',
+    'pexpect',
+    'atomicwrites',
+    'google-cloud-bigquery',
+    'uritemplate',
+    'pynacl',
+    'pycodestyle',
+    'elasticsearch',
+    'absl-py',
+    'aiohttp',
+    'redis',
+    'backports-functools-lru-cache',
+    'sklearn',
     'gevent',
-    'netaddr',
-    'pylint',
-    'logilab.common',
-    'twisted',
-    'msgpack',
-    'blessings',
-    'oauth2client',
+    'pymysql',
+    'wcwidth',
+    'tqdm',
+    'beautifulsoup4',
+    'functools32',
+    'configparser',
+    'gunicorn',
+    'typing',
     'ujson',
-    'mozrunner',
-    'googleapiclient',
-    'markdown',
-    'pyasn1',
+    'pyflakes',
+    'packaging',
+    'lazy-object-proxy',
+    'ipython-genutils',
+    'toolz',
+    'async-timeout',
+    'traitlets',
+    'kiwisolver',
+    'pathlib2',
+    'greenlet',
+    'networkx',
+    'opencv-python',
+    'termcolor',
+    'babel',
+    'google-auth-httplib2',
+    'django',
+    'pymemcache',
+    'scikit-image',
+    'pickleshare',
+    'flake8',
+    'cycler',
+    'requests-toolbelt',
+    'bleach',
+    'scandir',
+    'selenium',
+    'dask',
+    'websockets',
+    'isort',
+    'h5py',
+    'tabulate',
+    'tensorflow',
     'html5lib',
+    'pylint',
+    'tensorboard',
+    'docker-compose',
+    'astroid',
+    'trueskill',
+    'webencodings',
+    'defusedxml',
+    'pykube',
+    'pymongo',
+    'retrying',
+    'cached-property',
+    'zope-interface',
+    'singledispatch',
+    'azure-common',
+    'tzlocal',
+    'datadog',
+    'pyzmq',
+    'discord-py',
+    'apache-beam',
+    'subprocess32',
+    'astor',
+    'backports-ssl-match-hostname',
+    'entrypoints',
+    'gast',
+    'nose',
+    'smmap2',
+    'gitdb2',
     'isodate',
-    'tox',
-    'mozdevice',
-    'cython',
-    'raven',
-    'mozprocess',
-    'mozprofile',
-    'mozinfo',
-    'mozlog',
-    'pandas',
+    'pywavelets',
+    'simplegeneric',
+    'sortedcontainers',
+    'psycopg2-binary',
+    'pytest-cov',
+    'hiredis',
+    'elasticsearch-dsl',
+    'dill',
+    'keras-preprocessing',
+    'keras-applications',
+    'contextlib2',
+    'ordereddict',
+    'hdfs',
+    'jupyter-core',
+    'typed-ast',
+    'croniter',
+    'azure-nspkg',
+    'nbformat',
+    'xmltodict',
     'lockfile',
-    'pycurl',
-    'novaclient',
-    'ply',
-    'eventlet',
-    'suds',
-    'zmq',
-    'memcache',
-    'netifaces',
-    'repoze.lru',
-    'testtools',
-    'cups',
-    'oauth2',
-    'scipy',
-    'thrift',
-    'statsd',
-    'BeautifulSoup',
-    'bs4',
-    'sklearn',
-    'cmd2',
-    'alembic',
-    'gflags',
-    'gflags_validators',
-    'sqlparse',
-    'debug_toolbar',
-    'cov_core',
-    'unidecode',
-    'websocket',
-    'webtest',
-    'django_extensions',
-    'networkx',
-    'newrelic',
-    'pymysql',
-    'pytest_cov',
-    'matplotlib',
-    'djcelery',
-    'google.protobuf',
-    'compressor',
-    'aspen',
-    'subunit',
-    'libcloud',
-    'versiontools',
-    'chardet',
-    'blinker',
-    'pystache',
-    'passlib',
-    'xlrd',
-    'pygeoip',
-    'configobj',
-    'cjson',
-    'testrepository',
-    'zc.recipe.egg',
-    'jsonpointer',
-    'amqplib',
-    'rdflib',
-    'SPARQLWrapper',
-    'jsonpatch',
-    'mrjob',
-    'pexpect',
-    'google.apputils',
-    'ansible',
-    'django_nose',
-    'd2to1',
-    'routes',
-    'tempita',
-    'testscenarios',
-    'demjson',
-    'openid',
-    'uwsgidecorators',
-    'wtforms',
-    'dns',
-    'alabaster',
-    'M2Crypto',
-    'reportlab',
-    'feedparser',
-    'ldap',
-    'hgtools',
-    'whoosh',
-    'path',
-    'nosexcover',
-    'iptools',
-    'xlwt',
-    'keyring',
-    'termcolor',
-    'flask_sqlalchemy',
-    'httpretty',
-    'magic',
-    'leaderboard',
-    'sqlobject',
-    'nltk',
+    'arrow',
+    'parso',
+    'jsonpickle',
 
     # Skipped (name considered too generic):
     # - fixtures


Code review comments:
diff_hunk: @@ -21,198 +21,205 @@
 
 third_party_modules = {
     # From Python 3 Wall of Superpowers (https://python3wos.appspot.com/)
-    'simplejson',
-    'requests',
-    'six',
     'pip',
-    'virtualenv',
-    'boto',
-    'setuptools',
-    'dateutil',
-    'pytz',
+    'urllib3',
+    'six',
+    'botocore',
+    'python-dateutil',
path: mypy/moduleinfo.py
text: @user1:
This is the PyPI distribution name, but the actual package name is different, and the package name is required in this file. In this case the package name is `dateutil`. Unfortunately, some manual tweaking seems to be required to get the package name to import.

diff_hunk: @@ -21,198 +21,205 @@
 
 third_party_modules = {
     # From Python 3 Wall of Superpowers (https://python3wos.appspot.com/)
-    'simplejson',
-    'requests',
-    'six',
     'pip',
-    'virtualenv',
-    'boto',
-    'setuptools',
-    'dateutil',
-    'pytz',
+    'urllib3',
+    'six',
+    'botocore',
+    'python-dateutil',
+    's3transfer',
+    'pyyaml',
+    'requests',
     'pyasn1',
-    'nose',
-    'yaml',
-    'lxml',
-    'jinja2',
     'docutils',
-    'markupsafe',
-    'paramiko',
-    'Crypto',
+    'jmespath',
+    'certifi',
     'rsa',
-    'django',
-    'coverage',
-    'pika',
+    'setuptools',
+    'idna',
+    'awscli',
+    'futures',
     'colorama',
-    'ecdsa',
-    'psycopg2',
-    'httplib2',
-    'mock',
+    'chardet',
+    'wheel',
+    'simplejson',
+    'boto3',
+    'pytz',
+    'numpy',
+    'markupsafe',
+    'jinja2',
+    'cffi',
+    'cryptography',
+    'protobuf',
+    'awscli-cwlogs',
path: mypy/moduleinfo.py
text: @user1:
This is another PyPI distribution name that isn't a real Python package name. There are some other examples below.