Repository: kaizendorks/pymongo_inmemory (language: python)

Issue:
Update downloadable versions to include 4.2 and latest minor versions of others
Update download URL look up dictionary to include MongoDB v4.2.2 and also update minor versions of others.

https://www.mongodb.com/download-center/community


Discussion / hints:

Fix patch:
diff --git a/README.md b/README.md
index bcd5fd0..b3d44a8 100644
--- a/README.md
+++ b/README.md
@@ -9,32 +9,25 @@ A mongo mocking library with an ephemeral MongoDB running in memory.
 pip install pymongo-inmemory
 ```
 
-## Configuration
-| Config param     | Description                                                          | Optional? | Default                                          |
-|------------------|----------------------------------------------------------------------|-----------|--------------------------------------------------|
-| mongo_version    | Which MongoD version to download and use.                            | Yes       | 4.0.10                                           |
-| mongod_port      | Override port preference.                                            | Yes       | Automatically determined between 27017 and 28000 |
-| operating_system | This makes sense for Linux setting, where there are several flavours | Yes       | Automatically determined (Generic for Linux)     |
-
-
 ## Usage
 ### Configure
 There are several ways you can configure `pymongo_inmemory`.
 
-1. Insert a new section to your project's `setup.cfg` for the operating system and mongo
+1. Insert a new section titled `pymongo_inmemory` to your project's `setup.cfg`
 version you want to spin up:
     ```ini
     [pymongo_inmemory]
-    mongo_version = 4.0.10
+    operating_system = ubuntu
+    os_version = 18
     mongod_port = 27019
     ```
-2. Define an ALL_CAPS environment variable with prefix `PYMONGOIM__` (attention to trailing double
+2. Define an `ALL_CAPS` environment variables with prefix `PYMONGOIM__` (attention to trailing double
    underscores.) For instance, to override the port, set up an environment variable
    `PYMONGOIM__MONGOD_PORT`.
 
 ### Import and use
-`pymongo_inmemory` wraps the client that comes from `pymongo` and configures and ephemeral server.
-Then you can import `MongoClient` from `pymongo_inmemory` instead of `pymongo` and use it:
+`pymongo_inmemory` wraps the client class `MongoClient` that comes from `pymongo` and configures with an ephemeral MongoDB server.
+You can import this `MongoClient` from `pymongo_inmemory` instead of `pymongo` and use it to perform tests:
 
 ```python
 from pymongo_inmemory import MongoClient
@@ -50,6 +43,88 @@ with MongoClient() as client:
     # do stuff
 ```
 
+## Configuration
+| Config param       | Description                               | Optional? | Default                                          |
+|--------------------|-------------------------------------------|-----------|--------------------------------------------------|
+| `mongo_version`    | Which MongoD version to download and use. | Yes       | Latest for the OS                                            |
+| `mongod_port`      | Override port preference.                 | Yes       | Automatically picked between `27017` and `28000` after testing availability                |
+| `operating_system` | This makes sense for Linux setting, where there are several flavours         | Yes       | Automatically determined (Generic for Linux)*           |
+| `os_version`       | If an operating system has several versions use this parameter to select one | Yes       | Latest versoin of the OS will be selected from the list |
+| `download_url`     | If set, it won't attempt to determine which MongoDB to download. However there won't be a fallback either.| Yes       | Automatically determined from given parameters and using [internal URL bank](pymongo_inmemory/downloader/_patterns.py)**|
+| `ignore_cache`     | Even if there is a downloaded version in the cache, download it again. | Yes       | False               |
+||||
+
+* ****Note 1:*** Generic Linux version offering for MongoDB ends with version **4.0.23**. If the operating system is just `linux` and if selected MongoDB version is higher, it will default to `4.0.23`.
+* *****Note 2:*** URL bank is filled with URLs collected from [release list](https://www.mongodb.com/download-center/community/releases) and [archived released list](https://www.mongodb.com/download-center/community/releases/archive), so if a version is not in the bank you can use the same list to provide an official download link.
+
+
+## Available MongoDB versions
+There is an internal [URL bank](pymongo_inmemory/downloader/_patterns.py) that is filled with URLs collected from
+* [release list](https://www.mongodb.com/download-center/community/releases) and
+* [archived released list](https://www.mongodb.com/download-center/community/releases/archive)
+
+Below table is a summary of possible setting for `operating_system`, `os_version` and available MongoDB versions for
+them to set as `mongo_version` at `major.minor` level.
+
+Note that, not all `major.minor.patch` level is available for all OS versions. For exact patch level range, either see
+release pages of MongoDB or have a look at the internal [URL bank](pymongo_inmemory/downloader/_patterns.py).
+
+| `operating_system` | `os_version` | MongoDB versions (`major.minor`)|
+|-|-|-|
+|`osx`|`generic`*| `2.6`, `3.0`, `3.2`, `3.4`, `3.6`, `4.0`, `4.2`, `4.4`|
+|`windows`|`generic`*| `2.6`, `3.0`, `3.2`, `3.4`, `3.6`, `4.0`, `4.2`, `4.4`|
+|`linux`|`generic`*| `2.6`, `3.0`, `3.2`, `3.4`, `3.6`, `4.0`|
+|`amazon`|`1`| `3.0`, `3.2`, `3.4`, `3.6`, `4.0`, `4.2`, `4.4`|
+|`amazon`|`2`| `3.6`, `4.0`, `4.2`, `4.4`|
+|`debian`|`7`| `3.0`, `3.2`, `3.4`, `3.6`|
+|`debian`|`8`| `3.2`, `3.4`, `3.6`, `4.0`|
+|`debian`|`9`| `3.6`, `4.0`, `4.2`, `4.4`|
+|`debian`|`10`| `4.2`, `4.4`|
+|`rhel`|`5`| `3.0`, `3.2`|
+|`rhel`|`6`| `3.0`, `3.2`, `3.4`, `3.6`, `4.0`, `4.2`, `4.4`|
+|`rhel`|`7`| `3.0`, `3.2`, `3.4`, `3.6`, `4.0`, `4.2`, `4.4`|
+|`rhel`|`8`| `3.6`, `4.0`, `4.2`, `4.4`|
+|`suse`|`11`| `3.0`, `3.2`, `3.4`, `3.6`|
+|`suse`|`12`| `3.2`, `3.4`, `3.6`, `4.0`, `4.2`, `4.4`|
+|`suse`|`15`| `4.2`, `4.4`|
+|`ubuntu`|`12`| `3.0`, `3.2`, `3.4`, `3.6`|
+|`ubuntu`|`14`| `3.0`, `3.2`, `3.4`, `3.6`, `4.0`|
+|`ubuntu`|`16`| `3.2`, `3.4`, `3.6`, `4.0`, `4.2`, `4.4`|
+|`ubuntu`|`18`| `3.6`, `4.0`, `4.2`, `4.4`|
+|`ubuntu`|`20`| `4.4`|
+|`sunos`|`5`| `2.6`, `3.0`, `3.2`, `3.4`|
+||||
+
+****Note:*** No need to specify `generic`, as it will be chosen automatically since it's the only version for that OS.
+
+### How do we determine which MongoDB to download?
+There are two (three if it's a Linux flavour) bits of information we need to determine a MongoDB:
+operating system and MongoDB version.
+
+**Note:** You can always set `download_url` to provide an exact URL to download from.
+
+#### Operating System detection
+Python has limited tools in its standard library to determine the exact version of the operating
+system and operating system version. `pymongo_inmemory` basically reads output of [`platform.system()`](platform.system())
+to determine if underlying OS is Linux, MacOS or Windows.
+
+For Windows and MacOS, it will download only one flavour of OS for a particular MongoDB version (64bit and, for Windows, Windows Server version if there is one.)
+However, Linux has many flavours. Up to MongoDB `4.0.23`, a MongoDB for a generic Linux OS can still be downloaded, but for later
+versions of MongoDB, there are no such builds, hence you will need to explicitly set `operating_system`
+parameter if you want to use MongoDB versions higher than that.
+
+Operating system detection behaviour of `pymongo_inmemory` might change in the future, if there is a demand for more **magic**,
+but for now we are keeping things simple.
+
+#### Deciding MongoDB version
+* If no version is provided, highest version of MongoDB for the operating system is selected.
+* If only a **major** version is given, like `4`, then highest `minor.patch` version is selected, like 4.4.4.
+* If only **major.minor** version is given, like `4.0`, then highest `patch` version is selected, like 4.0.23.
+* If exact **major.minor.patch** version is given, like `4.0.22`, then that version is selected.
+* If patch version is not found. like `4.0.50`, highest `patch` version is selected, like `4.0.23`.
+* If minor version is not found. like `3.90.50`, highest `minor.patch` version is selected, like `3.6.22`.
+* If major version is not found. like `1.0.0`, highest `major.minor.patch` version is selected, like `4.4.4`.
+
 ## Supported Python versions
 Since `pytest` uses [`LocalPath`](https://py.readthedocs.io/en/latest/path.html) for path related
 operations and on python versions older than 3.6 `LocalPath` does not behave well with all path
@@ -82,5 +157,5 @@ If on NIX systems you can run further tests:
 bash tests/integrity/test_integrity.sh
 ```
 
-**See how you can wet your feet,** check out [good first
-issues](https://github.com/kaizendorks/pymongo_inmemory/contribute).
+### See how you can wet your feet
+Check out [good first issues](https://github.com/kaizendorks/pymongo_inmemory/contribute).
diff --git a/extract/bin/mongod b/extract/bin/mongod
new file mode 100644
index 0000000..e69de29
diff --git a/pymongo_inmemory/_utils.py b/pymongo_inmemory/_utils.py
index bf42da8..52611aa 100644
--- a/pymongo_inmemory/_utils.py
+++ b/pymongo_inmemory/_utils.py
@@ -1,8 +1,15 @@
 from configparser import ConfigParser
+from collections import namedtuple
+import logging
 import os
 import socket
 
 
+logger = logging.getLogger("PYMONGOIM_UTILS")
+
+SemVer = namedtuple("SemVer", ["major", "minor", "patch"])
+
+
 def find_open_port(sq):
     for port in sq:
         with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as soc:
@@ -10,9 +17,7 @@ def find_open_port(sq):
                 return port
 
 
-DEFAULT_CONF = {
-    "mongo_version": "4.0.10"
-}
+DEFAULT_CONF = {}
 
 
 def _check_environment_vars(option, fallback=None):
@@ -32,7 +37,26 @@ def _check_cfg(option, filename, fallback=None):
     )
 
 
-def conf(option, fallback=None):
+def make_semver(version=None):
+    if version is None:
+        return SemVer(None, None, None)
+
+    parts = [int(x) for x in version.split(".")]
+    if len(parts) >= 3:
+        major, minor, patch = parts[:3]
+    elif len(parts) == 2:
+        major, minor = parts
+        patch = None
+    elif len(parts) == 1:
+        major = parts[0]
+        minor = patch = None
+    else:
+        major = minor = patch = None
+
+    return SemVer(major, minor, patch)
+
+
+def conf(option, fallback=None, optional=True):
     """Retrieve asked `option` if possible. There are number of places that are checked.
     In the order of precedence,
     1. Environment variables
@@ -54,7 +78,7 @@ def conf(option, fallback=None):
     -------
     any or None: If there is a value, it'll return it otherwise it'll return `None`.
     """
-    return _check_environment_vars(
+    value = _check_environment_vars(
         option,
         fallback=_check_cfg(
             option, "setup.cfg",
@@ -64,3 +88,13 @@ def conf(option, fallback=None):
             )
         )
     )
+
+    if value is None and not optional:
+        raise ValueError(
+            (
+                "Can't determine the value of {} "
+                "and it is not an optional parameter.").format(option))
+
+    logger.debug("Value for {}=={}".format(option, value))
+
+    return value
diff --git a/pymongo_inmemory/downloader.py b/pymongo_inmemory/downloader.py
deleted file mode 100644
index 891c50e..0000000
--- a/pymongo_inmemory/downloader.py
+++ /dev/null
@@ -1,316 +0,0 @@
-"""Downloader module downloads appropriate version of mongo server TAR file.
-The TAR file to download comes from MongoDB Communiti Edition doownload page:
-    https://www.mongodb.com/download-center/community
-
-However, it's not dynamically queried for now. Instead we have a predefined
-patterns for download URLs.
-
-Currently Windows is ignored.
-
-Operating systems can be one of these:
-    osx
-    linux
-    amazonlinux
-    amazonlinux2
-    debian7
-    debian8
-    debian9
-    rhel5
-    rhel6
-    rhel7
-    suse11
-    suse12
-    ubuntu14
-    ubuntu16
-    ubuntu18
-    windows
-If OS can't be determined an exception is thrown.
-
-Versions should be one of these, but not all versions are available for any
-operating system, so it might not be downloaded:
-    4.0
-    3.6
-    3.4
-    3.2
-    3.0
-"""
-
-import glob
-import zipfile
-import logging
-import os
-import platform
-import stat
-import shutil
-import tarfile
-import tempfile
-import urllib.request as request
-from urllib.error import HTTPError
-
-from ._utils import conf
-
-
-TARFILE_PATTERN = "mongodb_archive_{ver}.tgz"
-ZIPFILE_PATTERN = "mongodb_archive_{ver}.zip"
-CACHE_FOLDER = os.path.join(os.path.dirname(__file__), ".cache")
-DOWNLOAD_URL_PATTERNS = {
-    "osx": {
-        "url": "https://fastdl.mongodb.org/osx/mongodb-osx-ssl-x86_64-{ver}.tgz",
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "linux": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-{ver}.tgz",
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "amazonlinux": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-amazon-{ver}.tgz",
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "amazonlinux2": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-amazon2-{ver}.tgz",
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "debian7": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian71-{ver}.tgz",  # noqa E501
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "debian8": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian81-{ver}.tgz",  # noqa E501
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "debian9": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian92-{ver}.tgz",  # noqa E501
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "rhel5": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel55-{ver}.tgz",
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "rhel6": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel62-{ver}.tgz",
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "rhel7": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-{ver}.tgz",
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "suse11": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-suse11-{ver}.tgz",
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "suse12": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-suse12-{ver}.tgz",
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "ubuntu14": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-{ver}.tgz",  # noqa E501
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "ubuntu16": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1604-{ver}.tgz",  # noqa E501
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "ubuntu18": {
-        "url": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-{ver}.tgz",  # noqa E501
-        "file_pattern": TARFILE_PATTERN,
-    },
-    "windows": {
-        "url": "https://fastdl.mongodb.org/win32/mongodb-win32-x86_64-2008plus-ssl-{ver}.zip",  # noqa E501
-        "file_pattern": ZIPFILE_PATTERN,
-    },
-}
-VERSIONS = {
-    "4.0": "4.0.10",
-    "3.6": "3.6.13",
-    "3.4": "3.4.21",
-    "3.2": "3.2.22",
-    "3.0": "3.0.15",
-}
-logger = logging.getLogger("PYMONGOIM_DOWNLOADER")
-
-
-class OperatingSystemNotFound(ValueError):
-    pass
-
-
-class CantDownload(Exception):
-    pass
-
-
-class InvalidDownloadedFile(Exception):
-    pass
-
-
-def _mkdir_ifnot_exist(folder_name):
-    if not os.path.isdir(CACHE_FOLDER):
-        os.mkdir(CACHE_FOLDER)
-    path = os.path.join(CACHE_FOLDER, folder_name)
-    if not os.path.isdir(path):
-        os.mkdir(path)
-    return path
-
-
-def _download_folder():
-    return conf("download_folder", _mkdir_ifnot_exist("download"))
-
-
-def _extract_folder():
-    return conf("extract_folder", _mkdir_ifnot_exist("extract"))
-
-
-def bin_folder():
-    return conf("bin_folder", _mkdir_ifnot_exist("bin"))
-
-
-def _dl_reporter(blocknum, block_size, total_size):
-    percent_dled = blocknum * block_size / total_size * 100
-    size_dlded = blocknum * block_size / 1024 / 1024  # MBs
-    total_size = total_size / 1024 / 1024  # MBs
-    if 0 <= percent_dled % 10 <= 0.01:
-        logger.info("{:.0f} % ({:.0f} MiB of {:.0f} MiB)".format(
-            percent_dled, size_dlded, total_size)
-        )
-
-
-def _copy_bins():
-    extract_folder = _extract_folder()
-    bin_path = bin_folder()
-    for binfile_path in glob.iglob(
-        os.path.join(extract_folder, "**/bin/*"), recursive=True
-    ):
-        binfile_name = os.path.basename(binfile_path)
-        logger.debug("Copying {} to bin folder".format(binfile_name))
-        target = os.path.join(bin_path, binfile_name)
-        shutil.copyfile(binfile_path, target)
-        os.chmod(target, (
-            stat.S_IRUSR
-            | stat.S_IXUSR
-            | stat.S_IRGRP
-            | stat.S_IXGRP
-            | stat.S_IROTH
-            | stat.S_IXOTH
-        ))
-        logger.debug("Copied {}".format(binfile_name))
-
-
-def _download_file(dl_url, dst_file):
-    dl_folder = _download_folder()
-
-    if not os.path.isdir(dl_folder):
-        logger.debug("Download folder doesn't exist, creating it.")
-        os.mkdir(dl_folder)
-
-    if os.path.isfile(dst_file):
-        logger.debug((
-            "There is already a downloaded file {}, "
-            "skipping download"
-        ).format(dst_file))
-        return
-
-    with tempfile.NamedTemporaryFile(delete=False) as temp:
-        logger.debug("Starting download to temporary location {}".format(temp.name))
-        try:
-            request.urlretrieve(dl_url, filename=temp.name, reporthook=_dl_reporter)
-        except HTTPError:
-            raise CantDownload((
-                "Can't download {url}, "
-                "make sure MongoDB provides it. "
-                "Possibly the version is not provided for the operating system."
-            ).format(url=dl_url))
-
-        logger.debug("Finished download.")
-        shutil.copyfile(temp.name, dst_file)
-        logger.debug("Copied file to {}".format(dst_file))
-
-
-def _extract(downloaded_file):
-    extract_folder = _extract_folder()
-    if tarfile.is_tarfile(downloaded_file):
-        _extract_tar(downloaded_file, extract_folder)
-    elif zipfile.is_zipfile(downloaded_file):
-        _extract_zip(downloaded_file, extract_folder)
-    else:
-        raise InvalidDownloadedFile("Expecting either a .tar or a .zip file.")
-
-
-def _extract_tar(tar_file, extract_folder):
-    with tarfile.open(tar_file, "r") as t:
-        logger.info("Starting extraction.")
-        for f in t.getnames():
-            logger.debug("Extracting {} to {}".format(f, extract_folder))
-            t.extract(f, extract_folder)
-            logger.debug("Done extracting {}".format(f))
-        logger.info("Extractiong finished.")
-
-
-def _extract_zip(zip_file, extract_folder):
-    with zipfile.ZipFile(zip_file) as z:
-        logger.info("Starting extraction.")
-        for f in z.namelist():
-            logger.debug("Extracting {} to {}".format(f, extract_folder))
-            z.extract(f, extract_folder)
-            logger.debug("Done extracting {}".format(f))
-        logger.info("Extractiong finished.")
-
-
-def download(opsys=None, version=None):
-    """Download MongoDB binaries.
-
-    Parameters
-    ----------
-    opsys: str
-        Operating system. Should be one of these: osx, linux, amazonlinux, amazonlinux2,
-        debian7, debian8, debian9, rhel5, rhel6, rhel7, suse11, suse12, ubuntu14,
-        ubuntu16, ubuntu18, windows
-        If `None`, then it'll try to determine based on `paltform.system()`, if can't
-        determined `OperatingSystemNotFound` will be raised
-    version: str
-        MongoDB version, should be one of these: 4.0, 3.6, 3.4, 3.2, 3.0
-        Not all versions are available for all operating systems. Check this URL:
-        https://www.mongodb.com/download-center/community
-
-    Raises
-    ------
-    OperatingSystemNotFound: If download pattern can't be determined.
-    """
-    if version is None:
-        version = str(conf("mongo_version"))
-    if opsys is None:
-        opsys = conf("operating_system")
-        if opsys is None:
-            _mapping = {"Darwin": "osx", "Linux": "linux", "Windows": "windows"}
-            opsys = _mapping.get(platform.system())
-            if opsys is None:
-                raise OperatingSystemNotFound("Can't find operating system.")
-
-    version = VERSIONS.get(version, "4.0.10")
-    dl_pattern = DOWNLOAD_URL_PATTERNS.get(opsys)["url"]
-    downloaded_file_pattern = DOWNLOAD_URL_PATTERNS.get(opsys)["file_pattern"]
-
-    dl_url = dl_pattern.format(ver=version)
-    dl_folder = _download_folder()
-    downloaded_file = os.path.join(
-        dl_folder,
-        downloaded_file_pattern.format(ver=version)
-    )
-    dst_file = os.path.join(dl_folder, downloaded_file_pattern.format(ver=version))
-
-    if (
-        os.path.isfile(os.path.join(bin_folder(), "mongod")) or
-        os.path.isfile(os.path.join(bin_folder(), "mongod.exe"))
-    ):
-        return
-
-    if not os.path.isfile(downloaded_file):
-        logger.info("Archive file is not found, {}".format(downloaded_file))
-        _download_file(dl_url, dst_file)
-    else:
-        # There is a downloaded file and mongod is missing, reextract
-        logger.info("Extracting from the archive, {}".format(downloaded_file))
-    _extract(downloaded_file)
-    _copy_bins()
-
-
-if __name__ == "__main__":
-    logging.basicConfig(level=logging.DEBUG)
-    download()
diff --git a/pymongo_inmemory/downloader/README.md b/pymongo_inmemory/downloader/README.md
new file mode 100644
index 0000000..0bb3da9
--- /dev/null
+++ b/pymongo_inmemory/downloader/README.md
@@ -0,0 +1,27 @@
+# Updating download URLs
+Run URL check tool from the root:
+
+```sh
+python -m pymongo_inmemory.downloader.check_urls
+```
+
+It will expnd the URL tree and check all the URLs by sending `HEAD` requests.
+
+The check is done concurrently, but in order to reduce server-sode throttling, there is a random
+sleep between 1 to 4 seconds for each check.
+
+In the end of the check it will report all the URLs that has some problem.
+
+Note that URL checker will not generate new URLs. For instance if there is a new MongoDB version,
+it won't be checked.
+
+The URLs are collected from:
+* [Releases](https://www.mongodb.com/download-center/community/releases) page
+* [Release archive](https://www.mongodb.com/download-center/community/releases/archive) page
+
+The last addition is MongoDB version 4.4.4.
+
+Best way to keep track of the new version is to subscribe to [MongoDB release alerts](https://www.mongodb.com/lp/newsletter/enterprise-release-announcements).
+
+Result of the URL checking should be reflected on the [download patterns](_patterns.py)
+and the main [README](../../README.md)
diff --git a/pymongo_inmemory/downloader/__init__.py b/pymongo_inmemory/downloader/__init__.py
new file mode 100644
index 0000000..ea6e05b
--- /dev/null
+++ b/pymongo_inmemory/downloader/__init__.py
@@ -0,0 +1,246 @@
+import glob
+import zipfile
+import logging
+import os
+from os import path
+import platform
+import stat
+import shutil
+import tarfile
+import tempfile
+import urllib.request as request
+from urllib.error import HTTPError
+
+from .._utils import conf
+from ._urls import best_url
+
+
+CACHE_FOLDER = path.join(path.dirname(__file__), "..", ".cache")
+
+logger = logging.getLogger("PYMONGOIM_DOWNLOADER")
+
+
+class OperatingSystemNotFound(ValueError):
+    pass
+
+
+class CantDownload(Exception):
+    pass
+
+
+class InvalidDownloadedFile(Exception):
+    pass
+
+
+def _mkdir_ifnot_exist(*folders):
+    current_path = path.join(folders[0])
+    if not path.isdir(current_path):
+        os.mkdir(current_path)
+    for x in folders[1:]:
+        current_path = path.join(current_path, x)
+        if not path.isdir(current_path):
+            os.mkdir(current_path)
+    return current_path
+
+
+def _download_folder():
+    return conf("download_folder", _mkdir_ifnot_exist(CACHE_FOLDER, "download"))
+
+
+def _extract_folder():
+    return conf("extract_folder", _mkdir_ifnot_exist(CACHE_FOLDER, "extract"))
+
+
+def _extracted_folder(archive_file):
+    "Defines a nested versioned folder in the extract base folder"
+    base_folder = _extract_folder()
+    base_name = path.basename(archive_file).split(".")
+    file_name = ".".join(base_name[:-1])
+    extension = base_name[-1]
+    archive_folder = "-".join([file_name, extension])
+    return _mkdir_ifnot_exist(base_folder, archive_folder)
+
+
+def bin_folder():
+    return conf("bin_folder", _mkdir_ifnot_exist(CACHE_FOLDER, "bin"))
+
+
+def _dl_reporter(blocknum, block_size, total_size):
+    percent_dled = blocknum * block_size / total_size * 100
+    size_dlded = blocknum * block_size / 1024 / 1024  # MBs
+    total_size = total_size / 1024 / 1024  # MBs
+    if 0 <= percent_dled % 10 <= 0.01:
+        logger.info("{:.0f} % ({:.0f} MiB of {:.0f} MiB)".format(
+            percent_dled, size_dlded, total_size)
+        )
+
+
+def _copy_bins(archive_file):
+    extract_folder = _extracted_folder(archive_file)
+    bin_path = bin_folder()
+    for binfile_path in glob.iglob(
+        path.join(extract_folder, "**/bin/*"), recursive=True
+    ):
+        binfile_name = path.basename(binfile_path)
+        logger.debug("Copying {} to bin folder".format(binfile_name))
+        target = path.join(bin_path, binfile_name)
+        shutil.copyfile(binfile_path, target)
+        os.chmod(target, (
+            stat.S_IRUSR
+            | stat.S_IXUSR
+            | stat.S_IRGRP
+            | stat.S_IXGRP
+            | stat.S_IROTH
+            | stat.S_IXOTH
+        ))
+        logger.debug("Copied {}".format(binfile_name))
+
+
+def _download_file(dl_url, destination_file):
+    dl_folder = _download_folder()
+
+    if not path.isdir(dl_folder):
+        logger.debug("Download folder doesn't exist, creating it.")
+        os.mkdir(dl_folder)
+
+    if path.isfile(destination_file):
+        logger.debug((
+            "There is already a downloaded file {}, "
+            "skipping download"
+        ).format(destination_file))
+        return
+
+    with tempfile.NamedTemporaryFile(delete=False) as temp:
+        logger.debug("Starting download to temporary location {}".format(temp.name))
+        try:
+            request.urlretrieve(dl_url, filename=temp.name, reporthook=_dl_reporter)
+        except HTTPError:
+            raise CantDownload((
+                "Can't download {url}, "
+                "make sure MongoDB provides it. "
+                "Possibly the version is not provided for the operating system."
+            ).format(url=dl_url))
+
+        logger.debug("Finished download.")
+        shutil.copyfile(temp.name, destination_file)
+        logger.debug("Copied file to {}".format(destination_file))
+
+
+def _extract(archive_file):
+    logger.info("Extracting from the archive, {}".format(archive_file))
+    extract_folder = _extracted_folder(archive_file)
+
+    if tarfile.is_tarfile(archive_file):
+        _extract_tar(archive_file, extract_folder)
+    elif zipfile.is_zipfile(archive_file):
+        _extract_zip(archive_file, extract_folder)
+    else:
+        raise InvalidDownloadedFile("Expecting either a .tar or a .zip file.")
+
+
+def _extract_tar(tar_file, extract_folder):
+    with tarfile.open(tar_file, "r") as t:
+        logger.info("Starting extraction.")
+        for f in t.getnames():
+            logger.debug("Extracting {} to {}".format(f, extract_folder))
+            t.extract(f, extract_folder)
+            logger.debug("Done extracting {}".format(f))
+        logger.info("Extractiong finished.")
+
+
+def _extract_zip(zip_file, extract_folder):
+    with zipfile.ZipFile(zip_file) as z:
+        logger.info("Starting extraction.")
+        for f in z.namelist():
+            logger.debug("Extracting {} to {}".format(f, extract_folder))
+            z.extract(f, extract_folder)
+            logger.debug("Done extracting {}".format(f))
+        logger.info("Extractiong finished.")
+
+
+def _collect_archive_name(url):
+    return url.split("/")[-1]
+
+
+def _has_mongod(extracted_folder):
+    for binfile_path in glob.iglob(
+        path.join(extracted_folder, "**/bin/*"), recursive=True
+    ):
+        binfile_name = path.basename(binfile_path)
+        try:
+            binfile_name.index("mongod")
+        except ValueError:
+            continue
+        else:
+            return True
+    return False
+
+
+def download(os_name=None, version=None, os_ver=None, ignore_cache=False):
+    """Download MongoDB binaries.
+    Available versions are collected form this URL:
+    https://www.mongodb.com/download-center/community/releases
+    and this one:
+    https://www.mongodb.com/download-center/community/releases/archive
+
+    Parameters
+    ----------
+    os_name: str
+        If `None`, then it'll try to determine based on `platform.system()`, if can't
+        determined `OperatingSystemNotFound` will be raised
+    version: str
+        MongoDB version
+    os_ver: str
+        Operating system version, if the OS has several versions
+    ignore_cache: bool
+        Download MongoDB, even if there is already one in the cache
+
+    Raises
+    ------
+    OperatingSystemNotFound: If OS is not MacOS, Windows or Linux variant.
+    """
+    if version is None:
+        version = conf("mongo_version")
+
+    if os_name is None:
+        os_name = conf("operating_system")
+        if os_name is None:
+            _mapping = {"Darwin": "osx", "Linux": "linux", "Windows": "windows"}
+            os_name = _mapping.get(platform.system())
+            if os_name is None:
+                raise OperatingSystemNotFound("Can't determine operating system.")
+
+    if os_name == "linux":
+        logger.warn((
+            "Starting from MongoDB 4.0.23 "
+            "there isn't a generic Linux version of MongoDB"
+            ))
+
+    if os_ver is None:
+        os_ver = conf("os_version")
+
+    dl_url = conf("download_url", best_url(
+        os_name,
+        version=version,
+        os_ver=os_ver
+    ))
+
+    logger.debug("Downloading MongoD from {}".format(dl_url))
+    dl_folder = _download_folder()
+    archive_file = path.join(dl_folder, _collect_archive_name(dl_url))
+
+    should_ignore_cache = bool(conf("ignore_cache", ignore_cache))
+    bin_dir = bin_folder()
+
+    if should_ignore_cache or not path.isfile(archive_file):
+        logger.info("Archive file is not found, {}".format(archive_file))
+        _download_file(dl_url, archive_file)
+        _extract(archive_file)
+
+    extracted_dir = _extracted_folder(archive_file)
+    if not _has_mongod(extracted_dir):
+        _extract(archive_file)
+
+    logger.debug("Removing the binary dir")
+    shutil.rmtree(bin_dir)
+    _copy_bins(archive_file)
diff --git a/pymongo_inmemory/downloader/__main__.py b/pymongo_inmemory/downloader/__main__.py
new file mode 100644
index 0000000..c573c11
--- /dev/null
+++ b/pymongo_inmemory/downloader/__main__.py
@@ -0,0 +1,8 @@
+import logging
+
+from . import download
+
+
+if __name__ == "__main__":
+    logging.basicConfig(level=logging.DEBUG)
+    download("ubuntu", os_ver="18")
diff --git a/pymongo_inmemory/downloader/_patterns.py b/pymongo_inmemory/downloader/_patterns.py
new file mode 100644
index 0000000..03bdb0a
--- /dev/null
+++ b/pymongo_inmemory/downloader/_patterns.py
@@ -0,0 +1,626 @@
+PATCH_RANGE = {
+    "2.6": list(range(13)),
+    "3.0": list(range(16)),
+    "3.2": list(range(23)),
+    "3.2-suse11": list(range(20)),
+    "3.2-ubuntu18": list(range(20, 23)),
+    "3.2-ubuntu16": list(range(7, 23)),
+    "3.2-ubuntu12": list(range(20)),
+    "3.2-sunos5": list(range(15)),
+    "3.2-debian8": list(range(8, 23)),
+    "3.2-debian7": list(range(21)),
+    "3.4": list(range(8)) + list(range(9, 25)),
+    "3.4-suse11": list(range(8)) + list(range(9, 15)),
+    "3.4-ubuntu14": list(range(8)) + list(range(9, 21)) + list(range(23, 25)),
+    "3.4-ubuntu12": list(range(8)) + list(range(9, 15)),
+    "3.4-sunos5": list(range(6)),
+    "3.4-debian7": list(range(8)) + list(range(9, 16)),
+    "3.6": list(range(23)),
+    "3.6-suse11": list(range(3)),
+    "3.6-ubuntu18": list(range(20, 23)),
+    "3.6-ubuntu14": list(range(13)) + list(range(14, 23)),
+    "3.6-ubuntu12": list(range(4)),
+    "3.6-rhel8": list(range(17, 23)),
+    "3.6-debian9": list(range(5, 23)),
+    "3.6-debian7": list(range(6)),
+    "3.6-amazon2": [22],
+    "4.0": list(range(24)),
+    "4.0-rhel8": list(range(14, 24)),
+    "4.0-ubuntu14": list(range(10)) + list(range(12, 24)),
+    "4.0-ubuntu18": list(range(1, 24)),
+    "4.2": list(range(4)) + list(range(5, 13)),
+    "4.2-suse15": list(range(1, 4)) + list(range(5, 13)),
+    "4.2-rhel8": list(range(1, 4)) + list(range(5, 13)),
+    "4.2-debian10": list(range(1, 4)) + list(range(5, 13)),
+    "4.4": list(range(5)),
+}
+
+PATTERNS = {
+    "windows32-x86_64": "https://fastdl.mongodb.org/win32/mongodb-win32-x86_64-{}.zip",
+    "windows-2008plus-ssl": "https://fastdl.mongodb.org/win32/mongodb-win32-x86_64-2008plus-ssl-{}.zip",  # noqa E501
+    "windows-2012plus": "https://fastdl.mongodb.org/win32/mongodb-win32-x86_64-2012plus-{}.zip",  # noqa E501
+    "windows-x86_64": "https://fastdl.mongodb.org/windows/mongodb-windows-x86_64-{}.zip",
+    "sunos5": "https://fastdl.mongodb.org/sunos5/mongodb-sunos5-x86_64-{}.tgz",
+    "osx": "https://fastdl.mongodb.org/osx/mongodb-osx-x86_64-{}.tgz",
+    "osx-ssl": "https://fastdl.mongodb.org/osx/mongodb-osx-ssl-x86_64-{}.tgz",
+    "macos": "https://fastdl.mongodb.org/osx/mongodb-macos-x86_64-{}.tgz",
+    "linux": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-{}.tgz",
+    "ubuntu20": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2004-{}.tgz",  # noqa E501
+    "ubuntu18": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-{}.tgz",  # noqa E501
+    "ubuntu16": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1604-{}.tgz",  # noqa E501
+    "ubuntu14": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-{}.tgz",  # noqa E501
+    "ubuntu12": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1204-{}.tgz",  # noqa E501
+    "suse15": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-suse15-{}.tgz",
+    "suse12": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-suse12-{}.tgz",
+    "suse11": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-suse11-{}.tgz",
+    "rhel8": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel80-{}.tgz",
+    "rhel7": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-{}.tgz",
+    "rhel6": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel62-{}.tgz",
+    "rhel5": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel55-{}.tgz",
+    "debian10": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian10-{}.tgz",
+    "debian9": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian92-{}.tgz",
+    "debian8": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian81-{}.tgz",
+    "debian7": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian71-{}.tgz",
+    "amazon2": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-amazon2-{}.tgz",
+    "amazon1": "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-amazon-{}.tgz",
+}
+
+# An index of URL patterns and patch ranges. First with OS and second with MongoDB
+# version, because we are limited by the OS we are running on first, MongoDB
+# version second.
+URLS = {
+    "amazon": {
+        "1": {
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["amazon1"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2"],
+                    "url": PATTERNS["amazon1"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4"],
+                    "url": PATTERNS["amazon1"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6"],
+                    "url": PATTERNS["amazon1"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["amazon1"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["amazon1"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["amazon1"],
+                },
+            },
+        },
+        "2": {
+            3: {
+                6: {
+                    "patches": PATCH_RANGE["3.6-amazon2"],
+                    "url": PATTERNS["amazon2"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["amazon2"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["amazon2"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["amazon2"],
+                },
+            },
+        },
+    },
+    "debian": {
+        "7": {
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["debian7"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2-debian7"],
+                    "url": PATTERNS["debian7"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4-debian7"],
+                    "url": PATTERNS["debian7"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6-debian7"],
+                    "url": PATTERNS["debian7"],
+                },
+            },
+        },
+        "8": {
+            3: {
+                2: {
+                    "patches": PATCH_RANGE["3.2-debian8"],
+                    "url": PATTERNS["debian8"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4"],
+                    "url": PATTERNS["debian8"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6"],
+                    "url": PATTERNS["debian8"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["debian8"],
+                },
+            },
+        },
+        "9": {
+            3: {
+                6: {
+                    "patches": PATCH_RANGE["3.6-debian9"],
+                    "url": PATTERNS["debian9"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["debian9"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["debian9"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["debian9"],
+                },
+            },
+        },
+        "10": {
+            4: {
+                2: {
+                    "patches": PATCH_RANGE["4.2-debian10"],
+                    "url": PATTERNS["debian10"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["debian10"],
+                },
+            },
+        },
+    },
+    "rhel": {
+        "5": {
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["rhel5"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2"],
+                    "url": PATTERNS["rhel5"],
+                },
+            },
+        },
+        "6": {
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["rhel6"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2"],
+                    "url": PATTERNS["rhel6"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4"],
+                    "url": PATTERNS["rhel6"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6"],
+                    "url": PATTERNS["rhel6"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["rhel6"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["rhel6"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["rhel6"],
+                },
+            },
+        },
+        "7": {
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["rhel7"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2"],
+                    "url": PATTERNS["rhel7"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4"],
+                    "url": PATTERNS["rhel7"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6"],
+                    "url": PATTERNS["rhel7"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["rhel7"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["rhel7"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["rhel7"],
+                },
+            },
+        },
+        "8": {
+            3: {
+                6: {
+                    "patches": PATCH_RANGE["3.6-rhel8"],
+                    "url": PATTERNS["rhel8"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0-rhel8"],
+                    "url": PATTERNS["rhel8"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2-rhel8"],
+                    "url": PATTERNS["rhel8"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["rhel8"],
+                },
+            },
+        },
+    },
+    "suse": {
+        "11": {
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["suse11"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2-suse11"],
+                    "url": PATTERNS["suse11"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4-suse11"],
+                    "url": PATTERNS["suse11"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6-suse11"],
+                    "url": PATTERNS["suse11"],
+                },
+            },
+        },
+        "12": {
+            3: {
+                2: {
+                    "patches": PATCH_RANGE["3.2"],
+                    "url": PATTERNS["suse12"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4"],
+                    "url": PATTERNS["suse12"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6"],
+                    "url": PATTERNS["suse12"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["suse12"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["suse12"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["suse12"],
+                },
+            },
+        },
+        "15": {
+            4: {
+                2: {
+                    "patches": PATCH_RANGE["4.2-suse15"],
+                    "url": PATTERNS["suse15"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["suse15"],
+                },
+            },
+        },
+    },
+    "ubuntu": {
+        "12": {
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["ubuntu12"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2-ubuntu12"],
+                    "url": PATTERNS["ubuntu12"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4-ubuntu12"],
+                    "url": PATTERNS["ubuntu12"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6-ubuntu12"],
+                    "url": PATTERNS["ubuntu12"],
+                },
+            },
+        },
+        "14": {
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["ubuntu14"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2"],
+                    "url": PATTERNS["ubuntu14"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4-ubuntu14"],
+                    "url": PATTERNS["ubuntu14"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6-ubuntu14"],
+                    "url": PATTERNS["ubuntu14"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0-ubuntu14"],
+                    "url": PATTERNS["ubuntu14"],
+                },
+            },
+        },
+        "16": {
+            3: {
+                2: {
+                    "patches": PATCH_RANGE["3.2-ubuntu16"],
+                    "url": PATTERNS["ubuntu16"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4"],
+                    "url": PATTERNS["ubuntu16"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6"],
+                    "url": PATTERNS["ubuntu16"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["ubuntu16"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["ubuntu16"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["ubuntu16"],
+                },
+            },
+        },
+        "18": {
+            3: {
+                6: {
+                    "patches": PATCH_RANGE["3.6-ubuntu18"],
+                    "url": PATTERNS["ubuntu18"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0-ubuntu18"],
+                    "url": PATTERNS["ubuntu18"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["ubuntu18"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["ubuntu18"],
+                },
+            },
+        },
+        "20": {
+            4: {
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["ubuntu20"],
+                },
+            },
+        },
+    },
+    "linux": {
+        "generic": {
+            2: {
+                6: {
+                    "patches": PATCH_RANGE["2.6"],
+                    "url": PATTERNS["linux"],
+                },
+            },
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["linux"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2"],
+                    "url": PATTERNS["linux"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4"],
+                    "url": PATTERNS["linux"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6"],
+                    "url": PATTERNS["linux"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["linux"],
+                },
+            },
+        },
+    },
+    "osx": {
+        "generic": {
+            2: {
+                6: {
+                    "patches": PATCH_RANGE["2.6"],
+                    "url": PATTERNS["osx"],
+                },
+            },
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["osx"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2"],
+                    "url": PATTERNS["osx-ssl"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4"],
+                    "url": PATTERNS["osx-ssl"],
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6"],
+                    "url": PATTERNS["osx-ssl"],
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["osx-ssl"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["macos"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["macos"],
+                },
+            },
+        },
+    },
+    "sunos": {
+        "5": {
+            2: {
+                6: {
+                    "patches": PATCH_RANGE["2.6"],
+                    "url": PATTERNS["sunos5"],
+                },
+            },
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["sunos5"],
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2-sunos5"],
+                    "url": PATTERNS["sunos5"],
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4-sunos5"],
+                    "url": PATTERNS["sunos5"],
+                },
+            },
+        },
+    },
+    "windows": {
+        "generic": {
+            2: {
+                6: {
+                    "patches": PATCH_RANGE["2.6"],
+                    "url": PATTERNS["windows32-x86_64"]
+                },
+            },
+            3: {
+                0: {
+                    "patches": PATCH_RANGE["3.0"],
+                    "url": PATTERNS["windows-2008plus-ssl"]
+                },
+                2: {
+                    "patches": PATCH_RANGE["3.2"],
+                    "url": PATTERNS["windows-2008plus-ssl"]
+                },
+                4: {
+                    "patches": PATCH_RANGE["3.4"],
+                    "url": PATTERNS["windows-2008plus-ssl"]
+                },
+                6: {
+                    "patches": PATCH_RANGE["3.6"],
+                    "url": PATTERNS["windows-2008plus-ssl"]
+                },
+            },
+            4: {
+                0: {
+                    "patches": PATCH_RANGE["4.0"],
+                    "url": PATTERNS["windows-2008plus-ssl"]
+                },
+                2: {
+                    "patches": PATCH_RANGE["4.2"],
+                    "url": PATTERNS["windows-2012plus"]
+                },
+                4: {
+                    "patches": PATCH_RANGE["4.4"],
+                    "url": PATTERNS["windows-x86_64"]
+                },
+            },
+        },
+    },
+}
diff --git a/pymongo_inmemory/downloader/_urls.py b/pymongo_inmemory/downloader/_urls.py
new file mode 100644
index 0000000..33ff5d2
--- /dev/null
+++ b/pymongo_inmemory/downloader/_urls.py
@@ -0,0 +1,83 @@
+from collections import namedtuple
+import logging
+
+from .._utils import make_semver
+from ._patterns import URLS
+
+
+logger = logging.getLogger("PYMONGOIM_DOWNLOAD_URL")
+
+ExpandedURL = namedtuple("ExpandedURL", [
+    "os_name",
+    "os_version",
+    "version",
+    "url"
+])
+
+
+class OperatingSystemNameNotFound(ValueError):
+    pass
+
+
+class OperatingSystemVersionNotFound(ValueError):
+    pass
+
+
+def best_url(os_name, version=None, os_ver=None, url_tree=None):
+    if url_tree is None:
+        url_tree = URLS
+
+    try:
+        os_branch = url_tree[str(os_name).lower()]
+    except KeyError:
+        raise OperatingSystemNameNotFound(
+            "Can't find a MongoDB for this OS: {}".format(os_name))
+
+    if os_ver is None:
+        os_ver = max(os_branch.keys())
+    else:
+        os_ver = str(os_ver).lower()
+
+    os_ver = str(os_ver)
+    if os_ver not in os_branch.keys():
+        raise OperatingSystemVersionNotFound(
+            (
+                "Can't find a MongoDB for OS {} "
+                "version {}, available OS versions: {}"
+            ).format(
+                os_name, os_ver, os_branch.keys())
+            )
+
+    version_branch = os_branch[os_ver]
+
+    major, minor, patch = make_semver(version)
+    if major not in version_branch.keys():
+        major = max(version_branch.keys())
+        minor = max(version_branch[major].keys())
+        patch = max(version_branch[major][minor]["patches"])
+    elif minor not in version_branch[major].keys():
+        minor = max(version_branch[major].keys())
+        patch = max(version_branch[major][minor]["patches"])
+    elif patch not in version_branch[major][minor]["patches"]:
+        patch = max(version_branch[major][minor]["patches"])
+
+    logger.info("Requested MongoDB version {}, found version: {}.{}.{}".format(
+        version, major, minor, patch
+    ))
+    version = "{}.{}.{}".format(major, minor, patch)
+    return version_branch[major][minor]["url"].format(version)
+
+
+def expand_url_tree(tree):
+    for os_name, os_leaf in tree.items():
+        for os_version, version_leaf in os_leaf.items():
+            for major, major_leaf in version_leaf.items():
+                for minor, minor_leaf in major_leaf.items():
+                    for patch in minor_leaf["patches"]:
+                        version = "{}.{}.{}".format(major, minor, patch)
+                        yield ExpandedURL(
+                            os_name,
+                            os_version,
+                            version,
+                            minor_leaf["url"].format(version)
+                        )
diff --git a/pymongo_inmemory/downloader/check_urls.py b/pymongo_inmemory/downloader/check_urls.py
new file mode 100644
index 0000000..7e6d78a
--- /dev/null
+++ b/pymongo_inmemory/downloader/check_urls.py
@@ -0,0 +1,78 @@
+from concurrent import futures
+from http.client import HTTPSConnection
+import logging
+import time
+from random import randint
+
+from ._urls import URLS, expand_url_tree, ExpandedURL
+
+
+logger = logging.getLogger("PYMONGOIM_URL_CHECKER")
+
+
+def split(url):
+    # Quick and dirty way to split host from rest, but it would work for our case
+    parts = url.split("/")
+    host = format(parts[2])
+    rest = "/".join([""] + parts[3:])
+    return host, rest
+
+
+def check_url(expanded: ExpandedURL):
+    host, rest = split(expanded.url)
+
+    # sleep some random time to prevent server side throttling
+    time.sleep(randint(1, 4))
+
+    conn = HTTPSConnection(host)
+    conn.request("HEAD", rest)
+    response = conn.getresponse()
+
+    if response.status == 200:
+        logger.debug("SUCCESS: URL check for {} {} MongoDB {}".format(
+            expanded.os_name,
+            expanded.os_version,
+            expanded.version,
+        ))
+    else:
+        logger.error((
+            "FAIL: URL check for {} {} "
+            "MongoDB {}, {}, reason: {} {}"
+        ).format(
+            expanded.os_name,
+            expanded.os_version,
+            expanded.version,
+            expanded.url,
+            response.status,
+            response.reason
+        ))
+        return expanded
+
+
+def failed_url_check(x: futures.Future):
+    return x.result() is not None and x.exception() is not None
+
+
+if __name__ == "__main__":
+    logging.basicConfig(level=logging.INFO)
+    promises = []
+    failed = []
+    logging.info("Starting checks.")
+    with futures.ThreadPoolExecutor() as executor:
+        for expanded in expand_url_tree(URLS):
+            promises.append(executor.submit(check_url, expanded))
+
+        futures.wait(promises)
+        logging.info("All checks done.")
+
+        failed = [x.result() for x in promises if failed_url_check(x)]
+
+    print("= FAILED CHECKS ============================================================")
+    for x in failed:
+        print(
+            expanded.os_name,
+            expanded.os_version,
+            expanded.version,
+            expanded.url,
+        )
+    print("============================================================================")
diff --git a/pyproject.toml b/pyproject.toml
index cf965eb..1d5825e 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "pymongo_inmemory"
-version = "0.1.5"
+version = "0.2.0"
 description = "A mongo mocking library with an ephemeral MongoDB running in memory."
 authors = [
     "Kaizen Dorks <kaizendorks@gmail.com>",


Test patch:
diff --git a/tests/functional/test_download_extract_permission.py b/tests/functional/test_downloader.py
similarity index 96%
rename from tests/functional/test_download_extract_permission.py
rename to tests/functional/test_downloader.py
index aaade7d..95459bc 100644
--- a/tests/functional/test_download_extract_permission.py
+++ b/tests/functional/test_downloader.py
@@ -46,7 +46,7 @@ def test_downloader(make_mongo_payload, urlretrieve_patcher, monkeypatch, tmpdir
     urlretrieve = urlretrieve_patcher(tmpdir / "test_archive.tar")
     monkeypatch.setattr(request, "urlretrieve", urlretrieve)
 
-    downloader.download(opsys="osx")
+    downloader.download(os_name="osx", os_ver="generic", version="4.0.1")
     expected_mongod_path = tmpdir / ".cache" / "bin" / "mongod"
     assert os.path.isfile(expected_mongod_path)
 
diff --git a/tests/unit/test_downloader.py b/tests/unit/test_downloader_functions.py
similarity index 67%
rename from tests/unit/test_downloader.py
rename to tests/unit/test_downloader_functions.py
index c5edb26..ea3e38a 100644
--- a/tests/unit/test_downloader.py
+++ b/tests/unit/test_downloader_functions.py
@@ -26,6 +26,7 @@ def test_default_bin_folder(monkeypatch, tmpdir):
         downloader.bin_folder(),
         path.join(tmpdir, "bin")
     )
+    assert path.exists(path.join(tmpdir, "bin"))
 
 
 def test_default_dl_folder(monkeypatch, tmpdir):
@@ -34,6 +35,7 @@ def test_default_dl_folder(monkeypatch, tmpdir):
         downloader._download_folder(),
         path.join(tmpdir, "download")
     )
+    assert path.exists(path.join(tmpdir, "download"))
 
 
 def test_default_extract_folder(monkeypatch, tmpdir):
@@ -42,14 +44,32 @@ def test_default_extract_folder(monkeypatch, tmpdir):
         downloader._extract_folder(),
         path.join(tmpdir, "extract")
     )
+    assert path.exists(path.join(tmpdir, "extract"))
 
 
-def test_make_folder(monkeypatch, tmpdir):
+def test_extracted_folder(monkeypatch, tmpdir):
     monkeypatch.setattr(downloader, "CACHE_FOLDER", tmpdir)
     assert path.samefile(
-        downloader._mkdir_ifnot_exist("test"),
+        downloader._extracted_folder("mongodb-amazon2-x86_64-1.1.1.tar"),
+        path.join(tmpdir, "extract", "mongodb-amazon2-x86_64-1.1.1-tar")
+    )
+    assert path.samefile(
+        downloader._extracted_folder("mongodb-windows-x86_64-1.1.1.zip"),
+        path.join(tmpdir, "extract", "mongodb-windows-x86_64-1.1.1-zip")
+    )
+
+
+def test_make_folder(monkeypatch, tmpdir):
+    assert path.samefile(
+        downloader._mkdir_ifnot_exist(tmpdir, "test"),
         path.join(tmpdir, "test")
     )
+    assert path.exists(path.join(tmpdir, "test"))
+    assert path.samefile(
+        downloader._mkdir_ifnot_exist(tmpdir, "test2", "nested"),
+        path.join(tmpdir, "test2", "nested")
+    )
+    assert path.exists(path.join(tmpdir, "test2", "nested"))
 
 
 def test_fails_if_os_unknown(monkeypatch):
diff --git a/tests/unit/test_urls.py b/tests/unit/test_urls.py
new file mode 100644
index 0000000..bbb832e
--- /dev/null
+++ b/tests/unit/test_urls.py
@@ -0,0 +1,33 @@
+# flake8: noqa E501
+
+import pytest
+
+import pymongo_inmemory.downloader._urls as utools
+
+def test_best_url():
+    """
+    - Given `major.minor.patch` version:
+        - Starting from `major` to `patch`
+        - If there is an exact match it should take it
+        - If there isn't, it should take the highest and go on with the highest
+    """
+    assert utools.best_url("linux") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-4.0.23.tgz", "Should find latest MongoDB for Linux:Generic"
+    assert utools.best_url("linux", "3") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.6.22.tgz", "Should find latest MongoDB 3 for Linux:Generic"
+    assert utools.best_url("linux", "3.4") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.4.24.tgz", "Should find latest MongoDB 3.4 for Linux:Generic"
+    assert utools.best_url("linux", "2.6.11") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-2.6.11.tgz", "Should find exact MongoDB 2.6.11 for Linux:Generic"
+
+    assert utools.best_url("linux", "4.4.4") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-4.0.23.tgz", "Linux:Generic support ends at 4.0.23, higher versions default to this."
+
+    assert utools.best_url("ubuntu") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu2004-4.4.4.tgz", "Should find latest MongoDB for latest Ubuntu"
+    assert utools.best_url("ubuntu", os_ver=18) == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-4.4.4.tgz", "Should find latest MongoDB for Ubuntu 18"
+    assert utools.best_url("ubuntu", os_ver="14", version="3.0.3") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-3.0.3.tgz", "Should find MongoDB 3.0.3 for Ubuntu 14"
+
+    with pytest.raises(utools.OperatingSystemNameNotFound):
+        utools.best_url("xubuntu", os_ver="14", version="3.0.3")
+
+    with pytest.raises(utools.OperatingSystemVersionNotFound):
+        utools.best_url("ubuntu", os_ver="10", version="3.0.3")
+
+    assert utools.best_url("ubuntu", os_ver="14", version="1.4.4") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-4.0.23.tgz", "If major version is not there should find latest major version of MongoDB for Ubuntu 14"
+    assert utools.best_url("ubuntu", os_ver="14", version="3.8.4") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-3.6.22.tgz", "If minor version is not there should find latest minor version of MongoDB for given major version, for Ubuntu 14"
+    assert utools.best_url("ubuntu", os_ver="14", version="3.4.22") == "https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-3.4.24.tgz", "If patch version is not there should find latest patch version of MongoDB for given major.minor version, for Ubuntu 14"
diff --git a/tests/unit/test_utils.py b/tests/unit/test_utils.py
index 3f52c9c..2c0dfad 100644
--- a/tests/unit/test_utils.py
+++ b/tests/unit/test_utils.py
@@ -35,3 +35,29 @@ def test_environment_var_option(monkeypatch):
     assert _utils.conf("some_value") == "42"
     assert _utils.conf("SOME_VALUE") == "42"
     assert _utils.conf("SOME_OTHER_VALUE") is None
+
+
+def test_make_semver():
+    expected = _utils.make_semver("1.2.3")
+    assert expected == _utils.SemVer(1, 2, 3)
+    assert expected.major == 1
+    assert expected.minor == 2
+    assert expected.patch == 3
+
+    expected = _utils.make_semver("4")
+    assert expected == _utils.SemVer(4, None, None)
+    assert expected.major == 4
+    assert expected.minor is None
+    assert expected.patch is None
+
+    expected = _utils.make_semver("4.1")
+    assert expected == _utils.SemVer(4, 1, None)
+    assert expected.major == 4
+    assert expected.minor == 1
+    assert expected.patch is None
+
+    expected = _utils.make_semver()
+    assert expected == _utils.SemVer(None, None, None)
+    assert expected.major is None
+    assert expected.minor is None
+    assert expected.patch is None