Repository: pypi/warehouse

Issue:
Inconsistent handling of packages with equivalent version numbers (e.g. v4.1 & v4.1.0)
## Situation
Yesterday I uploaded a new release (**v4.1**) of 'theonionbox' to `https://upload.pypi.org/legacy/`.
Short after the upload I discovered that I made a mistake in the packaging process. With the intension to fix this 'silently' = without (relevant) change of the version number, I created a **v4.1.0** package and pushed it to PyPI. PyPI accepted this without complaint.

## Finding
Today, the comparison of the legacy system [pypi.python.org](https://pypi.python.org/) vs the _warehouse_ based [new one](https://pypi.org/) reveals what looks like a difference in the version number sorting algorithm:

[pypi.python.org](https://pypi.python.org/pypi/theonionbox) offers v4.1.0 as the latest version...
<p align='center'>
<img width="80%" alt="4 1 0" src="https://user-images.githubusercontent.com/16342003/34959213-c6736858-fa35-11e7-8f05-11924201219a.png">
</p>

... whereas [pypi.org](https://pypi.org/project/theonionbox/) announces v4.1:
<p align='center'>
<img width="80%" alt="4 1" src="https://user-images.githubusercontent.com/16342003/34959257-02eec3a4-fa36-11e7-81f3-d71f56c5b750.png">
</p>

## Some further remarks

- The [Python source code](https://svn.python.org/projects/stackless/trunk/Lib/distutils/version.py) explains that v4.1 and v4.1.0 are equivalent:
<p align='center'>
<img width="80%" alt="strict" src="https://user-images.githubusercontent.com/16342003/34959520-298f806a-fa37-11e7-96a6-5c48bccae081.png">
</p>

- Performing a `pip install theonionbox` collects v4.1.

- Performing an explicit `pip install theonionbox==4.1.0` collects v4.1 as well.

- The sequencing in the [release history](https://pypi.org/project/theonionbox/#history) of pypi.org displays v4.1.0 prior to v4.1 - which is technically wrong with regard to the upload time:
<p align='center'>
<img width="80%" alt="history" src="https://user-images.githubusercontent.com/16342003/34959868-aaec8292-fa38-11e7-98d1-f3cae7d93fdb.png">
</p>

- Explicitly selecting v4.1.0, [pypi.org](https://pypi.org/project/theonionbox/4.1.0/) as well states that this is the 'Latest Version':
<p align='center'>
<img width="80%" alt="latest" src="https://user-images.githubusercontent.com/16342003/34960655-04808aee-fa3c-11e7-91d4-4eadcd63b649.png">
</p>

- After a click on this 'Latest Version' button ... pypi.org crashed. :anguished:

## Summary

- pypi.python.org and pypi.org are not consistent in the way they handle this special case of version numbering.

- One could argue that pypi.python.org is 'more correct' as it respects the sequencing of the file upload.

## Proposal
- Option A: Change _warehouse_ to follow the legacy algorithm.

- Option B: As the Python source explicitly states, that v4.1 and v4.1.0 are equivalent, _warehouse_ should treat them as equivalent and reject uploads of that kind.

My preference is **Option B**.


Discussion:
@ralphwetzel Thanks for your detailed and thorough bug report, and sorry for the slow response!

I spoke with @di about this and we agree with **Option B**:

~~~
>>> from packaging.version import parse
>>> parse('4.1') == parse('4.1.0')
True
~~~

You mentioned that pypi.org crashed when you tried to select ‘Latest Version’, and that button ought to go to the page you were already on. Do you remember any details about the crash? Was it an error 500?

Thanks and sorry again for the wait.

For future developers who want to fix this: We get the release for a file upload at https://github.com/pypa/warehouse/blob/master/warehouse/forklift/legacy.py#L798-L803 and the relevant line is `(Release.version == form.version.data)).one()` (assuming the version strings match, not equivalences like `4.1` == `4.1.0`).

Yea, in addition to the "display" version (that keeps track of what the author actually put) we should probably add a normalized version column which is part of the uniqueness of the table, which does things like strip padded zeros and the like.
I feel like this is something the `packaging` module should be able to do, so I made an issue: https://github.com/pypa/packaging/issues/121
@brainwane Yes, it was a 500.

Resolving patch:
diff --git a/warehouse/forklift/legacy.py b/warehouse/forklift/legacy.py
--- a/warehouse/forklift/legacy.py
+++ b/warehouse/forklift/legacy.py
@@ -811,53 +811,69 @@ def file_upload(request):
                             (Release.version == form.version.data)).one()
         )
     except NoResultFound:
-        release = Release(
-            project=project,
-            _classifiers=[
-                c for c in all_classifiers
-                if c.classifier in form.classifiers.data
-            ],
-            _pypi_hidden=False,
-            dependencies=list(_construct_dependencies(
-                form,
-                {
-                    "requires": DependencyKind.requires,
-                    "provides": DependencyKind.provides,
-                    "obsoletes": DependencyKind.obsoletes,
-                    "requires_dist": DependencyKind.requires_dist,
-                    "provides_dist": DependencyKind.provides_dist,
-                    "obsoletes_dist": DependencyKind.obsoletes_dist,
-                    "requires_external": DependencyKind.requires_external,
-                    "project_urls": DependencyKind.project_url,
-                }
-            )),
-            **{
-                k: getattr(form, k).data
-                for k in {
-                    # This is a list of all the fields in the form that we
-                    # should pull off and insert into our new release.
-                    "version",
-                    "summary", "description", "license",
-                    "author", "author_email", "maintainer", "maintainer_email",
-                    "keywords", "platform",
-                    "home_page", "download_url",
-                    "requires_python",
-                }
-            }
-        )
-        request.db.add(release)
-        # TODO: This should be handled by some sort of database trigger or a
-        #       SQLAlchemy hook or the like instead of doing it inline in this
-        #       view.
-        request.db.add(
-            JournalEntry(
-                name=release.project.name,
-                version=release.version,
-                action="new release",
-                submitted_by=request.user,
-                submitted_from=request.remote_addr,
-            ),
+        # We didn't find a release with the exact version string, try and see
+        # if one exists with an equivalent parsed version
+        releases = (
+            request.db.query(Release)
+            .filter(Release.project == project)
+            .all()
         )
+        versions = {
+            packaging.version.parse(release.version): release
+            for release in releases
+        }
+
+        try:
+            release = versions[packaging.version.parse(form.version.data)]
+        except KeyError:
+            release = Release(
+                project=project,
+                _classifiers=[
+                    c for c in all_classifiers
+                    if c.classifier in form.classifiers.data
+                ],
+                _pypi_hidden=False,
+                dependencies=list(_construct_dependencies(
+                    form,
+                    {
+                        "requires": DependencyKind.requires,
+                        "provides": DependencyKind.provides,
+                        "obsoletes": DependencyKind.obsoletes,
+                        "requires_dist": DependencyKind.requires_dist,
+                        "provides_dist": DependencyKind.provides_dist,
+                        "obsoletes_dist": DependencyKind.obsoletes_dist,
+                        "requires_external": DependencyKind.requires_external,
+                        "project_urls": DependencyKind.project_url,
+                    }
+                )),
+                **{
+                    k: getattr(form, k).data
+                    for k in {
+                        # This is a list of all the fields in the form that we
+                        # should pull off and insert into our new release.
+                        "version",
+                        "summary", "description", "license",
+                        "author", "author_email",
+                        "maintainer", "maintainer_email",
+                        "keywords", "platform",
+                        "home_page", "download_url",
+                        "requires_python",
+                    }
+                }
+            )
+            request.db.add(release)
+            # TODO: This should be handled by some sort of database trigger or
+            #       a SQLAlchemy hook or the like instead of doing it inline in
+            #       this view.
+            request.db.add(
+                JournalEntry(
+                    name=release.project.name,
+                    version=release.version,
+                    action="new release",
+                    submitted_by=request.user,
+                    submitted_from=request.remote_addr,
+                ),
+            )
 
     # TODO: We need a better solution to this than to just do it inline inside
     #       this method. Ideally the version field would just be sortable, but


Test patch:
diff --git a/tests/unit/forklift/test_legacy.py b/tests/unit/forklift/test_legacy.py
--- a/tests/unit/forklift/test_legacy.py
+++ b/tests/unit/forklift/test_legacy.py
@@ -2253,6 +2253,52 @@ def test_upload_succeeds_creates_release(self, pyramid_config, db_request):
             ),
         ]
 
+    def test_equivalent_version_one_release(self, pyramid_config, db_request):
+        '''
+        Test that if a release with a version like '1.0' exists, that a future
+        upload with an equivalent version like '1.0.0' will not make a second
+        release
+        '''
+        pyramid_config.testing_securitypolicy(userid=1)
+
+        user = UserFactory.create()
+        project = ProjectFactory.create()
+        release = ReleaseFactory.create(project=project, version="1.0")
+        RoleFactory.create(user=user, project=project)
+
+        db_request.user = user
+        db_request.remote_addr = "10.10.10.20"
+        db_request.POST = MultiDict({
+            "metadata_version": "1.2",
+            "name": project.name,
+            "version": "1.0.0",
+            "summary": "This is my summary!",
+            "filetype": "sdist",
+            "md5_digest": "335c476dc930b959dda9ec82bd65ef19",
+            "content": pretend.stub(
+                filename="{}-{}.tar.gz".format(project.name, "1.0.0"),
+                file=io.BytesIO(b"A fake file."),
+                type="application/tar",
+            ),
+        })
+
+        storage_service = pretend.stub(store=lambda path, filepath, meta: None)
+        db_request.find_service = lambda svc: storage_service
+
+        resp = legacy.file_upload(db_request)
+
+        assert resp.status_code == 200
+
+        # Ensure that a Release object has been created.
+        releases = (
+            db_request.db.query(Release)
+            .filter(Release.project == project)
+            .all()
+        )
+
+        # Asset that only one release has been created
+        assert releases == [release]
+
     def test_upload_succeeds_creates_project(self, pyramid_config, db_request):
         pyramid_config.testing_securitypolicy(userid=1)