internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: google/osv.dev

Issue:
Include `modified` timestamp in bulk query results. 
This allows clients to cache results reliably by using ID+modified as the key.


Discussion:

Resolving patch:
diff --git a/gcp/api/server.py b/gcp/api/server.py
--- a/gcp/api/server.py
+++ b/gcp/api/server.py
@@ -147,7 +147,7 @@ def bug_to_response(bug, include_details=True):
   if include_details:
     return bug.to_vulnerability(include_source=True)
 
-  return {'id': bug.id()}
+  return bug.to_vulnerability_minimal()
 
 
 def _get_bugs(bug_ids, to_response=bug_to_response):
diff --git a/osv/models.py b/osv/models.py
--- a/osv/models.py
+++ b/osv/models.py
@@ -516,6 +516,16 @@ def update_from_vulnerability(self, vulnerability):
       self.database_specific = json_format.MessageToDict(
           vulnerability.database_specific, preserving_proto_field_name=True)
 
+  def to_vulnerability_minimal(self):
+    """Convert to Vulnerability proto (minimal)."""
+    if self.last_modified:
+      modified = timestamp_pb2.Timestamp()
+      modified.FromDatetime(self.last_modified)
+    else:
+      modified = None
+
+    return vulnerability_pb2.Vulnerability(id=self.id(), modified=modified)
+
   def to_vulnerability(self, include_source=False):
     """Convert to Vulnerability proto."""
     package = None


Test patch:
diff --git a/gcp/api/integration_tests.py b/gcp/api/integration_tests.py
--- a/gcp/api/integration_tests.py
+++ b/gcp/api/integration_tests.py
@@ -290,13 +290,15 @@ def test_query_batch(self):
             'results': [
                 {
                     'vulns': [{
-                        'id': 'GHSA-qc84-gqf4-9926'
+                        'id': 'GHSA-qc84-gqf4-9926',
+                        'modified': '2022-06-08T15:22:39Z'
                     }]
                 },
                 {},
                 {
                     'vulns': [{
-                        'id': 'OSV-2020-744'
+                        'id': 'OSV-2020-744',
+                        'modified': '2022-04-13T03:04:39.780694Z'
                     }]
                 },
             ]
Repository: google/osv.dev

Issue:
Debian: overlapping versions
e.g. on https://osv.dev/vulnerability/DSA-5087-1

"2.1.27+dfsg-1+deb10u2" is the fixed version for Debian 10. However, Debian 11 includes "2.1.27+dfsg-1+deb10u2" in the list of affected versions. 

We may need to do some filtering based on the +debXXuX" string here while enumerating for a particular Debian release. 


Discussion:
Some of the older debian releases actually don't use the `+debXXuX` syntax, but use `+<codename>uX`. For example:
`2.1.22.dfsg1-23+squeeze1`.

Resolving patch:
diff --git a/osv/ecosystems.py b/osv/ecosystems.py
--- a/osv/ecosystems.py
+++ b/osv/ecosystems.py
@@ -342,10 +342,16 @@ def enumerate_versions(self, package, introduced, fixed, limits=None):
       ) from ex
 
     response = json.loads(text_response)
-    versions = [x['version'] for x in response['result']]
+    versions: list[str] = [x['version'] for x in response['result']]
     # Sort to ensure it is in the correct order
     versions.sort(key=self.sort_key)
 
+    # The only versions with +deb
+    versions = [
+        x for x in versions
+        if '+deb' not in x or f'+deb{self.debian_release_ver}' in x
+    ]
+
     if introduced == '0':
       # Update introduced to the first version of the debian version
       introduced = debian_version_cache.get_first_package_version(


Test patch:
diff --git a/osv/ecosystems_test.py b/osv/ecosystems_test.py
--- a/osv/ecosystems_test.py
+++ b/osv/ecosystems_test.py
@@ -93,6 +93,14 @@ def test_debian(self):
     self.assertGreater(
         ecosystem.sort_key('1.13.6-2'), ecosystem.sort_key('1.13.6-1'))
 
+    # Test Ecosystem remover
+    ecosystem = ecosystems.get('Debian:10')
+    versions = ecosystem.enumerate_versions('cyrus-sasl2', '2.1.27+dfsg-1',
+                                            None)
+    self.assertIn('2.1.27+dfsg-1+deb10u1', versions)
+    self.assertNotIn('2.1.27~101-g0780600+dfsg-3+deb9u1', versions)
+    self.assertNotIn('2.1.27~101-g0780600+dfsg-3+deb9u2', versions)
+
     with self.assertRaises(ecosystems.EnumerateError):
       ecosystem.next_version('doesnotexist123456', '1')
Repository: google/osv.dev

Issue:
Missing HTML escaping in advisory description
See https://osv.dev/vulnerability/GHSA-prc3-vjfx-vhm9 for example, the XSS example is actually interpreted as HTML and breaks the page.


Discussion:

Resolving patch:
diff --git a/gcp/appengine/frontend_handlers.py b/gcp/appengine/frontend_handlers.py
--- a/gcp/appengine/frontend_handlers.py
+++ b/gcp/appengine/frontend_handlers.py
@@ -403,7 +403,8 @@ def group_versions(versions):
 def markdown(text):
   """Render markdown."""
   if text:
-    return markdown2.markdown(text, extras=['fenced-code-blocks'])
+    return markdown2.markdown(
+        text, safe_mode='escape', extras=['fenced-code-blocks'])
 
   return ''
 


Test patch:
Repository: google/osv.dev

Issue:
CVE aliases and reverse aliases are not being shown
Example:
 https://osv.dev/vulnerability/CVE-2022-27449
and 
https://osv.dev/vulnerability/DLA-3114-1

should have links referencing each other, but does not.


Discussion:

Resolving patch:
diff --git a/gcp/appengine/frontend_handlers.py b/gcp/appengine/frontend_handlers.py
--- a/gcp/appengine/frontend_handlers.py
+++ b/gcp/appengine/frontend_handlers.py
@@ -38,7 +38,6 @@
 _PAGE_SIZE = 16
 _PAGE_LOOKAHEAD = 4
 _REQUESTS_PER_MIN = 30
-_MAX_LINKING_ALIASES = 8
 
 if utils.is_prod():
   redis_host = os.environ.get('REDISHOST', 'localhost')
@@ -196,25 +195,25 @@ def add_related_aliases(bug: osv.Bug, response):
   """Add links to other osv entries that's related through aliases"""
   # Add links to other entries if they exist
   aliases = {}
-  for alias in bug.aliases:
-    # only if there aren't too many, otherwise skip this
-    if len(bug.aliases) <= _MAX_LINKING_ALIASES:
-      result = bug.get_by_id(alias)
-    else:
-      result = False
-    aliases[alias] = {'exists': result, 'same_alias_entries': []}
+  if bug.aliases:
+    directly_refed = osv.Bug.query(osv.Bug.db_id.IN(bug.aliases))
+    is_directly_refed = {dr.db_id for dr in directly_refed}
+    for alias in bug.aliases:
+      aliases[alias] = {
+          'exists': alias in is_directly_refed,
+          'same_alias_entries': []
+      }
 
   # Add links to other entries that have the same alias or references this
-  if bug.aliases:
-    query = osv.Bug.query(osv.Bug.aliases.IN(bug.aliases + [bug.id()]))
-    for other in query:
-      if other.id() == bug.id():
-        continue
-      for other_alias in other.aliases:
-        if other_alias in aliases:
-          aliases[other_alias]['same_alias_entries'].append(other.id())
-      if bug.id() in other.aliases:
-        aliases[other.id()] = {'exists': True, 'same_alias_entries': []}
+  query = osv.Bug.query(osv.Bug.aliases.IN(bug.aliases + [bug.id()]))
+  for other in query:
+    if other.id() == bug.id():
+      continue
+    for other_alias in other.aliases:
+      if other_alias in aliases:
+        aliases[other_alias]['same_alias_entries'].append(other.id())
+    if bug.id() in other.aliases:
+      aliases[other.id()] = {'exists': True, 'same_alias_entries': []}
 
   # Remove self if it was added
   aliases.pop(bug.id(), None)


Test patch:
Repository: google/osv.dev

Issue:
Missing result from query
```
curl -X POST -d '{"package": {"ecosystem": "npm", "name": "ws"}, "version": "7.1.1"}' https://api.osv.dev/v1/query
```

Should return at least https://osv.dev/vulnerability/GHSA-6fc8-4gx4-v693


Discussion:

Resolving patch:
diff --git a/gcp/api/server.py b/gcp/api/server.py
--- a/gcp/api/server.py
+++ b/gcp/api/server.py
@@ -363,6 +363,9 @@ def _is_semver_affected(affected_packages, package_name, ecosystem,
             event.value):
           affected = False
 
+      if affected:
+        return affected
+
   return affected
 
 


Test patch:
diff --git a/gcp/api/integration_tests.py b/gcp/api/integration_tests.py
--- a/gcp/api/integration_tests.py
+++ b/gcp/api/integration_tests.py
@@ -320,6 +320,22 @@ def test_query_semver_multiple_package(self):
     self.assertCountEqual(['GO-2021-0061', 'GO-2020-0036'],
                           [vuln['id'] for vuln in response_json['vulns']])
 
+    response = requests.post(
+        _api() + '/v1/query',
+        data=json.dumps({
+            'version': '7.1.1',
+            'package': {
+                'name': 'ws',
+                'ecosystem': 'npm',
+            }
+        }),
+        timeout=_TIMEOUT)
+
+    response_json = response.json()
+    self.assertEqual(1, len(response_json['vulns']))
+    self.assertCountEqual(['GHSA-6fc8-4gx4-v693'],
+                          [vuln['id'] for vuln in response_json['vulns']])
+
   def test_query_purl(self):
     """Test querying by PURL."""
     expected = [
Repository: google/osv.dev

Issue:
Add a News tab with a blog
Basic thinking: adding a "News" tab to [osv.dev](http://osv.dev/) and making it an OSV-specific blog for more technical posts less suited to the [Google Security Blog](https://security.googleblog.com/). The example that spawned this thought was communicating about #771 once it's completed.

From an internal conversation about this, @oliverchang said: "this has come up a few times, both from our UX designer and from other folks"




Discussion:
@lumjjb FYI :) 
🎉 🚀 
We can probably use something like https://github.com/gohugoio/hugo to get a blog going pretty quickly. This lets us write blog posts in markdown and have that be generated into static HTML as part of deployment. The only challenge here is getting the styling consistent with osv.dev. 

Resolving patch:
diff --git a/gcp/appengine/frontend_handlers.py b/gcp/appengine/frontend_handlers.py
--- a/gcp/appengine/frontend_handlers.py
+++ b/gcp/appengine/frontend_handlers.py
@@ -16,8 +16,10 @@
 import json
 import os
 import math
+import re
 
 from flask import abort
+from flask import current_app
 from flask import Blueprint
 from flask import make_response
 from flask import redirect
@@ -38,6 +40,8 @@
 _PAGE_SIZE = 16
 _PAGE_LOOKAHEAD = 4
 _REQUESTS_PER_MIN = 30
+_VALID_BLOG_NAME = re.compile(r'^[\w-]+$')
+_BLOG_CONTENTS_DIR = 'blog'
 
 if utils.is_prod():
   redis_host = os.environ.get('REDISHOST', 'localhost')
@@ -52,6 +56,17 @@ def check_rate_limit():
       abort(429)
 
 
+def _load_blog_content(name):
+  """Load blog content."""
+  path = os.path.join(current_app.static_folder, _BLOG_CONTENTS_DIR, name)
+  if not os.path.exists(path):
+    abort(404)
+    return None
+
+  with open(path) as handle:
+    return handle.read()
+
+
 @blueprint.before_request
 def check_cors_preflight():
   """Handle CORS preflight requests."""
@@ -88,6 +103,29 @@ def index():
       'home.html', ecosystem_counts=osv_get_ecosystem_counts_cached())
 
 
+@blueprint.route('/blog/', strict_slashes=False)
+def blog():
+  return render_template('blog.html', index=_load_blog_content('index.html'))
+
+
+@blueprint.route('/blog/index.xml')
+def blog_rss():
+  return current_app.send_static_file(
+      os.path.join(_BLOG_CONTENTS_DIR, 'index.xml'))
+
+
+@blueprint.route('/blog/posts/<blog_name>/', strict_slashes=False)
+def blog_post(blog_name):
+  if not _VALID_BLOG_NAME.match(blog_name):
+    abort(404)
+    return None
+
+  return render_template(
+      'blog_post.html',
+      content=_load_blog_content(
+          os.path.join('posts', blog_name, 'index.html')))
+
+
 @blueprint.route('/about')
 def about():
   return render_template('about.html')


Test patch:
Repository: google/osv.dev

Issue:
Change in affected versions for GHSA-5mg8-w23w-74h3 not picked up by OSV
For Maven package `com.google.guava:guava`, the affected version range was changed from <= 30.0 to <= 29.0 both on GHSA as well as on NVD, but OSV did not update the list of affected versions.
It seems the change is registered in the `Last affected` field, but not in the list of affected versions.

See the following references:
- https://osv.dev/vulnerability/GHSA-5mg8-w23w-74h3
- https://github.com/github/advisory-database/blob/main/advisories/github-reviewed/2021/03/GHSA-5mg8-w23w-74h3/GHSA-5mg8-w23w-74h3.json
- https://nvd.nist.gov/vuln/detail/CVE-2020-8908


Discussion:
Thanks for flagging! we'll take a look. 

Resolving patch:
diff --git a/osv/ecosystems.py b/osv/ecosystems.py
--- a/osv/ecosystems.py
+++ b/osv/ecosystems.py
@@ -334,7 +334,8 @@ def enumerate_versions(self,
                          limits=None):
     """Enumerate versions."""
     if use_deps_dev:
-      return self._deps_dev_enumerate(package, introduced, fixed, limits=limits)
+      return self._deps_dev_enumerate(
+          package, introduced, fixed, last_affected, limits=limits)
 
     get_versions = self._get_versions
     if shared_cache:


Test patch:
diff --git a/osv/ecosystems_test.py b/osv/ecosystems_test.py
--- a/osv/ecosystems_test.py
+++ b/osv/ecosystems_test.py
@@ -252,5 +252,23 @@ def test_cache(self, cache_mock: mock.MagicMock):
     cache_mock.set.assert_called_once()
 
 
+class EnumerateTest(unittest.TestCase):
+  """Enumerate test."""
+
+  @unittest.skipUnless(os.getenv('DEPSDEV_API_KEY'), 'Requires API key')
+  def test_maven_deps_dev(self):
+    """Test Maven using deps.dev."""
+    ecosystems.use_deps_dev = True
+    ecosystems.deps_dev_api_key = os.getenv('DEPSDEV_API_KEY')
+
+    ecosystem = ecosystems.get('Maven')
+    self.assertEqual(['10.0', '10.0.1', '11.0-rc1', '11.0'],
+                     ecosystem.enumerate_versions(
+                         'com.google.guava:guava', '10.0',
+                         last_affected='11.0'))
+
+    ecosystems.use_deps_dev = False
+
+
 if __name__ == '__main__':
   unittest.main(failfast=True)
Repository: google/osv.dev

Issue:
Create shorter redirect link format for vulnerabilities
e.g. `https://osv.dev/v/PYSEC-foo` should redirect to the full `https://osv.dev/vulnerability/PYSEC-foo`.

This can help with osv-scanner's human readable output, where horizontal space is a premium.  


Discussion:
The number of times I've tried and failed to use just osv.dev/id, could we support that for maximum URL brevity? Feels more intuitive.
Yeah that's certainly another possibility, just wonder if it creates any complexities in the future since it introduces potential routing clashes with other top level OSV.dev pages. 
Yes, there's a (small, IMO) risk, given all of the IDs should be fairly unambiguously regex-able...

Resolving patch:
diff --git a/gcp/appengine/frontend_handlers.py b/gcp/appengine/frontend_handlers.py
--- a/gcp/appengine/frontend_handlers.py
+++ b/gcp/appengine/frontend_handlers.py
@@ -40,7 +40,9 @@
 _PAGE_SIZE = 16
 _PAGE_LOOKAHEAD = 4
 _REQUESTS_PER_MIN = 30
-_VALID_BLOG_NAME = re.compile(r'^[\w-]+$')
+_WORD_CHARACTERS_OR_DASH = re.compile(r'^[\w-]+$')
+_VALID_BLOG_NAME = _WORD_CHARACTERS_OR_DASH
+_VALID_VULN_ID = _WORD_CHARACTERS_OR_DASH
 _BLOG_CONTENTS_DIR = 'blog'
 
 if utils.is_prod():
@@ -171,6 +173,21 @@ def vulnerability(vuln_id):
   return render_template('vulnerability.html', vulnerability=vuln)
 
 
+@blueprint.route('/<potential_vuln_id>')
+def vulnerability_redirector(potential_vuln_id):
+  """Convenience redirector for /VULN-ID to /vulnerability/VULN-ID."""
+  if not _VALID_VULN_ID.match(potential_vuln_id):
+    abort(404)
+    return None
+
+  vuln = osv_get_by_id(potential_vuln_id)
+  if vuln:
+    return redirect(f'/vulnerability/{potential_vuln_id}')
+
+  abort(404)
+  return None
+
+
 def bug_to_response(bug, detailed=True):
   """Convert a Bug entity to a response object."""
   response = osv.vulnerability_to_dict(bug.to_vulnerability())


Test patch:
Repository: google/osv.dev

Issue:
The purl of Alpine ecosystem is inconsistent with purl-spec
According to [**purl-type `apk`**](https://github.com/package-url/purl-spec/blob/master/PURL-TYPES.rst#apk) ([PR](https://github.com/package-url/purl-spec/pull/171)), it seems that the purl type of `Alpine` should be `apk` and the purl namespace can be `alpine`. In this project, however, the purl type of `Alpine` is `alpine`.

As a result, the purl of a package is **different** when apply different standards. For example, the purl of the `curl` package is `pkg:apk/alpine/curl` according to purl-spec while it is `pkg:alpine/curl` in this project.




Discussion:

Resolving patch:
diff --git a/osv/purl_helpers.py b/osv/purl_helpers.py
--- a/osv/purl_helpers.py
+++ b/osv/purl_helpers.py
@@ -16,6 +16,7 @@
 from urllib.parse import quote
 
 PURL_ECOSYSTEMS = {
+    'Alpine': 'apk',
     'crates.io': 'cargo',
     'Debian': 'deb',
     'Hex': 'hex',
@@ -48,8 +49,12 @@ def package_to_purl(ecosystem, package_name):
     # PURLs use / to separate the group ID and the artifact ID.
     package_name = package_name.replace(':', '/', 1)
 
-  if purl_type == 'deb':
+  if purl_type == 'deb' and ecosystem == 'Debian':
     package_name = 'debian/' + package_name
     suffix = '?arch=source'
 
+  if purl_type == 'apk' and ecosystem == 'Alpine':
+    package_name = 'alpine/' + package_name
+    suffix = '?arch=source'
+
   return f'pkg:{purl_type}/{_url_encode(package_name)}{suffix}'


Test patch:
diff --git a/osv/purl_helpers_test.py b/osv/purl_helpers_test.py
--- a/osv/purl_helpers_test.py
+++ b/osv/purl_helpers_test.py
@@ -42,6 +42,11 @@ def test_debian(self):
     self.assertEqual('pkg:deb/debian/nginx?arch=source',
                      purl_helpers.package_to_purl('Debian', 'nginx'))
 
+  def test_alpine(self):
+    """Test PURL generation for alpine."""
+    self.assertEqual('pkg:apk/alpine/nginx?arch=source',
+                     purl_helpers.package_to_purl('Alpine', 'nginx'))
+
 
 if __name__ == '__main__':
   unittest.main()
Repository: google/osv.dev

Issue:
Maven 'zero' versions causes errors when comparing
Maven versions that are 'zero' (e.g. `0.0.0` and `0.0.0-X.Y.Z`) cause assertion errors when comparing with non-zero versions, because the comparisons assume there's always an initial token without a `.` or `-` 'prefix'.

This seems to be causing some of the missing vulnerabilities in #1018


Discussion:

Resolving patch:
diff --git a/osv/ecosystems/maven.py b/osv/ecosystems/maven.py
--- a/osv/ecosystems/maven.py
+++ b/osv/ecosystems/maven.py
@@ -221,7 +221,7 @@ def from_string(cls, str_version):
     # Then, starting from the end of the version, the trailing "null" values
     # (0, "", "final", "ga") are trimmed.
     i = len(version.tokens) - 1
-    while i >= 0:
+    while i > 0:  # We always want at least one token for comparison
       if version.tokens[i].value in _TO_TRIM:
         version.tokens.pop(i)
         i -= 1


Test patch:
diff --git a/osv/ecosystems/maven_test.py b/osv/ecosystems/maven_test.py
--- a/osv/ecosystems/maven_test.py
+++ b/osv/ecosystems/maven_test.py
@@ -249,6 +249,15 @@ def test_versions_equal(self):
 
     self.check_versions_equal('1', '01', '001')
 
+  def test_version_zero(self):
+    """Test comparison and equality with versions 0.0.0"""
+    self.check_versions_equal('0.0.0', '0.0', '0')
+    self.check_versions_equal('0.0.0-0.0.0', '0-final-ga', '0')
+    self.check_versions_order('0', '1')
+
+    # actual case from com.graphql-java:graphql-java
+    self.check_versions_order('0.0.0-2021-05-17T01-01-51-5ec03a8b', '20.0.0')
+
 
 class MavenEcosystemTest(unittest.TestCase):
   """Maven ecosystem helper tests."""
Repository: google/osv.dev

Issue:
"withdrawn" entries not getting exported correctly
Identified by @andrewpollock : 

https://github.com/google/osv.dev/blob/26050deb42785bc5a4dc7d802eac8e7f95135509/docker/exporter/exporter.py#L94

withdrawn entries are marked as status = INVALID in our DB, so they're not included.

They should be included when we export. 


Discussion:

Resolving patch:
diff --git a/docker/exporter/exporter.py b/docker/exporter/exporter.py
--- a/docker/exporter/exporter.py
+++ b/docker/exporter/exporter.py
@@ -91,7 +91,7 @@ def _export_ecosystem_to_bucket(self, ecosystem, tmp_dir):
     zip_path = os.path.join(tmp_dir, 'all.zip')
     with zipfile.ZipFile(zip_path, 'w', zipfile.ZIP_DEFLATED) as zip_file:
       for bug in osv.Bug.query(osv.Bug.ecosystem == ecosystem):
-        if not bug.public or not bug.status == osv.BugStatus.PROCESSED:
+        if not bug.public or bug.status == osv.BugStatus.UNPROCESSED:
           continue
 
         file_path = os.path.join(tmp_dir, bug.id() + '.json')


Test patch:
Repository: google/osv.dev

Issue:
Don't suggest using -X POST
On https://osv.dev/#use-the-api is the instruction
```
Query by commit hash

curl -X POST -d \
  '{"commit": "6879efc2c1596d11a6a6ad296f80063b558d5e0f"}' \
  "https://api.osv.dev/v1/query"
```

Using `-X POST` here is unnecessary, redundant and potentially dangerous as people cut and paste this into more places. curl will actually tell you this if you add `-v` to this command:

`Note: Unnecessary use of -X or --request, POST is already inferred.`

See also https://daniel.haxx.se/blog/2015/09/11/unnecessary-use-of-curl-x/


Discussion:
Thanks @dfandrich for bringing this to our attention!
This issue is also applicable at https://github.com/google/osv.dev#using-the-api

Resolving patch:
diff --git a/docs/build.py b/docs/build.py
--- a/docs/build.py
+++ b/docs/build.py
@@ -100,10 +100,10 @@ def main():
       'lang':
           'Curl example',
       'source':
-          ('curl -X POST -d \\\n'
+          ('curl -d \\\n'
            '  \'{"commit": "6879efc2c1596d11a6a6ad296f80063b558d5e0f"}\' \\\n'
            '  "https://api.osv.dev/v1/query"\n\n'
-           'curl -X POST -d \\\n'
+           'curl -d \\\n'
            '  \'{"package": {"name": "mruby"}, "version": "2.1.2rc"}\' \\\n'
            '  "https://api.osv.dev/v1/query"')
   }]
@@ -111,8 +111,7 @@ def main():
   spec['paths']['/v1/querybatch']['post']['x-code-samples'] = [{
       'lang':
           'Curl example',
-      'source':
-          ("""cat <<EOF | curl -X POST -d @- "https://api.osv.dev/v1/querybatch"
+      'source': ("""cat <<EOF | curl -d @- "https://api.osv.dev/v1/querybatch"
 {
   "queries": [
     {


Test patch: