Repository: lukasschwab/arxiv.py

Issue:
`get_short_id` incorrect for pre-March 2007 arXiv identifiers: missing archive
<!-- See [Stack Overflow's "How do I ask a good question?"](https://stackoverflow.com/help/how-to-ask) -->

Error: 

```
/usr/local/lib/python3.7/dist-packages/arxiv/arxiv.py in results(self, search)
    552             ))
    553             page_url = self._format_url(search, offset, page_size)
--> 554             feed = self._parse_feed(page_url, first_page)
    555             if first_page:
    556                 # NOTE: this is an ugly fix for a known bug. The totalresults

/usr/local/lib/python3.7/dist-packages/arxiv/arxiv.py in _parse_feed(self, url, first_page)
    635         # Feed was never returned in self.num_retries tries. Raise the last
    636         # exception encountered.
--> 637         raise err
    638 
    639 

HTTPError: arxiv.HTTPError(Page request resulted in HTTP 400)
```




Code for parsing id from arxiv result object-
`
id = urlparse(result.entry_id).path.split('/')[-1].split('v')[0]
`

code to reproduce -

```
ids = ['1911.10854', '1905.00256', '0112019', '1202.2184', '1708.03109', '0205137', '1610.08147', '2003.05245', '0406182', '0708.3630', '0503148', '1111.6170', '1612.04479', '0307110', '0306127', '1307.2727', '0402059', '1012.4706', '1906.01999', '0101032']

papers = arxiv.Search(id_list=ids).get()
```

invalid ids are `'0112019', '0205137'` etc 

respective pdf urls still accessible, for example : [https://arxiv.org/pdf/quant-ph/0112019.pdf](https://arxiv.org/pdf/quant-ph/0112019.pdf)

The same error is referenced in another open issue but from the perspective of huge id arrays. [ issue ID : #15]

Apologies if I simply lack sufficient knowledge about identifier naming conventions but it should download from all research fields right? 


Discussion:

Patch:
diff --git a/arxiv/arxiv.py b/arxiv/arxiv.py
index 60d5d97..95fea41 100644
--- a/arxiv/arxiv.py
+++ b/arxiv/arxiv.py
@@ -170,10 +170,18 @@ class Result(object):
         """
         Returns the short ID for this result.
 
-        If the result URL is `"http://arxiv.org/abs/quant-ph/0201082v1"`,
-        `result.get_short_id()` returns `"0201082v1"`.
+        + If the result URL is `"http://arxiv.org/abs/2107.05580v1"`,
+        `result.get_short_id()` returns `2107.05580v1`.
+
+        + If the result URL is `"http://arxiv.org/abs/quant-ph/0201082v1"`,
+        `result.get_short_id()` returns `"quant-ph/0201082v1"` (the pre-March
+        2007 arXiv identifier format).
+
+        For an explanation of the difference between arXiv's legacy and current
+        identifiers, see [Understanding the arXiv
+        identifier](https://arxiv.org/help/arxiv_identifier).
         """
-        return self.entry_id.split('/')[-1]
+        return self.entry_id.split('arxiv.org/abs/')[-1]
 
     def _get_default_filename(self, extension: str = "pdf") -> str:
         """
diff --git a/docs/index.html b/docs/index.html
index e2415be..02a2215 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -637,10 +637,18 @@ arxiv<wbr>.arxiv    </h1>
         <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Returns the short ID for this result.</span>
 
-<span class="sd">        If the result URL is `&quot;http://arxiv.org/abs/quant-ph/0201082v1&quot;`,</span>
-<span class="sd">        `result.get_short_id()` returns `&quot;0201082v1&quot;`.</span>
+<span class="sd">        + If the result URL is `&quot;http://arxiv.org/abs/2107.05580v1&quot;`,</span>
+<span class="sd">        `result.get_short_id()` returns `2107.05580v1`.</span>
+
+<span class="sd">        + If the result URL is `&quot;http://arxiv.org/abs/quant-ph/0201082v1&quot;`,</span>
+<span class="sd">        `result.get_short_id()` returns `&quot;quant-ph/0201082v1&quot;` (the pre-March</span>
+<span class="sd">        2007 arXiv identifier format).</span>
+
+<span class="sd">        For an explanation of the difference between arXiv&#39;s legacy and current</span>
+<span class="sd">        identifiers, see [Understanding the arXiv</span>
+<span class="sd">        identifier](https://arxiv.org/help/arxiv_identifier).</span>
 <span class="sd">        &quot;&quot;&quot;</span>
-        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">entry_id</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
+        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">entry_id</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;arxiv.org/abs/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
 
     <span class="k">def</span> <span class="nf">_get_default_filename</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">extension</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;pdf&quot;</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
         <span class="sd">&quot;&quot;&quot;</span>
@@ -1370,10 +1378,18 @@ arxiv<wbr>.arxiv    </h1>
         <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Returns the short ID for this result.</span>
 
-<span class="sd">        If the result URL is `&quot;http://arxiv.org/abs/quant-ph/0201082v1&quot;`,</span>
-<span class="sd">        `result.get_short_id()` returns `&quot;0201082v1&quot;`.</span>
+<span class="sd">        + If the result URL is `&quot;http://arxiv.org/abs/2107.05580v1&quot;`,</span>
+<span class="sd">        `result.get_short_id()` returns `2107.05580v1`.</span>
+
+<span class="sd">        + If the result URL is `&quot;http://arxiv.org/abs/quant-ph/0201082v1&quot;`,</span>
+<span class="sd">        `result.get_short_id()` returns `&quot;quant-ph/0201082v1&quot;` (the pre-March</span>
+<span class="sd">        2007 arXiv identifier format).</span>
+
+<span class="sd">        For an explanation of the difference between arXiv&#39;s legacy and current</span>
+<span class="sd">        identifiers, see [Understanding the arXiv</span>
+<span class="sd">        identifier](https://arxiv.org/help/arxiv_identifier).</span>
 <span class="sd">        &quot;&quot;&quot;</span>
-        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">entry_id</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
+        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">entry_id</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;arxiv.org/abs/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
 
     <span class="k">def</span> <span class="nf">_get_default_filename</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">extension</span><span class="p">:</span> <span class="nb">str</span> <span class="o">=</span> <span class="s2">&quot;pdf&quot;</span><span class="p">)</span> <span class="o">-&gt;</span> <span class="nb">str</span><span class="p">:</span>
         <span class="sd">&quot;&quot;&quot;</span>
@@ -1783,18 +1799,35 @@ Taxonomy</a>.</p>
         <span class="sd">&quot;&quot;&quot;</span>
 <span class="sd">        Returns the short ID for this result.</span>
 
-<span class="sd">        If the result URL is `&quot;http://arxiv.org/abs/quant-ph/0201082v1&quot;`,</span>
-<span class="sd">        `result.get_short_id()` returns `&quot;0201082v1&quot;`.</span>
+<span class="sd">        + If the result URL is `&quot;http://arxiv.org/abs/2107.05580v1&quot;`,</span>
+<span class="sd">        `result.get_short_id()` returns `2107.05580v1`.</span>
+
+<span class="sd">        + If the result URL is `&quot;http://arxiv.org/abs/quant-ph/0201082v1&quot;`,</span>
+<span class="sd">        `result.get_short_id()` returns `&quot;quant-ph/0201082v1&quot;` (the pre-March</span>
+<span class="sd">        2007 arXiv identifier format).</span>
+
+<span class="sd">        For an explanation of the difference between arXiv&#39;s legacy and current</span>
+<span class="sd">        identifiers, see [Understanding the arXiv</span>
+<span class="sd">        identifier](https://arxiv.org/help/arxiv_identifier).</span>
 <span class="sd">        &quot;&quot;&quot;</span>
-        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">entry_id</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
+        <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">entry_id</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;arxiv.org/abs/&#39;</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
 </pre></div>
 
         </details>
 
             <div class="docstring"><p>Returns the short ID for this result.</p>
 
-<p>If the result URL is <code>"http://arxiv.org/abs/quant-ph/0201082v1"</code>,
-<code>result.get_short_id()</code> returns <code>"0201082v1"</code>.</p>
+<ul>
+<li><p>If the result URL is <code>"http://arxiv.org/abs/2107.05580v1"</code>,
+<code>result.get_short_id()</code> returns <code>2107.05580v1</code>.</p></li>
+<li><p>If the result URL is <code>"http://arxiv.org/abs/quant-ph/0201082v1"</code>,
+<code>result.get_short_id()</code> returns <code>"quant-ph/0201082v1"</code> (the pre-March
+2007 arXiv identifier format).</p></li>
+</ul>
+
+<p>For an explanation of the difference between arXiv's legacy and current
+identifiers, see <a href="https://arxiv.org/help/arxiv_identifier">Understanding the arXiv
+identifier</a>.</p>
 </div>
 
 
diff --git a/setup.py b/setup.py
index e4f7c13..3a6eb9e 100644
--- a/setup.py
+++ b/setup.py
@@ -1,6 +1,6 @@
 from setuptools import setup
 
-version = '1.3.0'
+version = '1.4.0'
 
 with open('README.md', 'r') as fh:
     long_description = fh.read()


Test patch:
diff --git a/tests/test_result.py b/tests/test_result.py
index 82a12f3..7b54ced 100644
--- a/tests/test_result.py
+++ b/tests/test_result.py
@@ -5,7 +5,7 @@ import time
 from datetime import datetime, timezone
 
 
-class TestAPI(unittest.TestCase):
+class TestResult(unittest.TestCase):
 
     def assert_nonempty(self, s):
         self.assertIsNotNone(s)
@@ -95,3 +95,8 @@ class TestAPI(unittest.TestCase):
         self.assertTrue(arxiv.Result.Link(href) == link)
         self.assertFalse(link == arxiv.Result.Link("other"))
         self.assertFalse(link == id)
+
+    def test_legacy_ids(self):
+        full_legacy_id = "quant-ph/0201082v1"
+        result = next(arxiv.Search(id_list=[full_legacy_id]).results())
+        self.assertEqual(result.get_short_id(), full_legacy_id)