Repository: biopython/biopython

Issue:
Align.IO fails to parse a stockholm alignment file. /Bio/AlignIO/StockholmIO.py ValueError
### Setup

I am reporting a problem with Biopython version, Python version, and operating
system as follows:

```
$ python
Python 3.8.1 (default, Jan  8 2020, 22:29:32) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; print(sys.version)
3.8.1 (default, Jan  8 2020, 22:29:32) 
[GCC 7.3.0]
>>> import platform; print(platform.python_implementation()); print(platform.platform())
CPython
Linux-5.4.0-37-generic-x86_64-with-glibc2.10
>>> import Bio; print(Bio.__version__)
1.76

```


### Expected behaviour


`align = AlignIO.read(stockholm_file_location, "stockholm")` should parse a Stockholm alignment file to an object called align that can be iterated through for records within the object.


### Actual behaviour

`align = AlignIO.read(stockholm_file_location, "stockholm")` in some Stockholm alignments yields:

```
Traceback (most recent call last):
  File "manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/django/core/management/__init__.py", line 395, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/django_extensions/management/email_notifications.py", line 63, in run_from_argv
    super(EmailNotificationCommand, self).run_from_argv(argv)
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/django/core/management/base.py", line 328, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/django_extensions/management/email_notifications.py", line 75, in execute
    super(EmailNotificationCommand, self).execute(*args, **options)
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/django/core/management/base.py", line 369, in execute
    output = self.handle(*args, **options)
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/django_extensions/management/utils.py", line 59, in inner
    ret = func(self, *args, **kwargs)
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/django_extensions/management/commands/runscript.py", line 238, in handle
    run_script(mod, *script_args)
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/django_extensions/management/commands/runscript.py", line 148, in run_script
    mod.run(*script_args)
  File "/mnt/brain/VarTMH/scripts/populate_funfams.py", line 147, in run
    stockholm_to_database(a_query, superfamily_id, funfam_id, stockholm_file)
  File "/mnt/brain/VarTMH/scripts/populate_funfams.py", line 90, in stockholm_to_database
    align = AlignIO.read(stockholm_file_location, "stockholm")
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/Bio/AlignIO/__init__.py", line 447, in read
    first = next(iterator)
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/Bio/AlignIO/__init__.py", line 394, in parse
    for a in i:
  File "/home/james/miniconda3/envs/vartmh/lib/python3.8/site-packages/Bio/AlignIO/StockholmIO.py", line 460, in __next__
    seq_id, feature, text = line[5:].strip().split(None, 2)
ValueError: not enough values to unpack (expected 3, got 2)
```

### Steps to reproduce

Save the Stockholm alignment attached here: [634591.sth.zip](https://github.com/biopython/biopython/files/4794175/634591.sth.zip)

This is an example of an alignment that causes the error, but nothing seems amiss to me. 

Run:


```
from Bio import AlignIO
align = AlignIO.read(stockholm_file_location, "stockholm")`
```



Discussion:
Helllo.

I can reproduce this. The exception is while handling the ``#=GS <seqname>
<feature> <free text>`` lines, and appears to be a line without any value.
It turns out to be this line:

```text
#=GS G4YF51/316-544              OS
```

Using any value there fixes it:

```text
#=GS G4YF51/316-544              OS .
```

I'd have to dig into the specifications to check if this is officially
valid or not, but it ought not to be too complicated to tolerate this
variation.

Where did this file come from?

Peter

>

Thanks for looking into this.

This is from CATH at UCL as part of their [FunFam](https://github.com/UCLOrengoGroup/cath-funfam-docs) project. 
You can directly view the alignment at the [CATH database here](http://www.cathdb.info/version/v4_2_0/superfamily/3.40.50.300/funfam/634591/files/stockholm)

From http://sonnhammer.sbc.su.se/Stockholm.html:
> ## The alignment mark-up:
> 
> #=GS `<seqname> <feature> <Generic per-Sequence annotation, free text>`
> 
>...
>
>  ##  Magic or recommended features:
> 
>  #=GS
>
>     Pfam uses these features:
> 
>     Feature                    Description
>     ---------------------      -----------
>     AC <accession>             ACcession number
>     DE <freetext>              DEscription
>     DR <db>; <accession>;      Database Reference
>     OS <organism>              OrganiSm (species)
>     OC <clade>                 Organism Classification (clade, etc.)
>     LO <look>                  Look (Color, etc.)
> 
>...
>
> ## Example
> 
> #=GS O31698/18-71 AC O31698
> #=GS O83071/192-246 AC O83071
> #=GS O83071/259-312 AC O83071
> #=GS O31698/88-139 AC O31698
> #=GS O31698/88-139 OS Bacillus subtilis

My interpretation is that a #=GS line should contain the free text section, but cannot be white space.

Thanks for quoting the specification reference, "free text" was my recollection - so arguably if the free text was empty, the line should end with a trailing space - but the current parser would break anyway with ``"#=GS G4YF51/316-544              OS "`` so we need to make a small change anyway.

Being pragmatic, I think we should cope with the missing value, and assign an empty string as the annotation value.

Is this something you would be interested in working on, and submitting a pull request for?
> Being pragmatic, I think we should cope with the missing value, and assign an empty string as the annotation value.

This sounds like a good solution.

> Is this something you would be interested in working on, and submitting a pull request for?

I'll give it a go!
Excellent. Thanks!

Resolving patch:
diff --git a/Bio/AlignIO/StockholmIO.py b/Bio/AlignIO/StockholmIO.py
--- a/Bio/AlignIO/StockholmIO.py
+++ b/Bio/AlignIO/StockholmIO.py
@@ -456,7 +456,13 @@ def __next__(self):
                 elif line[:5] == "#=GS ":
                     # Generic per-Sequence annotation, free text
                     # Format: "#=GS <seqname> <feature> <free text>"
-                    seq_id, feature, text = line[5:].strip().split(None, 2)
+                    try:
+                        seq_id, feature, text = line[5:].strip().split(None, 2)
+                    except ValueError:
+                        # Free text can sometimes be empty, which a one line split throws an error for.
+                        # See https://github.com/biopython/biopython/issues/2982 for more details
+                        seq_id, feature = line[5:].strip().split(None, 1)
+                        text = ""
                     # if seq_id not in ids:
                     #    ids.append(seq_id)
                     if seq_id not in gs:


Test patch:
diff --git a/Tests/Stockholm/funny.sth b/Tests/Stockholm/funny.sth
--- a/Tests/Stockholm/funny.sth
+++ b/Tests/Stockholm/funny.sth
@@ -7,6 +7,7 @@
 #=GF CC in 2 or four copies within a protein. 
 #=GF SQ 67
 #=GS O31698/18-71 AC O31698
+#=GS O83071/192-246 OS
 #=GS O83071/192-246 AC O83071
 #=GS O83071/259-312 AC O83071
 #=GS O31698/88-139 AC O31698