Repository: biopython/biopython

Issue:
Parsing problem between organism name and taxonomy
### Context 
Hello, I am using Biopython to extract informations from GenBank files with the function "SeqIO.parse()". 
Today, I got an issue with the accession "UZN22335". When I parsed this protein, the taxonomy is present but in the organism name. 

### Output
{'topology': 'linear', 'data_file_division': 'BCT', 'date': '15-NOV-2022', 'accessions': ['UZN22335'], 'sequence_version': 1, 'db_source': 'accession CP046447.1', 'keywords': [''], 'source': 'bacterium 3DAC', **'organism': 'bacterium 3DAC Bacteria.', 'taxonomy': [],** 'references': [Reference(title='Discovery of a novel thermophilic, piezophilic sulfur-reducing bacterial phylum provides insights into the evolutionary history of thermophilic bacteria', ...), Reference(title='Direct Submission', ...)], 'comment': 'The annotation was added by the NCBI Prokaryotic Genome Annotation\nPipeline (PGAP). Information about PGAP can be found here:\nhttps://www.ncbi.nlm.nih.gov/genome/annotation_prok/', 'structured_comment': defaultdict(<class 'dict'>, {'Genome-Annotation-Data': {'Annotation Provider': 'NCBI', 'Annotation Date': '12/03/2019 18:59:54', 'Annotation Pipeline': 'NCBI Prokaryotic Genome Annotation Pipeline (PGAP)', 'Annotation Method': 'Best-placed reference protein set; GeneMarkS-2+', 'Annotation Software revision': '4.10', 'Features Annotated': 'Gene; CDS; rRNA; tRNA; ncRNA; repeat_region', 'Genes (total)': '1,554', 'CDSs (total)': '1,496', 'Genes (coding)': '1,491', 'CDSs (with protein)': '1,491', 'Genes (RNA)': '58', 'rRNAs': '3, 3, 3 (5S, 16S, 23S)', 'complete rRNAs': '3, 3, 3 (5S, 16S, 23S)', 'tRNAs': '46', 'ncRNAs': '3', 'Pseudo Genes (total)': '5', 'CDSs (without protein)': '5', 'Pseudo Genes (ambiguous residues)': '0 of 5', 'Pseudo Genes (frameshifted)': '0 of 5', 'Pseudo Genes (incomplete)': '3 of 5', 'Pseudo Genes (internal stop)': '2 of 5', 'CRISPR Arrays': '1'}}), 'molecule_type': 'protein'}

As you can see the taxonomy is empty. I think the problem is the only presence of "Bacteria" in the taxonomy. 

### Setup
I am reporting a problem with Biopython version 1.80, Python version 3.8.10, and operating
system as follows:
3.8.10 (default, Nov 14 2022, 12:59:47) 
[GCC 9.4.0]
CPython
Linux-5.15.0-56-generic-x86_64-with-glibc2.29
1.80

Regards, 
Matthieu Boulinguiez. 


Discussion:
Does your GenBank file look like https://www.ncbi.nlm.nih.gov/protein/UZN22335 and start:

```
LOCUS       UZN22335                 450 aa            linear   BCT 15-NOV-2022
DEFINITION  chromosomal replication initiator protein DnaA [bacterium 3DAC].
ACCESSION   UZN22335
VERSION     UZN22335.1
DBLINK      BioProject: PRJNA591439
            BioSample: SAMN13381960
DBSOURCE    accession CP046447.1
KEYWORDS    .
SOURCE      bacterium 3DAC
  ORGANISM  bacterium 3DAC
            Bacteria.
REFERENCE   1  (residues 1 to 450)
  AUTHORS   Wang,Y. and Xiao,X.
...
```

Note the ``ORGANISM`` line and taxonomic information following it is minimal, and seems to me to explain the annotation dictionary content.
Ah - you mean even this minimal taxonomy of "Bacteria" is not reported?
This is down to the lack of a semi-colon in the "taxonomy line" which is taken to indicate the start of the taxonomy, and not just more of the organism name.

https://www.ncbi.nlm.nih.gov/genbank/samplerecord/#OrganismB

I'd have to dig out more test cases as these alternative interpretations may not be easily distinguished.
Thank you for your fast answer. 

Yes my file starts like this and yes for me it's a problem from the parsing of your function or a NCBI problem when this protein was released. 

I also tried with other proteins (UYG54991) and the result was different:
{'topology': 'linear', 'data_file_division': 'BCT', 'date': '18-NOV-2022', 'accessions': ['UYG54991'], 'sequence_version': 1, 'db_source': 'accession OP125508.1', 'keywords': [''], 'source': 'Candidatus Lariskella guizhouensis', **'organism': 'Candidatus Lariskella guizhouensis', 'taxonomy': ['Bacteria', 'Proteobacteria', 'Alphaproteobacteria', 'Rickettsiales', 'Candidatus Midichloriaceae', 'Candidatus Lariskella'],** 'references': [Reference(title='Prevalence of Spotted Fever Group Rickettsia and Candidatus Lariskella in Multiple Tick Species from Guizhou Province, China', ...), Reference(title='Direct Submission', ...)], 'structured_comment': defaultdict(<class 'dict'>, {'Assembly-Data': {'Sequencing Technology': 'Sanger dideoxy sequencing'}}), 'molecule_type': 'protein'}

Here the organism name is good and the taxonomy fill up. 

But on this protein (UZN22335) there is a mix between the organism name and the taxonomy, surely due to the lack of a semi-colon as you told. 

To have more examples like UZN22335 you can check this link :
https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Tree&id=49928&lvl=3&lin=f&keep=1&srchmode=1&unlock 
There is a lot of organisms with only Bacteria in the taxonomy. I think the same problem can happen with other unclassified organisms like eukaryota and archea. 

That example looks normal, yes:

```
LOCUS       UYG54991                 150 aa            linear   BCT 18-NOV-2022
DEFINITION  GltA, partial [Candidatus Lariskella guizhouensis].
ACCESSION   UYG54991
VERSION     UYG54991.1
DBSOURCE    accession OP125508.1
KEYWORDS    .
SOURCE      Candidatus Lariskella guizhouensis
  ORGANISM  Candidatus Lariskella guizhouensis
            Bacteria; Proteobacteria; Alphaproteobacteria; Rickettsiales;
            Candidatus Midichloriaceae; Candidatus Lariskella.
REFERENCE   1  (residues 1 to 150)
...
```

The current behaviour was discussed on our RedMine Tracker as issue 2951, when the NCBI started using multi-line organism names, test case: https://www.ncbi.nlm.nih.gov/nuccore/NC_011147

That starts:

```
LOCUS       NC_011147            4581797 bp    DNA     circular CON 24-DEC-2022
DEFINITION  Salmonella enterica subsp. enterica serovar Paratyphi A str.
            AKU_12601, complete sequence.
ACCESSION   NC_011147
VERSION     NC_011147.1
DBLINK      BioProject: PRJNA224116
            BioSample: SAMEA1705953
            Assembly: GCF_000026565.1
KEYWORDS    RefSeq; complete genome.
SOURCE      Salmonella enterica subsp. enterica serovar Paratyphi A str.
            AKU_12601
  ORGANISM  Salmonella enterica subsp. enterica serovar Paratyphi A str.
            AKU_12601
            Bacteria; Proteobacteria; Gammaproteobacteria; Enterobacterales;
            Enterobacteriaceae; Salmonella.
REFERENCE   1
```

Quoting some of the old issue discussion:

>I just received the wording that will appear in Section 3.4.2 of gbrel.txt
>for this month's release:
>
>   ORGANISM     - Formal scientific name of the organism (first line)
>and taxonomic classification levels (second and subsequent lines).
>Mandatory subkeyword in all annotated entries/two or more records.
>
>   In the event that the organism name exceeds 68 characters (80 - 13 +
>1)
>   in length, it will be line-wrapped and continue on a second line,
>   prior to the taxonomic classification. Unfortunately, very long
>   organism names were not anticipated when the fixed-length GenBank
>   flatfile format was defined in the 1980s. The possibility of linewraps
>   makes the job of flatfile parsers more difficult : essentially, one
>   cannot be sure that the second line is truly a classification/lineage
>   unless it consists of multiple tokens, delimited by semi-colons.
>   The long-term solution to this problem is to introduce an additional
>   subkeyword, probably 'LINEAGE' . This might occur sometime in 2009
>   or 2010.

Quoting https://www.ncbi.nlm.nih.gov/genbank/release/252/ from October 15 2022,

>    ORGANISM	- Formal scientific name of the organism (first line)
> and taxonomic classification levels (second and subsequent lines).
> Mandatory subkeyword in all annotated entries/two or more records.
> 
>    In the event that the organism name exceeds 68 characters (80 - 13 + 1)
>    in length, it will be line-wrapped and continue on a second line,
>    prior to the taxonomic classification. Unfortunately, very long 
>    organism names were not anticipated when the fixed-length GenBank
>    flatfile format was defined in the 1980s. The possibility of linewraps
>    makes the job of flatfile parsers more difficult : essentially, one
>    cannot be sure that the second line is truly a classification/lineage
>    unless it consists of multiple tokens, delimited by semi-colons.
>    The long-term solution to this problem is to introduce an additional
>    subkeyword, possibly 'LINEAGE'. But because changes like this can be
>    very disruptive for customers, implementation has not yet been scheduled.

Perhaps we can also consider ``Bacteria.``, ``Archaea.`` and ``Eukaryota.`` as special case single entry lineages?
Yes, I think it's a good idea to place Bacteria., Archaea. and Eukaryota. as special cases but I don't know if there is other special cases with only one ancestor except these three. 
Possibly there are some virus examples?

I will make a pull request with that workaround, which I hope you can test.

Resolving patch:
diff --git a/Bio/GenBank/Scanner.py b/Bio/GenBank/Scanner.py
--- a/Bio/GenBank/Scanner.py
+++ b/Bio/GenBank/Scanner.py
@@ -576,7 +576,7 @@ def parse_cds_features(
                         # sub features...
                         annotations["raw_location"] = location_string.replace(" ", "")
 
-                        for (qualifier_name, qualifier_data) in qualifiers:
+                        for qualifier_name, qualifier_data in qualifiers:
                             if (
                                 qualifier_data is not None
                                 and qualifier_data[0] == '"'
@@ -1706,7 +1706,21 @@ def _feed_header_lines(self, consumer, lines):
                     while True:
                         line = next(line_iter)
                         if line[0 : self.GENBANK_INDENT] == self.GENBANK_SPACER:
-                            if lineage_data or ";" in line:
+                            if (
+                                lineage_data
+                                or ";" in line
+                                or line[self.GENBANK_INDENT :].strip()
+                                in (
+                                    "Bacteria.",
+                                    "Archaea.",
+                                    "Eukaryota.",
+                                    "Unclassified.",
+                                    "Viruses.",
+                                    "cellular organisms.",
+                                    "other sequences.",
+                                    "unclassified sequences.",
+                                )
+                            ):
                                 lineage_data += " " + line[self.GENBANK_INDENT :]
                             elif line[self.GENBANK_INDENT :].strip() == ".":
                                 # No lineage data, just . place holder


Test patch: