Repository: aphp/edsnlp (language: python)

Issue:
Feature request: Pollution
## Feature type

Add new patterns

## Description

Add pattern to match text like :

```
2/2Pat : <NOM> <Prenom> le <date> IPP <ipp> Intitulé RCP : Urologie HMN le <date>

1/2Pat : <NOM> <Prenom> le <date> IPP <ipp> Intitulé RCP : Urologie HMN le <date>
```



Discussion / hints:

Fix patch:
diff --git a/changelog.md b/changelog.md
index 180774f0b..607af3041 100644
--- a/changelog.md
+++ b/changelog.md
@@ -6,6 +6,9 @@
 
 - Add nephew, niece and daughter to family qualifier patterns
 
+### Added
+- Add new pattern (footer) to pipeline normalisation (pollution)
+
 ## v0.7.0 (2022-09-06)
 
 ### Added
diff --git a/docs/pipelines/core/normalisation.md b/docs/pipelines/core/normalisation.md
index 982102f84..19b4c7435 100644
--- a/docs/pipelines/core/normalisation.md
+++ b/docs/pipelines/core/normalisation.md
@@ -217,6 +217,7 @@ nlp.add_pipe(
 | `doctors`     | List of doctor names and specialities, often found in left-side note margins. Also source of potential *false positives*. | "... Dr ABC - Diabète/Endocrino ..."                                                                       | `True`              |
 | `web`         | Webpages URL and email adresses. Also source of potential *false positives*.                                              | "... www.vascularites.fr ..."                                                                              | `True`              |
 | `coding`      | Subsection containing ICD-10 codes along with their description. Also source of potential *false positives*.              | "... (2) E112 + Oeil (2) E113 + Neuro (2) E114 Démence (2) F03 MA (2) F001+G301 DCL G22+G301 Vasc (2) ..." | `False`             |
+`footer`| Footer of new page| "2/2Pat : NOM Prenom le 2020/01/01 IPP 12345678 Intitulé RCP : Urologie HMN le <date>"| `True`
 
 #### Custom pollution
 
diff --git a/edsnlp/pipelines/core/normalizer/pollution/factory.py b/edsnlp/pipelines/core/normalizer/pollution/factory.py
index aac296919..11ea82e5f 100644
--- a/edsnlp/pipelines/core/normalizer/pollution/factory.py
+++ b/edsnlp/pipelines/core/normalizer/pollution/factory.py
@@ -14,6 +14,7 @@
         doctors=True,
         web=True,
         coding=False,
+        footer=True,
     ),
 )
 
diff --git a/edsnlp/pipelines/core/normalizer/pollution/patterns.py b/edsnlp/pipelines/core/normalizer/pollution/patterns.py
index ff3f34c5e..525ced1c4 100644
--- a/edsnlp/pipelines/core/normalizer/pollution/patterns.py
+++ b/edsnlp/pipelines/core/normalizer/pollution/patterns.py
@@ -23,6 +23,9 @@
 # Subsection with ICD-10 Codes
 coding = r".*?[a-zA-Z]\d{2,4}.*?(\n|[a-zA-Z]\d{2,4})"
 
+# New page
+footer = r"(?i)^\d\/\d\s?pat.*ipp.*"
+
 pollution = dict(
     information=information,
     bars=bars,
@@ -30,4 +33,5 @@
     doctors=doctors,
     web=web,
     coding=coding,
+    footer=footer,
 )


Test patch:
diff --git a/tests/pipelines/core/test_normalisation.py b/tests/pipelines/core/test_normalisation.py
index b2e30dd43..b324ac1e0 100644
--- a/tests/pipelines/core/test_normalisation.py
+++ b/tests/pipelines/core/test_normalisation.py
@@ -87,3 +87,10 @@ def test_normalization_pollution(nlp_factory, text):
     norm = get_text(doc, attr="NORM", ignore_excluded=True)
 
     assert norm == "L'aïeul ʺnˊest pas malade”, écrit-il. Fièvre jaune."
+
+    text2 = "2/2Pat : <NOM> <Prenom> le <date> IPP <ipp> Intitulé RCP"
+    text2 += " : Urologie HMN le <date>\nRéunion de Concertation"
+    text2 += " Pluridisciplinaire\nHôpital Henri Mondor"
+    doc = nlp(text2)
+    norm = get_text(doc, attr="NORM", ignore_excluded=True)
+    assert norm == "\nRéunion de Concertation Pluridisciplinaire\nHôpital Henri Mondor"