Repository: nipy/heudiconv

Issue:
Participants.tsv will null values
<!-- DO NOT DELETE THIS! 
This template is used to facilitate issue resolution.
All text in <!-> tags will not be displayed.
-->

### Summary
<!-- If you are having conversion troubles, please share as much
relevant information as possible. This includes, but is not limited to:
- log of conversion
- heuristic
-->
In participants.tsv age and sex have null values (empty cells) instead of a 'n/a', which causes bids validation to fail.

I'm not sure these N/A replacements catch all the edge cases:
https://github.com/nipy/heudiconv/blob/master/heudiconv/bids.py#L285

I would've expected age to be fixed by this commit:
https://github.com/nipy/heudiconv/commit/7a7ef5c5f9852ef2b479aafcd5e09e462a08193c#diff-9f5ed69cb0b31e87bd735166c54cbd41b50710d545dc0e48d4d42f21269271c2

But I believe the n/a replacement for sex has been around awhile, and is in the version I'm using (0.9.0), but sex is still missing from participants.tsv.

ex:
```
participant_id	age	sex	group
sub-111			        control
```
Yarik, you can checkout the UIC sub-travhuman data for an example.

I sorted this out on my own with pandas and numpy:
```
import pandas as pd
import numpy as np
participants_df = pd.read_csv('participants.tsv', sep='\t')
participants_df = participants_df.replace(np.nan, 'n/a')
participants_df.to_csv('participants.tsv', sep="\t")
```
but it's probably worth doing within heudiconv. 


### Platform details:

Choose one:
- [ ] Local environment
<!-- If selected, please provide OS and python version -->
- [x] Container
<!-- If selected, please provide container name and tag"-->
Docker container: nipy/heudiconv:0.9.0

- Heudiconv version: 0.9.0
<!-- To check: run heudiconv with just the --version flag -->


Thanks,
Joshua

Comments:

Patch:
diff --git a/heudiconv/bids.py b/heudiconv/bids.py
index e07c347..fc630aa 100644
--- a/heudiconv/bids.py
+++ b/heudiconv/bids.py
@@ -47,6 +47,36 @@ class BIDSError(Exception):
 BIDS_VERSION = "1.4.1"
 
 
+def maybe_na(val):
+    """Return 'n/a' if non-None value represented as str is not empty
+
+    Primarily for the consistent use of lower case 'n/a' so 'N/A' and 'NA'
+    are also treated as 'n/a'
+    """
+    if val is not None:
+        val = str(val)
+        val = val.strip()
+    return 'n/a' if (not val or val in ('N/A', 'NA')) else val
+
+
+def treat_age(age):
+    """Age might encounter 'Y' suffix or be a float"""
+    age = str(age)
+    if age.endswith('M'):
+        age = age.rstrip('M')
+        age = float(age) / 12
+        age = ('%.2f' if age != int(age) else '%d') % age
+    else:
+        age = age.rstrip('Y')
+    if age:
+        # strip all leading 0s but allow to scan a newborn (age 0Y)
+        age = '0' if not age.lstrip('0') else age.lstrip('0')
+        if age.startswith('.'):
+            # we had float point value, let's prepend 0
+            age = '0' + age
+    return age
+
+
 def populate_bids_templates(path, defaults={}):
     """Premake BIDS text files with templates"""
 
@@ -278,12 +308,13 @@ def add_participant_record(studydir, subject, age, sex):
                             "control group)")])),
                 ]),
                 sort_keys=False)
+
     # Add a new participant
     with open(participants_tsv, 'a') as f:
         f.write(
           '\t'.join(map(str, [participant_id,
-                              age.lstrip('0').rstrip('Y') if age else 'N/A',
-                              sex if sex else 'n/a',
+                              maybe_na(treat_age(age)),
+                              maybe_na(sex),
                               'control'])) + '\n')
 
 


Test patch:
diff --git a/heudiconv/tests/test_bids.py b/heudiconv/tests/test_bids.py
new file mode 100644
index 0000000..a980e35
--- /dev/null
+++ b/heudiconv/tests/test_bids.py
@@ -0,0 +1,26 @@
+"""Test functions in heudiconv.bids module.
+"""
+
+from heudiconv.bids import (
+    maybe_na,
+    treat_age,
+)
+
+
+def test_maybe_na():
+    for na in '', ' ', None, 'n/a', 'N/A', 'NA':
+        assert maybe_na(na) == 'n/a'
+    for notna in 0, 1, False, True, 'value':
+        assert maybe_na(notna) == str(notna)
+
+
+def test_treat_age():
+    assert treat_age(0) == '0'
+    assert treat_age('0') == '0'
+    assert treat_age('0000') == '0'
+    assert treat_age('0000Y') == '0'
+    assert treat_age('000.1Y') == '0.1'
+    assert treat_age('1M') == '0.08'
+    assert treat_age('12M') == '1'
+    assert treat_age('0000.1') == '0.1'
+    assert treat_age(0000.1) == '0.1'
\ No newline at end of file