Repository: mir-evaluation/mir_eval
Issue: Load time series no longer uses comma delimiter for .csv
Root cause: Architectural Constraint

In it's early iteration [load_time_series](https://github.com/craffel/mir_eval/blob/master/mir_eval/io.py#L214) selected the delimiter based on the filename extension, using whitespace for .txt and comma for .csv files:

```
if os.path.splitext(filename)[1] == '.csv':
        delimiter = ','
```

It seems that this functionality has been dropped in the new load_time_series function in io.py. Apart from breaking my code, I was wondering whether there was a specific reason for this? 

My thinking is - if the file extension is .csv, surely we can assume the delimiter is a comma, no? Does anyone oppose to adding this functionality back to load_time_series? (i.e. using comma for .csv files by default?)


CSV files often have tab delimiters.


but ... aren't then then tsv files?  Excel will save tab-delimited
files, but it gives them a .txt extension.

  DAn.

On Sun, Aug 31, 2014 at 5:14 PM, Brian McFee notifications@github.com wrote:

> CSV files often have tab delimiters.
> 
> —
> Reply to this email directly or view it on GitHub.


OK, wikipedia says: ( http://en.wikipedia.org/wiki/Comma-separated_values )

   For example, many so-called "CSV" files in fact use the tab
character instead of comma
   (such files can be more precisely referred to as "TSV" for tab
separated values);

Can't we have .csv default to a comma separator, with some option to override?

  DAn.

On Sun, Aug 31, 2014 at 5:23 PM, Dan Ellis dpwe@ee.columbia.edu wrote:

> but ... aren't then then tsv files?  Excel will save tab-delimited
> files, but it gives them a .txt extension.
> 
>   DAn.
> 
> On Sun, Aug 31, 2014 at 5:14 PM, Brian McFee notifications@github.com wrote:
> 
> > CSV files often have tab delimiters.
> > 
> > —
> > Reply to this email directly or view it on GitHub.


load_time_series supports overriding the delimiter, but the default behavior (for all files) is one or more spaces. As dan suggests, I'd default to a comma separator for .csv (if it's tab-separated then white space won't work anyway (?), but at least it will work for some (most?) csv files)


More generally, CSV is a nightmare, and we really shouldn't be in the business of parsing it directly. See http://tburette.github.io/blog/2014/05/25/so-you-want-to-write-your-own-CSV-code/

We should probably be using an existing csv library, but as I recall, all of the existing ones have issues with Unicode. (This points to a larger issue, that we should migrate to python 3 for better Unicode support, but that's getting a little off topic.)


Be the solution what it may, I do think we need to support (at least to an extent) loading in CSV timeseries - it's a commonly used format for storing f0 output from e.g. Sonic Visualiser, Sonic Annotator, etc., and I think many melody extraction folks (myself included) need this functionality. It's also a relatively well behaved case - two columns of purely floats, no unicode. (I know the point is to find a robust solution, but I'm saying that in the meantime something that works for the constrained case of melody extraction would be good to have)


Use the delimiter argument.  If you want it to be able to read a white-space delimited or a csv file, just use the regexp
`'\s+|,'`
(\s = whitespace, + = any amount, | = or , , = comma) instead of the default '\s+'.

If you think that there are more melody annotations which use comma-separated files than whitespace separated, then feel free to change the default to `','`.  If some melody annotations are comma separated and some are whitespace separated, that's ugly and I'd rather not encourage that; plus, as it is `mir_eval` can accommodate this behavior by making a minor change to a keyword arg.


The reality is that some annotations use white space and some use commas, and whilst I'd rather not encourage this either (--> JAMS), it's the way things are. The reason I'd be keen to support both is less for myself (I know I can change the code) but more for non-experienced users who don't know python and are hoping to use mir_eval out of the box. In fact, I started this whole thread after being contacted by a student who was trying to use mir_eval to evaluate melody extraction and ran into trouble when trying to use csv files exported from Sonic Visualiser.

Would it be really bad if the default regex for load_time_series was `delimiter=r'\s+|,'` ? Or, we could even only use it in the melody evaluator?