Repository: mir-evaluation/mir_eval
Issue: py.test fails
Root cause: Validation Gap

current master 72bb77a fails at two tests:

```
==================================================================== test session starts =====================================================================
platform darwin -- Python 2.7.9 -- py-1.4.26 -- pytest-2.6.4
plugins: cache, cov, pep8
collected 166 items 

tests/test_beat.py .......
tests/test_chord.py ..............................................F....................................................................................
tests/test_melody.py ....F....
tests/test_onset.py .
tests/test_pattern.py ......
tests/test_segment.py .......
tests/test_separation.py .
tests/test_util.py ....

========================================================================== FAILURES ==========================================================================
_______________________________________________________________________ test_encode[3] _______________________________________________________________________

function = <function encode at 0x10801aed8>, parameters = ('G:dim(4)/6',), exception = <class 'mir_eval.chord.InvalidChordException'>

    def __check_exception(function, parameters, exception):
        ''' Makes sure the provided function throws the provided
        exception given the provided input '''
>       nose.tools.assert_raises(exception, function, *parameters)

tests/test_chord.py:28: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py:473: in assertRaises
    callableObj(*args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <unittest.case._AssertRaisesContext object at 0x109af8bd0>, exc_type = None, exc_value = None, tb = None

    def __exit__(self, exc_type, exc_value, tb):
        if exc_type is None:
            try:
                exc_name = self.expected.__name__
            except AttributeError:
                exc_name = str(self.expected)
            raise self.failureException(
>               "{0} not raised".format(exc_name))
E           AssertionError: InvalidChordException not raised

/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/case.py:116: AssertionError
____________________________________________________________________ test_to_cent_voicing ____________________________________________________________________

    def test_to_cent_voicing():
        # We'll just test a few values from one of the test annotations
>       ref_file = sorted(glob.glob(REF_GLOB))[0]
E       IndexError: list index out of range

tests/test_melody.py:65: IndexError
============================================================ 2 failed, 164 passed in 0.95 seconds ============================================================
```


I'm not getting the first failure.  What is the result when you run the following code?

```
In [1]: import mir_eval

In [2]: mir_eval.chord.STRICT_BASS_INTERVALS = True

In [3]: mir_eval.chord.encode('G:dim(4)/6')
```

You should get an invalid chord exception (this is what the test which failed for you is checking for).  cc @ejhumphrey 

For the second failure, it looks like glob.glob(REF_GLOB) is empty, which means that `'data/melody/ref*.txt'` is empty/nonexistant from where you're running tests.  What command are you using to run the tests, and from which directory?  The intention is to run nosetests from the test directory.


Ah, interestingly, if I run the tests with py.test instead of nose, the `InvalidChordException` is not raised.  Maybe py.test somehow ignores this line?

```
    mir_eval.chord.STRICT_BASS_INTERVALS = True
```


One option which should fix this would be to make it so that rather than having a constant `STRICT_BASS_INTERVALS`, we just have a keyword argument `strict_bass_intervals` in `chord.encode` (the only place this constant is used).  I actually like this more than having a module-level constant.  @ejhumphrey  what do you think?  Why did you use a constant in the first place?


> @ejhumphrey https://github.com/ejhumphrey what do you think?
> 
> I can dig it.​
> 
> Why did you use a constant in the first place?
> 
> ​Originally I was punting on a decision I wanted to resolve​ definitively;
> now, I guess I see the value to both modes of operation, and an argument
> makes sense.
> 
> —
> Reply to this email directly or view it on GitHub
> https://github.com/craffel/mir_eval/issues/108#issuecomment-84083076.


OK, separate issue created #109.