Benp/total event report

A few questions:

* Is there an easy way to remove the warnings, by specifying client:
hadoopcli as it desires?   It's already in the client.cfg file, which I
thought was merged with other config entries, so I don't know why the
warning is even happening on regular runs.  But it should be
straight-forward to add it to config/test.cfg, just to be sure it's there
as well.

* And why does a warning like that trigger cleanup?

* And why do other tests that use addCleanup() seem to work fine?  (E.g.
test_answer_dist, test_enrollment_validation, test_mapreduce,
test_mysql_load, test_sqoop, test_user_location...)


On Tue, Dec 23, 2014 at 6:30 PM, Ben Patterson <notifications@github.com>
wrote:

> In edx/analytics/tasks/reports/tests/test_total_events_report.py
> <https://github.com/edx/edx-analytics-pipeline/pull/52#discussion-diff-22240627>
> :
>
> > +
> > +    def test_multilines_sorted(self):
> > +        source = """
> > +        2014-10-03 300
> > +        2015-11-01 7000
> > +        2014-10-02 15
> > +        """
> > +        out = self.run_task(source)
> > +        # Info for the 2015 date should appear first
> > +        self.assertEqual(out.loc[1]['count'], '7000')
> > +        self.assertEqual(out.loc[1]['date'], '2015-11-01')
> > +        self.assertEqual(out.loc[2]['count'], '300')
> > +        self.assertEqual(out.loc[2]['date'], '2014-10-03')
> > +
> > +    def tearDown(self):
> > +        self.cleanup(self.temp_rootdir)
>
> Ok, after some digging, it looks like addCleanup will not work for us.
> There is an underlying configuration that emits a warning:
>
> /Users/benpatterson/workspaces/edx-analytics-pipeline/venv/src/luigi/luigi/hdfs.py:539: UserWarning: Deprecated: Specify 'client: hadoopcli' in config
>   warnings.warn("Deprecated: Specify 'client: hadoopcli' in config")
>
> This happens during load, which means addCleanup is called immediately,
> rather than at the end of the tests. I've tried a few different
> configurations in test.cfg but this setting is not immediately apparent to
> me (and I'm not sure I'd want to override default hdfs client behavior,
> either).
>
> So...took a good look but I think we'll be alright with tearDown instead,
> even if it's suboptimal.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/edx/edx-analytics-pipeline/pull/52/files#r22240627>.
>