Paper:
Measuring Mutual Information (MI) between high-dimensional, continuous, random variables from observed samples has wide theoretical and practical applications. Recent works have developed accurate MI estimators through provably lowbias approximations and tight variational lower bounds assuming abundant supply of samples, but require an unrealistic number of samples to guarantee statistical significance of the estimation. In this work, we focus on improving data efficiency and propose a Data-Efficient MINE Estimator (DEMINE) that can provide a tight lower confident interval of MI under limited data, through adding cross-validation to the MINE lower bound (Belghazi et al., 2018). Hyperparameter search is employed and a novel meta-learning approach with task augmentation is developed to increase robustness to hyperparameters, reduce overfitting and improve accuracy. With improved data-efficiency, our DEMINE estimator enables statistical testing of dependency at practical dataset sizes. We demonstrate the effectiveness of DEMINE on synthetic benchmarks and real world fMRI data, with application of inter-subject correlation analysis.

Review:
Second update:
We thank the authors for updating their paper.
The work is now improving, and is on the right track for publication at a future conference.  There are a few comments on the new results, and suggestions for further improvement:
* The issue of possible false positives due to sample dependence in fMRI data has again been ignored in the rebuttal. Without careful treatment of this effect, these results are vacuous.
* There is still no comparison with competing nonparametric tests on the fMRI data.
* the results linking the COCO and MINE estimators are interesting. Some statements don't make sense, however, eg. "HSIC-based estimators tend to have loose confidence intervals due to the need to bound generalization error of kernels f and g on unseen data points." First, f and g are functions, not kernels. Second, testing with HSIC or COCO does not require generalisation to unseen data points: this is why testing is an easier problem than regression.
For HSIC testing, I am surprised to read in the footnote that the Gamma approximation report significantly more than 5% errors, especially given the Table 4 results that show the correct level. In any case, I very strongly suggest using a permutation approach to obtain the test threshold for HSIC, which is by far the most robust and reliable method. The Gamma approximation has no statistical guarantees, as stated explicitly in the HSIC testing paper of NeurIPS 2007. Permutation gives a guarantee of the correct level. See the NeurIPS paper for details. Once you have verified the correct false positive rate for the permutation threshold, then you can compute the p-value on the alternative.
While the comparison with HSIC is a helpful one, it is also required to compare with the competing method closest to yours, i.e. the Berrett  and Samworth test. In addition, HSIC is a non-adaptive test, but your test is adaptive, so a fairer comparison would be to a modern adaptive test such as "An Adaptive Test of Independence with Analytic Kernel Embeddings."
* The false positive rate in the sanity check is far below the design level of 0.05. This is as I expected, given the use of the Hoeffding bound. This should be stated clearly in the main text, and not disclosed in the final sentence of the final page of the appendix.
====================
Update: thank you for your rebuttal. 
"the necessary and sufficient condition of dependency, is more general and is complementary to other techniques that make stronger assumptions about the data. "
The alternative tests listed are nonparametric. That is to say, unlike Pearson correlation, they do not make specific parametric assumptions on the nature of the dependence. Rather they make generic smoothness assumptions (your test also makes such assumptions by the choice of neural network architecture). Thus, comparison with the prior work in Statistics and Machine Learning is relevant, since these tests have the same aims and scope as your tests. 
The cited Berrett and Samworth MI test uses a permutation approach to obtaining the test threshold, not an asymptotic approach  (see the results of Section 4 of that paper).   Several of the other cited tests also use a permutation approach for the test threhsold. These tests are therefore relevant prior work.  
" If things change very little from one second to the next, the signals could be very similar and may not really be, intuitively, independent samples and may bias result of the study. However, which independence assumptions to use is not in scope for our paper,  because our fMRI study is trying to show that dependency testing works "
In the work cited by Chwialkowski et al, failure to account for the dependence between samples results in excessive false positives. This is because, for dependent data, the effective sample size is reduced, and the tests must be made more conservative to correct for this effect. It is therefore the case that the fMRI results may be false positives.
Re level:"This proof could be experimentally verified ..."  This should be verified. In particular, Hoeffding can be very loose in practice, which is likely to be observed in experiments. 
======
The authors propose a procedure for improving neural mutual information estimates, via a combination of data augmentation and cross validation. They then use these estimates in hypothesis testing, on low dimensional toy datasets and on high dimensional real-world fMRI data.
The improved training procedures for MI estimation are of interest, however the hypothesis testing parts of the paper could still be improved.
In hypothesis testing, it is important to verify that the test has the correct level (false positive rate). This is all the more essential when the estimate has required optimisation over parameters. It is not clear from the presentation that this has been confirmed.
There are a number of prior approaches to testing for multivariate statistical dependence in the machine learning and statistics literature (including a 2017 paper which uses mutual information). A small selection is given below, although a literature search will reveal many more papers. In the absence of citation or comparison with any of the prior work on multivariate statistical dependence testing, the current submission is not suitable for publication. 
In statistics:
---------------
https://arxiv.org/abs/1711.06642
Nonparametric independence testing via mutual information
Thomas B. Berrett, Richard J. Samworth
2017
Measuring and testing dependence by correlation of distances
Gábor J. Székely, Maria L. Rizzo, and Nail K. Bakirov
Ann. Statist.
Volume 35, Number 6 (2007), 2769-2794.
Large-scale kernel methods for independence testing
Qinyi ZhangEmail Sarah Filippi, Arthur Gretton, Dino Sejdinovic
Statistics and Computing
January 2018, Volume 28, Issue 1, pp 113–130| Cite as
In machine learning:
---------------------
Multivariate tests of association based on univariate tests
Heller, Ruth and Heller, Yair
Advances in Neural Information Processing Systems 29
2016
A Kernel Statistical Test of Independence
Gretton, Arthur and Fukumizu, Kenji and Choon H. Teo and Song, Le and Sch\"{o}lkopf, Bernhard and Alex J. Smola
Advances in Neural Information Processing Systems 20
2008
http://papers.nips.cc/paper/3201-a-kernel-statistical-test-of-independence.pdf
http://proceedings.mlr.press/v70/jitkrittum17a/jitkrittum17a.pdf
An Adaptive Test of Independence with Analytic Kernel Embeddings
Wittawat Jitkrittum, Zoltán Szabó, Arthur Gretton ; ICML 2017, PMLR 70:1742-1751
Time dependence
----------------
It is also the case that if the variables have time dependence, then appropriate corrections must be made for the test threshold, to avoid excessive false positives. Does the fMRI data exhibit time dependence?  For the case of multivariate statistical dependence testing, such corrections are described e.g. in:
https://papers.nips.cc/paper/5452-a-wild-bootstrap-for-degenerate-kernel-tests.pdf
A Wild Bootstrap for Degenerate Kernel Tests
Kacper Chwialkowski, Dino Sejdinovic, Arthur Gretton
NeurIPS 2014