Repository: rasbt/mlxtend (Python)

Issue:
McNemar's test mcnemar_table off diagonal entries are swapped
Hi Sebastian, 

the tb[1,0] and tb[0,1] entries in the 2x2 contigency table for McNemar's test in _mcnemar_table_ are swapped: https://github.com/rasbt/mlxtend/blob/master/mlxtend/evaluate/mcnemar.py#L63 

While it should not affect the test statistic, the layout is not consistent with the one you describe here: https://arxiv.org/abs/1811.12808 (Figure 19 and Figure 20)

Best,
Marius



Discussion:
Yes, i faced the same issue. tb[0,1] and tb[1,0] are swapped. 
I am just seeing this issue, thanks @mmarius and @sankaran45 . So basically you are suggesting to change the current ordering:

![mcnemar_table_13_0](https://user-images.githubusercontent.com/5618407/94343084-4fcd5980-ffdb-11ea-9ca1-d2523f3588ec.png)

such that it matches this ordering:

![mcnemar_contingency_table](https://user-images.githubusercontent.com/5618407/94343090-5956c180-ffdb-11ea-9b93-cb09bd590bfb.png)

Sure, that can be done. If you'd like to submit a PR, that'd be great.



Patch:
diff --git a/docs/sources/CHANGELOG.md b/docs/sources/CHANGELOG.md
index 5b6f16ada..4d28e8195 100755
--- a/docs/sources/CHANGELOG.md
+++ b/docs/sources/CHANGELOG.md
@@ -27,6 +27,7 @@ The CHANGELOG for the current development version is available at
 ##### Changes
 
 - `permutation_test` (`mlxtend.evaluate.permutation`) ìs corrected to give the proportion of permutations whose statistic is *at least as extreme* as the one observed. ([#721](https://github.com/rasbt/mlxtend/pull/721) via [Florian Charlier](https://github.com/DarthTrevis))
+- Fixes the McNemar confusion matrix layout to match the convention (and documentation), swapping the upper left and lower right cells. ([#744](https://github.com/rasbt/mlxtend/pull/744) via [mmarius](https://github.com/mmarius))
 
 ##### Bug Fixes
 
diff --git a/mlxtend/evaluate/mcnemar.py b/mlxtend/evaluate/mcnemar.py
index 12f7e1cf8..517e15b3a 100644
--- a/mlxtend/evaluate/mcnemar.py
+++ b/mlxtend/evaluate/mcnemar.py
@@ -59,9 +59,9 @@ def mcnemar_table(y_target, y_model1, y_model2):
     tb = np.zeros((2, 2), dtype=int)
 
     tb[0, 0] = np.sum(plus_true == 2)
+    tb[0, 1] = np.sum(minus_true == 1)
+    tb[1, 0] = np.sum(minus_true == -1)
     tb[1, 1] = np.sum(plus_true == 0)
-    tb[1, 0] = np.sum(minus_true == 1)
-    tb[0, 1] = np.sum(minus_true == -1)
 
     return tb
 
@@ -148,9 +148,9 @@ def mcnemar_tables(y_target, *y_model_predictions):
         minus_true = model1_vs_true - model2_vs_true
 
         tb[0, 0] = np.sum(plus_true == 2)
+        tb[0, 1] = np.sum(minus_true == 1)
+        tb[1, 0] = np.sum(minus_true == -1)
         tb[1, 1] = np.sum(plus_true == 0)
-        tb[1, 0] = np.sum(minus_true == 1)
-        tb[0, 1] = np.sum(minus_true == -1)
 
         name_str = 'model_%s vs model_%s' % (comb[0], comb[1])
         tables[name_str] = tb


Test patch:
diff --git a/mlxtend/evaluate/tests/test_mcnemar_table.py b/mlxtend/evaluate/tests/test_mcnemar_table.py
index 4498af9ec..09f13d605 100644
--- a/mlxtend/evaluate/tests/test_mcnemar_table.py
+++ b/mlxtend/evaluate/tests/test_mcnemar_table.py
@@ -75,8 +75,8 @@ def test_input_binary():
     tb = mcnemar_table(y_target=y_target,
                        y_model1=y_model1,
                        y_model2=y_model2)
-    expect = np.array([[4, 1],
-                       [2, 3]])
+    expect = np.array([[4, 2],
+                       [1, 3]])
 
     np.testing.assert_array_equal(tb, expect)
 
@@ -89,7 +89,7 @@ def test_input_nonbinary():
     tb = mcnemar_table(y_target=y_target,
                        y_model1=y_model1,
                        y_model2=y_model2)
-    expect = np.array([[4, 1],
-                       [2, 3]])
+    expect = np.array([[4, 2],
+                       [1, 3]])
 
     np.testing.assert_array_equal(tb, expect)
diff --git a/mlxtend/evaluate/tests/test_mcnemar_tables.py b/mlxtend/evaluate/tests/test_mcnemar_tables.py
index 9b981f862..6d79972e1 100644
--- a/mlxtend/evaluate/tests/test_mcnemar_tables.py
+++ b/mlxtend/evaluate/tests/test_mcnemar_tables.py
@@ -96,11 +96,11 @@ def test_input_binary():
                         y_model2,
                         y_model3)
 
-    expect1 = np.array([[4, 1],
-                       [2, 3]])
+    expect1 = np.array([[4, 2],
+                       [1, 3]])
 
-    expect2 = np.array([[5, 0],
-                        [1, 4]])
+    expect2 = np.array([[5, 1],
+                        [0, 4]])
 
     expect3 = np.array([[4, 1],
                         [1, 4]])
@@ -121,11 +121,11 @@ def test_input_nonbinary():
                         y_model2,
                         y_model3)
 
-    expect1 = np.array([[4, 1],
-                       [2, 3]])
+    expect1 = np.array([[4, 2],
+                       [1, 3]])
 
-    expect2 = np.array([[5, 0],
-                        [1, 4]])
+    expect2 = np.array([[5, 1],
+                        [0, 4]])
 
     expect3 = np.array([[4, 1],
                         [1, 4]])