Repository: saulpw/visidata

Issue:
[freqtbl] toggling rows selects/unselects an inconsistent number of rows
**Small description**
In a FreqTableSheet where its source sheet has many rows, when rows are toggled, the number of rows that is selected/unselected on the source sheet is inconsistent.

**Expected result**
Every time the toggle is performed, the number of rows selected/unselected on the source sheet should be the same.

**Actual result with screenshot**
The number of rows selected on the source sheet is different every time. Sometimes, the number of unselected rows is negative.
https://asciinema.org/a/ksxBTGy0E1tGh5unX2Gsu0eeR

**Steps to reproduce with sample data and a .vd**
`python -c "n = 200_000; s = 'line\n'; print(f'{s*n}',end='')" |vd`
hit `F` and then repeatedly hit `t` (or `gt` or `gzt` or, in a sheet with more than one kind of cell, `zt`)

**Additional context**
VisiData v2.10.2

Workaround:
Use `s` and `u` to select/unselect rows instead of toggling.

There is a simple fix for the inconsistent behavior, by adding vd.sync() to selectRow() and unselectRow().
https://github.com/midichef/visidata/commit/52d8596ed01fed17bca8b5215130a77801f1e3b3
However, I'm unsure if this is the correct solution for 2 reasons.
First, I don't have a good understanding of how threading works in the visidata codebase.
Second, it's not clear to me how FreqTableSheet toggling should behave. Suppose that some rows on the source sheet are already selected, but not all of them. The way it works now is that after a toggle, all matching rows in the source sheet will end up in the same state: either all are selected, or all are unselected. But arguably the effect should be to toggle the selected status of all the matching rows in the source sheet, leaving some selected and some unselected.


Discussion:
Wow, this is an interesting bug, @midichef !

I agree with you that the behavior should be consistent, above all else.  We can *probably* add the vd.sync() to selectRow/unselectRow as you suggest, since it seems that every caller is either an asyncthread function already, or is only selecting/unselecting a single row.  But maybe a better solution ultimately is to prevent executing any commands that call asyncthread functions, if any asyncthread functions are already running on that sheet, since in that case we are just asking for a race condition.

To your second point, I think (but am not sure) that the toggle operation on the FreqTableSheet should act on the individual rows of the FreqTableSheet, which then cascade to the source sheet (as it currently does), instead of acting independently on the source sheet selection.  I could be convinced otherwise, but to be honest, this "cascading selection" of the FreqTableSheet itself is almost too tricky, and I've thought more than once that maybe it should be removed. But then I want to delete a bunch of rows off the source sheet, based on what I see in the FreqTableSheet, and there's not a good way to do this otherwise.
@saulpw I regularly use toggle on the FreqTableSheet.  My use case is usually for set difference.   I  generate a Frequency sheet, on the column(s) of interest, then  select a couple rows that I don't want and then use `gt` to get the inverse selection on the base sheet, then I quit the Freq sheet, and  use `"` `dup-sheet` to get the interesting subset. I think there are some options to do this without toggle, This is what my fingers know.
@frosencrantz Good to know!  So do you think that toggle on the FreqTableSheet should actually(a) toggle the selection of every source row in that bin, or (b) toggle the bin itself, and make the selection status of every source row in that bin conform to the bin selection status?
> (b) toggle the bin itself, and make the selection status of every source row in that bin conform to the bin selection status

Which after re-reading now is what you were already suggesting.  I wasn't fully following it before. 
This should be fixed in df4f51e884cd4e45faf88a2e6f2c24b9ec125c8f along with #1148. Commands that would spawn threads will fail if other threads are already running.

Resolving patch:
diff --git a/visidata/selection.py b/visidata/selection.py
--- a/visidata/selection.py
+++ b/visidata/selection.py
@@ -23,8 +23,10 @@ def isSelected(self, row):
 def toggle(self, rows):
     'Toggle selection of given *rows*.  Async.'
     self.addUndoSelection()
-    for r in Progress(rows, 'toggling', total=len(self.rows)):
-        if not self.unselectRow(r):
+    for r in Progress(rows, 'toggling', total=len(rows)):
+        if self.isSelected(r):  #1671
+            self.unselectRow(r)
+        else:
             self.selectRow(r)
 
 
@@ -39,7 +41,9 @@ def select_row(self, row):
 def toggle_row(self, row):
     'Toggle selection of given *row*.'
     self.addUndoSelection()
-    if not self.unselectRow(row):
+    if self.isSelected(row):
+        self.unselectRow(row)
+    else:
         self.selectRow(row)
 
 


Test patch: