internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: TabbycatDebate/tabbycat

Issue:
Add button for direct preformed panel allocation
As an alternative for the Hungarian-based algorithm, as it is somewhat common to just want all in the same order of the panels, and deal with conflicts afterwards.


Discussion:
This seems to be a quite straightforward task that shows the interaction between the many components Tabbycat uses, including workers, Django, and Vue.js. It's a good introductory task.

To accomplish this task, there are a few steps:

1.  In `tabbycat/templates/modals/ModalForAllocating.vue`, add a modal on line 20 with a `v-if` the preformed panels allocator was selected. In this modal, you're given the option for Hungarian or Direct.
2. Add a variable to that file to distinguish the type of allocation sought
3. In `tabbycat/adjallocation/consumers.py`, import `DirectPreformedPanelAllocator` from `.preformed.direct` and on lines 76+, use the variable defined in (2) to decide which allocator to use.

Resolving patch:
diff --git a/tabbycat/adjallocation/consumers.py b/tabbycat/adjallocation/consumers.py
--- a/tabbycat/adjallocation/consumers.py
+++ b/tabbycat/adjallocation/consumers.py
@@ -17,6 +17,7 @@
 from .models import PreformedPanel
 from .preformed import copy_panels_to_debates
 from .preformed.anticipated import calculate_anticipated_draw
+from .preformed.direct import DirectPreformedPanelAllocator
 from .preformed.hungarian import HungarianPreformedPanelAllocator
 from .serializers import (EditPanelAdjsPanelSerializer,
                           SimpleDebateAllocationSerializer, SimpleDebateImportanceSerializer,
@@ -37,7 +38,7 @@ class AdjudicatorAllocationWorkerConsumer(EditDebateOrPanelWorkerMixin):
     def _apply_allocation_settings(self, round, settings):
         t = round.tournament
         for key, value in settings.items():
-            if key == "usePreformedPanels":
+            if key in ("usePreformedPanels", "allocationMethod"):
                 # Passing this here is much easier than splitting the function
                 continue # (Not actually a preference; just a toggle from Vue)
             # No way to force front-end to only accept floats/integers :(
@@ -73,7 +74,10 @@ def allocate_debate_adjs(self, event):
 
             debates = round.debate_set.all()
             panels = round.preformedpanel_set.all()
-            allocator = HungarianPreformedPanelAllocator(debates, panels, round)
+            if event['extra']['settings']['allocationMethod'] == 'hungarian':
+                allocator = HungarianPreformedPanelAllocator(debates, panels, round)
+            else:
+                allocator = DirectPreformedPanelAllocator(debates, panels, round)
 
             debates, panels = allocator.allocate()
             copy_panels_to_debates(debates, panels)


Test patch:
Repository: TabbycatDebate/tabbycat

Issue:
Update BP preset for inclusion of both speeches in team score



Discussion:

Resolving patch:
diff --git a/tabbycat/options/presets.py b/tabbycat/options/presets.py
--- a/tabbycat/options/presets.py
+++ b/tabbycat/options/presets.py
@@ -119,7 +119,7 @@ class BritishParliamentaryPreferences(PreferencesPreset):
     scoring__score_max                         = 99.0
     scoring__score_step                        = 1.0
     scoring__maximum_margin                    = 0.0
-    scoring__teamscore_includes_ghosts         = False  # WUDC 34.9.3.2
+    scoring__teamscore_includes_ghosts         = True  # WUDC 34.9.3.2
     # Debate Rules
     debate_rules__substantive_speakers         = 2
     debate_rules__teams_in_debate              = 'bp'


Test patch:
Repository: TabbycatDebate/tabbycat

Issue:
Expose site timezone in API
Could be at the root level.


Discussion:

Resolving patch:
diff --git a/tabbycat/api/serializers.py b/tabbycat/api/serializers.py
--- a/tabbycat/api/serializers.py
+++ b/tabbycat/api/serializers.py
@@ -132,6 +132,7 @@ class RootLinksSerializer(serializers.Serializer):
         v1 = serializers.HyperlinkedIdentityField(view_name='api-v1-root')
 
     _links = RootLinksSerializer(source='*', read_only=True)
+    timezone = serializers.CharField(allow_blank=False, read_only=True)
     version = serializers.CharField()
 
 
diff --git a/tabbycat/api/views.py b/tabbycat/api/views.py
--- a/tabbycat/api/views.py
+++ b/tabbycat/api/views.py
@@ -65,6 +65,7 @@ def get(self, request, format=None):
             "_links": {
                 "v1": reverse('api-v1-root', request=request, format=format),
             },
+            "timezone": settings.TIME_ZONE,
             "version": settings.TABBYCAT_VERSION,
         })
 


Test patch:
Repository: TabbycatDebate/tabbycat

Issue:
Create preset for using private URLs
Private URLs are becoming more commonly used, but could be hard to find. A settings preset could be created to enable all uses of private URLs (and with #2162 could be disabled).


Discussion:

Resolving patch:
diff --git a/tabbycat/options/presets.py b/tabbycat/options/presets.py
--- a/tabbycat/options/presets.py
+++ b/tabbycat/options/presets.py
@@ -354,3 +354,13 @@ class TabRelease(PreferencesPreset):
     public_features__public_team_standings     = False
     public_features__public_draw               = 'off'
     public_features__public_break_categories   = False
+
+
+class PrivateURLS(PreferencesPreset):
+    name = _("Use Private URLs")
+    show_in_list = False
+    description = _("Enables participant data entry through private URLs.")
+
+    data_entry__participant_ballots             = 'private-urls'
+    data_entry__participant_feedback            = 'private-urls'
+    data_entry__public_checkins_submit          = True


Test patch:
Repository: TabbycatDebate/tabbycat

Issue:
Automatically generate private URL key and checkin identifier on participant import
It would reduce the number of steps required to use the checkins and private URLs features if people's/rooms keys could be automatically generated when importing them.


Discussion:

Resolving patch:
diff --git a/tabbycat/importer/forms.py b/tabbycat/importer/forms.py
--- a/tabbycat/importer/forms.py
+++ b/tabbycat/importer/forms.py
@@ -8,7 +8,9 @@
 from django.core.validators import validate_email
 from django.utils.translation import gettext as _, ngettext, ngettext_lazy
 
+from checkins.models import PersonIdentifier, VenueIdentifier
 from participants.models import Adjudicator, Institution, Speaker, Team
+from privateurls.utils import populate_url_keys
 from venues.models import Venue
 
 logger = logging.getLogger(__name__)
@@ -150,6 +152,12 @@ class Meta:
         model = Venue
         fields = ('name', 'priority')
 
+    def save(self, commit=True):
+        venue = super().save(commit=False)
+        if commit:
+            VenueIdentifier.objects.create(venue=venue)
+        return venue
+
 
 class BaseInstitutionObjectDetailsForm(BaseTournamentObjectDetailsForm):
     """Adds a hidden input for the institution and automatic detection of the
@@ -280,7 +288,11 @@ def save(self, commit=True):
         if commit:
             team.save()
             for name, email in zip_longest(self.cleaned_data['speakers'], self.cleaned_data['emails']):
-                team.speaker_set.create(name=name, email=email)
+                speaker = team.speaker_set.create(name=name, email=email)
+
+                PersonIdentifier.objects.create(person=speaker)
+                populate_url_keys([speaker])
+
             team.break_categories.set(team.tournament.breakcategory_set.filter(is_general=True))
 
             if team.institution:
@@ -320,8 +332,13 @@ def clean_base_score(self):
 
     def save(self, commit=True):
         adj = super().save(commit=commit)
-        if commit and adj.institution:
-            adj.adjudicatorinstitutionconflict_set.create(institution=adj.institution)
+
+        if commit:
+            if adj.institution:
+                adj.adjudicatorinstitutionconflict_set.create(institution=adj.institution)
+            PersonIdentifier.objects.create(person=adj)
+            populate_url_keys([adj])
+
         return adj
 
 
diff --git a/tabbycat/privateurls/utils.py b/tabbycat/privateurls/utils.py
--- a/tabbycat/privateurls/utils.py
+++ b/tabbycat/privateurls/utils.py
@@ -1,6 +1,6 @@
 import logging
 import string
-from typing import TYPE_CHECKING
+from typing import Iterable, TYPE_CHECKING
 
 from participants.models import Person
 from utils.misc import generate_identifier_string
@@ -11,7 +11,7 @@
 logger = logging.getLogger(__name__)
 
 
-def populate_url_keys(people: 'QuerySet[Person]', length: int = 8, num_attempts: int = 10) -> None:
+def populate_url_keys(people: 'Iterable[Person]', length: int = 8, num_attempts: int = 10) -> None:
     """Populates the URL key field for every instance in the given QuerySet."""
     chars = string.ascii_lowercase + string.digits
 


Test patch:
Repository: TabbycatDebate/tabbycat

Issue:
Ballots' column in private URL table is not in the same order



Discussion:

Resolving patch:
diff --git a/tabbycat/utils/tables.py b/tabbycat/utils/tables.py
--- a/tabbycat/utils/tables.py
+++ b/tabbycat/utils/tables.py
@@ -849,12 +849,15 @@ def add_debate_ballot_link_column(self, debates, show_ballot=False):
             self.add_column(ballot_links_header, ballot_links_data)
 
         elif self.private_url:
-            debates = Debate.objects.filter(pk__in=[d.pk for d in debates]).select_related('round').annotate(
+            debateqs = Debate.objects.filter(pk__in=[d.pk for d in debates]).select_related('round').annotate(
                 has_ballot=Exists(BallotSubmission.objects.filter(debate_id=OuterRef('id')).exclude(discarded=True)),
             ).prefetch_related(
-                Prefetch('ballotsubmission_set', queryset=BallotSubmission.objects.exclude(discarded=True), to_attr='nondiscard_ballots'))
+                Prefetch('ballotsubmission_set', queryset=BallotSubmission.objects.exclude(discarded=True), to_attr='nondiscard_ballots'),
+            ).all()
+            annotated_debates = {d.pk: d for d in debateqs}
             ballot_links_data = []
-            for debate in debates:
+            for o_debate in debates:
+                debate = annotated_debates[o_debate.pk]
                 if not debate.has_ballot:
                     ballot_links_data.append(no_ballot)
                 elif not get_result_class(debate.nondiscard_ballots[0], debate.round, self.tournament).uses_speakers:


Test patch:
Repository: google/fuzzbench

Issue:
Eclipser maxfilelen value
I experienced the same problem that you had in choosing maxfilelen to evaluate Eclipser some time ago.
I found that they used 1048576 in their experiments (https://github.com/SoftSec-KAIST/Eclipser-Artifact/blob/master/docker-scripts/experiment-scripts/package-exp/run_eclipser.sh#L25), so maybe you want to use this value to produce consistent results with the paper.


Discussion:
That sounds good.
@jchoi2022 - would you be willing to look at our integration in https://github.com/google/fuzzbench/blob/master/fuzzers/eclipser/fuzzer.py and verify if we configured things correctly. Would be very helpful to get your contributions here.

Resolving patch:
diff --git a/fuzzers/eclipser/fuzzer.py b/fuzzers/eclipser/fuzzer.py
--- a/fuzzers/eclipser/fuzzer.py
+++ b/fuzzers/eclipser/fuzzer.py
@@ -66,7 +66,9 @@ def fuzz(input_corpus, output_corpus, target_binary):
         '-f',
         'foo',
         '--maxfilelen',
-        str(10 * 1024 * 1024),  # Increase since default is too low (8 bytes).
+        # Default is too low (8 bytes), match experiment config at:
+        # https://github.com/SoftSec-KAIST/Eclipser-Artifact/blob/6aadf02eeadb0416bd4c5edeafc8627bc24ebc82/docker-scripts/experiment-scripts/package-exp/run_eclipser.sh#L25
+        '1048576',
     ]
     if os.listdir(input_corpus):  # Important, otherwise Eclipser crashes.
         command += ['-i', input_corpus]


Test patch:
Repository: google/fuzzbench

Issue:
[reports] Data.csv.gz don't need to contain id column
It has these columns because data.csv.gz contains data from a join query of snapshots on trials.
time_started and time_ended are from trials but they are probably not useful for the analysis people want to do so they just take up space at this point.


Discussion:
Actually we do use these, the only thing we can drop is id which is the same as trial_id

Resolving patch:
diff --git a/analysis/queries.py b/analysis/queries.py
--- a/analysis/queries.py
+++ b/analysis/queries.py
@@ -26,4 +26,8 @@ def get_experiment_data(experiment_names):
         sqlalchemy.orm.joinedload('trial')).filter(
             models.Snapshot.trial.has(
                 models.Trial.experiment.in_(experiment_names)))
-    return pd.read_sql_query(snapshots_query.statement, db_utils.engine)
+
+    # id must be loaded to do the join but get rid of it now since
+    # trial_id provides the same info.
+    data = pd.read_sql_query(snapshots_query.statement, db_utils.engine)
+    return data.drop(columns=['id'])


Test patch:
Repository: google/fuzzbench

Issue:
wpantund benchmark can initiate network connections
The wpantund benchmark can initiate networking connections to remote networks under some conditions. Attaching a testcase for that:

[1788a8b166e1fe4e854bdbb1572e2bb6.00000080.honggfuzz.cov.txt](https://github.com/google/fuzzbench/files/4391761/1788a8b166e1fe4e854bdbb1572e2bb6.00000080.honggfuzz.cov.txt)

```shell
$ strace -f ~/fuzz/wpantund/wpantund-fuzz ~/fuzz/wpantund/corpus/1788a8b166e1fe4e854bdbb1572e2bb6.00000080.honggfuzz.cov
...
openat(AT_FDCWD, "/tmp", O_RDWR|O_EXCL|O_TMPFILE, 0600) = 6
fcntl(6, F_GETFL)                       = 0x418002 (flags O_RDWR|O_LARGEFILE|O_TMPFILE)
fstat(6, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
write(6, "\235mfProxy:EnabiG:NNN\305SocketPath \""..., 127) = 127
lseek(6, 0, SEEK_SET)                   = 0
read(6, "\235mfProxy:EnabiG:NNN\305SocketPath \""..., 4096) = 127
read(6, "", 4096)                       = 0
close(6)                                = 0
brk(0xb518000)                          = 0xb518000
socket(AF_INET6, SOCK_STREAM, IPPROTO_IP) = 6
connect(6, {sa_family=AF_INET6, sin6_port=htons(4951), inet_pton(AF_INET6, "::ffff:0.0.3.122", &sin6_addr), sin6_flowinfo=htonl(0), sin6_scope_id=0}, 28^C) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
..
```

I know nothing about wpantund internals, so maybe this can be fixed (if you want to fix it) in the fuzzer harness code (maybe some API option needs to be set).

If not on API level, a quick hack could be to add something like

```C
unshare(CLONE_NEWUSER|CLONE_NEWNET);
```

at the beginning of the file.

I'm not sure if we want to leave current behavior as-is. On one hand it might be a useful case to test fuzzing timeouts, on the other, benchmarks initiating connections over non-loopback sound somehow wrong (and they also add unnecessary timeouts).


Discussion:
Thanks for noticing this. We could try one of the solutions you propose. Or we could maybe find another benchmark. We can get pretty much anything from OSS-Fuzz, but new benchmarks may have their own problems.

Resolving patch:
diff --git a/.github/workflows/build.py b/.github/workflows/build.py
--- a/.github/workflows/build.py
+++ b/.github/workflows/build.py
@@ -42,7 +42,6 @@
     're2-2014-12-09',
     'vorbis-2017-12-11',
     'woff2-2016-05-06',
-    'wpantund-2018-02-27',
 ]
 
 
diff --git a/fuzzers/afl/fuzzer.py b/fuzzers/afl/fuzzer.py
--- a/fuzzers/afl/fuzzer.py
+++ b/fuzzers/afl/fuzzer.py
@@ -73,7 +73,7 @@ def run_afl_fuzz(input_corpus,
     # Spawn the afl fuzzing process.
     # FIXME: Currently AFL will exit if it encounters a crashing input in seed
     # corpus (usually timeouts). Add a way to skip/delete such inputs and
-    # re-run AFL. This currently happens with a seed in wpantund benchmark.
+    # re-run AFL.
     print('[run_fuzzer] Running target with afl-fuzz')
     command = [
         './afl-fuzz',


Test patch:
Repository: google/fuzzbench

Issue:
Should local docker run restrict cpu to 1 to match FuzzBench prod environment ?
See also
https://github.com/google/fuzzbench/issues/173#issuecomment-605283610


Discussion:
I have to look into it but I think we attempt this somewhere, it might not be possible.
Yes, I think doing this would be a good idea. Setting `--cpus=1` might be sufficient.
https://docs.docker.com/config/containers/resource_constraints/#configure-the-default-cfs-scheduler
Hmm...in production we do [--cpuset-cpus=0](https://github.com/google/fuzzbench/blob/master/experiment/scheduler.py#L251) I wonder if this isn't sufficient or wrong, because I think the same code path was taken when Honggfuzz is used in a local experiment and I think Honggfuzz still used multiple threads.
> Hmm...in production we do [--cpuset-cpus=0](https://github.com/google/fuzzbench/blob/master/experiment/scheduler.py#L251) I wonder if this isn't sufficient or wrong, because I think the same code path was taken when Honggfuzz is used in a local experiment and I think Honggfuzz still used multiple threads.

Threads are independent from the CPU usage: ie you can run multiple threads on a single CPU. (In our case multiple threads are okay as long as a fuzzer doesn't go over 100% cpu usage.)

I think the --cpuset-cpu=0 flag means "use only the first cpu of the machine". That's a no op on our single cpu production vms. For local runs I think what we want is "don't use more than one cpu", which should be the --cpus=1 flag. (Otherwise all docker runs would try to use the same cpu)
Production and local experiments now use --cpus=1 thanks to #8, so fixing this means just changing the Makefile to do the same.

Resolving patch:
diff --git a/common/fuzzer_utils.py b/common/fuzzer_utils.py
--- a/common/fuzzer_utils.py
+++ b/common/fuzzer_utils.py
@@ -20,7 +20,6 @@
 
 from common import logs
 from common import utils
-from common import yaml_utils
 
 DEFAULT_FUZZ_TARGET_NAME = 'fuzz-target'
 FUZZ_TARGET_SEARCH_STRING = b'LLVMFuzzerTestOneInput'
@@ -80,6 +79,10 @@ def validate(fuzzer):
 
 def get_fuzzer_configs(fuzzers=None):
     """Returns the list of all fuzzers."""
+    # Import it here to avoid yaml dependency in runner.
+    # pylint: disable=import-outside-toplevel
+    from common import yaml_utils
+
     fuzzers_dir = os.path.join(utils.ROOT_DIR, 'fuzzers')
     fuzzer_configs = []
     for fuzzer in os.listdir(fuzzers_dir):


Test patch:
Repository: google/fuzzbench

Issue:
make debug/test-run/run don't respect FORCE_LOCAL and FORCE_NOT_LOCAL
This breaks when someone needs to use FORCE_LOCAL (see https://github.com/google/fuzzbench/issues/82).
I think the quick-and-dirty solution here is to always pass FORCE_LOCAL to these docker runs since they are never run in production. The real fix is making this unnecessary.


Discussion:
@tanq16 discovered this.

Resolving patch:
diff --git a/docker/generate_makefile.py b/docker/generate_makefile.py
--- a/docker/generate_makefile.py
+++ b/docker/generate_makefile.py
@@ -184,6 +184,7 @@
     --cap-add SYS_NICE \\
     --cap-add SYS_PTRACE \\
     -e FUZZ_OUTSIDE_EXPERIMENT=1 \\
+    -e FORCE_LOCAL=1 \\
     -e TRIAL_ID=1 \\
     -e FUZZER={fuzzer} \\
     -e BENCHMARK={benchmark} \\
@@ -195,6 +196,7 @@
     --cap-add SYS_NICE \\
     --cap-add SYS_PTRACE \\
     -e FUZZ_OUTSIDE_EXPERIMENT=1 \\
+    -e FORCE_LOCAL=1 \\
     -e TRIAL_ID=1 \\
     -e FUZZER={fuzzer} \\
     -e BENCHMARK={benchmark} \\
@@ -209,6 +211,7 @@
     --cap-add SYS_NICE \\
     --cap-add SYS_PTRACE \\
     -e FUZZ_OUTSIDE_EXPERIMENT=1 \\
+    -e FORCE_LOCAL=1 \\
     -e TRIAL_ID=1 \\
     -e FUZZER={fuzzer} \\
     -e BENCHMARK={benchmark} \\


Test patch:
Repository: google/fuzzbench

Issue:
Environment variable OUT not set for curl benchmark
The `curl_curl_fuzzer_http` benchmark container does not set the `OUT` environment variable.

Thanks
Tanishq


Discussion:

Resolving patch:
diff --git a/fuzzers/manul/fuzzer.py b/fuzzers/manul/fuzzer.py
new file mode 100644
--- /dev/null
+++ b/fuzzers/manul/fuzzer.py
@@ -0,0 +1,47 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Manul Integration"""
+import os
+import subprocess
+import shutil
+from fuzzers import utils
+from fuzzers.afl import fuzzer as afl_fuzzer
+
+
+def build():
+    """Build benchmark and copy fuzzer to $OUT."""
+    afl_fuzzer.prepare_build_environment()
+    utils.build_benchmark()
+    # Move manul base to /out.
+    shutil.move('/manul', os.environ['OUT'])
+
+
+def fuzz(input_corpus, output_corpus, target_binary):
+    """Run fuzzer.
+
+    Arguments:
+      input_corpus: Directory containing the initial seed corpus for
+                    the benchmark.
+      output_corpus: Output directory to place the newly generated corpus
+                     from fuzzer run.
+      target_binary: Absolute path to the fuzz target binary.
+    """
+    afl_fuzzer.prepare_fuzz_environment(input_corpus)
+    # Run fuzzer on the benchmark.
+    manul_directory = os.path.join(os.environ['OUT'], 'manul')
+    command = ([
+        'python3', 'manul.py', '-i', input_corpus, '-o', output_corpus, '-c',
+        os.path.join(manul_directory, 'manul_lin.config'), target_binary + ' @@'
+    ])
+    subprocess.check_call(command, cwd=manul_directory)


Test patch:
Repository: google/fuzzbench

Issue:
sql query optimization
To prevent https://github.com/google/fuzzbench/pull/444#discussion_r441080002, we later change the list creation to set creation in `set_up_coverage_binaries` for `benchmarks` by substituting `[ ]` to `{ }`.
```
def set_up_coverage_binaries(pool, experiment):
    """Set up coverage binaries for all benchmarks in |experiment|."""
    benchmarks = [
        trial.benchmark for trial in db_utils.query(models.Trial).distinct(
            models.Trial.benchmark).filter(
                models.Trial.experiment == experiment)
    ]
```

This may also be fixed by https://github.com/google/fuzzbench/issues/448#issuecomment-645507409.


Discussion:

Resolving patch:
diff --git a/experiment/measurer.py b/experiment/measurer.py
--- a/experiment/measurer.py
+++ b/experiment/measurer.py
@@ -570,11 +570,12 @@ def measure_snapshot_coverage(fuzzer: str, benchmark: str, trial_num: int,
 def set_up_coverage_binaries(pool, experiment):
     """Set up coverage binaries for all benchmarks in |experiment|."""
     # Use set comprehension to select distinct benchmarks.
-    benchmarks = {
-        trial.benchmark for trial in db_utils.query(models.Trial).distinct(
-            models.Trial.benchmark).filter(
-                models.Trial.experiment == experiment)
-    }
+    benchmarks = [
+        benchmark_tuple[0]
+        for benchmark_tuple in db_utils.query(models.Trial.benchmark).distinct(
+        ).filter(models.Trial.experiment == experiment)
+    ]
+
     coverage_binaries_dir = build_utils.get_coverage_binaries_dir()
     filesystem.create_directory(coverage_binaries_dir)
     pool.map(set_up_coverage_binary, benchmarks)


Test patch: