internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: openstates/openstates-scrapers

Issue:
UT failing since at least 2019-06-28
UT has been failing since 2019-06-28

Based on automated runs it appears that UT has not run successfully in 2 days (2019-06-28).


```
  loaded Open States pupa settings...
ut (scrape, import)
  bills: {}
08:00:53 CRITICAL pupa: no sessions from Utah.get_session_list()
```

Visit http://bobsled.openstates.org for more info.



Discussion:
Looks like they've made some significant markup changes. I'll try to tackle this week.

Resolving patch:
diff --git a/openstates/ut/__init__.py b/openstates/ut/__init__.py
--- a/openstates/ut/__init__.py
+++ b/openstates/ut/__init__.py
@@ -241,6 +241,6 @@ def get_organizations(self):
     def get_session_list(self):
         sessions = url_xpath(
                 'http://le.utah.gov/Documents/bills.htm',
-                '//p/a[contains(@href, "session")]/text()'
+                '//ul[contains(@class,"bills-alternate")]/li/a[contains(@href, "BillList")]/text()'
                 )
         return [re.sub(r'\s+', ' ', session.strip()) for session in sessions]
diff --git a/openstates/ut/bills.py b/openstates/ut/bills.py
--- a/openstates/ut/bills.py
+++ b/openstates/ut/bills.py
@@ -44,7 +44,7 @@ def scrape(self, session=None, chamber=None):
             session_search_text += "GS"
 
         sessions = sessions.xpath(
-            '//p/a[contains(@href, "{}")]'.format(session_search_text))
+            '//li/a[contains(@href, "{}")]'.format(session_search_text))
 
         session_url = ''
         S = [i for i, _ in enumerate(self.jurisdiction.legislative_sessions)


Test patch:
Repository: openstates/openstates-scrapers

Issue:
NH: people scraper broken
State: NH  (be sure to include in ticket title)

The NH people scraper is broken, because 
 http://www.gencourt.state.nh.us/downloads/Members.txt
no longer exists.  Instead, there is a 
 http://www.gencourt.state.nh.us/downloads/Members.csv

I'll own this.


Discussion:

Resolving patch:
diff --git a/openstates/nh/people.py b/openstates/nh/people.py
--- a/openstates/nh/people.py
+++ b/openstates/nh/people.py
@@ -1,11 +1,12 @@
 import re
-
+import csv
 from pupa.scrape import Person, Scraper
 from openstates.utils import LXMLMixin
+import requests
 
 
 class NHPersonScraper(Scraper, LXMLMixin):
-    members_url = 'http://www.gencourt.state.nh.us/downloads/Members.txt'
+    members_url = 'http://www.gencourt.state.nh.us/downloads/Members.csv'
     lookup_url = 'http://www.gencourt.state.nh.us/house/members/memberlookup.aspx'
     house_profile_url = 'http://www.gencourt.state.nh.us/house/members/member.aspx?member={}'
     senate_profile_url = 'http://www.gencourt.state.nh.us/Senate/members/webpages/district{}.aspx'
@@ -59,17 +60,6 @@ def _parse_person(self, row, chamber, seat_map):
             self.warning('Skipping {}, district is set to 0'.format(full_name))
             return
 
-        # Temporary fix for Kari Lerner
-        if district == 'Rockingham 0' and last_name == 'Lerner':
-            district = 'Rockingham 4'
-
-        # Temporary fix for Casey Conley
-        if last_name == 'Conley':
-            if district == '13':
-                district = 'Strafford 13'
-            elif district == 'Strafford 13':
-                self.info('"Temporary fix for Casey Conley" can be removed')
-
         person = Person(primary_org=chamber,
                         district=district,
                         name=full_name,
@@ -122,12 +112,13 @@ def _parse_person(self, row, chamber, seat_map):
         return person
 
     def _parse_members_txt(self):
-        lines = self.get(self.members_url).text.splitlines()
+        response = requests.get(self.members_url)
+        lines = csv.reader(response.text.strip().split('\n'), delimiter=',')
 
-        header = lines[0].split('\t')
+        header = next(lines)
 
-        for line in lines[1:]:
-            yield dict(zip(header, line.split('\t')))
+        for line in lines:
+            yield dict(zip(header, line))
 
     def _parse_seat_map(self):
         """Get mapping between seat numbers and legislator identifiers."""


Test patch:
Repository: openstates/openstates-scrapers

Issue:
NM-scrape failing since at least 2020-01-27
NM-scrape has failed 5 times since 2020-01-27

Logs:
```
Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/**BACKUPS/PGUSER**-vRcYrsYN-py3.7/bin/pupa", line 11, in <module>
    load_entry_point('pupa==0.10.0', 'console_scripts', 'pupa')()
  File "/root/.cache/pypoetry/virtualenvs/**BACKUPS/PGUSER**-vRcYrsYN-py3.7/lib/python3.7/site-packages/pupa/cli/__main__.py", line 68, in main
    subcommands[args.subcommand].handle(args, other)
  File "/root/.cache/pypoetry/virtualenvs/**BACKUPS/PGUSER**-vRcYrsYN-py3.7/lib/python3.7/site-packages/pupa/cli/commands/update.py", line 278, in handle
    return self.do_handle(args, other, juris)
  File "/root/.cache/pypoetry/virtualenvs/**BACKUPS/PGUSER**-vRcYrsYN-py3.7/lib/python3.7/site-packages/pupa/cli/commands/update.py", line 327, in do_handle
    report['scrape'] = self.do_scrape(juris, args, scrapers)
  File "/root/.cache/pypoetry/virtualenvs/**BACKUPS/PGUSER**-vRcYrsYN-py3.7/lib/python3.7/site-packages/pupa/cli/commands/update.py", line 175, in do_scrape
    report[scraper_name] = scraper.do_scrape(**scrape_args)
  File "/root/.cache/pypoetry/virtualenvs/**BACKUPS/PGUSER**-vRcYrsYN-py3.7/lib/python3.7/site-packages/pupa/scrape/base.py", line 112, in do_scrape
    for obj in self.scrape(**kwargs) or []:
  File "/opt/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/nm/bills.py", line 74, in scrape
    yield from self.scrape_chamber(chamber, session)
  File "/opt/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/nm/bills.py", line 171, in scrape_chamber
    self.scrape_actions(chamber_letter, bills)
  File "/opt/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/nm/bills.py", line 423, in scrape_actions
    action_name, action_type = action_map[action_code]
KeyError: '7651'
```
        


Discussion:
working on this.

Resolving patch:
diff --git a/openstates/nm/bills.py b/openstates/nm/bills.py
--- a/openstates/nm/bills.py
+++ b/openstates/nm/bills.py
@@ -328,6 +328,7 @@ def scrape_actions(self, chamber_letter, bills):
             "7645": ("motion to reconsider adopted", None),
             "7649": ("printed", None),
             "7650": ("not printed %s", None),
+            "7651": ("not printed, referred to committee", "referral-committee"),
             "7652": ("not printed, not referred to committee, tabled", None),
             "7654": ("referred to %s", "referral-committee"),
             "7656": ("referred to Finance committee", "referral-committee"),


Test patch:
Repository: openstates/openstates-scrapers

Issue:
KY-scrape failing since at least 2020-01-30
KY-scrape has failed 5 times since 2020-01-30

Logs:
```
    subcommands[args.subcommand].handle(args, other)
  File "/root/.cache/pypoetry/virtualenvs/**BACKUPS/PGUSER**-vRcYrsYN-py3.7/lib/python3.7/site-packages/pupa/cli/commands/update.py", line 270, in handle
    juris, module = self.get_jurisdiction(args.module)
  File "/root/.cache/pypoetry/virtualenvs/**BACKUPS/PGUSER**-vRcYrsYN-py3.7/lib/python3.7/site-packages/pupa/cli/commands/update.py", line 142, in get_jurisdiction
    module = importlib.import_module(module_name)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/ky/__init__.py", line 10, in <module>
    from .bills import KYBillScraper
  File "/opt/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/ky/bills.py", line 22, in <module>
    class KYBillScraper(Scraper, LXMLMixin):
  File "/opt/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/**BACKUPS/PGUSER**/ky/bills.py", line 36, in KYBillScraper
    ("1st reading", "reading-1"),
TypeError: 'tuple' object is not callable
```
        


Discussion:

Resolving patch:
diff --git a/openstates/ky/bills.py b/openstates/ky/bills.py
--- a/openstates/ky/bills.py
+++ b/openstates/ky/bills.py
@@ -32,7 +32,7 @@ class KYBillScraper(Scraper, LXMLMixin):
         (" to [A-Z]", "referral-committee"),
         ("reported favorably", "committee-passage"),
         ("adopted by voice vote", "passage"),
-        ("3rd reading, passed", ["reading-3", "passage"])
+        ("3rd reading, passed", ["reading-3", "passage"]),
         ("1st reading", "reading-1"),
         ("2nd reading", "reading-2"),
         ("3rd reading", "reading-3"),


Test patch:
Repository: openstates/openstates-scrapers

Issue:
OK-scrape failing since at least 2020-04-02
OK-scrape has failed 3 times since 2020-04-02

Logs:
```
loaded Open States pupa settings...
ok (scrape, import)
  bills: {}
01:01:37 CRITICAL pupa: Session(s) 2020 First Special Session were reported by Oklahoma.get_session_list() but were not found in Oklahoma.legislative_sessions or Oklahoma.ignored_scraped_sessions.
```
        


Discussion:

Resolving patch:
diff --git a/openstates/ok/__init__.py b/openstates/ok/__init__.py
--- a/openstates/ok/__init__.py
+++ b/openstates/ok/__init__.py
@@ -89,6 +89,13 @@ class Oklahoma(State):
             "name": "2020 Regular Session",
             "start_date": "2019-02-03",
         },
+        {
+            "_scraped_name": "2020 First Special Session",
+            "identifier": "2020SS1",
+            "name": "2020 First Special Session",
+            "start_date": "2020-04-02",
+            "end_date": "2020-04-03",
+        },
     ]
     ignored_scraped_sessions = [
         "2017 Regular Session",
diff --git a/openstates/ok/bills.py b/openstates/ok/bills.py
--- a/openstates/ok/bills.py
+++ b/openstates/ok/bills.py
@@ -27,6 +27,7 @@ class OKBillScraper(Scraper):
         "2017-2018": "1800",
         "2019-2020": "1900",
         "2020": "2000",
+        "2020SS1": "201X",
     }
 
     def scrape(self, chamber=None, session=None, only_bills=None):


Test patch:
Repository: openstates/openstates-scrapers

Issue:
OH-scrape failing since at least 2020-05-25
OH-scrape has failed 5 times since 2020-05-25

Logs:
```
Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/bin/os-update", line 10, in <module>
    sys.exit(main())
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/cli/update.py", line 318, in main
    report = do_update(args, other, juris)
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/cli/update.py", line 207, in do_update
    report["import"] = do_import(juris, args)
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/cli/update.py", line 117, in do_import
    report.update(vote_event_importer.import_directory(datadir))
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/importers/base.py", line 206, in import_directory
    return self.import_data(json_stream())
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/importers/base.py", line 243, in import_data
    obj_id, what = self.import_item(data)
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/importers/base.py", line 266, in import_item
    data = self.prepare_for_db(data)
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/importers/vote_events.py", line 87, in prepare_for_db
    data["bill_id"] = self.bill_importer.resolve_json_id(bill)
  File "/root/.cache/pypoetry/virtualenvs/openstates-scrapers-vRcYrsYN-py3.7/lib/python3.7/site-packages/openstates/importers/base.py", line 195, in resolve_json_id
    raise UnresolvedIdError("cannot resolve id: {}".format(json_id))
openstates.exceptions.UnresolvedIdError: cannot resolve id: 221300c8-9e45-11ea-ba8a-02c824419979
```
        


Discussion:

Resolving patch:
diff --git a/scrapers/oh/bills.py b/scrapers/oh/bills.py
--- a/scrapers/oh/bills.py
+++ b/scrapers/oh/bills.py
@@ -176,6 +176,13 @@ def scrape(self, session=None, chambers=None):
                     )
                 )
                 data = self.get(bill_api_url).json()
+                if len(data["items"]) == 0:
+                    self.logger.warning(
+                        "Data for bill {bill_id} has empty 'items' array,"
+                        " cannot process related information".format(bill_id=bill_id.lower().replace(" ", ""))
+                    )
+                    yield bill
+                    continue
 
                 # add title if no short title
                 if not bill.title:


Test patch:
Repository: google/clusterfuzz

Issue:
linting in CI works differently than locally
I'm pretty sure it is pylint 1.9.4.
See https://github.com/google/clusterfuzz/pull/185 for a discrepancy I noticed between running pylint locally and running it in CI.

When I upgraded my local copy of pylint to 1.9.4 I was able to discover the issue showing up in CI.


Discussion:
That's really strange...

```bash
$ sudo docker run --rm -ti -v $(pwd)/clusterfuzz:/workspace gcr.io/clusterfuzz-images/ci /bin/bash
root@6bcbbf462031:/workspace# pylint --version
No config file found, using default configuration
pylint 1.9.3, 
astroid 1.6.5
Python 2.7.12 (default, Nov 12 2018, 14:36:49) 
[GCC 5.4.0 20160609]
root@6bcbbf462031:/workspace# pylint src/appengine/handlers/upload_testcase.py
No config file found, using default configuration
************* Module src/appengine/handlers/upload_testcase.py
F:  1, 0: No module named src/appengine/handlers/upload_testcase.py (fatal)
```
My ENV version is also 1.9.3, but also cannot find this.
Actually, I may have spoke too soon about 1.9.4 being the fix.
But the part about being unable to repro locally is true. Maybe it's because we have some deps installed in CI that pylint is looking at, but not locally?
I think the issue may be PYTHONPATH.

In the CI, we have `/data/google_appengine:/workspace/src`. If I remove the app engine path, it no longer reports that issue. Maybe it's exiting early when it can't import the appengine modules.
>I think the issue may be PYTHONPATH.

Sounds correct to me. I don't actually think it has to do with using 1.9.4 (I think I thought this because of similar issues that came up when I used pylint in a fresh virtualenv)

Resolving patch:
diff --git a/src/local/butler/lint.py b/src/local/butler/lint.py
--- a/src/local/butler/lint.py
+++ b/src/local/butler/lint.py
@@ -16,11 +16,15 @@
 import os
 import sys
 
+from local.butler import appengine
 from local.butler import common
 
 
 def execute(_):
   """Lint changed code."""
+  pythonpath = os.getenv('PYTHONPATH', '')
+  os.environ['PYTHONPATH'] = appengine.find_sdk_path() + ':' + pythonpath
+
   if "GOOGLE_CLOUDBUILD" in os.environ:
     # Explicitly compare against master if we're running on the CI
     _, output = common.execute('git diff --name-only master FETCH_HEAD')


Test patch:
Repository: google/clusterfuzz

Issue:
Fuzzers page does not work well with large number of jobs
For libFuzzer, if we have a lot of existing jobs (>100) and want to add a new job and associate it. Submit button feels stuck, does not show updates, and take 1-2 min to finish. Can we show some update or something better to optimize this when only one job is updated.

@oliverchang as fyi.


Discussion:

Resolving patch:
diff --git a/src/python/fuzzing/fuzzer_selection.py b/src/python/fuzzing/fuzzer_selection.py
--- a/src/python/fuzzing/fuzzer_selection.py
+++ b/src/python/fuzzing/fuzzer_selection.py
@@ -37,20 +37,20 @@ def update_mappings_for_fuzzer(fuzzer, mappings=None):
   entities = ndb_utils.get_all_from_query(query)
   old_mappings = {}
   for entity in entities:
-    old_mappings[(entity.job, entity.platform)] = entity
+    old_mappings[entity.job] = entity
 
   new_mappings = []
   for job_name in mappings:
+    mapping = old_mappings.pop(job_name, None)
+    if mapping:
+      continue
+
     job = data_types.Job.query(data_types.Job.name == job_name).get()
     if not job:
       logs.log_error('An unknown job %s was selected for fuzzer %s.' %
                      (job_name, fuzzer.name))
       continue
 
-    mapping = old_mappings.pop((job_name, job.platform), None)
-    if mapping:
-      continue
-
     mapping = data_types.FuzzerJob()
     mapping.fuzzer = fuzzer.name
     mapping.job = job_name


Test patch:
Repository: google/clusterfuzz

Issue:
/fuzzer-weights cron is failing on OSS-Fuzz
@mbarbella-chromium I can get to this a bit later, but if you're interested to check this out, I'd be grateful


Discussion:
I can take a look.
This is a bit of a false positive. We originally included an error log to deal with an issue we saw shortly after deploying fuzzer weights which is only of any use if it spikes heavily. It hasn't been an issue since, so I'm going to remove the log. The good news is everything seems to be working correctly with weights regardless.

Keeping this open for now to track the work for removing the error log.
Thanks a lot for investigating!

Resolving patch:
diff --git a/src/appengine/handlers/cron/fuzzer_weights.py b/src/appengine/handlers/cron/fuzzer_weights.py
--- a/src/appengine/handlers/cron/fuzzer_weights.py
+++ b/src/appengine/handlers/cron/fuzzer_weights.py
@@ -251,8 +251,7 @@ def update_weight_for_target(fuzz_target_name, job, match):
   target_job = data_handler.get_fuzz_target_job(fuzz_target_name, job)
 
   if not target_job:
-    logs.log_error('FuzzTargetJob for target %s and job %s does not exist.' %
-                   (fuzz_target_name, job))
+    # Bail out. This is expected if any fuzzer/job combinations become outdated.
     return
 
   weight = match.new_weight


Test patch:
Repository: google/clusterfuzz

Issue:
fuzzer-common-data-bundles is not accessible
```
fs.js:145
    throw err;
    ^

Error: ENOENT: no such file or directory, scandir '/mnt/scratch0/bots/oss-fuzz-linux-zone1-host-high-end-b1b2-3/clusterfuzz/bot/inputs/fuzzer-common-data-bundles/WebKit/JSTests'
```

same error for other data bundles, e.g. spidermonkey


Discussion:
Oh, seems to affect internal too :/

Might be my change. Will take a look today. 

Resolving patch:
diff --git a/src/python/base/persistent_cache.py b/src/python/base/persistent_cache.py
--- a/src/python/base/persistent_cache.py
+++ b/src/python/base/persistent_cache.py
@@ -36,7 +36,7 @@ def initialize():
     os.makedirs(cache_directory_path)
 
 
-def clear_values():
+def clear_values(clear_all=False):
   """Remove all values."""
   cache_directory_path = environment.get_value('CACHE_DIR')
   if not os.path.exists(cache_directory_path):
@@ -44,12 +44,12 @@ def clear_values():
 
   for root_directory, _, filenames in os.walk(cache_directory_path):
     for filename in filenames:
-      if filename.endswith(PERSIST_FILE_EXTENSION):
+      if filename.endswith(PERSIST_FILE_EXTENSION) and not clear_all:
         continue
 
       file_path = os.path.join(root_directory, filename)
       persist_file_path = file_path + PERSIST_FILE_EXTENSION
-      if os.path.exists(persist_file_path):
+      if os.path.exists(persist_file_path) and not clear_all:
         continue
 
       os.remove(file_path)
diff --git a/src/python/system/shell.py b/src/python/system/shell.py
--- a/src/python/system/shell.py
+++ b/src/python/system/shell.py
@@ -105,7 +105,7 @@ def clear_data_directories():
   clear_testcase_directories()
   clear_mutator_plugins_directory()
 
-  persistent_cache.clear_values()
+  persistent_cache.clear_values(clear_all=True)
 
 
 def clear_data_directories_on_low_disk_space():


Test patch:
Repository: google/clusterfuzz

Issue:
Show time executed / cpu hours instead of number of runs in fuzzer stats.



Discussion:

Resolving patch:
diff --git a/src/local/butler/scripts/setup.py b/src/local/butler/scripts/setup.py
--- a/src/local/butler/scripts/setup.py
+++ b/src/local/butler/scripts/setup.py
@@ -135,7 +135,7 @@ def __init__(self):
 timeout_percent: "Percent of fuzzing runs that had testcases timeout (should be 0)"
 startup_crash_percent: "Percent of fuzzing runs that crashed on startup (should be 0)"
 avg_unwanted_log_lines: "Average number of unwanted log lines in fuzzing runs (should be 0)"
-runs_count: "Number of libFuzzer fuzzing runs or sessions (default is ~1 hr)"
+total_fuzzing_time_hrs: "Total time in hours for which the fuzzer(s) ran. Will be lower if fuzzer hits a crash frequently."
 logs: "Link to fuzzing logs"
 corpus_backup: "Backup copy of the minimized corpus generated based on code coverage"'''
 
@@ -155,7 +155,7 @@ def __init__(self):
 avg(t.timeout_count*100) as timeout_percent,
 avg(t.startup_crash_count*100) as startup_crash_percent,
 avg(t.log_lines_unwanted) as avg_unwanted_log_lines,
-COUNT(t.bad_instrumentation) as runs_count,
+sum(t.actual_duration/3600.0) as total_fuzzing_time_hrs,
 _FUZZER_RUN_LOGS as logs,
 _CORPUS_BACKUP as corpus_backup,"""
 
@@ -182,7 +182,7 @@ def __init__(self):
 timeout_percent: "Percent of fuzzing runs that had testcases timeout (should be 0)"
 startup_crash_percent: "Percent of fuzzing runs that crashed on startup (should be 0)"
 avg_unwanted_log_lines: "Average number of unwanted log lines in fuzzing runs (should be 0)"
-runs_count: "Number of libFuzzer fuzzing runs or sessions (default is ~1 hr)"
+total_fuzzing_time_hrs: "Total time in hours for which the fuzzer(s) ran. Will be lower if fuzzer hits a crash frequently."
 logs: "Link to fuzzing logs"
 corpus_backup: "Backup copy of the minimized corpus generated based on code coverage"'''
 
@@ -197,7 +197,7 @@ def __init__(self):
 avg(t.timeout_count*100) as timeout_percent,
 avg(t.startup_crash_count*100) as startup_crash_percent,
 avg(t.log_lines_unwanted) as avg_unwanted_log_lines,
-COUNT(t.bad_instrumentation) as runs_count,
+sum(t.actual_duration/3600.0) as total_fuzzing_time_hrs,
 _FUZZER_RUN_LOGS as logs,
 _CORPUS_BACKUP as corpus_backup,"""
 


Test patch:
Repository: google/clusterfuzz

Issue:
Crashes found from new engine impl should not be tested with old engine impl
This is probably the cause of minimize task exceptions on OSS-Fuzz since %TESTCASE% is not in minimized_arguments.

```
IOError: [Errno 2] No such file or directory: '-timeout=25'
at copyfile (/usr/lib/python2.7/shutil.py:82)
at copy (/usr/lib/python2.7/shutil.py:119)
at _chroot_testcase (/mnt/scratch0/clusterfuzz/src/python/bot/fuzzers/engine_common.py:492)
at __enter__ (/usr/lib/python2.7/contextlib.py:17)
at run_single_testcase (/mnt/scratch0/clusterfuzz/src/python/bot/fuzzers/libfuzzer.py:661)
at load_testcase_if_exists (/mnt/scratch0/clusterfuzz/src/python/bot/fuzzers/libFuzzer/launcher.py:318)
at main (/mnt/scratch0/clusterfuzz/src/python/bot/fuzzers/libFuzzer/launcher.py:625)
at <module> (/mnt/scratch0/clusterfuzz/src/python/bot/fuzzers/libFuzzer/launcher.py:855)
```


Discussion:

Resolving patch:
diff --git a/src/python/bot/tasks/setup.py b/src/python/bot/tasks/setup.py
--- a/src/python/bot/tasks/setup.py
+++ b/src/python/bot/tasks/setup.py
@@ -165,6 +165,12 @@ def prepare_environment_for_testcase(testcase):
   if app_args:
     environment.set_value('APP_ARGS', app_args)
 
+  # TODO(ochang): Remove once migration is completed. Testcases generated by the
+  # new pipeline aren't compatible with the legacy launchers.
+  if (environment.is_engine_fuzzer_job(testcase.job_type) and
+      '%TESTCASE%' not in testcase.minimized_arguments):
+    environment.set_value('USE_NEW_ENGINE_IMPL', True)
+
 
 def setup_testcase(testcase, fuzzer_override=None):
   """Sets up the testcase and needed dependencies like fuzzer,


Test patch: