internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: google/mobly

Issue:
When failure occurs in setup class "begin_time" for skipped tests is null
End time is not null.

Can we either make them equal to each other (i.e. skipped test takes 0 time) or both null?


Discussion:
How about making them both null since the test never began or ended?

Resolving patch:
diff --git a/mobly/records.py b/mobly/records.py
--- a/mobly/records.py
+++ b/mobly/records.py
@@ -89,7 +89,8 @@ def _test_end(self, result, e):
                 be any exception instance or of any subclass of
                 mobly.signals.TestSignal.
         """
-        self.end_time = utils.get_current_epoch_time()
+        if self.begin_time is not None:
+            self.end_time = utils.get_current_epoch_time()
         self.result = result
         if self.extra_errors:
             self.result = TestResultEnums.TEST_RESULT_ERROR


Test patch:
diff --git a/tests/mobly/base_test_test.py b/tests/mobly/base_test_test.py
--- a/tests/mobly/base_test_test.py
+++ b/tests/mobly/base_test_test.py
@@ -189,9 +189,11 @@ def on_fail(self, test_name, begin_time):
         bt_cls = MockBaseTest(self.mock_test_cls_configs)
         bt_cls.run()
         actual_record = bt_cls.results.error[0]
+        skipped_record = bt_cls.results.skipped[0]
+        self.assertIsNone(skipped_record.begin_time)
+        self.assertIsNone(skipped_record.end_time)
         utils.validate_test_result(bt_cls.results)
         self.assertEqual(actual_record.test_name, "setup_class")
-
         self.assertEqual(actual_record.details, MSG_EXPECTED_EXCEPTION)
         self.assertIsNone(actual_record.extras)
         expected_summary = ("Error 1, Executed 0, Failed 0, Passed 0, "
@@ -869,6 +871,8 @@ def test_func(self):
         bt_cls = MockBaseTest(self.mock_test_cls_configs)
         bt_cls.run(test_names=["test_func"])
         actual_record = bt_cls.results.skipped[0]
+        self.assertIsNotNone(actual_record.begin_time)
+        self.assertIsNotNone(actual_record.end_time)
         self.assertEqual(actual_record.test_name, "test_func")
         self.assertEqual(actual_record.details, MSG_EXPECTED_EXCEPTION)
         self.assertEqual(actual_record.extras, MOCK_EXTRA)
@@ -884,6 +888,8 @@ def test_func(self):
         bt_cls = MockBaseTest(self.mock_test_cls_configs)
         bt_cls.run(test_names=["test_func"])
         actual_record = bt_cls.results.skipped[0]
+        self.assertIsNotNone(actual_record.begin_time)
+        self.assertIsNotNone(actual_record.end_time)
         self.assertEqual(actual_record.test_name, "test_func")
         self.assertEqual(actual_record.details, MSG_EXPECTED_EXCEPTION)
         self.assertEqual(actual_record.extras, MOCK_EXTRA)
Repository: google/mobly

Issue:
Make it possible to add more info to controller infos
Use case: when testing an Android app, it would be nice to have the app's version be in the controller info of AndroidDevice as well.


Discussion:

Resolving patch:
diff --git a/mobly/controllers/android_device.py b/mobly/controllers/android_device.py
--- a/mobly/controllers/android_device.py
+++ b/mobly/controllers/android_device.py
@@ -351,9 +351,10 @@ def take_bug_reports(ads, test_name, begin_time):
     Args:
         ads: A list of AndroidDevice instances.
         test_name: Name of the test method that triggered this bug report.
-        begin_time: Logline format timestamp taken when the test started.
+        begin_time: timestamp taken when the test started, can be either
+            string or int.
     """
-    begin_time = mobly_logger.normalize_log_line_timestamp(begin_time)
+    begin_time = mobly_logger.normalize_log_line_timestamp(str(begin_time))
 
     def take_br(test_name, begin_time, ad):
         ad.take_bug_report(test_name, begin_time)
@@ -849,7 +850,7 @@ def take_bug_report(self, test_name, begin_time):
 
         Args:
             test_name: Name of the test method that triggered this bug report.
-            begin_time: Logline format timestamp taken when the test started.
+            begin_time: Timestamp of when the test started.
         """
         new_br = True
         try:


Test patch:
Repository: google/mobly

Issue:
Improper usage of `deepcopy` in `record`
`copy.deepcopy` does not always work with user-defined types, as described [here](https://stackoverflow.com/questions/10618956/copy-deepcopy-raises-typeerror-on-objects-with-self-defined-new-method).

So if a user-defined object is put into `record`, attempts to `deepcopy` the object would crash.


Discussion:
Narrowed it down to the exception object used in `ExceptionRecord`.
So `ExceptionRecord` needs to have custom logic for and `__deepcopy__`

Resolving patch:
diff --git a/mobly/records.py b/mobly/records.py
--- a/mobly/records.py
+++ b/mobly/records.py
@@ -172,6 +172,25 @@ def to_dict(self):
         result[TestResultEnums.RECORD_EXTRAS] = copy.deepcopy(self.extras)
         return result
 
+    def __deepcopy__(self, memo):
+        """Overrides deepcopy for the class.
+
+        If the exception object has a constructor that takes extra args, deep
+        copy won't work. So we need to have a custom logic for deepcopy.
+        """
+        try:
+            exception = copy.deepcopy(self.exception)
+        except TypeError:
+            # If the exception object cannot be copied, use the original
+            # exception object.
+            exception = self.exception
+        result = ExceptionRecord(exception, self.position)
+        result.stacktrace = self.stacktrace
+        result.details = self.details
+        result.extras = self.extras
+        result.position = self.position
+        return result
+
 
 class TestResultRecord(object):
     """A record that holds the information of a single test.


Test patch:
diff --git a/tests/mobly/records_test.py b/tests/mobly/records_test.py
--- a/tests/mobly/records_test.py
+++ b/tests/mobly/records_test.py
@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import copy
 import os
 import shutil
 import tempfile
@@ -25,6 +26,16 @@
 from tests.lib import utils
 
 
+class RecordTestError(Exception):
+    """Error class with constructors that take extra args.
+
+    Used for ExceptionRecord tests.
+    """
+
+    def __init__(self, something):
+        self._something = something
+
+
 class RecordsTest(unittest.TestCase):
     """This test class tests the implementation of classes in mobly.records.
     """
@@ -323,6 +334,16 @@ def test_summary_write_dump(self):
             self.assertEqual(content['Type'],
                              records.TestSummaryEntryType.RECORD.value)
 
+    def test_exception_record_deepcopy(self):
+        """Makes sure ExceptionRecord wrapper handles deep copy properly."""
+        try:
+            raise RecordTestError('Oh ha!')
+        except RecordTestError as e:
+            er = records.ExceptionRecord(e)
+        new_er = copy.deepcopy(er)
+        self.assertIsNot(er, new_er)
+        self.assertDictEqual(er.to_dict(), new_er.to_dict())
+
 
 if __name__ == "__main__":
     unittest.main()
Repository: google/mobly

Issue:
If `SnippetClient.connect` throws, adb port forwarding is not cleaned.
This leaves adb forwarded port there forever and requires manual clean up.


Discussion:

Resolving patch:
diff --git a/mobly/controllers/android_device.py b/mobly/controllers/android_device.py
--- a/mobly/controllers/android_device.py
+++ b/mobly/controllers/android_device.py
@@ -725,7 +725,17 @@ def load_snippet(self, name, package):
                     ' "%s".' % (package, client_name))
         client = snippet_client.SnippetClient(
             package=package, adb_proxy=self.adb, log=self.log)
-        client.start_app_and_connect()
+        try:
+            client.start_app_and_connect()
+        except Exception as e:
+            # If errors happen, make sure we clean up before raising.
+            try:
+                client.stop_app()
+            except:
+                self.log.exception(
+                    'Failed to stop app after failure to launch.')
+            # Raise the error from start app failure.
+            raise e
         self._snippet_clients[name] = client
         setattr(self, name, client)
 


Test patch:
Repository: google/mobly

Issue:
`AndroidDevice` adb logcat file name gets truncated
If device model name includes space, the output file name of adb logcat is incorrect.
For example, for model "aqua power hd-4g", we expect `adblog,aqua power hd-4g,usb:3-5.2.3.txt`, but we actually get `adblog,aqua`.


Discussion:

Resolving patch:
diff --git a/mobly/controllers/android_device.py b/mobly/controllers/android_device.py
--- a/mobly/controllers/android_device.py
+++ b/mobly/controllers/android_device.py
@@ -836,10 +836,8 @@ def start_adb_logcat(self, clear_log=True):
             extra_params = self.adb_logcat_param
         except AttributeError:
             extra_params = ''
-        cmd = '"%s" -s %s logcat -v threadtime %s >> %s' % (adb.ADB,
-                                                            self.serial,
-                                                            extra_params,
-                                                            logcat_file_path)
+        cmd = '"%s" -s %s logcat -v threadtime %s >> "%s"' % (
+            adb.ADB, self.serial, extra_params, logcat_file_path)
         process = utils.start_standing_subprocess(cmd, shell=True)
         self._adb_logcat_process = process
         self.adb_logcat_file_path = logcat_file_path


Test patch:
diff --git a/tests/mobly/controllers/android_device_test.py b/tests/mobly/controllers/android_device_test.py
--- a/tests/mobly/controllers/android_device_test.py
+++ b/tests/mobly/controllers/android_device_test.py
@@ -326,7 +326,7 @@ def test_AndroidDevice_take_logcat(self, stop_proc_mock, start_proc_mock,
         creat_dir_mock.assert_called_with(os.path.dirname(expected_log_path))
         adb_cmd = '"adb" -s %s logcat -v threadtime  >> %s'
         start_proc_mock.assert_called_with(
-            adb_cmd % (ad.serial, expected_log_path), shell=True)
+            adb_cmd % (ad.serial, '"%s"' % expected_log_path), shell=True)
         self.assertEqual(ad.adb_logcat_file_path, expected_log_path)
         expected_msg = (
             'Logcat thread is already running, cannot start another'
@@ -373,7 +373,7 @@ def test_AndroidDevice_take_logcat_with_user_param(
         creat_dir_mock.assert_called_with(os.path.dirname(expected_log_path))
         adb_cmd = '"adb" -s %s logcat -v threadtime -b radio >> %s'
         start_proc_mock.assert_called_with(
-            adb_cmd % (ad.serial, expected_log_path), shell=True)
+            adb_cmd % (ad.serial, '"%s"' % expected_log_path), shell=True)
         self.assertEqual(ad.adb_logcat_file_path, expected_log_path)
 
     @mock.patch(
Repository: google/mobly

Issue:
`utils_test` has an unclosed file warning
This warning is shown when running unit tests in py3.

```
tests/mobly/utils_test.py::UtilsTest::test_stop_standing_subproc
  /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py:601: ResourceWarning: unclosed file <_io.BufferedReader name=14>
    testMethod()
  /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py:601: ResourceWarning: unclosed file <_io.BufferedReader name=16>
    testMethod()
```


Discussion:
Need to turn on warnings of pytest to see

Resolving patch:
diff --git a/mobly/utils.py b/mobly/utils.py
--- a/mobly/utils.py
+++ b/mobly/utils.py
@@ -357,6 +357,11 @@ def stop_standing_subprocess(proc, kill_signal=signal.SIGTERM):
         logging.exception('Failed to kill standing subprocess %d', pid)
     if failed:
         raise Error('Failed to kill standing subprocesses: %s' % failed)
+    # Call wait and close pipes on the original Python object so we don't get
+    # runtime warnings.
+    proc.stdout.close()
+    proc.stderr.close()
+    proc.wait()
     logging.debug('Stopped standing subprocess %d', pid)
 
 


Test patch:
diff --git a/tests/mobly/utils_test.py b/tests/mobly/utils_test.py
--- a/tests/mobly/utils_test.py
+++ b/tests/mobly/utils_test.py
@@ -35,15 +35,23 @@ def setUp(self):
         self.sleep_cmd = 'timeout' if system == 'Windows' else 'sleep'
 
     def test_start_standing_subproc(self):
-        p = utils.start_standing_subprocess([self.sleep_cmd, '1'])
-        p1 = psutil.Process(p.pid)
-        self.assertTrue(p1.is_running())
+        try:
+            p = utils.start_standing_subprocess([self.sleep_cmd, '0.1'])
+            p1 = psutil.Process(p.pid)
+            self.assertTrue(p1.is_running())
+        finally:
+            p.stdout.close()
+            p.stderr.close()
+            p.wait()
 
     def test_stop_standing_subproc(self):
         p = utils.start_standing_subprocess([self.sleep_cmd, '4'])
         p1 = psutil.Process(p.pid)
+        start_time = time.time()
         utils.stop_standing_subprocess(p)
         self.assertFalse(p1.is_running())
+        stop_time = time.time()
+        self.assertTrue(stop_time - start_time < 0.1)
 
     @mock.patch(
         'mobly.controllers.android_device_lib.adb.list_occupied_adb_ports')
Repository: google/mobly

Issue:
AdbLogExcerpts filename is invalid on Windows
Windows filenames should not have colons in them:
https://support.microsoft.com/en-us/help/905231/information-about-the-characters-that-you-cannot-use-in-site-names-fol


Discussion:
E       IOError: [Errno 22] Invalid argument: 'c:\\users\\winter~1\\appdata\\local\\temp\\tmpuiwhtu\\SampleTestBed\\04-06-2018_12-16-57-490\\AndroidDevice1\\AdbLogExcerpts\\some_test,02-29 14:02:20.123,fakemodel,1.txt'

Resolving patch:
diff --git a/mobly/controllers/android_device.py b/mobly/controllers/android_device.py
--- a/mobly/controllers/android_device.py
+++ b/mobly/controllers/android_device.py
@@ -436,9 +436,8 @@ def __init__(self, serial=''):
         self._log_path = os.path.join(self._log_path_base,
                                       'AndroidDevice%s' % self._serial)
         self._debug_tag = self._serial
-        self.log = AndroidDeviceLoggerAdapter(logging.getLogger(), {
-            'tag': self.debug_tag
-        })
+        self.log = AndroidDeviceLoggerAdapter(logging.getLogger(),
+                                              {'tag': self.debug_tag})
         self.sl4a = None
         self.ed = None
         self._adb_logcat_process = None
@@ -937,6 +936,7 @@ def cat_adb_log(self, tag, begin_time):
         f_name = os.path.basename(self.adb_logcat_file_path)
         out_name = f_name.replace('adblog,', '').replace('.txt', '')
         out_name = ',%s,%s.txt' % (begin_time, out_name)
+        out_name = out_name.replace(':', '-')
         tag_len = utils.MAX_FILENAME_LEN - len(out_name)
         tag = tag[:tag_len]
         out_name = tag + out_name


Test patch:
diff --git a/tests/mobly/controllers/android_device_test.py b/tests/mobly/controllers/android_device_test.py
--- a/tests/mobly/controllers/android_device_test.py
+++ b/tests/mobly/controllers/android_device_test.py
@@ -630,7 +630,7 @@ def test_AndroidDevice_cat_adb_log(self, mock_timestamp_getter,
         ad.cat_adb_log('some_test', MOCK_ADB_LOGCAT_BEGIN_TIME)
         cat_file_path = os.path.join(
             ad.log_path, 'AdbLogExcerpts',
-            ('some_test,02-29 14:02:20.123,%s,%s.txt') % (ad.model, ad.serial))
+            ('some_test,02-29 14-02-20.123,%s,%s.txt') % (ad.model, ad.serial))
         with open(cat_file_path, 'r') as f:
             actual_cat = f.read()
         self.assertEqual(actual_cat, ''.join(MOCK_ADB_LOGCAT_CAT_RESULT))
diff --git a/tests/mobly/logger_test.py b/tests/mobly/logger_test.py
--- a/tests/mobly/logger_test.py
+++ b/tests/mobly/logger_test.py
@@ -1,11 +1,11 @@
 # Copyright 2016 Google Inc.
-# 
+#
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
-# 
+#
 #     http://www.apache.org/licenses/LICENSE-2.0
-# 
+#
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
Repository: google/mobly

Issue:
Use ro.debuggable for is_rootable check
Since at least the Android Donut release, this is the check that adb uses to determine if root access is available: https://android.googlesource.com/platform/system/core.git/+/donut-release/adb/services.c

Additionally, there exist 'user' build type devices with root permissions.


Discussion:

Resolving patch:
diff --git a/mobly/controllers/android_device.py b/mobly/controllers/android_device.py
--- a/mobly/controllers/android_device.py
+++ b/mobly/controllers/android_device.py
@@ -436,8 +436,9 @@ def __init__(self, serial=''):
         self._log_path = os.path.join(self._log_path_base,
                                       'AndroidDevice%s' % self._serial)
         self._debug_tag = self._serial
-        self.log = AndroidDeviceLoggerAdapter(logging.getLogger(),
-                                              {'tag': self.debug_tag})
+        self.log = AndroidDeviceLoggerAdapter(logging.getLogger(), {
+            'tag': self.debug_tag
+        })
         self.sl4a = None
         self.ed = None
         self._adb_logcat_process = None
@@ -783,15 +784,7 @@ def is_adb_root(self):
 
     @property
     def is_rootable(self):
-        """If the build type is 'user', the device is not rootable.
-
-        Other possible build types are 'userdebug' and 'eng', both are rootable.
-        We are checking the last four chars of the clean stdout because the
-        stdout of the adb command could be polluted with other info like adb
-        server startup message.
-        """
-        build_type_output = self.adb.getprop('ro.build.type').lower()
-        return build_type_output[-4:] != 'user'
+        return self.adb.getprop('ro.debuggable') == '1'
 
     @property
     def model(self):


Test patch:
diff --git a/tests/mobly/controllers/android_device_test.py b/tests/mobly/controllers/android_device_test.py
--- a/tests/mobly/controllers/android_device_test.py
+++ b/tests/mobly/controllers/android_device_test.py
@@ -706,7 +706,8 @@ def test_AndroidDevice__enable_logpersist_with_logpersist(
             self, MockFastboot, MockAdbProxy):
         mock_serial = '1'
         mock_adb_proxy = MockAdbProxy.return_value
-        mock_adb_proxy.getprop.return_value = 'userdebug'
+        # Set getprop to return '1' to indicate the device is rootable.
+        mock_adb_proxy.getprop.return_value = '1'
         mock_adb_proxy.has_shell_command.side_effect = lambda command: {
             'logpersist.start': True,
             'logpersist.stop': True, }[command]
Repository: google/mobly

Issue:
Log the contents of config file at the debug level early
This helps in debugging remote user's malformed json/yaml or configs that don't adhere to schema.


Discussion:
where do we want this to go, summary yaml or regular log?
I vote DEBUG log. Preferably as the read in string (to expose any poor json/yaml formatting).
And I assume the config here is the user config file content as is?
Yessir.
Hmm, so what kind of error do we want to catch with this?
If the config format is incorrect, the logger won't even get set up so we won't even get a log file...

Resolving patch:
diff --git a/mobly/config_parser.py b/mobly/config_parser.py
--- a/mobly/config_parser.py
+++ b/mobly/config_parser.py
@@ -16,6 +16,7 @@
 
 import copy
 import io
+import pprint
 import os
 import yaml
 
@@ -189,4 +190,7 @@ def copy(self):
         return copy.deepcopy(self)
 
     def __str__(self):
-        return str(self.__dict__)
+        content = dict(self.__dict__)
+        content.pop('summary_writer')
+        content.pop('register_controller')
+        return pprint.pformat(content)


Test patch:
diff --git a/mobly/test_runner.py b/mobly/test_runner.py
--- a/mobly/test_runner.py
+++ b/mobly/test_runner.py
@@ -381,8 +381,9 @@ def _run_test_class(self, config, test_class, tests=None):
             test_class: class, test class to execute.
             tests: Optional list of test names within the class to execute.
         """
-
         with test_class(config) as test_instance:
+            logging.debug('Executing test class "%s" with config: %s',
+                          test_class.__name__, config)
             try:
                 cls_result = test_instance.run(tests)
                 self.results += cls_result
diff --git a/tests/mobly/config_parser_test.py b/tests/mobly/config_parser_test.py
--- a/tests/mobly/config_parser_test.py
+++ b/tests/mobly/config_parser_test.py
@@ -56,6 +56,11 @@ def test__load_config_file_with_unicode(self):
         config = config_parser._load_config_file(tmp_file_path)
         self.assertEqual(config['TestBeds'][0]['Name'], u'\u901a')
 
+    def test_run_config_type(self):
+        config = config_parser.TestRunConfig()
+        self.assertNotIn('summary_writer', str(config))
+        self.assertNotIn('register_controller', str(config))
+
 
 if __name__ == "__main__":
     unittest.main()
Repository: google/mobly

Issue:
Log snippet client calls at debug level
Right now there is no log for calls made to snippet client side, which makes it difficult to debug should a snippet client call hangs.

On debug level, we should log messages sent and received by the client, same way we log messages received and sent by snippet server on devices.


Discussion:

Resolving patch:
diff --git a/mobly/controllers/android_device_lib/jsonrpc_client_base.py b/mobly/controllers/android_device_lib/jsonrpc_client_base.py
--- a/mobly/controllers/android_device_lib/jsonrpc_client_base.py
+++ b/mobly/controllers/android_device_lib/jsonrpc_client_base.py
@@ -19,7 +19,7 @@
 
     Request:
     {
-        "id": <monotonically increasing integer containing the ID of 
+        "id": <monotonically increasing integer containing the ID of
                this request>
         "method": <string containing the name of the method to execute>
         "params": <JSON array containing the arguments to the method>
@@ -239,6 +239,7 @@ def _client_send(self, msg):
         try:
             self._client.write(msg.encode("utf8") + b'\n')
             self._client.flush()
+            self.log.debug('Snippet sent %s.', msg)
         except socket.error as e:
             raise Error(
                 self._ad,
@@ -255,7 +256,9 @@ def _client_receive(self):
             Error: a socket error occurred during the read.
         """
         try:
-            return self._client.readline()
+            response = self._client.readline()
+            self.log.debug('Snippet received: %s', response)
+            return response
         except socket.error as e:
             raise Error(
                 self._ad,
@@ -299,7 +302,7 @@ def _rpc(self, method, *args):
         if not response:
             raise ProtocolError(self._ad,
                                 ProtocolError.NO_RESPONSE_FROM_SERVER)
-        result = json.loads(str(response, encoding="utf8"))
+        result = json.loads(str(response, encoding='utf8'))
         if result['error']:
             raise ApiError(self._ad, result['error'])
         if result['id'] != apiid:


Test patch: