internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: freedomofpress/securedrop

Issue:
Consider using an if statement
*This is a good first issue for new contributors to take on, if you have any questions, please ask on the task or in our [Gitter room](https://gitter.im/freedomofpress/securedrop)!*

In file `admin/securedrop_admin/__init__.py`, line 518 (https://github.com/freedomofpress/securedrop/blob/develop/admin/securedrop_admin/__init__.py#L518), we can use an `if` statement.

Current code:
```python
517        if type(default) is bool:
518            default = default and 'yes' or 'no'
```

Updated code:
```python
 517        if type(default) is bool:
 518            default = 'yes' if default else 'no'
```

Reason:
 Using `[condition] and [on_true] or [on_false]` may give wrong results when `on_true` has a false boolean value. Though I am not sure about it. However, the `if` expression is more readable and straight forward.

PS
Went through the codebase and found this one, thus opened the issue and not for the sake of the T-shirt!


Discussion:
Note that the code has moved down to https://github.com/freedomofpress/securedrop/blob/develop/admin/securedrop_admin/__init__.py#L569

I agree that the code as you've proposed is cleaner and easier to logically read.
I'll hop on this if it's still available!

Resolving patch:
diff --git a/admin/securedrop_admin/__init__.py b/admin/securedrop_admin/__init__.py
--- a/admin/securedrop_admin/__init__.py
+++ b/admin/securedrop_admin/__init__.py
@@ -567,7 +567,7 @@ def validated_input(
         self, prompt: str, default: Any, validator: Validator, transform: Optional[Callable]
     ) -> Any:
         if type(default) is bool:
-            default = default and "yes" or "no"
+            default = "yes" if default else "no"
         if type(default) is int:
             default = str(default)
         if isinstance(default, list):


Test patch:
Repository: freedomofpress/securedrop

Issue:
Translation-related errors in output when running `make dev`
Haven't tried to prune Docker files yet, so might be specific to my env (this is on latest `develop`). Seeing errors like this:

```
WARNING: Start of line didn't match any expected keyword.
WARNING: Problem on line 1258: '| msgid "Must be at least {num} character long."'
WARNING: Start of line didn't match any expected keyword.
WARNING: Problem on line 1259: '| msgid_plural "Must be at least {num} characters long."'
compiling catalog /home/erik/Code/securedrop/securedrop/translations/fr_FR/LC_MESSAGES/messages.po to /home/erik/Code/securedrop/securedrop/translations/fr_FR/LC_MESSAGES/messages.mo
```


Discussion:
Connfirmed on `develop`. Can repro after running `docker system prune --all`.
Not an issue on `release/2.5.0`, so probably related to the `sh` dependency removal in `i18n_tool.py`.
Yep :( bisected to 21ebdc8d9a04f228d7fb0643a3e81745cc17db90. Debugging now...
The warnings have always been there, the old code just suppressed them instead of letting them print. I'll restore that behavior in a few minutes.

The warnings itself are because of https://github.com/python-babel/babel/issues/566 and seem harmless.

Resolving patch:
diff --git a/securedrop/i18n_tool.py b/securedrop/i18n_tool.py
--- a/securedrop/i18n_tool.py
+++ b/securedrop/i18n_tool.py
@@ -119,7 +119,14 @@ def translate_messages(self, args: argparse.Namespace) -> None:
                 log.warning("messages translations are already up to date")
 
         if args.compile and len(os.listdir(args.translations_dir)) > 1:
-            subprocess.check_call(["pybabel", "compile", "--directory", args.translations_dir])
+            # Suppress all pybabel to hide warnings (e.g.
+            # https://github.com/python-babel/babel/issues/566) and verbose "compiling..." messages
+            subprocess.run(
+                ["pybabel", "compile", "--directory", args.translations_dir],
+                check=True,
+                stdout=subprocess.DEVNULL,
+                stderr=subprocess.DEVNULL,
+            )
 
     def translate_desktop(self, args: argparse.Namespace) -> None:
         messages_file = Path(args.translations_dir).absolute() / "desktop.pot"


Test patch:
Repository: freedomofpress/securedrop

Issue:
[2.6.0] locale switcher lists full locale name when `SUPPORTED_LOCALES` contains duplicates
## Description

If the `SUPPORTED_LOCALES` list contains duplicates, including just if an administrator explicitly adds the default `en_US` locale, then the locale switcher will unnecessarily list the full locale name including the country.

## Steps to Reproduce

1. `securedrop-admin sdconfig`: explicitly include `en_US` as a supported locale
2. `securedrop-admin install`

## Expected Behavior

`English` is listed in the locale switcher.

## Actual Behavior

`English (United States)` is listed in the locale switcher.

Courtesy of @legoktm:

![IMG_20230615_174621](https://github.com/freedomofpress/securedrop/assets/357435/7fa87841-46a6-4203-848e-a3e3b3f246b3)


Discussion:

Resolving patch:
diff --git a/securedrop/i18n.py b/securedrop/i18n.py
--- a/securedrop/i18n.py
+++ b/securedrop/i18n.py
@@ -172,13 +172,16 @@ def map_locale_display_names(
     like Chinese, we do need the additional detail.
     """
 
+    # Deduplicate before sorting.
+    supported_locales = sorted(list(set(config.SUPPORTED_LOCALES)))
+
     language_locale_counts = collections.defaultdict(int)  # type: Dict[str, int]
-    for l in sorted(config.SUPPORTED_LOCALES):
+    for l in supported_locales:
         locale = RequestLocaleInfo(l)
         language_locale_counts[locale.language] += 1
 
     locale_map = collections.OrderedDict()
-    for l in sorted(config.SUPPORTED_LOCALES):
+    for l in supported_locales:
         if Locale.parse(l) not in usable_locales:
             continue
 


Test patch:
diff --git a/securedrop/tests/test_i18n.py b/securedrop/tests/test_i18n.py
--- a/securedrop/tests/test_i18n.py
+++ b/securedrop/tests/test_i18n.py
@@ -456,3 +456,20 @@ def test_same_lang_diff_locale():
     html = resp.data.decode("utf-8")
     assert "português (Brasil)" in html
     assert "português (Portugal)" in html
+
+
+def test_duplicate_locales():
+    """
+    Verify that we don't display the full locale name for duplicate locales,
+    whether from user input or securedrop.sdconfig's enforcement of the
+    fallback locale.
+    """
+
+    # ["en_US", "en_US"] alone will not display the locale switcher, which
+    # *does* pass through set deduplication.
+    test_config = create_config_for_i18n_test(supported_locales=["en_US", "en_US", "ar"])
+
+    app = journalist_app_module.create_app(test_config).test_client()
+    resp = app.get("/", follow_redirects=True)
+    html = resp.data.decode("utf-8")
+    assert "English (United States)" not in html
Repository: freedomofpress/securedrop

Issue:
Tests failed because random_datetime generated invalid date on April 30
## Description

Tests failed because it randomly generated an invalid date.

## Steps to Reproduce

Wait until it is April 30th (March probably is the best time to trigger it), run tests a number of times

## Expected Behavior

They pass

## Actual Behavior

```
Traceback (most recent call last):
  File "./loaddata.py", line 483, in <module>
    load(parse_arguments())
  File "./loaddata.py", line 392, in load
    journalists = create_default_journalists()
  File "./loaddata.py", line 292, in create_default_journalists
    default_journalist = add_journalist("journalist", is_admin=True)
  File "./loaddata.py", line 149, in add_journalist
    attempt.timestamp = random_datetime(nullable=True)
  File "./loaddata.py", line 86, in random_datetime
    return datetime.datetime(
ValueError: day is out of range for month
```

https://github.com/freedomofpress/securedrop-client/actions/runs/8899756224/job/24440145469?pr=1980

## Comments

The `random_datetime` function does the following:
```python
    now = datetime.datetime.now()
    return datetime.datetime(
        year=random.randint(2013, now.year),
        month=random.randint(1, now.month),
        day=random.randint(1, now.day),
        hour=random.randint(0, 23),
        minute=random.randint(0, 59),
        second=random.randint(0, 59),
        microsecond=random.randint(0, 1000),
    )
```

I assume it randomly selected February as the month, and then picked a day that doesn't exist in February (30th, or 29th in a non-leap year).

A safer way to randomly generate the day in a safe manner is to generate a year, then generate a number between 1 and 365 (366 if a leap year) and then calculate month/day from that.


Discussion:

Resolving patch:
diff --git a/securedrop/loaddata.py b/securedrop/loaddata.py
--- a/securedrop/loaddata.py
+++ b/securedrop/loaddata.py
@@ -5,6 +5,7 @@
 """
 
 import argparse
+import calendar
 import datetime
 import io
 import math
@@ -83,10 +84,17 @@ def random_datetime(nullable: bool) -> Optional[datetime.datetime]:
         return None
 
     now = datetime.datetime.now()
+    year = random.randint(2013, now.year)
+    max_day = 366 if calendar.isleap(year) else 365
+    day = random.randint(1, max_day)
+
+    # Calculate the month/day given the year
+    date = datetime.date(year, 1, 1) + datetime.timedelta(days=day - 1)
+
     return datetime.datetime(
-        year=random.randint(2013, now.year),
-        month=random.randint(1, now.month),
-        day=random.randint(1, now.day),
+        year=year,
+        month=date.month,
+        day=date.day,
         hour=random.randint(0, 23),
         minute=random.randint(0, 59),
         second=random.randint(0, 59),


Test patch:
Repository: iterative/dvc

Issue:
Should `--help` give complete list of all commands?
```
>>> dvc --help
Unimplemented or unrecognized command: --help

usage: dvc [--version] [--help] command [<args>]

These are common DVC commands:

start a working area
    init           Initialize dvc over a directory (should already be a git dir).
    run            Run command.
    import         Import file to data directory.
    remove         Remove data item from data directory.

synchronize data between remote and local
    data sync      Synchronize data file with cloud (cloud settings already setup).
-
```


Discussion:
@talwrii Thank you!
It looks like --help actually does its job but I should remove the first line.
I'll fix that.

Resolving patch:
diff --git a/dvc/main.py b/dvc/main.py
--- a/dvc/main.py
+++ b/dvc/main.py
@@ -38,7 +38,7 @@ def print_usage():
     print('\n'.join(usage))
 
 def main():
-    cmds = ['--version', 'init', 'run', 'sync', 'repro', 'data', 'remove', 'import', 'lock', 'cloud', \
+    cmds = ['--help', '--version', 'init', 'run', 'sync', 'repro', 'data', 'remove', 'import', 'lock', 'cloud', \
             'cloud', 'cloud-run', 'cloud-instance-create', 'cloud-instance-remove', 'cloud-instance-describe', \
             'test', 'test-aws', 'test-gcloud', 'test-cloud']
 
@@ -50,7 +50,9 @@ def main():
 
     cmd = sys.argv[1]
 
-    if cmd == '--version':
+    if cmd == '--help':
+        print_usage()
+    elif cmd == '--version':
         print('dvc version {}'.format(VERSION))
     elif cmd == 'init':
         Runtime.run(CmdInit, parse_config=False)


Test patch:
Repository: iterative/dvc

Issue:
Sync cmd: 1/10 num prefix
The num prefix is constantly equal to 1/10.

```$ dvc sync data/
(1/10): [####################################### ] 99% model.p_51b6e1a
```


Discussion:
Could you share steps to reproduce?
It happens to me all the time when I sync something to cloud.
I went through the tutorial (~10 files in `.cache`) and `dvc sync data`. It syncs correctly but I constantly see 1/10 as a progress.
Thanks, I'll take a look shortly.

Resolving patch:
diff --git a/dvc/data_cloud.py b/dvc/data_cloud.py
--- a/dvc/data_cloud.py
+++ b/dvc/data_cloud.py
@@ -248,6 +248,7 @@ def sync_from_cloud(self, item):
         Logger.info('Downloading cache file from S3 "{}/{}"'.format(bucket.name, key_name))
         key.get_contents_to_filename(item.resolved_cache.relative,
                                      cb=create_cb(item.resolved_cache.relative))
+        progress.finish_target(os.path.basename(item.resolved_cache.relative))
         Logger.info('Downloading completed')
 
     def sync_to_cloud(self, data_item):
@@ -270,6 +271,7 @@ def sync_to_cloud(self, data_item):
         key = bucket.new_key(aws_key)
         key.set_contents_from_filename(data_item.resolved_cache.relative,
                                        cb=create_cb(data_item.resolved_cache.relative))
+        progress.finish_target(os.path.basename(data_item.resolved_cache.relateive))
 
     def remove_from_cloud(self, data_item):
         aws_file_name = self.cache_file_key(data_item.cache.dvc)


Test patch:
Repository: iterative/dvc

Issue:
Remove dvc.py from output
Users use `dvc`, not  `dvc.py`. Remove `dvc.py`. 

```$ dvc sync
usage: dvc.py sync [-h] [-q] [-v] [-G] [-j JOBS] targets [targets ...]
dvc.py sync: error: too few arguments```


Discussion:

Resolving patch:
diff --git a/dvc/cli.py b/dvc/cli.py
--- a/dvc/cli.py
+++ b/dvc/cli.py
@@ -38,9 +38,13 @@ def parse_args(argv=None):
                         default=False,
                         help='Skip all git actions including reproducibility check and commits.')
 
+    # Main parser
     desc = 'Data Version Control'
-    parser = argparse.ArgumentParser(description=desc, parents=[parent_parser],
-             formatter_class=argparse.RawTextHelpFormatter)
+    parser = argparse.ArgumentParser(
+                        prog='dvc',
+                        description=desc,
+                        parents=[parent_parser],
+                        formatter_class=argparse.RawTextHelpFormatter)
 
     parser.add_argument('-V',
                         '--version',


Test patch:
Repository: iterative/dvc

Issue:
Unconfigured git error
Git repository might be created but git might be not configured.
The results of this failure:
1. .gitignore file was not created.
2. change was not committed

```
c:\src\tmp>dvc init
Directories data/, .cache/ and .state/ were created
Error. Git command error (git commit -m DVC init. data dir data, cache dir .cache, state dir .state):
*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: empty ident name (for <(NULL)>) not allowed
```


Discussion:

Resolving patch:
diff --git a/dvc/git_wrapper.py b/dvc/git_wrapper.py
--- a/dvc/git_wrapper.py
+++ b/dvc/git_wrapper.py
@@ -46,6 +46,15 @@ def git_file_statuses():
 
         return GitWrapper.parse_porcelain_files(out)
 
+    @staticmethod
+    def git_config_get(name):
+        code, out, err = Executor.exec_cmd(['git', 'config', '--get', name])
+        Logger.debug('[dvc-git] "git config --get {}": code({}), out({}), err({})'.format(
+                                                               name, code, out, err))
+        if code != 0:
+            return None
+        return out
+
     @staticmethod
     def git_path_to_system_path(path):
         if os.name == 'nt':
@@ -86,6 +95,17 @@ def is_ready_to_go(self):
                 Logger.error("{} {}".format(status, file))
             return False
 
+        # Sanity check to make sure we will be able to commit
+        name = self.git_config_get('user.name')
+        if name == None:
+            Logger.error('[dvc-git] Please setup user.name in git config')
+            return False
+
+        email = self.git_config_get('user.email')
+        if email == None:
+            Logger.error('[dvc-git] Please setup user.email in git config')
+            return False
+
         return True
 
     @property


Test patch:
Repository: iterative/dvc

Issue:
Resume downloading on import
Considering that big files aren't something unusual in dvc scenarios, we should support resuming import.


Discussion:

Resolving patch:
diff --git a/dvc/cli.py b/dvc/cli.py
--- a/dvc/cli.py
+++ b/dvc/cli.py
@@ -223,6 +223,12 @@ def parse_args(argv=None):
                         type=int,
                         default=cpu_count(),
                         help='Number of jobs to run simultaneously.')
+    import_parser.add_argument('-c',
+                        '--continue',
+                        dest='cont',
+                        action='store_true',
+                        default=False,
+                        help='Resume downloading file from url')
     import_parser.set_defaults(func=CmdImportFile)
 
     # Lock
diff --git a/dvc/command/import_file.py b/dvc/command/import_file.py
--- a/dvc/command/import_file.py
+++ b/dvc/command/import_file.py
@@ -142,21 +142,45 @@ def download_file(self, from_url, to_file):
         """
         Download single file from url.
         """
-        r = requests.get(from_url, stream=True)
 
+        tmp_file = to_file + '.part'
         name = os.path.basename(from_url)
         chunk_size = 1024 * 100
         downloaded = 0
         last_reported = 0
         report_bucket = 100*1024*10
+
+        resume_header = None
+        mode = 'wb'
+
+        # Resume download if we can
+        if os.path.exists(tmp_file) and self.parsed_args.cont:
+            mode = 'ab'
+
+            downloaded = os.path.getsize(tmp_file)
+            resume_header = {'Range': 'bytes=%d-' % downloaded}
+
+            Logger.debug('found existing {} file, resuming download'.format(tmp_file))
+
+        r = requests.get(from_url, stream=True, headers=resume_header)
+
+        content_range = r.headers.get('content-range')
+        if resume_header and content_range == None:
+            mode = 'wb'
+            downloaded = 0
+
+            Logger.debug('\'range\' is not supported by the server. Can\'t resume download')
+
         total_length = r.headers.get('content-length')
+        if total_length == None:
+            Logger.debug('\'content-length\' is not supported by the server')
 
-        with open(to_file, 'wb') as f:
+        with open(tmp_file, mode) as f:
             for chunk in r.iter_content(chunk_size=chunk_size):
                 if not chunk:  # filter out keep-alive new chunks
                     continue
 
-                downloaded += chunk_size
+                downloaded += len(chunk)
 
                 last_reported += chunk_size
                 if last_reported >= report_bucket:
@@ -171,6 +195,8 @@ def download_file(self, from_url, to_file):
         # tell progress bar that this target is finished downloading
         progress.finish_target(name)
 
+        os.rename(tmp_file, to_file)
+
     def download_targets(self, targets):
         """
         Download targets in a number of threads.


Test patch:
Repository: iterative/dvc

Issue:
Add 'dvc cloud create' to https://github.com/dataversioncontrol/dvc/pull/149



Discussion:

Resolving patch:
diff --git a/dvc/cli.py b/dvc/cli.py
--- a/dvc/cli.py
+++ b/dvc/cli.py
@@ -263,10 +263,19 @@ def parse_args(argv=None):
                         help='Reset target.')
     target_parser.set_defaults(func=CmdTarget)
 
+    # Cloud
+    cloud_parser = subparsers.add_parser(
+                        'cloud',
+                        parents=[parent_parser],
+                        help='Cloud manipulation')
+    cloud_subparsers = cloud_parser.add_subparsers(
+                        dest='cmd',
+                        help='Use dvc cloud CMD --help for command-specific help')
+
     # Instance create
-    instance_create_parser = subparsers.add_parser('instance-create',
-                                                    parents=[parent_parser],
-                                                    help='Create cloud instance')
+    instance_create_parser = cloud_subparsers.add_parser(
+                        'create',
+                        help='Create cloud instance')
     instance_create_parser.add_argument('name',
                                         # metavar='',
                                         nargs='?',


Test patch:
Repository: iterative/dvc

Issue:
Don't throw unexpected exceptions



Discussion:

Resolving patch:
diff --git a/dvc/command/data_sync.py b/dvc/command/data_sync.py
--- a/dvc/command/data_sync.py
+++ b/dvc/command/data_sync.py
@@ -33,20 +33,28 @@ class CmdDataStatus(CmdBase):
     def __init__(self, settings):
         super(CmdDataStatus, self).__init__(settings)
 
+    def _show(self, status):
+        for s in status:
+            target, ret = s
+
+            if ret == cloud.STATUS_UNKNOWN or ret == cloud.STATUS_OK:
+                continue
+
+            prefix_map = {
+                cloud.STATUS_DELETED  : 'deleted: ',
+                cloud.STATUS_MODIFIED : 'modified:',
+                cloud.STATUS_NEW      : 'new file:',
+            }
+
+            Logger.info('\t{}\t{}'.format(prefix_map[ret], target.resolved_cache.dvc))
+
     def run(self):
         with DvcLock(self.is_locker, self.git):
             status = self.cloud.status(self.parsed_args.targets, self.parsed_args.jobs)
 
-            for s in status:
-                target, ret = s
-
-                if ret == cloud.STATUS_UNKNOWN or ret == cloud.STATUS_OK:
-                    continue
+            if len(status) != len(self.parsed_args.targets):
+                return 1
 
-                prefix_map = {
-                    cloud.STATUS_DELETED  : 'deleted: ',
-                    cloud.STATUS_MODIFIED : 'modified:',
-                    cloud.STATUS_NEW      : 'new file:',
-                }
+            self._show(status)
 
-                Logger.info('\t{}\t{}'.format(prefix_map[ret], target.resolved_cache.dvc))
+            return 0
diff --git a/dvc/main.py b/dvc/main.py
--- a/dvc/main.py
+++ b/dvc/main.py
@@ -6,6 +6,17 @@
 from dvc.settings import Settings
 
 def main():
-    settings = Settings(sys.argv[1:])
-    instance = settings._parsed_args.func(settings)
-    sys.exit(instance.run())
+    try:
+        settings = Settings(sys.argv[1:])
+        instance = settings._parsed_args.func(settings)
+    except Exception as e:
+        Logger.error("Exception caught while parsing settings", exc_info=True)
+        return 255
+
+    try:
+        ret = instance.run()
+    except Exception as e:
+        Logger.error("Exception caught in " + instance.__class__.__name__, exc_info=True)
+        return 254
+
+    return ret
diff --git a/dvc/utils.py b/dvc/utils.py
--- a/dvc/utils.py
+++ b/dvc/utils.py
@@ -66,6 +66,7 @@ def map_progress(func, targets, n_threads):
     """
     progress.set_n_total(len(targets))
     p = ThreadPool(processes=n_threads)
+    ret = []
 
     try:
         ret = p.map(func, targets)


Test patch: