internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: dpgmediamagazines/django-arctic

Issue:
QuickFiltersSelectMultiple add default attrs
For quick filters Arctic has 2 widget: `QuickFiltersSelect` and `QuickFiltersSelectMultiple`. They use same template and it requires `select_multiple` property in `attrs` to make it work checkbox-like. 
It would be much easier to have this attribute be set by deafult in `QuickFiltersSelectMultiple` to do not specify it each time manually. 


Discussion:

Resolving patch:
diff --git a/arctic/widgets.py b/arctic/widgets.py
--- a/arctic/widgets.py
+++ b/arctic/widgets.py
@@ -151,6 +151,10 @@ class QuickFiltersSelectMultiple(
     """
     This widget is used to be able to have a more than one active filters
     """
+    def __init__(self, attrs=None, **kwargs):
+        attrs = attrs or {}
+        attrs['select_multiple'] = True
+        super().__init__(attrs, **kwargs)
 
 
 class SearchInput(TextInput):


Test patch:
diff --git a/tests/test_layout_mixin.py b/tests/test_layout_mixin.py
--- a/tests/test_layout_mixin.py
+++ b/tests/test_layout_mixin.py
@@ -101,7 +101,7 @@ def test_layout_example_4(layout):
 
     assert layout[0]['fieldset']['title'] == 'fieldset'
     assert layout[0]['fieldset']['description'] is None
-    assert layout[0]['fieldset']['collapsible'] is 'closed'
+    assert layout[0]['fieldset']['collapsible'] == 'closed'
     assert layout[0]['rows'][0]['name'] == 'title'
     assert layout[0]['rows'][0]['column'] is None
     assert layout[0]['rows'][1]['name'] == 'title'
@@ -121,6 +121,6 @@ def test_layout_example_4(layout):
 
     assert layout[2]['fieldset']['title'] == 'fieldset3'
     assert layout[2]['fieldset']['description'] is None
-    assert layout[2]['fieldset']['collapsible'] is 'open'
+    assert layout[2]['fieldset']['collapsible'] == 'open'
     assert layout[2]['rows'][0]['name'] == 'published'
     assert layout[2]['rows'][0]['column'] is None
Repository: dpgmediamagazines/django-arctic

Issue:
Keep sidebar menu open on more urls
Currently the sidebar menu stays open if the user is in a child url of that section, but that stops happening as soon as the user moves away from those links.

Arctic already allows more links to be passed to keep the sidebar open
e.g.
```
(_('Articles'), 'articles:list', 'fa-newspaper-o', (
        (_('Articles overview'), 'articles:list', (
            'articles:feed-list',
            'articles:pending-list',
            'articles:create',
            'articles:position',
        )),
    )),
```
After this issue is fixed this would also be possible
```
e.g.
(_('Articles'), 'articles:list', 'fa-newspaper-o', (
        (_('Articles overview'), 'articles:list', (
            'articles:*',
        )),
    )),
```
This would keep the sidebar articles-list open if the url is part of articles:*


Discussion:
The syntax should be more like this:

    ARTIC_MENU =(
    (_('parent'), None, 'fa-download', (
            (_('child 1'), 'site:app:view1'),
            (_('child 2'),  'site:app:view2'),
            # keep menu section opened if the bellow urls also match, 
            # when clicked the first item will be displayed
            (_('child 3'),  ('site:app:view3', 'site:app:create-view1', 'site:app:create-view2')),
        )),
    )

Resolving patch:
diff --git a/arctic/utils.py b/arctic/utils.py
--- a/arctic/utils.py
+++ b/arctic/utils.py
@@ -34,9 +34,19 @@ def is_active(menu_entry, url_name):
                 elif isinstance(menu_item, (tuple, list)):
                     # suppose it is related urls names
                     related_urls += list(menu_item)
-
             related_urls += [submenu[1] for submenu in related_submenus]
-            return url_name in related_urls
+            if url_name in related_urls:
+                return True
+            # Check for * in the related url's
+            start_url = []
+            for related in related_urls:
+                if related.endswith('*'):
+                    start_url += [related]
+            if len(start_url):
+                url_base_name = url_name.split(':')
+                del url_base_name[-1]
+                url_base_name = ':'.join(url_base_name)
+                return [item.startswith(url_base_name) for item in start_url]
     return False
 
 


Test patch:
Repository: dpgmediamagazines/django-arctic

Issue:
Advanced search form is removed from the DOM after being closed
After closing the advanced search form popup the form is removed from the DOM.
This causes a weird behaviour with the toggle buttons because they will ignore the advanced popup form if the form was opened at least once


Discussion:

Resolving patch:
diff --git a/arctic/generics.py b/arctic/generics.py
--- a/arctic/generics.py
+++ b/arctic/generics.py
@@ -641,7 +641,7 @@ def csv_file_response(self):
         writer.writerow(titles)
 
         m2m_fields = [m2m_f.attname for m2m_f in model._meta.many_to_many]
-        for obj in self.get_queryset():
+        for obj in self.get_object_list():
             row = []
             for field in displayed_fields:
                 try:


Test patch:
Repository: microsoft/DeepSpeed

Issue:
Add representation method to optimizer like pytorch optimizer
Just an Idea, so Don't hesitate to close if you disagree and priority is so low.

I suggest that optimizer should have `__repr__`  method like[ pytorch's optimizer](https://github.com/pytorch/pytorch/blob/master/torch/optim/optimizer.py#L63) to see the parameters of optimizer human-readable.



Discussion:
Thanks. This is a good idea. Do you mind contributing a PR? Otherwise, we will keep this open until we get some spare cycles to devote to it. 
No, I will PR later.

Resolving patch:
diff --git a/deepspeed/pt/deepspeed_zero_optimizer.py b/deepspeed/pt/deepspeed_zero_optimizer.py
--- a/deepspeed/pt/deepspeed_zero_optimizer.py
+++ b/deepspeed/pt/deepspeed_zero_optimizer.py
@@ -531,3 +531,6 @@ def load_state_dict(self, state_dict, load_optimizer_states=True):
 
         for current, saved in zip(self.single_partition_of_fp32_groups, state_dict['single_partition_of_fp32_groups']):
             current.data.copy_(saved.data)
+
+    def __repr__(self):
+        return repr(self.optimizer)
diff --git a/deepspeed/pt/fp16_optimizer.py b/deepspeed/pt/fp16_optimizer.py
--- a/deepspeed/pt/fp16_optimizer.py
+++ b/deepspeed/pt/fp16_optimizer.py
@@ -337,3 +337,6 @@ def load_state_dict(self, state_dict, load_optimizer_states=True):
         # are guaranteed to exist, so we can just copy_() from the saved master params.
         for current, saved in zip(self.fp32_groups_flat, state_dict['fp32_groups_flat']):
             current.data.copy_(saved.data)
+
+    def __repr__(self):
+        return repr(self.optimizer)
diff --git a/deepspeed/pt/fp16_unfused_optimizer.py b/deepspeed/pt/fp16_unfused_optimizer.py
--- a/deepspeed/pt/fp16_unfused_optimizer.py
+++ b/deepspeed/pt/fp16_unfused_optimizer.py
@@ -325,3 +325,6 @@ def load_state_dict(self, state_dict, load_optimizer_states=True):
         for current_group, saved_group in zip(self.fp32_groups, state_dict['fp32_groups']):
             for current, saved in zip(current_group, saved_group):
                 current.data.copy_(saved.data)
+
+    def __repr__(self):
+        return repr(self.optimizer)


Test patch:
Repository: microsoft/DeepSpeed

Issue:
Pipeline throughput is incorrect
I think the throughput should be ```tput = self.train_batch_size() * self.steps_per_print() / elapsed ```. Right now it is not taking into account the logging frequency.

https://github.com/microsoft/DeepSpeed/blob/4ac9bf60a740a5a8433ee256c48527c0fba63beb/deepspeed/runtime/pipe/engine.py#L277


Discussion:

Resolving patch:
diff --git a/deepspeed/runtime/pipe/engine.py b/deepspeed/runtime/pipe/engine.py
--- a/deepspeed/runtime/pipe/engine.py
+++ b/deepspeed/runtime/pipe/engine.py
@@ -260,9 +260,8 @@ def train_batch(self, data_iter=None):
         self.module.train()
         self.total_loss = None
 
-        self.timers('train_batch').start()
-
         # Do the work
+        self.timers('train_batch').start()
         sched = schedule.TrainSchedule(micro_batches=self.micro_batches,
                                        stages=self.num_stages,
                                        stage_id=self.stage_id)
@@ -274,7 +273,7 @@ def train_batch(self, data_iter=None):
             if self.global_rank == 0:
                 elapsed = self.timers('train_batch').elapsed(reset=True)
                 iter_time = elapsed / self.steps_per_print()
-                tput = self.train_batch_size() / elapsed
+                tput = self.train_batch_size() / iter_time
                 print(f'steps: {self.global_steps} '
                       f'loss: {self.agg_train_loss:0.4f} '
                       f'iter time (s): {iter_time:0.3f} '


Test patch:
Repository: microsoft/DeepSpeed

Issue:
how to restore a client_state from a saved checkpoint?
According to this [doc](https://deepspeed.readthedocs.io/en/latest/model-checkpointing.html)

I can save a custom client_state:
```
 deepspeed.DeepSpeedEngine.save_checkpoint(self, save_dir, tag=None, client_state={}, save_latest=True)
```
but I don't see how to get it back:
```
deepspeed.DeepSpeedEngine.load_checkpoint(self, load_dir, tag=None, load_module_strict=True, load_optimizer_states=True, load_lr_scheduler_states=True)
```
What am I missing?

Unless it just calls `torch.save` behind the walls and then I just have to `torch.load` it manually. I guess the loader can't do anything with the client state since where would it put it. 

I trust I will see it in action tomorrow and it'll all become clear, I was just trying to understand the docs first before integrating it and was missing the counterpart.

Thank you!


Discussion:
load_checkpoint returns a [tuple ](https://github.com/microsoft/DeepSpeed/blob/46d2e2872b64ebccb8bf4eb5c8a3a55f9adaaa6c/deepspeed/runtime/engine.py#L1239-L1241) which includes the client state.
aha! that's why I failed to parse this sentence: 

> path of the loaded checkpoint. None if loading the checkpoint failed client_state: 

I couldn't make sense of "failed client_state".

It needs a dot and ideally a new line there, so it's easier to see that it returns 2 things. of course, best made into a bullet list like the incoming args.

Thank you, @tjruwase!
Actually, if you re-read that [doc link](https://deepspeed.readthedocs.io/en/latest/model-checkpointing.html) very carefully, **past the formatting mistakes :)**, you will see the following 


```
Returns:  Path of the loaded checkpoint. None if loading the checkpoint failed client_state: State dictionary used for loading required training states in the client code.
```

We are working on fixing the formatting. Thanks for catching this. 


Resolving patch:
diff --git a/deepspeed/runtime/engine.py b/deepspeed/runtime/engine.py
--- a/deepspeed/runtime/engine.py
+++ b/deepspeed/runtime/engine.py
@@ -1236,9 +1236,12 @@ def load_checkpoint(self,
             load_module_strict: Optional. Boolean to strictly enforce that the keys in state_dict of module and checkpoint match.
             load_optimizer_states: Optional. Boolean to load the training optimizer states from Checkpoint. Ex. ADAM's momentum and variance
             load_lr_scheduler_states: Optional. Boolean to add the learning rate scheduler states from Checkpoint.
-        Return:
-            load_path: Path of the loaded checkpoint. None if loading the checkpoint failed
-            client_state: State dictionary used for loading required training states in the client code.
+        Returns:
+            A tuple of ``load_path`` and ``client_state``.
+
+            *``load_path``: Path of the loaded checkpoint. ``None`` if loading the checkpoint failed.
+
+            *``client_state``: State dictionary used for loading required training states in the client code.
         """
 
         if tag is None:


Test patch:
Repository: microsoft/DeepSpeed

Issue:
fix format
This PR fixed a typo introduced in a previous merge. The bug was reported at https://github.com/microsoft/DeepSpeed/issues/1029.


Discussion:

Resolving patch:
diff --git a/deepspeed/runtime/engine.py b/deepspeed/runtime/engine.py
--- a/deepspeed/runtime/engine.py
+++ b/deepspeed/runtime/engine.py
@@ -826,9 +826,11 @@ def _configure_progressive_layer_drop(self):
 
         return pld
 
+    @staticmethod
     def is_map_style_dataset(obj):
         return hasattr(obj, "__getitem__") and hasattr(obj, "__len__")
 
+    @staticmethod
     def is_iterable_style_dataset(obj):
         return isinstance(obj,
                           torch.utils.data.IterableDataset


Test patch:
Repository: microsoft/DeepSpeed

Issue:
possible relaxation on the `tensorboardX` hardwired version requirement
I'm having a bit of a yo-yo conflicting requirements between deepspeed and a sub-dependency of `transformers`:

deepspeed wants `tensorboardX==1.8`

```
cd transformers
pip install -e .[dev]
[...]
  Attempting uninstall: tensorboardX
    Found existing installation: tensorboardX 1.8
    Uninstalling tensorboardX-1.8:
      Successfully uninstalled tensorboardX-1.8
Successfully installed tensorboardX-2.4
```

Traced it to: `ray`:

```
$ pip install tensorboardX==1.8; pip install ray[tune]
(as above)
```


https://github.com/ray-project/ray/blob/56619b955e627d044bd948c38c14816c3a00052a/python/requirements.txt#L41
```
tensorboardX >= 1.9
```

but deepspeed's is `tensorboardX==1.8`

Could deepspeed use `tensorboardX>=1.8,<2.0`


Discussion:

Resolving patch:
diff --git a/deepspeed/runtime/engine.py b/deepspeed/runtime/engine.py
--- a/deepspeed/runtime/engine.py
+++ b/deepspeed/runtime/engine.py
@@ -18,7 +18,6 @@
 from torch.optim import Optimizer
 from torch.optim.lr_scheduler import _LRScheduler
 from torch.distributed.distributed_c10d import _get_global_rank
-from tensorboardX import SummaryWriter
 
 from typing import Callable, Dict, Optional, Union, Iterable
 
@@ -533,6 +532,16 @@ def get_summary_writer(
             log_dir = os.path.join(base, summary_writer_dir_name, name)
 
         os.makedirs(log_dir, exist_ok=True)
+        try:
+            # torch.utils.tensorboard will fail if `tensorboard` is not available,
+            # see their docs for more details: https://pytorch.org/docs/1.8.0/tensorboard.html
+            import tensorboard
+        except ImportError:
+            print(
+                'If you want to use tensorboard logging please `pip install tensorboard`'
+            )
+            raise
+        from torch.utils.tensorboard import SummaryWriter
 
         return SummaryWriter(log_dir=log_dir)
 


Test patch:
Repository: microsoft/DeepSpeed

Issue:
cpu_offload == true does not converge.
https://github.com/microsoft/DeepSpeed/issues/493
https://github.com/microsoft/DeepSpeedExamples/issues/153
thanks.


Discussion:
I've tried the same example with deepspeed 0.5.7 and 0.5.8, neither version converges.

Resolving patch:
diff --git a/deepspeed/runtime/zero/stage_1_and_2.py b/deepspeed/runtime/zero/stage_1_and_2.py
--- a/deepspeed/runtime/zero/stage_1_and_2.py
+++ b/deepspeed/runtime/zero/stage_1_and_2.py
@@ -1225,6 +1225,9 @@ def reduce_ipg_grads(self):
                             param.grad = None  #only if self.partition_gradients
                     elif self.contiguous_gradients:
                         self.copy_grads_in_partition(param)
+                else:  # zero stage 1 - partition only optimizer state
+                    if self.contiguous_gradients:
+                        self.copy_grads_in_partition(param)
 
         self.grads_in_ipg_bucket = []
         self.params_in_ipg_bucket = []


Test patch:
Repository: microsoft/DeepSpeed

Issue:
[BUG] ImportError: cannot import name 'OrderedDict'
python3.6, master version
```
Traceback (most recent call last):
  File "/opt/conda/bin/ds_report", line 3, in <module>
    from deepspeed.env_report import cli_main
  File "/opt/conda/lib/python3.6/site-packages/deepspeed/__init__.py", line 16, in <module>
    from .runtime.engine import DeepSpeedEngine, DeepSpeedOptimizerCallable, DeepSpeedSchedulerCallable
  File "/opt/conda/lib/python3.6/site-packages/deepspeed/runtime/engine.py", line 30, in <module>
    from deepspeed.runtime.bf16_optimizer import BF16_Optimizer
  File "/opt/conda/lib/python3.6/site-packages/deepspeed/runtime/bf16_optimizer.py", line 5, in <module>
    from typing import OrderedDict
ImportError: cannot import name 'OrderedDict'
```


Discussion:
Dockerfile to reproduce
```docker
FROM nvcr.io/nvidia/pytorch:20.11-py3
RUN pip install deepspeed==0.7.2
RUN python -c "import deepspeed; print(deepspeed.__version__)"
```

Resolving patch:
diff --git a/deepspeed/runtime/bf16_optimizer.py b/deepspeed/runtime/bf16_optimizer.py
--- a/deepspeed/runtime/bf16_optimizer.py
+++ b/deepspeed/runtime/bf16_optimizer.py
@@ -2,7 +2,7 @@
 Copyright 2022 The Microsoft DeepSpeed Team
 """
 
-from typing import OrderedDict
+from collections import OrderedDict
 import torch
 import os
 from deepspeed import comm as dist


Test patch:
Repository: microsoft/DeepSpeed

Issue:
[BUG] /tmp/torch_extensions directory created without global write permission
**Describe the bug**
The directory (and associated sub-directories) /tmp/torch_extensions are created without global write permissions, e.g., only group and user have write permissions. On shared machines this can result in others being unable to run deepspeed as it fails at compile time.

Possibly related to this pytorch issue: https://github.com/pytorch/pytorch/issues/34238

Work around: `export TORCH_EXTENSIONS_DIR=/tmp/$USER/torch_extensions/`

**To Reproduce**
Steps to reproduce the behavior:
1. On a shared (linux) machine or cluster node user1 who is in group1 runs deepspeed, e.g. following https://www.deepspeed.ai/tutorials/inference-tutorial/
2. On the same (linux) machine or cluster node user2 who is in group2 tries to run deepspeed but gets permission error on export /tmp/torch_extensions/

**Expected behavior**
deepspeed either sets "other" write-permissions to the /tmp/torch_extensions/ directories or it creates separate directories per user.

**System info (please complete the following information):**
 - OS: 20.04.1-Ubuntu
 - GPU count and types: AWS cluster nodes with A100s

**Launcher context**
launched with command `deepspeed` at the command line




Discussion:
Hi @skiingpacmanm, this behavior is (mostly) inherited from torch. I believe in earlier versions of torch they had this path default to `/tmp/torch_extensions`. However for a while now this defaults to the 'user' tmp path which is something like this (depending on system): https://github.com/pytorch/pytorch/blob/56a37ea1a6e89a8aa31abc888127ccac647b92d4/torch/_appdirs.py#L318-L322 and https://github.com/pytorch/pytorch/blob/56a37ea1a6e89a8aa31abc888127ccac647b92d4/torch/utils/cpp_extension.py#L1821

However, looking now I see we have left some code in our op builder that attempts to create the path at `/tmp/torch_extensions/op-name` if `TORCH_EXTENSIONS_DIR` isn't set. I think we can remove this block of code now since the path isn't even being used anymore in modern torch versions.

Resolving patch:
diff --git a/op_builder/builder.py b/op_builder/builder.py
--- a/op_builder/builder.py
+++ b/op_builder/builder.py
@@ -487,13 +487,6 @@ def jit_load(self, verbose=True):
         self.jit_mode = True
         from torch.utils.cpp_extension import load
 
-        # Ensure directory exists to prevent race condition in some cases
-        ext_path = os.path.join(
-            os.environ.get('TORCH_EXTENSIONS_DIR',
-                           DEFAULT_TORCH_EXTENSION_PATH),
-            self.name)
-        os.makedirs(ext_path, exist_ok=True)
-
         start_build = time.time()
         sources = [self.deepspeed_src_path(path) for path in self.sources()]
         extra_include_paths = [


Test patch:
Repository: microsoft/DeepSpeed

Issue:
Checkpoints created prior to refactoring of bf16_optimizer not working
@stas00 @tjruwase fyi
Minimalistic working script for BLOOM-176B:
```python
>>> import torch
>>> torch.load("/net/llm-shared-nfs/data/BLOOM/models--bigscience--bloom-optimizer-states/snapshots/fffeb1434b96997490396f46df742fb0be8f7774/global_step95000/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/net/llm-shared-nfs/nfs/yelkurdi/conda/miniconda3/envs/bloom/lib/python3.8/site-packages/torch/serialization.py", line 712, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/net/llm-shared-nfs/nfs/yelkurdi/conda/miniconda3/envs/bloom/lib/python3.8/site-packages/torch/serialization.py", line 1049, in _load
    result = unpickler.load()
  File "/net/llm-shared-nfs/nfs/yelkurdi/conda/miniconda3/envs/bloom/lib/python3.8/site-packages/torch/serialization.py", line 1042, in find_class
    return super().find_class(mod_name, name)
AttributeError: Can't get attribute 'fragment_address' on <module 'deepspeed.runtime.bf16_optimizer' from '/net/llm-shared-nfs/nfs/mayank/DeepSpeed/deepspeed/runtime/bf16_optimizer.py'
```


Discussion:

Resolving patch:
diff --git a/deepspeed/runtime/bf16_optimizer.py b/deepspeed/runtime/bf16_optimizer.py
--- a/deepspeed/runtime/bf16_optimizer.py
+++ b/deepspeed/runtime/bf16_optimizer.py
@@ -4,6 +4,7 @@
 
 from collections import OrderedDict
 import torch
+import sys
 import os
 from deepspeed import comm as dist
 from deepspeed.runtime.constants import PIPE_REPLICATED
@@ -21,7 +22,7 @@
                                      is_model_parallel_parameter,
                                      see_memory_usage)
 
-from deepspeed.utils import link_hp_params
+from deepspeed.utils import link_hp_params, fragment_address
 from deepspeed.checkpoint import enable_universal_checkpoint
 from deepspeed.checkpoint.constants import (DS_VERSION,
                                             PARTITION_COUNT,
@@ -31,6 +32,8 @@
                                             GROUP_PADDINGS,
                                             PARAM_SLICE_MAPPINGS)
 
+setattr(sys.modules[__name__], 'fragment_address', fragment_address)
+
 
 class BF16_Optimizer(ZeROOptimizer):
     def __init__(self,
diff --git a/deepspeed/utils/__init__.py b/deepspeed/utils/__init__.py
--- a/deepspeed/utils/__init__.py
+++ b/deepspeed/utils/__init__.py
@@ -4,6 +4,6 @@
 from .init_on_device import OnDevice
 from .groups import *
 from .nvtx import instrument_w_nvtx
-from .tensor_fragment import tensor_fragment, get_full_hp_param, get_hp_fragment_mapping
+from .tensor_fragment import tensor_fragment, get_full_hp_param, get_hp_fragment_mapping, fragment_address
 from .mixed_precision_linkage import link_hp_params
 from deepspeed.runtime.dataloader import RepeatingLoader


Test patch: