internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: hpcaitech/ColossalAI

Issue:
[BUG]: colossalai run failed with unknown reason
### 🐛 Describe the bug

Some users have reported that they encounter launch failure when using `colossalai run`, but there is no error message to tell exactly what went wrong. A sample output is given below.

```text
Error: failed to run torchrun --nproc_per_node=4 --nnodes=1 --node_rank=0 --rdzv_backend=c10d --rdzv_endpoint=127.0.0.1:29500 --rdzv_id=colossalai-default-job train.py --config config.py -s on 127.0.0.1
```

### Environment

_No response_


Discussion:

Resolving patch:
diff --git a/colossalai/cli/launcher/multinode_runner.py b/colossalai/cli/launcher/multinode_runner.py
--- a/colossalai/cli/launcher/multinode_runner.py
+++ b/colossalai/cli/launcher/multinode_runner.py
@@ -1,8 +1,10 @@
-import fabric
-from .hostinfo import HostInfo, HostInfoList
 from multiprocessing import Pipe, Process
 from multiprocessing import connection as mp_connection
+
 import click
+import fabric
+
+from .hostinfo import HostInfo, HostInfoList
 
 
 def run_on_host(hostinfo: HostInfo, workdir: str, recv_conn: mp_connection.Connection,
@@ -45,8 +47,10 @@ def run_on_host(hostinfo: HostInfo, workdir: str, recv_conn: mp_connection.Conne
                             # execute on the remote machine
                             fab_conn.run(cmds, hide=False)
                     send_conn.send('success')
-            except:
-                click.echo(f"Error: failed to run {cmds} on {hostinfo.hostname}")
+            except Exception as e:
+                click.echo(
+                    f"Error: failed to run {cmds} on {hostinfo.hostname}, is localhost: {hostinfo.is_local_host}, exception: {e}"
+                )
                 send_conn.send('failure')
 
     # shutdown


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests

[devops] notify lark if 8-GPU test fails
According to the proposal #2573, we can hook up the Lark so that when the scheduled 8-GPU build and test fails, a message will be sent to notify the development team.


Discussion:

Resolving patch:
diff --git a/.github/workflows/scripts/send_message_to_lark.py b/.github/workflows/scripts/send_message_to_lark.py
new file mode 100644
--- /dev/null
+++ b/.github/workflows/scripts/send_message_to_lark.py
@@ -0,0 +1,20 @@
+import argparse
+
+import requests
+
+
+def parse_args():
+    parser = argparse.ArgumentParser()
+    parser.add_argument('-m', '--message', type=str)
+    parser.add_argument('-u', '--url', type=str)
+    return parser.parse_args()
+
+
+def send_message_to_lark(message, webhook_url):
+    data = {"msg_type": "text", "content": {"text": message}}
+    requests.post(webhook_url, json=data)
+
+
+if __name__ == '__main__':
+    args = parse_args()
+    send_message_to_lark(args.message, args.url)


Test patch:
diff --git a/test.sh b/test.sh
new file mode 100644
--- /dev/null
+++ b/test.sh
@@ -0,0 +1,6 @@
+avai=true
+for i in $(seq 0 7);
+do
+  gpu_used=$(nvidia-smi -i $i --query-gpu=memory.used --format=csv,noheader,nounits)
+  [ "$gpu_used" -le "10000" ] && avai=false
+done
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests

[BUG]: pip install failed when CUDA_EXT is 1
### 🐛 Describe the bug

```
$CUDA_EXT=1 python -m pip install --force-reinstall --no-deps colossalai -U
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting colossalai
  Downloading colossalai-0.2.0.tar.gz (697 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 697.8/697.8 kB 10.7 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... done
Discarding https://files.pythonhosted.org/packages/9c/c7/427dd2ed3769c6beb05cefdc18f8fad634673ed1472d5409afbdb768c7c3/colossalai-0.2.0.tar.gz (from https://pypi.org/simple/colossalai/) (requires-python:>=3.6): Requested colossalai from https://files.pythonhosted.org/packages/9c/c7/427dd2ed3769c6beb05cefdc18f8fad634673ed1472d5409afbdb768c7c3/colossalai-0.2.0.tar.gz has inconsistent version: expected '0.2.0', but metadata has '0.2.0+torch1.13cu11.3'
ERROR: Could not find a version that satisfies the requirement colossalai (from versions: 0.2.0)
ERROR: No matching distribution found for colossalai
```

### Environment

linux ubuntu


Discussion:



Hi @fuhengwu2021 , I am able to reproduce this problem. Thank you for discovering this bug. I will fix it tomorrow.

Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -117,14 +117,26 @@ def get_version():
 
     with open(version_txt_path) as f:
         version = f.read().strip()
-        if build_cuda_ext:
-            torch_version = '.'.join(torch.__version__.split('.')[:2])
-            cuda_version = '.'.join(get_cuda_bare_metal_version(CUDA_HOME)[1:])
-            version += f'+torch{torch_version}cu{cuda_version}'
 
     # write version into version.py
     with open(version_py_path, 'w') as f:
         f.write(f"__version__ = '{version}'\n")
+        if build_cuda_ext:
+            torch_version = '.'.join(torch.__version__.split('.')[:2])
+            cuda_version = '.'.join(get_cuda_bare_metal_version(CUDA_HOME)[1:])
+        else:
+            torch_version = None
+            cuda_version = None
+
+        if torch_version:
+            f.write(f'torch = "{torch_version}"\n')
+        else:
+            f.write('torch = None\n')
+
+        if cuda_version:
+            f.write(f'cuda = "{cuda_version}"\n')
+        else:
+            f.write('cuda = None\n')
 
     return version
 


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests

[BUG]: the release workflows don't work properly
### 🐛 Describe the bug

The PR #2602 has updated the version to v0.2.1. However, some workflows failed after the PR is merged. The problematic workflows are:
1. draft release post: failed with error
2. docker build: successful but the notification message is wrong
3. release to pypi: failed due to workflow yaml file error

More details can be found in https://github.com/hpcaitech/ColossalAI/pull/2602/checks.

### Environment

_No response_


Discussion:

Resolving patch:
diff --git a/.github/workflows/scripts/generate_release_draft.py b/.github/workflows/scripts/generate_release_draft.py
--- a/.github/workflows/scripts/generate_release_draft.py
+++ b/.github/workflows/scripts/generate_release_draft.py
@@ -57,7 +57,12 @@ def collate_release_info(commit_info_list):
 
     for commit_info in commit_info_list:
         author = commit_info['commit']['author']['name']
-        author_url = commit_info['author']['url']
+
+        try:
+            author_url = commit_info['author']['url']
+        except:
+            # author can be None
+            author_url = None
         msg = commit_info['commit']['message']
         match = re.search(pattern, msg)
 
@@ -86,7 +91,10 @@ def generate_release_post_markdown(current_version, last_version, release_info):
             # only keep the first line
             msg = msg.split('\n')[0]
 
-            item = f'{msg} by [{author}]({author_url})\n'
+            if author_url:
+                item = f'{msg} by [{author}]({author_url})\n'
+            else:
+                item = f'{msg} by {author}\n'
             text.append(f'- {item}')
 
         text.append('\n')


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests

[BUG]: Testing failed due to triton
### 🐛 Describe the bug

The build on PR workflow failed with the following errors:

<img width="1509" alt="Screenshot 2023-02-07 at 10 30 17" src="https://user-images.githubusercontent.com/31818963/217132926-fd6cffa1-2c4b-46aa-a6cc-1a3d10918411.png">


### Environment

_No response_


Discussion:

Resolving patch:
diff --git a/colossalai/amp/apex_amp/apex_amp.py b/colossalai/amp/apex_amp/apex_amp.py
--- a/colossalai/amp/apex_amp/apex_amp.py
+++ b/colossalai/amp/apex_amp/apex_amp.py
@@ -2,6 +2,7 @@
 # -*- encoding: utf-8 -*-
 
 import torch.nn as nn
+
 try:
     import apex.amp as apex_amp
 except ImportError:


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests

[DOC]: the sphinx theme is too old
### 📚 The doc issue

As stated in #2579 , we want to use Read the Docs to host our documentation. In this way, tutorials and API documentations will be visited from a single entry. This issue will mainly discuss the appearance of the RTD website. Ideally, we should use Tailwind for style consistency. However, it can take some time to implement a tailwind-based theme, therefore, we should use an existing theme which looks more modern first.


Discussion:

Resolving patch:
diff --git a/docs/conf.py b/docs/conf.py
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -23,8 +23,7 @@
 author = 'HPC-AI Technology Inc.'
 
 # The full version, including alpha/beta/rc tags
-release = '0.0.1'
-
+# release = '0.0.1'
 
 # -- General configuration ---------------------------------------------------
 
@@ -64,14 +63,14 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'sphinx_rtd_theme'
+html_theme = 'sphinx_book_theme'
 html_show_sourcelink = False
 html_theme_options = {
     'navigation_depth': 3,
 }
 
 html_context = {
-    'display_github': False,
+    'display_github': True,
     'github_user': 'hpcaitech',
     'github_repo': 'ColossalAI',
     #   'github_version': 'master/docs/',
@@ -90,7 +89,10 @@
 source_suffix = ['.rst', '.md', '.MD']
 
 import inspect
+
 import colossalai
+
+
 def linkcode_resolve(domain, info):
     """
     Determine the URL corresponding to Python object


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests



Discussion:

Resolving patch:
diff --git a/examples/images/diffusion/main.py b/examples/images/diffusion/main.py
--- a/examples/images/diffusion/main.py
+++ b/examples/images/diffusion/main.py
@@ -539,6 +539,8 @@ def on_train_epoch_end(self, trainer, pl_module):
         raise ValueError("-n/--name and -r/--resume cannot be specified both."
                          "If you want to resume training in a new log folder, "
                          "use -n/--name in combination with --resume_from_checkpoint")
+
+    ckpt = None
     if opt.resume:
         rank_zero_info("Resuming from {}".format(opt.resume))
         if not os.path.exists(opt.resume):


Test patch:
diff --git a/examples/images/diffusion/test_ci.sh b/examples/images/diffusion/test_ci.sh
old mode 100644
new mode 100755
--- a/examples/images/diffusion/test_ci.sh
+++ b/examples/images/diffusion/test_ci.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+set -euxo pipefail
+
+conda env create -f environment.yaml
+
+conda activate ldm
+
+conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
+pip install transformers diffusers invisible-watermark
+
+CUDA_EXT=1  pip install colossalai
+
+pip install pytorch-lightning
+
+wget https://huggingface.co/stabilityai/stable-diffusion-2-base/resolve/main/512-base-ema.ckpt
+
+python main.py --logdir /tmp --train --base configs/Teyvat/train_colossalai_teyvat.yaml --ckpt 512-base-ema.ckpt
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests



Discussion:

Resolving patch:
diff --git a/.github/workflows/scripts/generate_leaderboard_and_send_to_lark.py b/.github/workflows/scripts/generate_leaderboard_and_send_to_lark.py
--- a/.github/workflows/scripts/generate_leaderboard_and_send_to_lark.py
+++ b/.github/workflows/scripts/generate_leaderboard_and_send_to_lark.py
@@ -292,7 +292,13 @@ def generate_user_engagement_leaderboard_image(github_token: str, output_path: s
     y = []
 
     if len(total_engagement_count) > 0:
+        ranking = []
         for name, count in total_engagement_count.items():
+            ranking.append((name, count))
+
+        ranking.sort(key=lambda x: x[1], reverse=True)
+
+        for name, count in ranking:
             x.append(count)
             y.append(name)
 


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests



Discussion:

Resolving patch:
diff --git a/colossalai/zero/sharded_optim/low_level_optim.py b/colossalai/zero/sharded_optim/low_level_optim.py
--- a/colossalai/zero/sharded_optim/low_level_optim.py
+++ b/colossalai/zero/sharded_optim/low_level_optim.py
@@ -131,7 +131,10 @@ def __init__(
         # partition these param groups for data parallel training
         # and add buffers to parameter store for future access
         for group_id, param_group in enumerate(self.optim.param_groups):
-            group_params = param_group['params']
+            group_params = list()
+            for param in param_group['params']:
+                if param.requires_grad:
+                    group_params.append(param)
 
             # add the fp16 params to fp16_param_groups for bookkeeping
             self._fp16_param_groups[group_id] = group_params


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests

[BUG]: Gemini is not compatible with LoRA
### 🐛 Describe the bug

`loralib` provides LoRALinear layer:

![image](https://user-images.githubusercontent.com/23111350/218422859-ed8072a6-37a7-46c0-a1ec-b2fb428743f5.png)


When using `ColoInitContext`, it first convert `self.weigh` to `ColoParameter` after line 105. In line 112, `self.weight.new_zeros()` will return a `ColoTensor`. Then it will be wrapped by `Parameter`. In this case, `isinstance(self.lora_A, ColoParameter)` is `True`, but it does not run `ColoParameter.__init__()`.

`ColoInitContext` should ensure `type(p) is ColoParameter`.

### Environment

torch 1.12.0


Discussion:

Resolving patch:
diff --git a/colossalai/utils/model/colo_init_context.py b/colossalai/utils/model/colo_init_context.py
--- a/colossalai/utils/model/colo_init_context.py
+++ b/colossalai/utils/model/colo_init_context.py
@@ -32,7 +32,7 @@ def _convert_to_coloparam(param: torch.nn.Parameter,
                           default_pg: Optional[ProcessGroup] = None,
                           default_dist_spec: Optional[Any] = None) -> ColoParameter:
 
-    if isinstance(param, ColoParameter):
+    if type(param) is ColoParameter:
         return param
     # detaching tensor is necessary for optimizers.
     requires_grad = param.requires_grad
@@ -102,7 +102,7 @@ def _post_init_method(self, module: torch.nn.Module, *args, **kwargs):
         """
         name_list = []
         for name, param in _named_params_with_replica(module):
-            if isinstance(param, ColoTensor):
+            if type(param) is ColoParameter:
                 continue
 
             split = name.rfind('.')


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests



Discussion:

Resolving patch:
diff --git a/colossalai/nn/parallel/data_parallel.py b/colossalai/nn/parallel/data_parallel.py
--- a/colossalai/nn/parallel/data_parallel.py
+++ b/colossalai/nn/parallel/data_parallel.py
@@ -294,7 +294,7 @@ def _setup_grads_ptr(self):
                 continue
             p.grad = None
 
-    def _pre_bacward(self):
+    def _pre_backward(self):
         # set a visit label for all parameters
         # the label is used to check whether the parameter is correctly reduced
         for param in self.param2name:
@@ -318,7 +318,7 @@ def _post_backward(self):
         self.gemini_manager.post_iter()
 
     def backward(self, loss: torch.Tensor):
-        self._pre_bacward()
+        self._pre_backward()
         with self.param_op_hook.switch_to_backward(), ColoParamOpHookManager.use_hooks(self.param_op_hook):
             loss.backward()
         self._post_backward()


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests



Discussion:

Resolving patch:
diff --git a/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/strategy_generator.py b/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/strategy_generator.py
--- a/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/strategy_generator.py
+++ b/colossalai/auto_parallel/tensor_shard/deprecated/op_handler/strategy_generator.py
@@ -1,6 +1,7 @@
-from dataclasses import dataclass
 from abc import ABC, abstractmethod
-from typing import List, Dict
+from dataclasses import dataclass
+from typing import Dict, List
+
 from colossalai.device.device_mesh import DeviceMesh
 
 __all__ = ['IntermediateStrategy', 'StrategyGenerator']
@@ -9,7 +10,7 @@
 @dataclass
 class IntermediateStrategy:
     """
-    IntermediateStrategy contains the subset of meta information for ShardingStrategy. It is 
+    IntermediateStrategy contains the subset of meta information for ShardingStrategy. It is
     to store the essential information regarding the tensor sharding and leave other meta information to OperatorHandler.
 
     Args:
@@ -24,7 +25,7 @@ class IntermediateStrategy:
 
 class StrategyGenerator(ABC):
     """
-    StrategyGenerator is used to generate the same group of sharding strategies. 
+    StrategyGenerator is used to generate the same group of sharding strategies.
     """
 
     def __init__(self, device_mesh: DeviceMesh):
@@ -39,7 +40,7 @@ def generate(self) -> List[IntermediateStrategy]:
     @abstractmethod
     def validate(self, *args, **kwargs) -> bool:
         """
-        Validate if the operands are of desired shape. 
+        Validate if the operands are of desired shape.
         If True, means this generator can be used for the current operation.
         """
         pass


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests



Discussion:

Resolving patch:
diff --git a/applications/ChatGPT/setup.py b/applications/ChatGPT/setup.py
--- a/applications/ChatGPT/setup.py
+++ b/applications/ChatGPT/setup.py
@@ -22,7 +22,6 @@ def fetch_version():
     packages=find_packages(exclude=(
         'tests',
         'benchmarks',
-        'requirements',
         '*.egg-info',
     )),
     description='A RLFH implementation (ChatGPT) powered by ColossalAI',
@@ -30,7 +29,7 @@ def fetch_version():
     long_description_content_type='text/markdown',
     license='Apache Software License 2.0',
     url='https://github.com/hpcaitech/ChatGPT',
-    install_requires=fetch_requirements('requirements/requirements.txt'),
+    install_requires=fetch_requirements('requirements.txt'),
     python_requires='>=3.6',
     classifiers=[
         'Programming Language :: Python :: 3',


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests



Discussion:

Resolving patch:
diff --git a/colossalai/engine/gradient_handler/utils.py b/colossalai/engine/gradient_handler/utils.py
--- a/colossalai/engine/gradient_handler/utils.py
+++ b/colossalai/engine/gradient_handler/utils.py
@@ -1,29 +1,30 @@
-import torch.distributed as dist
-import torch.nn as nn
-from torch._utils import _flatten_dense_tensors, _unflatten_dense_tensors
-from typing import Iterable
-
-
-def bucket_allreduce(param_list: Iterable[nn.Parameter], group=None):
-    # get communication world size
-    comm_size = dist.get_world_size(group)
-    # bucketize and all-reduce
-    buckets = {}
-    # Pack the buckets.
-    for param in param_list:
-        if param.requires_grad and param.grad is not None:
-            tp = param.data.type()
-            if tp not in buckets:
-                buckets[tp] = []
-            buckets[tp].append(param)
-
-    # For each bucket, all-reduce and copy all-reduced grads.
-    for tp in buckets:
-        bucket = buckets[tp]
-        grads = [param.grad.data for param in bucket]
-        coalesced = _flatten_dense_tensors(grads)
-        coalesced /= comm_size
-
-        dist.all_reduce(coalesced, group=group)
-        for buf, synced in zip(grads, _unflatten_dense_tensors(coalesced, grads)):
-            buf.copy_(synced)
+from typing import Iterable
+
+import torch.distributed as dist
+import torch.nn as nn
+from torch._utils import _flatten_dense_tensors, _unflatten_dense_tensors
+
+
+def bucket_allreduce(param_list: Iterable[nn.Parameter], group=None):
+    # get communication world size
+    comm_size = dist.get_world_size(group)
+    # bucketize and all-reduce
+    buckets = {}
+    # Pack the buckets.
+    for param in param_list:
+        if param.requires_grad and param.grad is not None:
+            tp = param.data.type()
+            if tp not in buckets:
+                buckets[tp] = []
+            buckets[tp].append(param)
+
+    # For each bucket, all-reduce and copy all-reduced grads.
+    for tp in buckets:
+        bucket = buckets[tp]
+        grads = [param.grad.data for param in bucket]
+        coalesced = _flatten_dense_tensors(grads)
+        coalesced /= comm_size
+
+        dist.all_reduce(coalesced, group=group)
+        for buf, synced in zip(grads, _unflatten_dense_tensors(coalesced, grads)):
+            buf.copy_(synced)


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests



Discussion:

Resolving patch:
diff --git a/colossalai/cli/launcher/__init__.py b/colossalai/cli/launcher/__init__.py
--- a/colossalai/cli/launcher/__init__.py
+++ b/colossalai/cli/launcher/__init__.py
@@ -1,7 +1,9 @@
 import click
-from .run import launch_multi_processes
+
 from colossalai.context import Config
 
+from .run import launch_multi_processes
+
 
 @click.command(help="Launch distributed training on a single node or multiple nodes",
                context_settings=dict(ignore_unknown_options=True))


Test patch:
Repository: hpcaitech/ColossalAI

Issue:
[tensor] fix some unittests

[tensor] fix some unittests

[tensor] fix some unittests



Discussion:

Resolving patch:
diff --git a/colossalai/context/process_group_initializer/initializer_sequence.py b/colossalai/context/process_group_initializer/initializer_sequence.py
--- a/colossalai/context/process_group_initializer/initializer_sequence.py
+++ b/colossalai/context/process_group_initializer/initializer_sequence.py
@@ -3,9 +3,10 @@
 import torch.distributed as dist
 
 from colossalai.registry import DIST_GROUP_INITIALIZER
+
+from ..parallel_mode import ParallelMode
 from .initializer_tensor import Initializer_Tensor
 from .process_group_initializer import ProcessGroupInitializer
-from ..parallel_mode import ParallelMode
 
 
 @DIST_GROUP_INITIALIZER.register_module


Test patch: