Repository: hpcaitech/ColossalAI

Issue:
[BUG]: 运行chatgpt推理示例报错
### 🐛 Describe the bug

(https://github.com/hpcaitech/ColossalAI/tree/main/applications/ChatGPT)/examples/    运行inference.py 抛出OSError:
![image](https://user-images.githubusercontent.com/51046864/222064682-05cc668a-aee6-4f05-a2d7-3379169a1835.png)


### Environment

_No response_
[tensor] fix some unittests

[tensor] fix some unittests



Discussion:
 
> Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿      
----  
**Title:** [BUG]: Run the chatgpt reasoning example and report an error    
      
meet the same issue.
一样
 
> Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿         
----    
Same  
      
Hi @peng06051126 @hurun @JingxinLee Thanks for reminding. Sorry for the trouble. We have fixed it.
Please refer to the screenshot to check if there is any problem with the inference code. I see that you only modified README.md.
> Please refer to the screenshot to check if there is any problem with the inference code. I see that you only modified README.md.

Sorry, We are also fixing the bug of code.



Resolving patch:
diff --git a/applications/ChatGPT/examples/inference.py b/applications/ChatGPT/examples/inference.py
--- a/applications/ChatGPT/examples/inference.py
+++ b/applications/ChatGPT/examples/inference.py
@@ -9,30 +9,34 @@
 def eval(args):
     # configure model
     if args.model == 'gpt2':
-        model = GPTActor(pretrained=args.pretrain).to(torch.cuda.current_device())
+        actor = GPTActor().to(torch.cuda.current_device())
     elif args.model == 'bloom':
-        model = BLOOMActor(pretrained=args.pretrain).to(torch.cuda.current_device())
+        actor = BLOOMActor().to(torch.cuda.current_device())
     elif args.model == 'opt':
-        model = OPTActor(pretrained=args.pretrain).to(torch.cuda.current_device())
+        actor = OPTActor().to(torch.cuda.current_device())
     else:
         raise ValueError(f'Unsupported model "{args.model}"')
 
+    state_dict = torch.load(args.pretrain)
+    actor.model.load_state_dict(state_dict)
+    
+    
     # configure tokenizer
     if args.model == 'gpt2':
         tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
         tokenizer.pad_token = tokenizer.eos_token
     elif args.model == 'bloom':
-        tokenizer = AutoTokenizer.from_pretrained(args.pretrain)
+        tokenizer = AutoTokenizer.from_pretrained('bigscience/bloom-560m')
         tokenizer.pad_token = tokenizer.eos_token
     elif args.model == 'opt':
-        tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m")
+        tokenizer = AutoTokenizer.from_pretrained('facebook/opt-350m')
     else:
         raise ValueError(f'Unsupported model "{args.model}"')
 
-    model.eval()
+    actor.eval()
     input = args.input
     input_ids = tokenizer.encode(input, return_tensors='pt').to(torch.cuda.current_device())
-    outputs = model.generate(input_ids,
+    outputs = actor.generate(input_ids,
                              max_length=args.max_length,
                              do_sample=True,
                              top_k=50,
@@ -46,7 +50,7 @@ def eval(args):
     parser = argparse.ArgumentParser()
     parser.add_argument('--model', default='gpt2', choices=['gpt2', 'bloom', 'opt'])
     parser.add_argument('--pretrain', type=str, default=None)
-    parser.add_argument('--input', type=str, default='Q: How are you ? A:')
+    parser.add_argument('--input', type=str, default='Question: How are you ? Answer:')
     parser.add_argument('--max_length', type=int, default=100)
     args = parser.parse_args()
     eval(args)


Test patch: