internlm/SWE-Fixer-Train-110K huggingface dataset:
Repository: blakeblackshear/frigate

Issue:
[UI/nit] Logs, "Jump to Bottom" button colors
Recently and on dev-e91f3d8 I noticed this button may not have the right style? I could see how it could potentially be the right style, but something seems a little off by it, maybe just BG color, maybe just FG color, hard for me to know for sure (I also haven't checked all themes, but maybe it is just the Nord theme I happen to be using?)

![image](https://github.com/blakeblackshear/frigate/assets/4219348/734ae553-8814-4442-a62e-27e3ab7612ed)

Definitely close this if it is intended but I wanted to just point it out in case


Discussion:
No that seems like a remnant of the old colors 
This is another one maybe, when checking I checked the default dark theme and saw this bright blue, another one that I am honestly only 50% sure if it is intentional. I think the blue is a highlight for the active selected window too so maybe it is right but just wanted to report it in case as well

<img width="1014" alt="Screenshot 2024-05-21 at 7 09 24 PM" src="https://github.com/blakeblackshear/frigate/assets/4219348/54f6f441-90a6-4cb5-8c5c-d90fbdb8edb7">

That is correct, same as the other dialogs in the UI 

Resolving patch:
diff --git a/frigate/api/event.py b/frigate/api/event.py
--- a/frigate/api/event.py
+++ b/frigate/api/event.py
@@ -227,9 +227,9 @@ def events():
         elif sort == "score_desc":
             order_by = Event.data["score"].desc()
         elif sort == "date_asc":
-            Event.start_time.asc()
+            order_by = Event.start_time.asc()
         elif sort == "date_desc":
-            Event.start_time.desc()
+            order_by = Event.start_time.desc()
     else:
         order_by = Event.start_time.desc()
 


Test patch:
Repository: blakeblackshear/frigate

Issue:
API: access secrets directly
Currently there is a bug affecting, e.g., NixOS: https://github.com/blakeblackshear/frigate/discussions/11324

This issue stems from these lines: https://github.com/blakeblackshear/frigate/blob/962d213699dd6a08fb2be59fc3e8b2736321b465/frigate/api/auth.py#L90-L96

Certain distributions like NixOS create a per-process user to services like frigate. Often, secrets in the run directory are then given on per-process basis. Coincidentally, this means that the read capabilities of the `/run/secrets` folder is reserved for root only. Instead, the access pattern is such that services read their corresponding environment variables directly from the folder, such as `/run/secrets/FRIGATE_JWT_SECRET`.

To address this issue, the listing logic should be removed. This seems to make little sense anyway since the file is read directly later anyway: if the file does not exist, the read should fail.

Without this change, frigate process on NixOS and distributions with similar secret management logic will run into a Python crash on startup, because frigate tries to scan secrets it should not have access to.


Discussion:
Thanks for reporting. 

Genuinely honest question as we are trying to maintain a good workflow around bug reports. How did you submit this issue without selecting a template?

We have a specific link for bug reports on the new issue page, but this was submitted as a generic issue instead.

![Screenshot_20240611-062515.png](https://github.com/blakeblackshear/frigate/assets/569905/1088ce21-d349-44b0-a7b5-0ff1d863cc69)


This issue was submitted by going to the code, lining up a piece of code, and then clicking on "submit issue" using the GitHub web UI.
Thanks.

I'm not very familiar with NixOS. Does it look like the following would work?

```python
    elif os.path.isdir("/run/secrets") and os.path.isfile(
        os.path.join("/run/secrets", JWT_SECRET_ENV_VAR)
    ):
```
Personally, I would just try to open the filepath directly and fallthrough if it does not open up. I.e., to me, checking whether the directory exists seems redundant if you are then reading a file within it anyway (in other words, reading a file successfully implies that the directory exists). 

Aside of that, yes, that looks good to me! Also, thanks for the swift response. 

Resolving patch:
diff --git a/frigate/api/auth.py b/frigate/api/auth.py
--- a/frigate/api/auth.py
+++ b/frigate/api/auth.py
@@ -87,11 +87,7 @@ def get_jwt_secret() -> str:
         )
         jwt_secret = os.environ.get(JWT_SECRET_ENV_VAR)
     # check docker secrets
-    elif (
-        os.path.isdir("/run/secrets")
-        and os.access("/run/secrets", os.R_OK)
-        and JWT_SECRET_ENV_VAR in os.listdir("/run/secrets")
-    ):
+    elif os.path.isfile(os.path.join("/run/secrets", JWT_SECRET_ENV_VAR)):
         logger.debug(f"Using jwt secret from {JWT_SECRET_ENV_VAR} docker secret file.")
         jwt_secret = Path(os.path.join("/run/secrets", JWT_SECRET_ENV_VAR)).read_text()
     # check for the addon options file


Test patch:
Repository: PaddlePaddle/PaddleNLP

Issue:
fix perf and sample code
<!-- Demo: https://github.com/PaddlePaddle/PaddleNLP/pull/26 -->
### PR types
<!-- One of [ New features | Bug fixes | Function optimization | Performance optimization | Breaking changes | Others ] -->
Others
### PR changes
<!-- One of [ Models | APIs | Docs | Others ] -->
APIs
### Description
<!-- Describe what this PR does -->
fix perf and sample code


Discussion:

Resolving patch:
diff --git a/examples/language_model/gpt/faster_gpt/export_model.py b/examples/language_model/gpt/faster_gpt/export_model.py
--- a/examples/language_model/gpt/faster_gpt/export_model.py
+++ b/examples/language_model/gpt/faster_gpt/export_model.py
@@ -119,6 +119,7 @@ def do_predict(args):
             tokenizer.eos_token_id,
             tokenizer.eos_token_id,
             tokenizer.pad_token_id,
+            None,  # forced_eos_token_id
             args.temperature,
         ])
 
diff --git a/paddlenlp/ops/faster_transformer/sample/gpt_export_model_sample.py b/paddlenlp/ops/faster_transformer/sample/gpt_export_model_sample.py
--- a/paddlenlp/ops/faster_transformer/sample/gpt_export_model_sample.py
+++ b/paddlenlp/ops/faster_transformer/sample/gpt_export_model_sample.py
@@ -119,6 +119,7 @@ def do_predict(args):
             tokenizer.eos_token_id,
             tokenizer.eos_token_id,
             tokenizer.pad_token_id,
+            None,  # forced_eos_token_id
             args.temperature,
         ])
 


Test patch:
Repository: PaddlePaddle/PaddleNLP

Issue:
fix perf and sample code
<!-- Demo: https://github.com/PaddlePaddle/PaddleNLP/pull/26 -->
### PR types
<!-- One of [ New features | Bug fixes | Function optimization | Performance optimization | Breaking changes | Others ] -->
Others
### PR changes
<!-- One of [ Models | APIs | Docs | Others ] -->
APIs
### Description
<!-- Describe what this PR does -->
fix perf and sample code


Discussion:

Resolving patch:
diff --git a/paddlenlp/ops/faster_transformer/transformer/faster_transformer.py b/paddlenlp/ops/faster_transformer/transformer/faster_transformer.py
--- a/paddlenlp/ops/faster_transformer/transformer/faster_transformer.py
+++ b/paddlenlp/ops/faster_transformer/transformer/faster_transformer.py
@@ -676,7 +676,7 @@ def forward(self,
                                  dtype="int32")
 
             if bos_token_id == pad_token_id and paddle.sum(
-                    paddle.any(input_ids == pad_token_id)) > 0:
+                    paddle.any(input_ids == pad_token_id), dtype="int64") > 0:
                 seq_len = seq_len + 1
 
         if num_return_sequences > 1:


Test patch:
Repository: PaddlePaddle/PaddleNLP

Issue:
[amp] refine transformer fp16 train
### PR types
Performance optimization

### PR changes
Models

### Description
优化transformer动态图fp16速度

1、优化点：

- Adam优化器使用multi_tensor策略

- clear_grad使用set_to_zero=False策略

- dataloader num_workers支持>0 ：--num_workers

- 优化一些代码调用顺序

2、性能测试：

![图片](https://user-images.githubusercontent.com/82555433/148742137-aa3cdeff-333b-4efb-9b9b-5be95c9092dc.png)




Discussion:

Resolving patch:
diff --git a/examples/machine_translation/transformer/train.py b/examples/machine_translation/transformer/train.py
--- a/examples/machine_translation/transformer/train.py
+++ b/examples/machine_translation/transformer/train.py
@@ -158,7 +158,8 @@ def do_train(args):
         amp_level = 'O2' if args.use_pure_fp16 else 'O1'
         scaler = paddle.amp.GradScaler(
             enable=True, init_loss_scaling=args.scale_loss)
-        transformer = paddle.amp.decorate(models=transformer, level=amp_level)
+        transformer = paddle.amp.decorate(
+            models=transformer, level=amp_level, save_dtype='float32')
 
     # for distributed training
     if trainer_count > 1:
@@ -260,15 +261,23 @@ def do_train(args):
                 with paddle.no_grad():
                     for input_data in eval_loader:
                         (src_word, trg_word, lbl_word) = input_data
-                        with paddle.amp.auto_cast(
-                                custom_black_list={
-                                    'scale', 'reduce_sum', 'elementwise_div'
-                                } if amp_level == 'O2' else {},
-                                level=amp_level):
+                        if args.use_amp:
+                            with paddle.amp.auto_cast(
+                                    custom_black_list={
+                                        'scale', 'reduce_sum', 'elementwise_div'
+                                    } if amp_level == 'O2' else {},
+                                    level=amp_level):
+                                logits = transformer(
+                                    src_word=src_word, trg_word=trg_word)
+                                sum_cost, avg_cost, token_num = criterion(
+                                    logits, lbl_word)
+
+                        else:
                             logits = transformer(
                                 src_word=src_word, trg_word=trg_word)
                             sum_cost, avg_cost, token_num = criterion(logits,
                                                                       lbl_word)
+
                         total_sum_cost += sum_cost.numpy()
                         total_token_num += token_num.numpy()
                         total_avg_cost = total_sum_cost / total_token_num


Test patch:
Repository: PaddlePaddle/PaddleNLP

Issue:
FewClue中的cluewsc子数据集为空
<img width="1430" alt="image" src="https://user-images.githubusercontent.com/33639025/160518734-d033a125-0d38-4422-9d9d-73998753e5c3.png">



Discussion:

Resolving patch:
diff --git a/paddlenlp/datasets/fewclue.py b/paddlenlp/datasets/fewclue.py
--- a/paddlenlp/datasets/fewclue.py
+++ b/paddlenlp/datasets/fewclue.py
@@ -584,10 +584,12 @@ class FewCLUE(DatasetBuilder):
                     os.path.join('fewclue_cluewsc', 'dev_few_all.json'),
                     '9d5e5066758ac6ff24534b13dd2ed1ba'
                 ],
-                'unlabeled': [
-                    os.path.join('fewclue_cluewsc', 'unlabeled.json'),
-                    'd41d8cd98f00b204e9800998ecf8427e'
-                ],
+                # Note: FewCLUE cluewsc unlabeled.json() is an empty file.
+                # https://github.com/CLUEbenchmark/FewCLUE/blob/main/datasets/cluewsc/unlabeled.json
+                #'unlabeled': [
+                #    os.path.join('fewclue_cluewsc', 'unlabeled.json'),
+                #    'd41d8cd98f00b204e9800998ecf8427e'
+                #],
                 'test': [
                     os.path.join('fewclue_cluewsc', 'test.json'),
                     '0e9e8ffd8ee90ddf1f58d6dc2e02de7b'


Test patch:
Repository: PaddlePaddle/PaddleNLP

Issue:
tokenizer 的 from_pretrained 接口加载报错（tokenizer_config.json 文件含有 special_tokens 信息）
1. 示例代码：AutoTokenizer.from_pretrained("saibo/legal-roberta-base")。
2. 分析：在 `saibo/legal-roberta-base` 的 tokenizer_config.json 文件中，special_tokens 的信息通过以 dict 格式保存的。通过 from_pretrained 进行加载时，会出现 TypeError: unhashable type: 'dict'。



Discussion:

Resolving patch:
diff --git a/paddlenlp/transformers/tokenizer_utils.py b/paddlenlp/transformers/tokenizer_utils.py
--- a/paddlenlp/transformers/tokenizer_utils.py
+++ b/paddlenlp/transformers/tokenizer_utils.py
@@ -254,6 +254,9 @@ class AddedToken:
     def __getstate__(self):
         return self.__dict__
 
+    def __str__(self):
+        return self.content
+
 
 class Trie:
     """
@@ -1216,7 +1219,7 @@ def save_vocabulary(filepath, vocab):
 
     def __getattr__(self, name):
         if name.endswith('_token'):
-            return self.special_tokens_map[name]
+            return str(self.special_tokens_map[name])
         elif name.endswith('_token_id'):
             return self._convert_token_to_id(self.special_tokens_map[name[:-3]])
         raise AttributeError("'{}' object has no attribute '{}'".format(


Test patch:
Repository: PaddlePaddle/PaddleNLP

Issue:
【BUG】clue tnews 在不指定 split 参数的情况下，load_datasets 会报错
clue tnews 在不指定 split 参数的情况下，load_datasets 会报错， 如图
<img width="1037" alt="05d0aeb5abfdcda9138422002998d26f" src="https://user-images.githubusercontent.com/16911935/162115458-a164d824-7fe3-4ac3-9973-319a71d07775.png">



Discussion:
这个定位到tnews的splits有个test1.0的问题，这个问题@smallv0221 已经在看了

Resolving patch:
diff --git a/paddlenlp/datasets/dataset.py b/paddlenlp/datasets/dataset.py
--- a/paddlenlp/datasets/dataset.py
+++ b/paddlenlp/datasets/dataset.py
@@ -63,20 +63,26 @@ def load_from_ppnlp(path, *args, **kwargs):
 
 class DatasetTuple:
     def __init__(self, splits):
-        self.tuple_cls = namedtuple('datasets', splits)
+        self.identifier_map, identifiers = self._gen_identifier_map(splits)
+        self.tuple_cls = namedtuple('datasets', identifiers)
         self.tuple = self.tuple_cls(* [None for _ in splits])
 
     def __getitem__(self, key):
         if isinstance(key, (int, slice)):
             return self.tuple[key]
         if isinstance(key, str):
-            return getattr(self.tuple, key)
-
-    def __repr__(self):
-        return self.tuple.__repr__()
+            return getattr(self.tuple, self.identifier_map[key])
 
     def __setitem__(self, key, value):
-        self.tuple = self.tuple._replace(**{key: value})
+        self.tuple = self.tuple._replace(**{self.identifier_map[key]: value})
+
+    def _gen_identifier_map(self, splits):
+        identifier_map = {}
+        identifiers = []
+        for i in range(len(splits)):
+            identifiers.append('splits_' + str(i))
+            identifier_map[splits[i]] = 'splits_' + str(i)
+        return identifier_map, identifiers
 
     def __len__(self):
         return len(self.tuple)


Test patch:
Repository: PaddlePaddle/PaddleNLP

Issue:
ann_utils 当 output_emb_size = 0 时报错
欢迎您反馈PaddleNLP使用问题，非常感谢您对PaddleNLP的贡献！
在留下您的问题时，辛苦您同步提供如下信息：
- 版本、环境信息
1）PaddleNLP和PaddlePaddle版本：请提供您的PaddleNLP和PaddlePaddle版本号，例如PaddleNLP 2.0.4，PaddlePaddle2.1.1
2）系统环境：请您描述系统类型，例如Linux/Windows/MacOS/，python版本
- 复现信息：如为报错，请给出复现环境、复现步骤



Discussion:
只有当 output_emb_size > 0 的时候，才表示用户自己进行了降维度。
否则，dim 应该设置为预训练模型自身的 hidden_size。
![image](https://user-images.githubusercontent.com/6829601/167383808-c9cc93f3-635a-469e-bcab-1c1e7ad01c04.png)


Resolving patch:
diff --git a/applications/neural_search/recall/in_batch_negative/ann_util.py b/applications/neural_search/recall/in_batch_negative/ann_util.py
--- a/applications/neural_search/recall/in_batch_negative/ann_util.py
+++ b/applications/neural_search/recall/in_batch_negative/ann_util.py
@@ -21,7 +21,9 @@
 
 def build_index(args, data_loader, model):
 
-    index = hnswlib.Index(space='ip', dim=args.output_emb_size)
+    index = hnswlib.Index(
+        space='ip',
+        dim=args.output_emb_size if args.output_emb_size > 0 else 768)
 
     # Initializing index
     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded
diff --git a/applications/question_answering/faq_finance/ann_util.py b/applications/question_answering/faq_finance/ann_util.py
--- a/applications/question_answering/faq_finance/ann_util.py
+++ b/applications/question_answering/faq_finance/ann_util.py
@@ -19,7 +19,9 @@
 
 def build_index(args, data_loader, model):
 
-    index = hnswlib.Index(space='ip', dim=args.output_emb_size)
+    index = hnswlib.Index(
+        space='ip',
+        dim=args.output_emb_size if args.output_emb_size > 0 else 768)
 
     # Initializing index
     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded
diff --git a/applications/question_answering/faq_system/ann_util.py b/applications/question_answering/faq_system/ann_util.py
--- a/applications/question_answering/faq_system/ann_util.py
+++ b/applications/question_answering/faq_system/ann_util.py
@@ -19,7 +19,9 @@
 
 def build_index(args, data_loader, model):
 
-    index = hnswlib.Index(space='ip', dim=args.output_emb_size)
+    index = hnswlib.Index(
+        space='ip',
+        dim=args.output_emb_size if args.output_emb_size > 0 else 768)
 
     # Initializing index
     # max_elements - the maximum number of elements (capacity). Will throw an exception if exceeded


Test patch:
Repository: PaddlePaddle/PaddleNLP

Issue:
semantic-search用CPU跑时出现ValueError: (InvalidArgument) Variable value (input) of OP(fluid.layers.embedding) expected >= 0 and < 513
centos：7.5
anaconda->python：3.8.5
PaddlePaddle2.3.0（CPU）

尝试搭建：https://github.com/PaddlePaddle/PaddleNLP/tree/develop/applications/experimental/pipelines/examples/semantic-search中的CPU版本时：
unset CUDA_VISIBLE_DEVICES
python examples/semantic-search/semantic_search_example.py --device cpu，之后，出现如下错误：
![微信截图_20220704214439](https://user-images.githubusercontent.com/100854336/177167449-a3d31416-3886-439d-8e11-3c5d79394f7f.png)

请问什么情况？如何处理？谢谢



Discussion:
单看报错信息是因为输入的 id 超过了 embedding 所能表示的词表最大的大小，可以确认下输入的 id 是否都在 embedding 所能表示的 id 范围内。

Resolving patch:
diff --git a/applications/experimental/pipelines/pipelines/data_handler/processor.py b/applications/experimental/pipelines/pipelines/data_handler/processor.py
--- a/applications/experimental/pipelines/pipelines/data_handler/processor.py
+++ b/applications/experimental/pipelines/pipelines/data_handler/processor.py
@@ -861,7 +861,8 @@ def _convert_queries(self, baskets: List[SampleBasket]):
             if "query" in basket.raw:
                 try:
                     query = self._normalize_question(basket.raw["query"])
-                    query_inputs = self.query_tokenizer(query)
+                    query_inputs = self.query_tokenizer(
+                        text=query, max_seq_len=self.max_seq_len_query)
                     tokenized_query = self.query_tokenizer.convert_ids_to_tokens(
                         query_inputs["input_ids"])
 
@@ -940,7 +941,8 @@ def _convert_contexts(self, baskets: List[SampleBasket]):
                         len(all_ctx))
 
                     # [text] -> tokenize -> id
-                    ctx_inputs = self.passage_tokenizer(all_ctx[0])
+                    ctx_inputs = self.passage_tokenizer(
+                        text=all_ctx[0], max_seq_len=self.max_seq_len_passage)
 
                     # get tokens in string format
                     tokenized_passage = [


Test patch:
Repository: PaddlePaddle/PaddleNLP

Issue:
DuEE sequence_labeling.py导包错误
欢迎您反馈PaddleNLP使用问题，非常感谢您对PaddleNLP的贡献！
在留下您的问题时，辛苦您同步提供如下信息：
- 版本、环境信息
1）PaddleNLP和PaddlePaddle版本：PaddleNLP 2.3.4，PaddlePaddle2.1.1
2）系统环境：Linux
- 报错信息：![image](https://user-images.githubusercontent.com/50394665/180337159-dbaf45fd-135e-4169-b8ba-62237bf167e4.png)
- 修改建议：改成`AutoModelForTokenClassification`

https://github.com/PaddlePaddle/PaddleNLP/blob/4d18e4d648fadde639575b6de3e0b3c3a38bb906/examples/information_extraction/DuEE/sequence_labeling.py#L30

https://github.com/PaddlePaddle/PaddleNLP/blob/4d18e4d648fadde639575b6de3e0b3c3a38bb906/tests/ernie_information_extraction/export_model.py#L37-L38


Discussion:
Great post for this issue, I think you can fire a pr to fix it. How do you think about?

Resolving patch:
diff --git a/examples/information_extraction/DuEE/sequence_labeling.py b/examples/information_extraction/DuEE/sequence_labeling.py
--- a/examples/information_extraction/DuEE/sequence_labeling.py
+++ b/examples/information_extraction/DuEE/sequence_labeling.py
@@ -27,7 +27,7 @@
 import paddle.nn.functional as F
 from paddlenlp.data import Stack, Tuple, Pad
 from paddlenlp.transformers import LinearDecayWithWarmup
-from paddlenlp.transformers import AutoModelTokenClassification, AutoTokenizer
+from paddlenlp.transformers import AutoModelForTokenClassification, AutoTokenizer
 from paddlenlp.metrics import ChunkEvaluator
 from utils import read_by_lines, write_by_lines, load_dict
 


Test patch:
diff --git a/tests/ernie_information_extraction/export_model.py b/tests/ernie_information_extraction/export_model.py
--- a/tests/ernie_information_extraction/export_model.py
+++ b/tests/ernie_information_extraction/export_model.py
@@ -34,7 +34,7 @@
     # The number of labels should be in accordance with the training dataset.
     label_vocab = load_dict(os.path.join(args.data_dir, 'tag.dic'))
 
-    model = AutoModelTokenClassification.from_pretrained(
+    model = AutoModelForTokenClassification.from_pretrained(
         "ernie-3.0-medium-zh", num_classes=len(label_vocab))
 
     if args.params_path and os.path.isfile(args.params_path):
Repository: PaddlePaddle/PaddleNLP

Issue:
Support more model outputs for BERT/ERNIE/RoBERTa
<!-- Demo: https://github.com/PaddlePaddle/PaddleNLP/pull/26 -->
### PR types
<!-- One of [ New features | Bug fixes | Function optimization | Performance optimization | Breaking changes | Others ] -->
New features
### PR changes
<!-- One of [ Models | APIs | Docs | Others ] -->
APIs
### Description
<!-- Describe what this PR does -->
#2583 is reverted because of compatibility, this PR fix and then push it again.


Discussion:

Resolving patch:
diff --git a/paddlenlp/transformers/model_outputs.py b/paddlenlp/transformers/model_outputs.py
--- a/paddlenlp/transformers/model_outputs.py
+++ b/paddlenlp/transformers/model_outputs.py
@@ -91,7 +91,7 @@ def _transformer_encoder_fwd(self,
         if output_attentions:
             all_attentions.append(outputs[-1])
         if cache is not None:
-            new_caches.append(outputs[1])
+            new_caches.append(outputs[0])
 
     if self.norm is not None:
         output = self.norm(output)


Test patch: