mteb/LocBenchRR huggingface dataset:
Slow sending messages with data channels
When I try to send 10 messages per second in datachannel I observe that messages are sent much slower, at 5 message per second.

I did a docker example to reproduce: https://gist.github.com/le-chat/844272e8d0f91dcbb61ba98ca635cd6b

From logs:
```
08:32:51.808407 sent №100, buffered 1750, queues: _sent 1, _outbound 0, _data_channel 50; _flight_size 35
08:33:01.895772 sent №200, buffered 3600, queues: _sent 1, _outbound 1, _data_channel 100; _flight_size 35
08:33:01.982040, 100 received: №99 at 2022-09-28 08:32:51.808277 (buffered 0)
08:33:11.973638 sent №300, buffered 5436, queues: _sent 1, _outbound 0, _data_channel 151; _flight_size 36
08:33:22.056580 sent №400, buffered 7200, queues: _sent 1, _outbound 1, _data_channel 200; _flight_size 36
08:33:22.160382, 200 received: №199 at 2022-09-28 08:33:01.895633 (buffered 0)
08:33:32.146720 sent №500, buffered 9036, queues: _sent 1, _outbound 0, _data_channel 251; _flight_size 36
08:33:42.228501 sent №600, buffered 10836, queues: _sent 1, _outbound 1, _data_channel 301; _flight_size 36
08:33:42.523312, 300 received: №299 at 2022-09-28 08:33:11.973609 (buffered 0)
08:33:52.314587 sent №700, buffered 12672, queues: _sent 1, _outbound 0, _data_channel 352; _flight_size 36
```
we can see messages are stuck on `_data_channel_queue`, though flight size is low and both `_outbound_queue` and `_sent_queue` are almost empty.

It looks like the second check in https://github.com/aiortc/aiortc/blob/1.3.2/src/aiortc/rtcsctptransport.py#L1651 erroneously breaks the loop in the very beginning and is redundant.
[Bug]: plt.hist takes significantly more time with torch and jax arrays
### Bug summary

Hi,

Time taken to plot `plt.hist` directly on `jax` or `torch` arrays is significantly more than **combined time taken to first convert them to `numpy` and then using `plt.hist`**. Shouldn't `matplotlib` internally convert them to `numpy` arrays before plotting?

To reproduce the bug, directly run the following snippet on Google Colab.

### Code for reproduction

```python
from time import time
import numpy as np

import torch

import jax
import jax.random as jr
import jax.numpy as jnp

import matplotlib.pyplot as plt

jax_array = jr.normal(jr.PRNGKey(0), (1000, 150))
torch_array = torch.randn(1000, 150)

def plot_hist(array):
    init = time()
    plt.figure()
    plt.hist(array)
    print(f"Time to plot: {time() - init:.2f} s")
    plt.show()
    
plot_hist(jax_array.ravel())
plot_hist(torch_array.ravel())
plot_hist(np.array(jax_array.ravel()))
plot_hist(np.array(torch_array.ravel()))
```


### Actual outcome

Time to plot: 4.19 s
![image](https://github.com/matplotlib/matplotlib/assets/59758528/29f7e6e6-4c91-4801-8472-24eaaae591cc)

Time to plot: 2.61 s
![image](https://github.com/matplotlib/matplotlib/assets/59758528/cf0b12c2-fabf-45fb-8e59-89206bb6ed68)

Time to plot: 0.03 s
![image](https://github.com/matplotlib/matplotlib/assets/59758528/9a39702e-b0ba-48ba-9ec6-5731f3d4b372)

Time to plot: 0.04 s
![image](https://github.com/matplotlib/matplotlib/assets/59758528/5e99fa5c-c614-4422-9b98-ff62fa3d3563)


### Expected outcome

Time to plot: 0.03 s

Time to plot: 0.04 s

Time to plot: 0.03 s

Time to plot: 0.04 s

### Additional information

> What are the conditions under which this bug happens? input parameters, edge cases, etc?

It is happening with all kinds of shapes.

> Has this worked in earlier versions?

Tested with default colab `matplotlib` version 3.7.1 and also with 3.6.3.

> Do you know why this bug is happening?

Not exactly sure.

> Do you maybe even know a fix?

Maybe convert any python object to a `numpy` array before plotting?

### Operating system

Ubuntu 20.04.5 LTS

### Matplotlib Version

3.7.1

### Matplotlib Backend

module://matplotlib_inline.backend_inline

### Python version

3.10.11

### Jupyter version

6.4.8

### Installation

None
“Default” is a keyword, but it was converted into a string value after translation.
```
Python 3.13.0 | packaged by Anaconda, Inc. | (main, Oct  7 2024, 16:25:56) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlglot
>>> sqlglot.transpile("insert into t(i) values (default)", read="mysql", write="duckdb")[0]
'INSERT INTO t (i) VALUES ("default")'
```


“Default” is a keyword, but it was converted into a string value after translation.
Detect systemd service start/stop events to replace hard coded delays
The original implementation used conservative hard coded delays to sync fapolicyd starts/stops {via dbus} with the invocation and termination of the Profiling target executables. The focus at that time was correct functionality however it's time for a quick first pass optimization to improve the end-user experience. We can totally eliminate or minimize the hard coded delays if we wait/block while monitoring systemd's journal for fapolicyd events of interest e.g. wait after starting the fapolicyd profiling instance until journalctl reports that the fapd_profiling.service has started, then invoke the profiling target executable. 

This is not a blocker to the systemd profiler work, and will be done in parallel.
Profiler testing over Rhel86 observations
## Summary ##

1. Directly invoking `fapolicyd --debug --permissive` outputs to stderr not stdout wrt Profiler Tool Target Output pane section titles.
2. Specifying a `user` w/o specifying a `working directory` sets the working directory to the `root` user's HOME.
3. Specifying only the basename of a target binary while specifying a working directory, and a `PATH` env var results in no output in the Profiler Tool Target Output pane

Please see https://github.com/ctc-oss/fapolicy-analyzer/pull/770 for details and images.
IndexError: list index out of range in add_word_timestamps function
Hi,
 I found a rare condition, with a specific wav file, specific language and prompt, when I try to transcribe with word_timestamps=True, there is a list index out of range error in add_word_timestamps function:

```
  File "/usr/local/src/transcriber/lib/python3.11/site-packages/faster_whisper/transcribe.py", line 1574, in add_word_timestamps
    median_duration, max_duration = median_max_durations[segment_idx]
                                    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
IndexError: list index out of range
```

It seems in the median_max_durations list we have less elements than in the segments list.

I'm using large-v3-turbo model with these transcibe settings:
```
segments, _ = asr_model.transcribe(audio_to_analize, language="fr", condition_on_previous_text=False, initial_prompt="Free", task='transcribe', word_timestamps=True, suppress_tokens=[-1, 12], beam_size=5) 
segments = list(segments)  # The transcription will actually run here.
```

As I see, the median_max_durations is populated from alignments, so something is maybe wrong there? If i change language or prompt, or use another sound file, then there is no issue.

Thank you
[BUG] Security Vulnerability
Please check it here https://huntr.com/bounties/e3d7a994-bfd6-4772-ac9b-9aee1aa16a5f/
Improving IsolationForest predict time
### Discussed in https://github.com/scikit-learn/scikit-learn/discussions/25142

<div type='discussions-op-text'>

<sup>Originally posted by **fsiola** December  8, 2022</sup>
Hi, 

When using [IsolationForest predict](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/ensemble/_iforest.py#L341), we go down the path to [_compute_score_samples](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/ensemble/_iforest.py#L464). This executes `tree.apply` and `tree.decision_path`. Both calls will iterate over the tree for each sample in `X`. So we are evaluation the tree 2 times.

`tree.decision_path` returns a [csr matrix](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/tree/_tree.pyx#L962) containing the nodes indexes that were visited in the tree, to them later just have the [count of indexes summed later](https://github.com/scikit-learn/scikit-learn/blob/main/sklearn/ensemble/_iforest.py#L488).

We can save time in predict if instead of calling `tree.decision_path`, a `tree.decision_path_length` that return an integer exists. But that would required  changing the `_tree.pyx` file. Some changes could also avoid the call to `tree.apply`, avoiding 2 times iterating on the tree.

Is this something that would be accepted as PR, or changing the tree cpython files for this would not be accepted?</div>
Streaming audio to and from AVS
While waiting for Echo push notifications for a year now, I did a small extension of https://github.com/sammachin/AlexaPi to essentially enable push by submitting pre-recorded questions to AVS when triggered by an event. It worked pretty well, but the problem I faced is that response time is slow, typically 5 or even 10 seconds. I believe the approach used in https://github.com/alexa/alexa-avs-sample-app address those performance issues by streaming the audio to AVS (and maybe also streaming the response back, I asked that question, hoping to hear back from them). The approach is explained here - https://forums.developer.amazon.com/questions/10982/avs-takes-too-much-time-to-respond.html. 

Is streaming audio to and from AVS in the plan for the new evolving AlexaPi?
DataCollatorForChatML unexpected generation prompt
### System Info

- Platform: macOS-15.1.1-arm64-arm-64bit
- Python version: 3.10.15
- PyTorch version: 2.4.1
- CUDA device(s): not available
- Transformers version: 4.45.2
- Accelerate version: 1.0.1
- Accelerate config: not found
- Datasets version: 3.0.1
- HF Hub version: 0.25.2
- TRL version: 0.12.2
- bitsandbytes version: not installed
- DeepSpeed version: 0.15.2
- Diffusers version: 0.30.3
- Liger-Kernel version: not installed
- LLM-Blender version: not installed
- OpenAI version: 1.51.2
- PEFT version: 0.13.2

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder
- [X] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoTokenizer
from trl.trainer.utils import DataCollatorForChatML

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B-Instruct")
tokenizer.pad_token_id = tokenizer.eos_token_id
tokenizer.pad_token = tokenizer.eos_token

data_collator = DataCollatorForChatML(tokenizer)
examples = [
    {
        "messages": [
            {"role": "system", "content": "You are a professional translator."},
            {"role": "user", "content": "Hello!"},
            {"role": "assistant", "content": "Hi there! How can I help you today?"},
        ],
    },
]
batch = data_collator(examples)

print(tokenizer.decode(batch["input_ids"][0]))

label = batch["labels"][0]
label[label == -100] = tokenizer.eos_token_id
print(tokenizer.decode(label))
```

outputs:

```


system


Cutting Knowledge Date: December 2023
Today Date: 07 Dec 2024

You are a professional translator.

user


Hello!

assistant


Hi there! How can I help you today?

assistant



!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Hi there! How can I help you today?

assistant

```


### Expected behavior

When processing instruction tuning, the model is not expected to generate `
assistant
` after `
`. The correct model response should be `Hi there! How can I help you today?
`.

We should change `trl/trainer/utils.py L276:277`

```python
formatted_message = self.tokenizer.apply_chat_template(
    # message, tokenize=False, add_generation_prompt=True,
    message, tokenize=False, add_generation_prompt=False
)
```

### Checklist

- [X] I have checked that my issue isn't already filed (see [open issues](https://github.com/huggingface/trl/issues?q=is%3Aissue))
- [X] I have included my system information
- [X] Any code provided is minimal, complete, and reproducible ([more on MREs](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks))
- [X] Any code provided is properly formatted in code blocks, (no screenshot, [more on code blocks](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks))
- [X] Any traceback provided is complete
TSQL fails on parsing DATEDIFF function with literals
Hello,
I just discovered that this code generates an error while parsing DATEDIFF second argument:
```
import sqlglot
statement = """
SELECT DATEADD(DAY, DATEDIFF(DAY, -3, CREATION_TIME_NEW), '08:00:00')
FROM y
"""

ast = sqlglot.parse_one(statement, 'tsql')
print(ast.sql('tsql'))
```

Error:
```
Python312\Lib\site-packages\sqlglot\dialects\tsql.py:235, in _build_date_delta.<locals>._builder(args)

    232 if start_date and start_date.is_number:
    233     # Numeric types are valid DATETIME values
    234     if start_date.is_int:
--> 235         adds = DEFAULT_START_DATE + datetime.timedelta(days=int(start_date.this))
    236         start_date = exp.Literal.string(adds.strftime("%F"))
    237     else:
    238         # We currently don't handle float values, i.e. they're not converted to equivalent DATETIMEs.
    239         # This is not a problem when generating T-SQL code, it is when transpiling to other dialects.

TypeError: int() argument must be a string, a bytes-like object or a real number, not 'Literal'
```
start_date at this point is:
```
Neg(
  this=Literal(this=3, is_string=False))
```

So `int(start_date.this)` fails.
[ENH] Change detection
### 😕 Addressed Problem

If i change something like the text and try to republish it does not work if the publish interval is not met

### 💡 Suggested Solution

Save a hash from the config file to have change detection
#### Problems:
- Does not detect changes in images if they have the same name as before

### 🔎 Alternatives

_No response_

### 🗺️ Additional Context

_No response_

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct
Identification of images in docx
Hello everyone,
im working on docx document type with docling.
My first test was very poor.
I have a docx file which is only a testfile and this contains one image.
The standard procedure doesnt even recognise the image in the file...
Standard is this:

from docling.document_converter import DocumentConverter

source = "data/example_small.docx"
converter = DocumentConverter()
result = converter.convert(source)
for item in result.document:
print(item)

The example file is contained here!
Am im wrong with the code or is this a bug?

Greetings
[example_small.docx](https://github.com/user-attachments/files/17662530/example_small.docx)
[word_sample.docx](https://github.com/user-attachments/files/17662531/word_sample.docx)
[ENH] `polars` schema checks - address performance warnings
The current schema checks for lazy `polars` based data types raise performance warnings, e.g.,

```
sktime/datatypes/tests/test_check.py::test_check_metadata_inference[Table-polars_lazy_table-fixture:1]
  /home/runner/work/sktime/sktime/sktime/datatypes/_adapter/polars.py:234: PerformanceWarning: Determining the width of a LazyFrame requires resolving its schema, which is a potentially expensive operation. Use `LazyFrame.collect_schema().len()` to get the width without this warning.
    metadata["n_features"] = obj.width - len(index_cols)
```

These should be addressed.

The tests to execute to check whether these warnings persist are those in the `datatypes` module - these are automatically executed for a change in the impacted file, on remote CI.
make buffer size of SampleSplitter independently settable for each subsequent object of the processing chain
Currently, the `buffer_size` attribute of class `SampleSplitter` is of type `Int`. In fact, the SampleSplitter creates a block buffer of the same size for each subsequent object of the processing chain that has the `SampleSplitter` as a source. This is undesirable if one of these objects wants to obtain only the newest block from the buffer, which could be easily and efficiently achieved using `buffer_size=1` only for this object. Thus, `buffer_size` should also allow setting a `Dict` as a mapping between the calling object and the `buffer_size`. Alternatively, one could add a `return_policy` attribute to determine if the last or the first block is taken from the buffer for each calling object.
Unify FSLeyes commands for multiclass DeepSeg models
FSLeyes commands outputted by `sct_deepseg -i t2.nii.gz -task seg_sc_lesion_t2w_sci` differ from the FSLeyes command in [our tutorial](https://github.com/spinalcordtoolbox/sct_tutorial_data/blob/8b91e72c6575cbf40c5a66898fe3a13d9c26790c/single_subject/batch_single_subject.sh#L128):

<img width="1406" alt="Screenshot 2024-11-12 at 11 16 44 AM" src="https://github.com/user-attachments/assets/5204f40c-6d93-4147-adc1-252e99ff292b">

We could consider unifying the commands. I personally like the command from our tutorial, which opens both spinal cord and lesion segmentation in a single FSLeyes window:

```console
fsleyes t2.nii.gz -cm greyscale t2_sc_seg.nii.gz -cm red -a 70.0 t2_lesion_seg.nii.gz -cm blue-lightblue -a 70.0 &
```
[Postgres] Parse error for XMLElement
Postgres XMLElement returns parse error:
```
SELECT xmlelement(name foo);
# > <foo/>
```

**Fully reproducible code snippet**
```
from sqlglot import parse_one
from sqlglot.optimizer.qualify import qualify

qualify(parse_one(sql = """
 SELECT xmlelement(name "foo")
""", dialect="postgres"), schema=None)

# > sqlglot.errors.ParseError: Expecting ). Line 2, Col: 29.
```

**Official Documentation**
https://www.postgresql.org/docs/current/functions-xml.html#FUNCTIONS-PRODUCING-XML-XMLELEMENT
Allow using a subquery in QuerySet.filter()
Description
		
(last modified by MikiSoft)		
The following function is used for filtering by generic relation (and also by one column in the model where it is) which isn't natively supported by Django.
APP_LABEL = os.path.basename(os.path.dirname(__file__))
def generic_rel_filter(model, target, column, id):
	return model.objects.extra(where=['''
		{app_label}_{model}.id in (select object_id
		from {app_label}_{target}
		where content_type_id = (select id from django_content_type where model = '{model}')
			and {column} = {id})'''.format(app_label=APP_LABEL, model=model.__name__.lower(), target=target, column=column, id=id)])
Example: If I have Event and Like model, and the second one has generic relation to the first one (i.e. it has content_type, object_id and content_object fields), then if I want to get all events which current user liked, I would just make this call in a view: generic_rel_filter(Event, 'like', 'person', self.request.user.pk)
Note that this function isn't intended to be used with user specified parameters, otherwise it's prone to SQL injection attacks.
P.S. It can be done with ORM but then it would go with three queries, which is much slower than the method above (which uses only one query to do the same): Event.objects.filter(pk__in=Like.objects.filter(content_type=ContentType.objects.get(model='event'), person=self.request.user).values_list('object_id', flat=True))
Allow using a subquery in QuerySet.filter()
Description
		
(last modified by MikiSoft)		
The following function is used for filtering by generic relation (and also by one column in the model where it is) which isn't natively supported by Django.
APP_LABEL = os.path.basename(os.path.dirname(__file__))
def generic_rel_filter(model, target, column, id):
	return model.objects.extra(where=['''
		{app_label}_{model}.id in (select object_id
		from {app_label}_{target}
		where content_type_id = (select id from django_content_type where model = '{model}')
			and {column} = {id})'''.format(app_label=APP_LABEL, model=model.__name__.lower(), target=target, column=column, id=id)])
Example: If I have Event and Like model, and the second one has generic relation to the first one (i.e. it has content_type, object_id and content_object fields), then if I want to get all events which current user liked, I would just make this call in a view: generic_rel_filter(Event, 'like', 'person', self.request.user.pk)
Note that this function isn't intended to be used with user specified parameters, otherwise it's prone to SQL injection attacks.
P.S. It can be done with ORM but then it would go with three queries, which is much slower than the method above (which uses only one query to do the same): Event.objects.filter(pk__in=Like.objects.filter(content_type=ContentType.objects.get(model='event'), person=self.request.user).values_list('object_id', flat=True))
feat: constraints files
the skill installer via bus events (eg, ggwave) should use the constraints files from [ovos-releases](https://github.com/OpenVoiceOS/ovos-releases)

it currently has a [hardcoded path](https://github.com/OpenVoiceOS/ovos-core/blob/dev/ovos_core/skill_installer.py#L25) that was inherited from msm, this should instead be made configurable and default to fetch from the releases repo

we can also allow users to change between stable and testing if desired, eg, via a companion skill or cli util
feat: constraints files
the skill installer via bus events (eg, ggwave) should use the constraints files from [ovos-releases](https://github.com/OpenVoiceOS/ovos-releases)

it currently has a [hardcoded path](https://github.com/OpenVoiceOS/ovos-core/blob/dev/ovos_core/skill_installer.py#L25) that was inherited from msm, this should instead be made configurable and default to fetch from the releases repo

we can also allow users to change between stable and testing if desired, eg, via a companion skill or cli util
Speed up from O(n^2) iteration over all left/right primer hits in `OffTargetDetector._to_amplicons`
`OffTargetDetector._to_amplicons` uses `itertools.product` over the left and right primer hits and evaluates each of them in a loop to identify valid left/right hit combinations for the pair.

Suggestion:

1. Get hits by primer sequence from `mappings_of`
2. Split into left and right hits
3. Group hits by refname
4. For each refname, split into left +, right -, left -, right+ hits
5. Build amplicons from left + and right - (amplicon strand +), left - and right + (amplicon strand -)

Building amplicons - dealing with only positive & negative in correct orientation and known to be on the same reference, with known amplicon strand
1. `itertools.product` over positive & negative primer hits
2. Check product size against min/max amplicon size range
3. If in range, return the product hit as positive start to negative end with the known reference name and amplicon strand
Extend CLI to make GUI calls
It would be helpful to be able to run some common GUI functions from the CLI.  This could help with some manual test development as well as provide an interface that can be used by test frameworks that rely on running commands remotely.

At a minimum, I think we need cli access to the following GUI functions:
- assert_text
- click_on
- check_home_screen
Extend CLI to make GUI calls
It would be helpful to be able to run some common GUI functions from the CLI.  This could help with some manual test development as well as provide an interface that can be used by test frameworks that rely on running commands remotely.

At a minimum, I think we need cli access to the following GUI functions:
- assert_text
- click_on
- check_home_screen