mteb/LocBenchRR huggingface dataset:
NanoBeirEvaluator returns a mix of floats and numpy.float64 in the results.
I was writing test cases and noticed that the results were returned as a mix of floats and np floats. Should we aim to convert these to floats as well?

```
{'NanoQuoraRetrieval_cosine_accuracy@1': 0.1,
 'NanoQuoraRetrieval_cosine_accuracy@3': 0.18,
 'NanoQuoraRetrieval_cosine_accuracy@5': 0.24,
 'NanoQuoraRetrieval_cosine_accuracy@10': 0.28,
 'NanoQuoraRetrieval_cosine_precision@1': np.float64(0.1),
 'NanoQuoraRetrieval_cosine_precision@3': np.float64(0.06666666666666667),
 'NanoQuoraRetrieval_cosine_precision@5': np.float64(0.052000000000000005),
 'NanoQuoraRetrieval_cosine_precision@10': np.float64(0.034),
 'NanoQuoraRetrieval_cosine_recall@1': np.float64(0.09),
 'NanoQuoraRetrieval_cosine_recall@3': np.float64(0.18),
 'NanoQuoraRetrieval_cosine_recall@5': np.float64(0.21666666666666665),
 'NanoQuoraRetrieval_cosine_recall@10': np.float64(0.27),
 'NanoQuoraRetrieval_cosine_ndcg@10': np.float64(0.1759841616710832),
 'NanoQuoraRetrieval_cosine_mrr@10': 0.14733333333333334,
 'NanoQuoraRetrieval_cosine_map@100': np.float64(0.15211020997551106),
 'NanoMSMARCO_cosine_accuracy@1': 0.0,
 'NanoMSMARCO_cosine_accuracy@3': 0.02,
 'NanoMSMARCO_cosine_accuracy@5': 0.04,
 'NanoMSMARCO_cosine_accuracy@10': 0.06,
 'NanoMSMARCO_cosine_precision@1': np.float64(0.0),
 'NanoMSMARCO_cosine_precision@3': np.float64(0.006666666666666666),
 'NanoMSMARCO_cosine_precision@5': np.float64(0.008),
 'NanoMSMARCO_cosine_precision@10': np.float64(0.006000000000000001),
 'NanoMSMARCO_cosine_recall@1': np.float64(0.0),
 'NanoMSMARCO_cosine_recall@3': np.float64(0.02),
 'NanoMSMARCO_cosine_recall@5': np.float64(0.04),
 'NanoMSMARCO_cosine_recall@10': np.float64(0.06),
 'NanoMSMARCO_cosine_ndcg@10': np.float64(0.02702231788278665),
 'NanoMSMARCO_cosine_mrr@10': 0.016857142857142855,
 'NanoMSMARCO_cosine_map@100': np.float64(0.019069983940167533),
 'NanoBEIR_mean_cosine_accuracy@1': np.float64(0.05),
 'NanoBEIR_mean_cosine_accuracy@3': np.float64(0.09999999999999999),
 'NanoBEIR_mean_cosine_accuracy@5': np.float64(0.13999999999999999),
 'NanoBEIR_mean_cosine_accuracy@10': np.float64(0.17),
 'NanoBEIR_mean_cosine_precision@1': np.float64(0.05),
 'NanoBEIR_mean_cosine_precision@3': np.float64(0.03666666666666667),
 'NanoBEIR_mean_cosine_precision@5': np.float64(0.030000000000000002),
 'NanoBEIR_mean_cosine_precision@10': np.float64(0.02),
 'NanoBEIR_mean_cosine_recall@1': np.float64(0.045),
 'NanoBEIR_mean_cosine_recall@3': np.float64(0.09999999999999999),
 'NanoBEIR_mean_cosine_recall@5': np.float64(0.12833333333333333),
 'NanoBEIR_mean_cosine_recall@10': np.float64(0.165),
 'NanoBEIR_mean_cosine_ndcg@10': np.float64(0.10150323977693491),
 'NanoBEIR_mean_cosine_mrr@10': np.float64(0.0820952380952381),
 'NanoBEIR_mean_cosine_map@100': np.float64(0.0855900969578393)}
```

Who can help?
@tomaarsen
Fix code scanning alert - SQL doesn't use :symbols
Tracking issue for:
- [x] https://github.com/jobatabs/textec/security/code-scanning/3
Port `UnitarySynthesis` to Rust
This issue tracks porting the `UnitarySynthesis` pass to rust as part of the #12208 epic. This pass in particular will still always require a fairly large python component, as the unitary synthesis plugin interface necessitates a Python execution mode. But when the a plugin is not specified we shouldn't need to use python at all. The other option is we can call Python from rust to pass a unitary array to the plugins and then operate exclusively from rust otherwise as well.
Poor performance on Connection.recv() with large values of bufsiz.
Discovered while investigating kennethreitz/requests#3729.

When allocating a buffer using CFFI's `new` method (as `Connection.recv` does via `_ffi.new("char[]", bufsiz)`, CFFI kindly zeroes that buffer for us. That means this call has a performance cost that is linear in `bufsiz`. This can lead to nasty negative behaviour if really large values of `bufsiz` are used, e.g. 1GB. 

We don't need a zeroed buffer for this: `SSL_read` returns a length, so we know exactly how many bytes to copy out. Unfortunately though, CFFI doesn't provide any method to obtain a non-zeroed buffer, at least not as far as I can see.

Instead, a coping method might be to silently clamp the maximum value of `bufsiz` to the return value of `self._socket.getsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF)`. This is the size of the socket receive buffer, and therefore represents the maximum amount of data the kernel can possibly return in one `recv` call. On my OS X machine, this is ~129 kB by default. An alternative option is to clamp to 16kB, the maximum TLS record size, or to clamp to the nearest multiple of 16kB near `SO_RCVBUF`. Either of these will greatly shrink the allocations. Finally, we can just pick an arbitrary value: a few hundred KB is probably safe.

I should note, however, that this coping method *still* imposes a substantial cost when downloading data that is not incurred by the standard library's `ssl` module. Specifically, CFFI will require that we zero at least as many bytes as the size of the download. This is *all* wasted effort, and for multi-gigabyte files represents really quite a substantial waste of time. It would be best to do both of these things: to shrink the size of our allocations *and* to try to avoid zeroing data when we don't need to.
GradientBoostingClassifier.fit accepts sparse X, but .predict does not
I have a sparse dataset that is too large for main memory if I call `X.todense()`.  If I understand correctly, `GradientBoostingClassifier.fit` will accept my sparse `X`, but it is not currently possible to use `GradientBoostingClassifier.predict` on the results.  It would be great if that were not the case.

Here is a minimal example of the issue:

```
from scipy import sparse
from sklearn.datasets.samples_generator import make_classification
from sklearn.ensemble import GradientBoostingClassifier

X, y = make_classification(n_samples=20, n_features=5, random_state=0)
X_sp = sparse.coo_matrix(X)

clf = GradientBoostingClassifier()
clf.fit(X,y)
clf.predict(X)  # works

clf.fit(X_sp, y)  # works
clf.predict(X_sp)  # fails with TypeError: A sparse matrix was passed, but dense data is required.
```
S3 get_object_attributes fails for GLACIER objects
Environment:

Python 3.11
Windows 11 Pro
moto==5.0.20
pytest==8.3.3

My failing test

   ```
 def test_get_glacier_object_attributes(self) -> None:
        s3 = client(
            "s3",
            region_name="eu-west-1"
        )
        s3.create_bucket(
            Bucket="test-bucket",
            CreateBucketConfiguration={"LocationConstraint": "eu-west-1"},
        )

        csv_body = "field_1,field_2\n1,2"

        s3.put_object(
            Body=csv_body.encode("UTF-8"),
            Bucket="test-bucket",
            Key="file.csv",
            StorageClass="GLACIER",
        )

        attributes = s3.get_object_attributes(
            Bucket="test-bucket",
            Key="file.csv",
            ObjectAttributes=["Checksum", "ObjectSize", "StorageClass"],
        )

        assert attributes is not None
```

The error 

```
....
if http.status_code >= 300:
            error_info = parsed_response.get("Error", {})
            error_code = error_info.get("QueryErrorCode") or error_info.get(
                "Code"
            )
            error_class = self.exceptions.from_code(error_code)
>           raise error_class(parsed_response, operation_name)
E           botocore.errorfactory.InvalidObjectState: An error occurred (InvalidObjectState) when calling the GetObjectAttributes operation: The operation is not valid for the object's storage class


..\..\.venv\Lib\site-packages\botocore\client.py:1009: InvalidObjectState
```


Expected behaviour is that the call to s3..get_object_attributes(
                    Bucket=bucket,
                    Key=key,
                    ObjectAttributes=["Checksum", "ObjectSize", "StorageClass"],
                )
returns a valid response as it does for non-glacier storage class object
binance myLiquidations uninitialized before accessed
### Operating System

Ubuntu

### Programming Languages

Python

### CCXT Version

4.4.27

### Description

Got following error while watching binance websocket.

### Code



```
2024-11-26 05:56:31,267 - 875 - exchanges.binance - ERROR - 'NoneType' object does not support item assignment
Traceback (most recent call last):
  File "/home/sytd/test/src/exchanges/future_exchange_base.py", line 118, in watch_balances
    event = await self._exchange.watch_balance()
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sytd/test/.venv/lib/python3.11/site-packages/ccxt/pro/binance.py", line 2503, in watch_balance
    return await self.watch(url, messageHash, message, type)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/sytd/test/.venv/lib/python3.11/site-packages/ccxt/async_support/base/ws/fast_client.py", line 27, in handler
    self.handle_message(message)
  File "/home/sytd/test/.venv/lib/python3.11/site-packages/ccxt/async_support/base/ws/aiohttp_client.py", line 34, in handle_message
    self.handle_text_or_binary_message(message.data)
  File "/home/sytd/test/.venv/lib/python3.11/site-packages/ccxt/async_support/base/ws/aiohttp_client.py", line 29, in handle_text_or_binary_message
    self.on_message_callback(self, decoded)
  File "/home/sytd/test/.venv/lib/python3.11/site-packages/ccxt/pro/binance.py", line 3966, in handle_message
    method(client, message)
  File "/home/sytd/test/.venv/lib/python3.11/site-packages/ccxt/pro/binance.py", line 3396, in handle_order_update
    self.handle_my_liquidation(client, message)
  File "/home/sytd/test/.venv/lib/python3.11/site-packages/ccxt/pro/binance.py", line 518, in handle_my_liquidation
    self.myLiquidations[symbol] = myLiquidations
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^
TypeError: 'NoneType' object does not support item assignment
```
DOC: Convert v to conv_val in function for pytables.py
### Pandas version checks

- [X] I have checked that the issue still exists on the latest versions of the docs on `main` [here](https://pandas.pydata.org/docs/dev/)


### Location of the documentation

pandas\pandas\core\computation\pytables.py

### Documentation problem

Many instances of just v in this function. Wanted to clarify throughout

### Suggested fix for documentation

Change v to conv_val
UnicodeEncodeError when running keras.model.predict
The following problem occurs in keras version 3.1.1 (tensorflow version 2.16.1)
UnicodeEncodeError: 'charmap' codec can't encode characters in position 19-38: character maps to <undefined>

prediction = trained_nn_model.predict(last_sequence)
A bit more details:
  File "...\lib\site-packages\keras\src\utils\traceback_utils.py", line 122, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "...\.pyenv\pyenv-win\versions\3.10.5\lib\encodings\cp1252.py", line 20, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 19-38: character maps to <undefined>

(The positions varies when tried with different test data).
The code works ok with keras version 2.15.0 (tensorflow 2.15.0)
prompt_tokens_details not being populated correctly in response model
- [ - ] This is actually a bug report.
- [ ] I am not getting good LLM Results
- [ - ] I have tried asking for help in the community on discord or discussions and have not received a response.
- [ - ] I have tried searching the documentation and have not found an answer.

**What Model are you using?**

- [ -] gpt-3.5-turbo
- [ -] gpt-4-turbo
- [- ] gpt-4
- [ ] Other (please specify)

**Describe the bug**
I'm unable to see how many tokens were cached when I try to use client.chat.completions.create_with_completion. completion.usage.prompt_token_details is always completion.usage.prompt_token_details

**To Reproduce**
```python
import instructor
from time import sleep
from pydantic import BaseModel, field_validator
from openai import OpenAI
class FakeClass(BaseModel):
    insight: str
client = instructor.from_openai(OpenAI(api_key=OPENAI_API_KEY))
fake_str = "This is a Test string" * 1000

for x in range(0, 3):
    sleep(1)
    insight, completion = client.chat.completions.create_with_completion(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": fake_str,
            },
            {"role": "user", "content": "Test User Prompt"},
        ],
        max_retries=1,
        response_model=FakeClass,
    )
 
    print(completion.usage.prompt_token_details.model_dump())
```

I get this:
```python 
{'audio_tokens': 0, 'cached_tokens': 0}
```

**Expected behavior**
I expect to see how many token have been cached. I know OpenAI is caching it since I tried it without wrapping it around Instructor
```python

open_ai_client = OpenAI(api_key=OPENAI_API_KEY)
open_ai_client.chat.completions.create

for x in range(0, 3):
    sleep(1)
    chat_completion = open_ai_client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {
                "role": "system",
                "content": fake_str,
            },
            {"role": "user", "content": "Test User Prompt"},
        ],
    )

    cached_tokens = chat_completion.usage.prompt_tokens_details.cached_tokens
    if cached_tokens == 0:
        print("No cached tokens")
    else:
        print(f"Tokens used: {cached_tokens}")
        print(chat_completion.usage.prompt_tokens_details.model_dump())
```
I get this output
```
Tokens used: 4864
{'audio_tokens': 0, 'cached_tokens': 4864}
````


**Screenshots**
If applicable, add screenshots to help explain your problem.
Postgresql: ArrayField with Unicode characters gets serialized as string of "\u XXXX" characters
Description
	 
In ArrayField.value_to_string(self, obj) the value is encoded using JSON.dumps(values), which produces escaped Unicode \u XXXX by default.
For example, an ArrayField with 3 elements ["один", "два", "три"] (1,2,3 in Russian) will produce
["\u043e\u0434\u0438\u043d", "\u0434\u0432\u0430", "\u0442\u0440\u0438"]
While this is not a bug per se, this becomes a nuisance when viewing on result of "dumpdata" management command:
The ArrayField fields will be encoded differently from other text fields.
Perhaps there should be an option to turn on/off the ensure_ascii parameter in JSON.dumps(values, ensure_ascii=option)) ?
The option can be enabled by default, as we do for 'hstore' field, or perhaps enabled conditionally:
in the field settings ArrayField(name='numbers', ascii_only=False)
in settings.py ( ARRAY_FIELD_ENSURE_ASCII )
I will be glad to submit a patch.
MoviePy 2.0 throws exception on loading video previous version worked with
#### Expected Behavior

MoviePy should continue to work with the same videos it did previously, even if those videos aren't fully compliant (e.g. are missing some metadata).


#### Actual Behavior

The same video crashes on MoviePy 2.0 but works with MoviePy 1.x.


#### Steps to Reproduce the Problem

See the `corrupt_video.mp4` file from https://github.com/Breakthrough/PySceneDetect/tree/resources/tests/resources and the associated unit test in https://github.com/Breakthrough/PySceneDetect/blob/95d20ddca57bb8cba77354697cc092643bd04afb/tests/test_video_stream.py#L359

The issue seems to come from the ffmpeg info parser assuming the presence of certain metadata fields.  Instead of failing, MoviePy should probably try to set some reasonable default value for these fields if they are not critical, to improve compatibility with media files.


#### Specifications

Tested on a wide variety of OS and Python versions:
 -  os: [macos-13, macos-14, ubuntu-20.04, ubuntu-latest, windows-latest]
 - python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]

MoviePy 2.0 throws exception on loading video previous version worked with
#### Expected Behavior

MoviePy should continue to work with the same videos it did previously, even if those videos aren't fully compliant (e.g. are missing some metadata).


#### Actual Behavior

The same video crashes on MoviePy 2.0 but works with MoviePy 1.x.


#### Steps to Reproduce the Problem

See the `corrupt_video.mp4` file from https://github.com/Breakthrough/PySceneDetect/tree/resources/tests/resources and the associated unit test in https://github.com/Breakthrough/PySceneDetect/blob/95d20ddca57bb8cba77354697cc092643bd04afb/tests/test_video_stream.py#L359

The issue seems to come from the ffmpeg info parser assuming the presence of certain metadata fields.  Instead of failing, MoviePy should probably try to set some reasonable default value for these fields if they are not critical, to improve compatibility with media files.


#### Specifications

Tested on a wide variety of OS and Python versions:
 -  os: [macos-13, macos-14, ubuntu-20.04, ubuntu-latest, windows-latest]
 - python-version: ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
XSS vulnerability
Auto-reviewers: @NiharikaRay @matthewwardrop @earthmancash @danfrankj 

Hello, guys!

There is a cross-site scripting (XSS) vulnerability in the Knowledge Repo 0.7.4 (other versions may be affected as well) which allows remote attackers to inject arbitrary JavaScript via post comments functionality.

Steps to reproduce:
Just open any post like this https://site.com/post/posts/new_report.kp
and add the following code as a comment/PoC:
<script>alert("Client side attacks is possible here!");</script>

Impact: An unauthenticated evil user can leverage the vulnerability to conduct various types of client side attacks, for example, conducting a browser mining, redirecting users to malicious sites or hijacking the user’s browser using malware.

As an irrefutable evidence please take a look at the attached screen-shot.

Mitigation: Details on how to prevent XSS can be found here: https://www.owasp.org/index.php/XSS_(Cross_Site_Scripting)_Prevention_Cheat_Sheet

PoC:
![xss](https://user-images.githubusercontent.com/6551435/41200588-f832ab3c-6caf-11e8-9f4f-117d153a5473.png)
XSS vulnerabilities in comments marked as safe
Since all comments are marked as safe in `markdown-rendered.html`, someone could write Javascript as a comment and it gets interpreted as raw JS, opening up an XSS vulnerability. It seems like this is done so comments can be written in markdown, possibly for LaTeX as well (though removing `|safe` still seems to render LaTeX correctly). Do you consider the XSS vulnerability a problem? Would you accept a PR to fix it? If so, I was thinking that script tags could be stripped at time of submission, prior to posting to the database. You could also potentially strip all HTML period. It's not clear whether raw HTML should be necessary for any of the supported comment types. 

Thank you!

Auto-reviewers: @NiharikaRay @matthewwardrop @earthmancash ancash @danfrankj
Improve returned data when syncing a Git repository via the REST API
### As ...

Austin - Network Automation Engineer

### I want ...

to be able to see more useful return data when syncing a Git repository via the REST API. Today, e.g.:

```
curl -s -X POST \
-H "Authorization: Token $TOKEN" \
-H "Content-Type: application/json" \
-H "Accept: application/json; version=2.4" \
http://nautobot/api/extras/git-repositories/2ecb8556-db58-466d-8278-860b8fd74627/sync/
```

returns

```
{
  "message": "Repository demo-git-datasource sync job added to queue."
}
```

I would want to see more data in that return data, including, for example:
- job-result ID
- selected job queue
- worker ID

### So that ...

I can make better programmatic decisions based on the return data.

### I know this is done when...

A more detailed JSON structure is returned, perhaps e.g.:
```
{
  "message": "Repository demo-git-datasource sync job added to queue.",
  "job-id": "XXXXXX",
  "job-queue": "XXXXXX"
  etc.
}
```

### Optional - Feature groups this request pertains to.

- [ ] Automation
- [ ] Circuits
- [ ] DCIM
- [ ] IPAM
- [x] Misc (including Data Sources)
- [ ] Organization
- [x] Apps (and other Extensibility)
- [ ] Security (Secrets, etc)
- [ ] Image Management
- [ ] UI/UX
- [ ] Documentation
- [ ] Other (not directly a platform feature)

### Database Changes

Already exists.

### External Dependencies

No
Improve returned data when syncing a Git repository via the REST API
### As ...

Austin - Network Automation Engineer

### I want ...

to be able to see more useful return data when syncing a Git repository via the REST API. Today, e.g.:

```
curl -s -X POST \
-H "Authorization: Token $TOKEN" \
-H "Content-Type: application/json" \
-H "Accept: application/json; version=2.4" \
http://nautobot/api/extras/git-repositories/2ecb8556-db58-466d-8278-860b8fd74627/sync/
```

returns

```
{
  "message": "Repository demo-git-datasource sync job added to queue."
}
```

I would want to see more data in that return data, including, for example:
- job-result ID
- selected job queue
- worker ID

### So that ...

I can make better programmatic decisions based on the return data.

### I know this is done when...

A more detailed JSON structure is returned, perhaps e.g.:
```
{
  "message": "Repository demo-git-datasource sync job added to queue.",
  "job-id": "XXXXXX",
  "job-queue": "XXXXXX"
  etc.
}
```

### Optional - Feature groups this request pertains to.

- [ ] Automation
- [ ] Circuits
- [ ] DCIM
- [ ] IPAM
- [x] Misc (including Data Sources)
- [ ] Organization
- [x] Apps (and other Extensibility)
- [ ] Security (Secrets, etc)
- [ ] Image Management
- [ ] UI/UX
- [ ] Documentation
- [ ] Other (not directly a platform feature)

### Database Changes

Already exists.

### External Dependencies

No
Structural Pattern Matching Exhaustiveness check when using OR
<!--
If you're not sure whether what you're experiencing is a mypy bug, please see the "Question and Help" form instead.
Please consider:

- checking our common issues page: https://mypy.readthedocs.io/en/stable/common_issues.html
- searching our issue tracker: https://github.com/python/mypy/issues to see if it's already been reported
- asking on gitter chat: https://gitter.im/python/typing
-->

**Bug Report**

Mypy fails to identify missing value comparisons in my `match` when I use `|`

**To Reproduce**

[Gist URL](https://gist.github.com/mypy-play/cba9c66b56fcbfbd383ca559518ebac5)
[Playground URL](https://mypy-play.net/?mypy=latest&python=3.12&flags=strict&gist=cba9c66b56fcbfbd383ca559518ebac5)

```python
from typing import Literal, assert_never

Color = Literal["blue", "green", "red"]


def print_color(color: Color) -> None:
  match color:
      case "blue":
          print("blue")
      case "green" | "kangaroo":
          print("green")
      case _:
          assert_never(color)
```

**Expected Behavior**

I expected mypy to error because I missed checking against `red`
```console
colors.py:13: error: Argument 1 to "assert_never" has incompatible type "Literal['red']"; expected "Never"  [arg-type]
Found 1 error in 1 file (checked 1 source file)
```

**Actual Behavior**

Mypy doesn't raise any errors

```console
Success: no issues found in 1 source file
```

** Extra Context **

This only happens when I use `|`.
I don't use pyright but ran it to see if I was the one missing something but I believe they do flag it correctly
```console
colors.py:13:24 - error: Argument of type "Literal['red']" cannot be assigned to parameter "arg" of type "Never" in function "assert_never"
    Type "Literal['red']" is not assignable to type "Never" (reportArgumentType)
```

**Your Environment**

- Mypy version used: `mypy 1.13.0 (compiled: yes)`
- Mypy command-line flags: `--strict`
- Python version used: `Python 3.11.6`