mteb/LocBenchRR huggingface dataset:
Spill quickly from device to host
I'm curious how fast we are at moving data back and forth between device and host memory.  I suspect that currently we end up serializing with standard Dask serialization, which may not be aware of device memory.  We should take a look at using the `"cuda"` serialization family within Dask, that was added as part of the UCX work.   


```python
from distributed.protocol import serialize, deserialize
x = cupy.arange(100)
header, frames = serialize(x, serializers=("cuda", "dask", "pickle"))
y = deserialize(header, frames, deserializers=("cuda", "dask", "pickle",    "error"))
```

(more here https://github.com/dask/distributed/tree/master/distributed/protocol/tests)

And then we should probably do a bit of profiling.  I would focus this on just the `DeviceHostDisk` class, rather than engaging the full dask.distributed cluster machinery.

This is relevant for a couple of applications that people care about today

1.  Larger than memory image processing (a bit)
2.  Out-of-core sorts and joins (cc @randerzander @kkraus14) (more relevant)

@pentschev , you seem like the obvious person to work on this if you have some time.  I think it'd be useful to know how fast we are today, then look at serialization with the new "cuda" Dask serializer, and then see how fast we can get things relative to theoretical hardware speeds.
IDLE maxosc.overrideRootMenu: remove unused menudict
BPO | [33964](https://bugs.python.org/issue33964)
--- | :---
Nosy | @terryjreedy, @csabella
PRs | <li>python/cpython#7926</li>
Dependencies | <li>bpo-33963: IDLE macosx: add tests.</li>

<sup>*Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.*</sup>

<details><summary>Show more details</summary><p>

GitHub fields:
```python
assignee = 'https://github.com/terryjreedy'
closed_at = None
created_at = <Date 2018-06-26.04:01:56.869>
labels = ['expert-IDLE', '3.7', '3.8', 'performance']
title = 'IDLE maxosc.overrideRootMenu: remove unused menudict'
updated_at = <Date 2019-03-20.19:52:30.187>
user = 'https://github.com/terryjreedy'
```

bugs.python.org fields:
```python
activity = <Date 2019-03-20.19:52:30.187>
actor = 'cheryl.sabella'
assignee = 'terry.reedy'
closed = False
closed_date = None
closer = None
components = ['IDLE']
creation = <Date 2018-06-26.04:01:56.869>
creator = 'terry.reedy'
dependencies = ['33963']
files = []
hgrepos = []
issue_num = 33964
keywords = ['patch']
message_count = 2.0
messages = ['320470', '338509']
nosy_count = 2.0
nosy_names = ['terry.reedy', 'cheryl.sabella']
pr_nums = ['7926']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'performance'
url = 'https://bugs.python.org/issue33964'
versions = ['Python 3.7', 'Python 3.8']
```

</p></details>
getting UnboundLocalError: cannot access local variable 'aggregate' where it is not associated with a value when running sqlglot.planner.Plan
**Before you file an issue**
- Make sure you specify the "read" dialect eg. `parse_one(sql, read="spark")`
- Make sure you specify the "write" dialect eg. `ast.sql(dialect="duckdb")`
- Check if the issue still exists on main

**Fully reproducible code snippet**
```
import sqlglot
import sqlglot.planner

r = 'select suma from ( select sum(a) as suma from table1) order by suma'
parsed = sqlglot.parse_one(r, dialect='snowflake')
p = sqlglot.planner.Plan(parsed)
```
Throws:
```
File venv/lib/python3.11/site-packages/sqlglot/planner.py:14, in Plan.__init__(self, expression)
     12 def __init__(self, expression: exp.Expression) -> None:
     13     self.expression = expression.copy()
---> 14     self.root = Step.from_expression(self.expression)
     15     self._dag: t.Dict[Step, t.Set[Step]] = {}

File venv/lib/python3.11/site-packages/sqlglot/planner.py:213, in Step.from_expression(cls, expression, ctes)
    210         if extract_agg_operands(exp.alias_(ordered.this, f"_o_{i}", quoted=True)):
    211             ordered.this.replace(exp.column(f"_o_{i}", step.name, quoted=True))
--> 213     set_ops_and_aggs(aggregate)
    215 sort = Sort()
    216 sort.name = step.name

UnboundLocalError: cannot access local variable 'aggregate' where it is not associated with a value
```

On sqlglot 25.33.0

dialect seems to be irrelevant, same error with Athena

**Official Documentation**
Please include links to official SQL documentation related to your issue.
Validate MD5 checksum of downloaded ARFF in fetch_openml
`fetch_openml` downloads the `data_set_description` from openml.org which includes an `md5_checksum` field. This should correspond to the MD5 checksum of the downloaded ARFF data. We should (optionally?) use `hashlib` to verify that the MD5 matches.
Support multiline csv value in INI config
- `flake8` config typically has multiline config value such as `per-file-ignores`
- each line has value that is usually comma separated

We should support this type of config for INI format as it's frequently used to incrementally fine-tune flake8 rules
Add an option to skip the running `flutter doctor` when `flet build` fails
### Discussed in https://github.com/flet-dev/flet/discussions/4359

<div type='discussions-op-text'>

<sup>Originally posted by **DFNJKD-98** November 13, 2024</sup>
### Question

Due to some reasons, my country cannot freely access GitHub, so Flutter Doctor always reports errors. This hinders me from using Fleet to build Windows applications. Can I skip the Flutter Doctor phase using the 'flex build windows' command? I'm not sure if this is necessary.

### Code sample

_No response_

### Error message

```bash
As shown below, the Flutter Doctor command may encounter some issues, but building a Windows application directly using Flutter is not a problem. I can use Flutter normally in VSCode.
Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel stable, 3.22.1, on Microsoft Windows [version 1.0.19045.4046], local zh CN)
[√] Windows Version (Installed version of Windows is version 10 or higher)
[√] Chrome - develop for the web
[√] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.7.3)
[!] Flutter IDE Support (No supported IDEs installed)
[√] Connected device (3 available)
[!] Network resources
X An HTTP error occurred while checking " https://github.com/ ": Connection closed before full header was received

! Doctor found issues in 2 categories.
```
</div>
[community] Improving docstrings and type hints
There are many instances in the codebase where our docstring/typing convention is not followed. We'd like to work on improving this with your help!

Our convention looks like:

```python3
def function_name(parameter_1: Union[str, List[str]], parameter_2: Optional[int] = None, parameter_3: float = 42.0) -> Civilization:
    r"""
    Function that creates a simulation.

    Args:
        parameter_1 (`str` or `List[str]`):
            Description of game level.
        parameter_2 (`int`, *optional*):
            Kardashev scale of civilization.
        parameter_3 (`float`, defaults to `42.0`):
            Difficulty scale.

    Returns:
        [`~simulations.objects.Civilization`]
            A civilization simulation with provided initialization parameters.
    """
```

Some examples that don't follow the docstring convention are:
- [this](https://github.com/huggingface/diffusers/blob/c4a8979f3018fbffee33304c1940561f7a5cf613/src/diffusers/models/embeddings.py#L89): missing explanations
- [this](https://github.com/huggingface/diffusers/blob/33fafe3d143ca8380a9e405e7acfa69091d863fb/src/diffusers/pipelines/stable_diffusion_3/pipeline_stable_diffusion_3.py#L132): does not contain mixin-related documentation whereas as [this](https://github.com/huggingface/diffusers/blob/33fafe3d143ca8380a9e405e7acfa69091d863fb/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py#L154) does
- [this](https://github.com/huggingface/diffusers/blob/c4a8979f3018fbffee33304c1940561f7a5cf613/src/diffusers/utils/import_utils.py#L672): function explanation after "Args", but should be before
- [this](https://github.com/huggingface/diffusers/blob/c4a8979f3018fbffee33304c1940561f7a5cf613/src/diffusers/pipelines/deepfloyd_if/pipeline_output.py#L14): same reason as above
- [this](https://github.com/huggingface/diffusers/blob/c4a8979f3018fbffee33304c1940561f7a5cf613/src/diffusers/models/embeddings.py#L518): incorrect indentation

There are also many places where docstrings are completely missing or inadequately explained. If you feel something needs an improvement, you can open a PR with your suggestions too! Additionally, type hints are not appropriate/correctly used at many occurrences and mismatch the accompanying docstrings - these could use an improvement too!

Please limit your PRs to changes to a single file in each PR. Changes must be only related to docstrings/type hints. Feel free to ping either @yiyixuxu, @stevhliu or me for reviews.
[Feature]: Allow Ray placement group more time to wait for resources to be ready
### 🚀 The feature, motivation and pitch

```
ray start --head --dashboard-host 0.0.0.0 --disable-usage-stats --block  --num-gpus 1

python -m vllm.entrypoints.openai.api_server     --model Qwen/Qwen2.5-1.5B-Instruct     --trust-remote-code     --distributed-executor-backend ray     --tensor-parallel-size 2
```

I am using a ray cluster as the distributed executor backend. However, my cluster sometimes takes time to be ready. there are multiple reasons
- image downloading speed is different, some node has image cache and some does not.
- node short of resources, node level autoscaler will make decision to bring up more nodes but it takes time for new ray node to join and form a cluster

I do not want the engine to early exit. 

![image](https://github.com/user-attachments/assets/7a621963-21fb-4f16-9573-71299e951bad)

![image](https://github.com/user-attachments/assets/5b06c173-1465-4675-a5a0-2627a3b8b18d)


### Alternatives

_No response_

### Additional context

_No response_

### Before submitting a new issue...

- [X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the [documentation page](https://docs.vllm.ai/en/latest/), which can answer lots of frequently asked questions.
yg-privacyenhanced.py: excessive CPU usage
At first startup of the Yield Generator, CPU usage seems acceptable, but as it sits running for a few days, it starts using excessive amounts of CPU time. It seems to depend on how many CoinJoins it has participated in. Case in point: I have two instances of `yg-privacyenhanced.py` running (with separate JM wallets and separate Bitcoin Core wallets), one running with native SegWit and the other with transitional SegWit. Both were started at the same time, about 154 hours ago. One has used about 25 minutes of CPU time. The other has used about 75 hours of CPU time — i.e., nearly half of the wall-clock time that it has been running!

Attaching to the well-behaved instance with `strace` shows that it is making one `getblockcount` RPC call and one `listtransactions` RPC call about every 4-5 seconds. Not great, but okay. (Polling is evil.) Attaching `strace` to the misbehaving instance shows that it is obsessively (many times per second!) calling `gettransaction`, all for the same transaction ID. The responses from Bitcoin Core show that the specified transaction has over 200 confirmations, so why is JoinMarket obsessively hammering requests for it?

If I restart the `yg-privacyenhanced.py` script, then the new instance is well behaved for a while, but eventually it will start obsessively hammering requests for some particular transaction ID again.
Problems with "expected overloaded function" when method hinted via Callback Protocol (⇝`numpy`)
**Bug Report**

`mypy` complains about expecting an overloaded function, when a method is hinted by a Callback-Protocol. This technique is used in the `numpy` stubs, and causes `mypy` to complain when trying to describe numpy scalars with simple protocols.

**To Reproduce**

Below is a MWE [[mypy-playground]](https://mypy-play.net/?mypy=latest&python=3.12&gist=8041e0145f43507bed8c86104a5fb5b2), based on how the [numpy stubs declare the methods on scalars](https://github.com/numpy/numpy/blob/bd1f6067a13f8bbdf99871277426dfaad9f7602d/numpy/__init__.pyi#L3582). Note that [[pyright-playground]](https://pyright-play.net/?code=GYJw9gtgBALgngBwJYDsDmUkQWEMoAK4MYAxmADYA0UAygKYXA1gBu9IFYAhgCYBQ-UhW4BnUVAByAV2xwAYl24wAXPygaoAfS19eOlVABEinjADyBI4OFiJp5ZYAURMCXIUAlGs1QAAmwcSgK%2BvPTA2lqk3BQUOk6ijMxQbgAWHIbASjCeUAC0AHxSsggK2YYAdFXqmgHsnDwhmmEROtGx8YlMLDDpIIYycg45%2BUWDpcOV1UIi4nTSCDh4ogCCvLwuxGSU3jUadUGNe1AtkXqdST19mdm5hXRJUxXHBw18x6c651oJlym9GWKQ1uoweTCeggAHoZaAsljBVusoABeIETbJOTz8IA) does not report any issues.

```python
from typing import Protocol, Self, overload

class NumpyFloat:
    __add__: "FloatOP"

class FloatOP(Protocol):
    @overload
    def __call__(self, other: float) -> NumpyFloat: ...
    @overload
    def __call__(self, other: NumpyFloat) -> NumpyFloat: ...

class SupportsAdd(Protocol):
    @overload
    def __add__(self, other: float) -> Self: ...
    @overload
    def __add__(self, other: NumpyFloat) -> Self: ...

x: SupportsAdd = NumpyFloat()  # expected overloaded function, got "FloatOP"
```

Concrete example with `numpy==2.1.3` (again, `pyright` reports no errors):
 
```python
from typing import Protocol, Self, overload
import numpy as np

class TimeDelta(Protocol):
    def __sub__(self, other: Self, /) -> Self: ...

class TimeStamp[TD: TimeDelta](Protocol):
    @overload
    def __sub__(self, other: Self, /) -> TD: ...
    @overload
    def __sub__(self, other: TD, /) -> Self: ...

x: TimeStamp[np.timedelta64] = np.datetime64("2021-01-01")  # ✅
y: TimeStamp[np.float64] = np.float64(10.0)  # expected overloaded function, got "_FloatOp[_64Bit]"
```
Could not run with argurment "-s lambda" or "-s awslambda"
### Steps to Reproduce

export LOG_LEVEL=INFO && python prowler.py aws -s lambda



### Expected behavior

Prowler should scan for awslambda service

### Actual Result with Screenshots or Logs

2024-11-27 13:57:25,877 [File: __main__.py:280]         [Module: __main__]       ERROR: There are no checks to execute. Please, check your input arguments


### How did you install Prowler?

Cloning the repository from github.com (git clone)

### Environment Resource

Local PC

### OS used

MacOS Sonoma 14.6.1

### Prowler version

Prowler 5.0.0 (You are running the latest version, yay!)

### Pip version

pip 24.3.1 

### Context

_No response_
Proposal(CLI): after_instantiate_classes hook
### Description & Motivation

Adds a `after_instantiate_classes` hook to the Lightning CLI, called after `self.instantiate_classes()` during the initalization of `LightningCLI`.

### Pitch

While having the Lightning CLI is great, it is not perfect for each use case out-of-the-box. Hence, you included hooks like `before_instantiate_classes` and describe in the docs how to extend the CLI. Problem is, you cannot extend this feature without hacks or substantial copy-pasta.

I think, to further improve the CLI, without adding any complexity, it makes sense to add a `after_instantiate_classes` hook, too.

### Alternatives

1. Hacks
 - Extend the Lightning CLI and run the `after_instantiate_classes` function before the `self._run_subcommand` function.
 - Problems: it's not intuitive that the function is called there, won't be called if `self.subcommand is None`
2. Copy-Pasta
 - Extend the Lightning CLI and replace the original `__init__` with the proposed one.
 - Problems: could break with any update, lots of code duplication


### Additional context

_No response_

cc @borda @tchaton @justusschock @awaelchli @mauvilsa
Dependency python-Jose appears to be unmaintained
Hey - just a heads-up that it appears this library is using `python-jose` as a dependency, which hasn't been updated in ~2 years. Maintainers haven't shown any activity in GitHub for issues or pull requests in quite a while, either. It would probably be prudent to pivot to PyJWT or JWCrypto, before CVEs start cropping up against the abandoned library.
Dependency python-Jose appears to be unmaintained
Hey - just a heads-up that it appears this library is using `python-jose` as a dependency, which hasn't been updated in ~2 years. Maintainers haven't shown any activity in GitHub for issues or pull requests in quite a while, either. It would probably be prudent to pivot to PyJWT or JWCrypto, before CVEs start cropping up against the abandoned library.
`jax.random.beta` 3 orders of magnitude slower from 0.4.36 on GPU
### Description

My code runs substantially slower from one month ago, and I figued out a key bottleneck: sampling from beta distribution has gotten around 1000 times slower on GPU.

On Colab, I run the following code on different versions of jax
```
@jax.jit
def sample_beta(rng_key):
  return jax.random.beta(key=rng_key, a=1, b=1, shape=(1000, 2))

seed = jrand.PRNGKey(1)
sample_beta(seed)

%timeit sample_beta(seed)
```

* Time take on version 0.4.35: **0.784ms**
* Time take on version 0.4.36: **351ms**
* Time take on version 0.4.37: **354ms**


![image](https://github.com/user-attachments/assets/e7dd4600-192c-4308-b4d4-52eb7ca762a3)

![image](https://github.com/user-attachments/assets/0e8f6952-50b2-4ba5-8fda-6b3533a94b86)

![image](https://github.com/user-attachments/assets/ce088de7-89e4-466a-b331-3a79cdf77ae2)



### System info (python version, jaxlib version, accelerator, etc.)

jax:    0.4.36
jaxlib: 0.4.36
numpy:  1.26.4
python: 3.10.12 (main, Nov  6 2024, 20:22:13) [GCC 11.4.0]
device info: Tesla T4-1, 1 local devices"
process_count: 1
platform: uname_result(system='Linux', node='d36852658d94', release='6.1.85+', version='#1 SMP PREEMPT_DYNAMIC Thu Jun 27 21:05:47 UTC 2024', machine='x86_64')


$ nvidia-smi
Fri Dec 13 13:13:50 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  Tesla T4                       Off | 00000000:00:04.0 Off |                    0 |
| N/A   72C    P0              31W /  70W |    109MiB / 15360MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+
[Data] Ray 2.40 `add_column` Erroneous Assertion
### What happened + What you expected to happen

Ray 2.40 introduces an assertion that a value is of type `pd.Series`, but this is erroneous. A dataframe may have a constant assigned as in the following example or be a geopandas series (which should subclass).

```python
>>> import pandas as pd
>>>
>>> # Create a sample DataFrame
...     "A": [1, 2, 3],
...     "B": [4, 5, 6]
... })
>>>
>>> # Assign a constant value to a new column "C"
>>> df["C"] = 10
>>>
>>> print(df)
   A  B   C
0  1  4  10
1  2  5  10
2  3  6  10
>>>
```

As a result of the change valid expressions like:
```python
q_ds = q_ds.add_column("_q", lambda _: True)
s_ds = s_ds.add_column("_q", lambda _: False)
```
no longer succeed.

### Versions / Dependencies

Ray 2.40

### Reproduction script
```python
import pandas as pd
import ray

df= pd.DataFrame({"geometry": [1, 2, 3]})

# Create Ray datasets
ds = ray.data.from_pandas(df)

ds = ds.add_column("works", lambda _: True)
ds = ds.add_column("works", lambda _: True)

# evaluates and fails
ds.materialize()
ds.show()
```
### Issue Severity

Low: It annoys or frustrates me.
SOCKS server listens on all interfaces by default
### Configuration  
impacket version:  v0.10.1.dev1+20230207.182628.6cd68a05
Python version:  3.9.2
Target OS:  NA
 
By default, the SOCKS server used by ntlmrelayx.py listens on all interfaces (0.0.0.0)1080, which is dangerous.

Please see:
https://github.com/fortra/impacket/blob/6328a9b74a1df675fb47d8cc3a626bddc1f49b1d/impacket/examples/ntlmrelayx/servers/socksserver.py#L456

Any malicious agent on the same network could take advantage of the listening SOCKS proxy.
This service should only be exposed on localhost by default, and a supplemental flag should be provided in order to make it listen "publicly".
Significant mAP Drop When Using Bottom-Right Padding Instead of Center Padding in YOLOv8 Training
### Search before asking

- [X] I have searched the Ultralytics YOLO [issues](https://github.com/ultralytics/ultralytics/issues) and [discussions](https://github.com/ultralytics/ultralytics/discussions) and found no similar questions.


### Question

Hi, I'm training a YOLOv8 model on the same dataset, but noticed a significant difference in mAP when changing the padding strategy.

    When using center padding (default), the mAP@50 after the first epoch is around 0.88.
    When using bottom-right padding instead, the mAP@50 drops to 0.0001.

I ensured that:

    The same data augmentation and other settings were used in both cases.

However, the bottom-right padding leads to poor performance. What could be causing such a drastic performance drop? Is it related to padding affecting feature distribution, anchor design, or augmentation strategies? Any suggestions for improving performance in this case would be appreciated!

Thanks!
I changed "center=True" to "center=False",augment.py line1500
![2024-11-23 16-09-01 的屏幕截图](https://github.com/user-attachments/assets/b2c6c27d-2518-45cf-819b-40e6e6de87ce)
![2024-11-23 16-09-40 的屏幕截图](https://github.com/user-attachments/assets/fa10d8b8-2a61-4212-b709-2c213133bac4)
![2024-11-23 16-09-29 的屏幕截图](https://github.com/user-attachments/assets/2b2c9fd2-f53d-47ca-977f-7db6cd00f1c8)


### Additional

_No response_
Unnecessary `._copartition()` on binary operations
<b>quote from [this comment](https://github.com/modin-project/modin/issues/6948#issuecomment-1969082652):</b>

This block with binary operations unnecessary triggers lazy executions because of [`._copartition()`](https://github.com/modin-project/modin/blob/271d98b260f03b345ae2384001f2df83b98322bf/modin/core/dataframe/pandas/dataframe/dataframe.py#L3682-L3684) call that ensures that indices/partitioning of both arguments are equal before performing a binary operation:
```python
df['a'] = (df.l_extendedprice) * (1 - (df.l_discount))
df['b'] = (((df.l_extendedprice) * (1 - (df.l_discount))) * (1 + (df.l_tax)))
```
The current implementation of `._copartition()` simply triggers computation of actual indexing and actual row lengths for both dataframes and then compares them. But in this case we know that all arguments are views of the same dataframe, meaning that both indexing and partitioning are identical, so we can potentially skip this check. The mechanism of detecting sibling frames when comparing indices/partitioning was already implemented in #6491. What we can do here is simply enable it for `._copartition()` method as well.