eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Following codes gives different results each time, sometimes in correct human readable ASCII, but other times in some other non-ASCII encoding format. HEADERS = ({'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36', 'Accept-Language': 'en-US,en;q=0.9', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7'}) page = requests.get('https://www.powerball.com/', headers=HEADERS) print(page.text) print(page.encoding)  The encoding of the page is always utf-8. What could be the reason for the difference? Tried copy http headers from the request sent from browser but getting the same result.

Answer: The page.headers dictionary contains 'Content-Encoding': 'br'. This indicates Brotli compression and is not supported by default from requests (as of version 2.31.0 anyway). Per the requests documentation:  When either the brotli or brotlicffi package is installed, requests also decodes Brotli-encoded responses.  # Note: pip install brotli import requests import brotli HEADERS = ({'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36', 'Accept-Language': 'en-US,en;q=0.9', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng;q=0.8,application/signed-exchange;v=b3;q=0.7', 'Accept-Encoding': 'gzip'}) page = requests.get('https://www.powerball.com/', headers=HEADERS) print(page.text)
Question: I'm building a radial chart in python but can't order the values of the plot based on the 'categoria' values. I already tried to sort the df and force through domain and sort in the altair code but can't get the desired result. What I'm doing wrong?  Here's a sample of my current dataframe:    index da_tipo_servicio_salud categoria porcentaje rubro     57 En ambos, público y privado Menos de $200 pesos 44.44444444444444 Transporte   36 En ambos, público y privado Menos de $200 pesos 7.142857142857142 Medicamentos   15 En ambos, público y privado Menos de $200 pesos 3.571428571428571 Citas médicas   48 En ambos, público y privado Entre $200 y $500 pesos 29.629629629629626 Transporte    and the altair code I'm using:  orden_monto = ['Menos de $200 pesos', 'Entre $200 y $500 pesos', 'Entre $500 y $800 pesos', 'Entre $800 y $1,000 pesos', 'Entre $1,000 y $1,500 pesos', 'Entre $1,500 y $2,000 pesos', 'Más de $2,000 pesos'] base = alt.Chart(df_combinado_orden_ambos).transform_filter( alt.datum.rubro == 'Medicamentos' ).encode( theta=alt.Theta(field='porcentaje', type='quantitative', stack=True), radius=alt.Radius(field='porcentaje', type='quantitative', scale=alt.Scale(type='sqrt', zero=True, rangeMin=100)), color=alt.Color('categoria:N', scale=alt.Scale(scheme='blues', domain=orden_monto), sort=orden_monto) ).properties( title={ 'text': ['Distribución porcentual por categoría'], 'subtitle': ['Medicamentos'], 'anchor': 'start', 'offset': 20 } ) c1 = base.mark_arc(innerRadius=20, cornerRadius=5, stroke="#fff") c2 = base.mark_text(radiusOffset=25).encode(text=alt.Text('porcentaje:Q', format='.0f')) final_chart = c1 + c2 final_chart.display() 

Answer: Try using the order encoding and set order='categoria'.
Question: I've got some input data, which has a structure such that items can be a single entry, but also a str that is actually an array, but was not yet converted. A basic example would look like this: d = { 'col1': ['[5.02, 1, 3, 2]', '[2.002, 3.14, 4.5, 6.000153, 3]'], 'col2': ['[1.01, 1.02, 2, 4, 5, 10]', '[2, 3.05, 4.001, 7]'], 'col3': [5.2, 3] } df = pd.DataFrame(d)  Output:  col1 col2 col3 0 [5.02, 1, 3, 2] [1.01, 1.02, 2, 4, 5, 10] 5.2 1 [2.002, 3.14, 4.5, 6.000153, 3] [2, 3.05, 4.001, 7] 3.0  Each row corresponds to a dataset, each column represents one type of measurement within that dataset and the entries in each item correspond to the actual measurement data. Currently, the strings are converted to numpy arrays, but each item in col1 and col2 will still hold an array. So a lot of the code that follows with make use apply to perform calculations on the datasets, which I suspect is highly inefficient (confirmation would be good here) as it takes quite some time. Should note that in the real data there are many more rows and columns, so efficiency is important. Thus, I think it might make sense to handle this data differently and instead convert the data to a multiindex, where the first index selects the dataset (so the row in the above example) and the second index refers to the data point in each dataset. It is however important to know that I can't rely on all of those datasets having the same length, I have to assume a variable length. Since I have only very little experience with pandas multi index, can someone help me out here how to convert this data in an efficient way into a pandas multi index? The result should look like this:  col1 col2 col3 dataset data point 0 0 5.02 1.01 5.2 1 1 1.02 2 3 2 3 2 4 4 5 5 10 1 0 2.002 2 3.0 1 3.14 3.05 2 4.5 4.001 3 6.000153 7 4 3  Any help / suggestions on this would be highly appreciated.

Answer: One option taking advantage of the fact that str.extractall adds a second level: out = pd.concat({c: df[c].str.extractall('(\d+\.?\d*)')[0].astype(float) if df[c].dtype == object else df[c].set_axis( pd.MultiIndex.from_product([df[c].index, [0]]) ) for c in df}, axis=1 ).sort_index().rename_axis(['dataset', 'data point'])  Or: out = pd.concat({c: df[c].astype(str) .str.extractall('(\d+\.?\d*)')[0] .astype(float) for c in df}, axis=1 ).sort_index().rename_axis(['dataset', 'data point'])  Or using ast.literal_eval and explode: from ast import literal_eval def expand(s): if s.dtype == object: s = s.apply(literal_eval).explode() return s.set_axis(pd.MultiIndex .from_arrays([s.index, s.groupby(level=0).cumcount(), ], names=['dataset', 'data point']) ) out = pd.concat({c: expand(df[c]) for c in df}, axis=1)  Output:  col1 col2 col3 dataset data point 0 0 5.02 1.01 5.2 1 1 1.02 NaN 2 3 2 NaN 3 2 4 NaN 1 0 2.002 2 3.0 1 3.14 3.05 NaN 2 4.5 4.001 NaN 3 6.000153 7 NaN 4 3 NaN NaN 0 4 NaN 5 NaN 5 NaN 10 NaN
Question: I can't «persuade» Jupyter to use another directory then the default one for storing its projects. I edited the following parameter of the jupyter_notebook_config.py file (I tried all three variants one-by-one, none worked): c.NotebookApp.notebook_dir = 'D:\\_Travail\\Python\\Jupyter_projects' c.NotebookApp.notebook_dir = r'D:\_Travail\Python\Jupyter_projects' c.NotebookApp.notebook_dir = 'D:/_Travail/Python/Jupyter_projects'  Besides, I tried to rename the config-file (and names of parameters therein) exactly as described in the only answer in this thread (I also have this message 'notebook_dir' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. in console). I wonder if it is important but the disk D: where I want files stored is a network drive connected to another machine on LAN.

Answer: You can launch the Jupyter in the directory you want. Open Anaconda Prompt and enter the following statement: jupyter notebook --notebook-dir "your-directory"
Question: I want to know how to use the Streaming function in Autogen. The code below uses Autogen to enable the Agent to have streaming function. llm_config = { "config_list" : config_list, "timeout" : 120, "seed" : random.randint(1, 100), "stream" : True } user_proxy.initiate_chat( manager, message="hello, the weather is nice today", llm_config=llm_config, )  The Streaming function that I know of is to print and display each letter as it is created before the sentence is completed, but even if I use the option("stream" : True), the result is that the entire text is output on the terminal. I'm not sure if I'm misunderstanding the streaming function or if I know how to use it properly. Could you please help? Thank you :)

Answer: Currently, there isn't a direct solution provided by AutoGen for the streaming issue you've described. This is evidenced by the lack of solutions in their open issues on GitHub, specifically related to streaming (https://github.com/microsoft/autogen/issues?q=is%3Aissue+is%3Aopen+streaming). However, a workaround exists that involves using "monkey patching" to modify the behavior of the function responsible for printing received messages. This method allows you to intercept and alter the output behavior to achieve the streaming effect you're looking for. The monkey patching technique can be implemented by overwriting the _print_received_message method in the library. This approach is detailed in https://youtu.be/iNPCB6b5gvA, which provides step-by-step instructions on how to apply this method effectively.
Question: I know that torch.argmax(x, dim = 0) returns the index of the first maximum value in x along dimension 0. But is there an efficient way to return the indexes of the first n maximum values? If there are duplicate values I also want the index of those among the n indexes. As a concrete example, say x=torch.tensor([2, 1, 4, 1, 4, 2, 1, 1]). I would like a function generalized_argmax(xI torch.tensor, n: int)  such that generalized_argmax(x, 4) returns [0, 2, 4, 5] in this example.

Answer: To acquire all you need you have to go over the whole tensor. The most efficient should therefore be to use argsort afterwards limited to n entries. >>> x=torch.tensor([2, 1, 4, 1, 4, 2, 1, 1]) >>> x.argsort(dim=0, descending=True)[:n] [2, 4, 0, 5]  Sort it again to get [0, 2, 4, 5] if you need the ascending order of indices.
Question: Scikit learn import statements in their tutorials are on the form from sklearn.decomposition import PCA  Another versions that works is import sklearn.decomposition pca = sklearn.decomposition.PCA(n_components = 2)  However import sklearn pca = sklearn.decomposition.PCA(n_components = 2)  does not, and complains AttributeError: module 'sklearn' has no attribute 'decomposition'  Why is this, and how can I predict which ones will work and not so i don't have to test around? If the understanding and predictiveness extends to python packages in general that would be the best.

Answer: sklearn doesn't automatically import its submodules. If you want to use sklearn.<SUBMODULE>, then you will need to import it explicitly e.g. import sklearn.<SUBMODULE>. Then you can use it without any further imports like result = sklearn.<SUBMODULE>.function(...). Large packages often behave this way where they don't automatically import all the submodules. Memory and load-time efficiency become worse if the submodules are automatically loaded; by specifying the submodule explicitly it saves on memory consumption and minimises the start-up time. I think namespace cluttering is another consideration, where explicit imports reduce the chance of naming conflicts and help maintain clarity about the specific functionality being used.
Question: As the title says. The warnings are: docstring of liesel.goose.EpochType.from_bytes:9: WARNING: Inline interpreted text or phrase reference start-string without end-string. docstring of liesel.goose.EpochType.to_bytes:8: WARNING: Inline interpreted text or phrase reference start-string without end-string.  I am using Sphinx version 7.2.6. My investigation yielded the following: The class EpochType is derived from enum.IntEnum. IntEnum is derived partly from int (source code on GitHub). As far as I can see, the methods from_bytes and to_bytes originate from the corresponding methods on int. I do not understand why sphinx is throwing this warning and/or what I can do about it.

Answer: I ran into this problem too, and @mzjn's explanation seems to be right. A fix to use apostrophes for the opening quotes got merged and will be in the upcoming Python 3.13. release: https://github.com/python/cpython/pull/117847 I also tried to fix a handful other similar cases of this in the standard library in https://github.com/python/cpython/pull/119231 so hopefully people won't run into it with other classes. Until you can switch to Python 3.13, you can probably work around this by writing :inherited-members: int instead of :inherited-members:, telling Sphinx to stop documenting inherited members once it hits class int. See autodoc docs for details.
Question: I'm trying to run code that writes on a SQL Server database. The code is working fine on my machine (Microsoft). dockerfile: # syntax=docker/dockerfile:1 FROM ubuntu:22.04 WORKDIR /app RUN apt-get update RUN apt-get install -y apt-utils \ && echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections RUN apt-get install -y locales \ && localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 \ && localedef -i it_IT -c -f UTF-8 -A /usr/share/locale/locale.alias it_IT.UTF-8 ENV LANG it_IT.utf8 RUN apt-get install -y python3 python3-pip RUN apt-get install -y unixodbc-dev libodbc2 COPY ./src . RUN pip3 install -r requirements.txt CMD [ "python3", "-m" , "flask", "run", "--host=0.0.0.0", "--port=5000"]  application-docker.cfg file: SECRET_KEY = 'xxx' ALLOWED_FILES = ['xls'] MAX_RUNNING_JOBS = 3 STORAGE_PATH = '/tmp' DRIVER = 'SQL Server' SERVER = 'xxx' DB_NAME = 'xxx' USER_ID = 'xxx' PSW = 'xxx'  It raises this error:  sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found (0) (SQLDriverConnect)")  I've tried to replace DRIVER with DRIVER = '/opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.3.so.1.1' but it raises quite the same error:  sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql18/lib64/libmsodbcsql-18.3.so.1.1' : file not found (0) (SQLDriverConnect)")  Tried also by using DRIVER = 'ODBC Driver 17 for SQL Server' and in the docker file RUN apt-get install -y msodbcsql17 mssql-tools  but when I build the image:  0.915 E: Unable to locate package msodbcsql17. 0.915 E: Unable to locate package mssql-tools 

Answer: If you must have mssql-tools inside the container then you have to follow : the Microsoft instructions (note different Ubuntu version) or get an existing Docker container with mssql-tools in it. The basic commands (Ubuntu 22.04 version) are: curl https://packages.microsoft.com/keys/microsoft.asc | sudo tee /etc/apt/trusted.gpg.d/microsoft.asc curl https://packages.microsoft.com/config/ubuntu/22.04/prod.list | sudo tee /etc/apt/sources.list.d/mssql-release.list sudo apt-get update sudo apt-get install mssql-tools18 unixodbc-dev  One can also try one of existing mssql-tools Docker images but these may be outdated (most popular is 7 years old) / unknown quality / different Linux distros
Question: I am trying to create a Python script that waits for a message on a RabbitMQ queue to start a task in a subprocess. During task execution, the script continues to consume another queue for a cancel order that would stop the task. I use kombu package to handle interactions with RabbitMQ. I call message.ack() when the task terminates (whether normally or because of cancellation). Despite the call to message.ack(), the start message is not removed from the queue (I can tell by using rabbitmqctl). This causes the message to be redelivered even if the task ran to completion. I created a sample repository to show the problem. The README file shows the reproduction steps. I do not know where the problem could be. I realize there are many moving parts but this is a slightly simplified version of a real project with its own constraints (like the Python version being fixed to 3.8). I am open to any suggestion to make a better use of kombu or asyncio because I am quite new to AMQP and async Python.

Answer: I finally found the cause of my problem. Actually there were two causes:  I did not set channels when creating the Consumers, so they were both using the same channel. Since I had set noAck=True for the "stop" consumer, calling message.ack() had no effect. The strange thing is that noAck=True should have caused the ACK to be sent at reception, even for start message, but it was not. I got mistaken with the closure of the on_done callback. I thought the closure contained the message at the moment it was created, but it wasn't. Instead, it contained the last message received, i.e. the "stop" message. I'm quite new to Python so I am not sure how closures work. I assumed it worked like JavaScript but it seems not. Overall I stored the message when creating the task, gave it back as an argument to the on_done callback and it worked.  I could not find the solution until I used the debugger to inspect the execution step by step. This shows once again the superiority of the debugger when troubleshooting. I should have gone with it earlier.
Question: Given a string of a person's name: "Richard David"  I'd like to create a list of all combinations of this name where up to all but one of the individual names are replaced with initials: [ "Richard David", "R David", "Richard D", ]  An example with three individual names: "Richard Anthony David"  Would output: [ # replace no initials "Richard Anthony David", # replace one name with initial "Richard Anthony D", "Richard A David", "R Anthony David", # replace two names with initials "R A David", "R Anthony D", "Richard A D", ]  itertools is usually my go to for combinations but I can't quite work out how to build in the replacement of a name with an initial.

Answer: Using the fact that the order of the elements returned by itertools.product is predictable, as guaranteed in its documentation. from itertools import product def all_names(fullname): pool = [(word, word[0]) for word in fullname.split()] return [' '.join(name) for name in product(*pool)][:-1] print(*all_names("Richard Anthony David"), sep='\n') # Richard Anthony David # Richard Anthony D # Richard A David # Richard A D # R Anthony David # R Anthony D # R A David
Question: When I try to install openai on my Google Colab, I get this error: from openai import OpenAI --------------------------------------------------------------------------- ImportError Traceback (most recent call last) <ipython-input-6-5ee9a4e68b89> in <cell line: 1>() ----> 1 from openai import OpenAI 5 frames /usr/local/lib/python3.10/dist-packages/openai/_utils/_streams.py in <module> 1 from typing import Any ----> 2 from typing_extensions import Iterator, AsyncIterator 3 4 5 def consume_sync_iterator(iterator: Iterator[Any]) -> None: ImportError: cannot import name 'Iterator' from 'typing_extensions' (/usr/local/lib/python3.10/dist-packages/typing_extensions.py) --------------------------------------------------------------------------- NOTE: If your import is failing due to a missing package, you can manually install dependencies using either !pip or !apt. To view examples of installing some common dependencies, click the "Open Examples" button below.  My details to reproduce: Python 3.10.12 OS is Windows 11 typing_extensions is 4.9.0 Edit: my typing_extensions details: 

Answer: I found a related issue on OpenAI forum: https://community.openai.com/t/error-while-importing-openai-from-open-import-openai/578166/26 Solution 1: Credit to @Aymane_oub pip install --force-reinstall typing-extensions==4.5 pip install --force-reinstall openai==1.8   Solution 2: Credit to @Shannon-21 After navigating through the answers I found that I can change the import of the file that is trying to import Iterator from typing_extensions in this way (On Google Colab): %%writefile /usr/local/lib/python3.10/dist-packages/openai/_utils/_streams.py from typing import Any from typing_extensions import AsyncIterator from typing import Iterator # import Iterator from the correct library def consume_sync_iterator(iterator: Iterator[Any]) -> None: for _ in iterator: ... async def consume_async_iterator(iterator: AsyncIterator[Any]) -> None: async for _ in iterator: ...  #output  After running the above code on my colab cell, I was able to use:  Link to answer : https://community.openai.com/t/error-while-importing-openai-from-open-import-openai/578166/27?u=rubeenatalha313
Question: In the code below, if the upload_file_2_sp function is called, I'm unsure of how the offset argument is being assigned a value. The code will run successfully as is, but I'm trying to add a second argument to the print_upload_progress function to pass each file in my file list loop to the print_upload_progress function, so I can display the upload progress as the for loop iterates through my list of files. Not sure how to do this, because I'm not sure how the assignment of the offset argument is currently working. local_path = '/sso_win_mounts/win_appsprod/CRMDAGROUP/phones/reporting/customer_care/prod/weekly/kpm/test 3.xlsb' #display progress of file upload to sharepoint def print_upload_progress(offset): print(str(offset)) # type: (int) -> None file_size = os.path.getsize(local_path) print( "Uploaded '{0}' bytes from '{1}'...[{2}%]".format( offset, file_size, round(offset / file_size * 100, 2) ) ) #upload files to sharepoint def upload_file_2_sp(report_list, base_url, target_url): ctx = sharepointConnection(base_url) target_folder = ctx.web.get_folder_by_server_relative_url(target_url) size_chunk = 1000000 for file in report_list: uploaded_file = target_folder.files.create_upload_session( file, size_chunk, print_upload_progress ).execute_query() print("File {0} has been uploaded successfully".format(uploaded_file.serverRelativeUrl)) 

Answer: The arguments are passed automatically by create_upload_session(), you can't change what it passes. You could use a lambda to pass additional arguments to your function.  for file in report_list: uploaded_file = target_folder.files.create_upload_session( file, size_chunk, lambda offset, file=file: print_upload_progress(offset, file) ).execute_query() print("File {0} has been uploaded successfully".format(uploaded_file.serverRelativeUrl))  See Creating functions (or lambdas) in a loop (or comprehension) for the explanation of the file=file argument to the lambda.