eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm trying to plot a time series on python using plotly, here is the result without the dates on the x-axis :  And when I add the date :  Here is a view of my table : {'sasdate': {4: Timestamp('1959-01-09 00:00:00'), 5: Timestamp('1959-01-12 00:00:00'), 6: Timestamp('1960-01-03 00:00:00'), 7: Timestamp('1960-01-06 00:00:00'), 8: Timestamp('1960-01-09 00:00:00'), 9: Timestamp('1960-01-12 00:00:00'), 10: Timestamp('1961-01-03 00:00:00'), 11: Timestamp('1961-01-06 00:00:00'), 12: Timestamp('1961-01-09 00:00:00'), 13: Timestamp('1961-01-12 00:00:00')}, 'CDT': {4: 0.9633000000000003, 5: 0.35329999999999995, 6: 0.6134, 7: 1.2666999999999997, 8: 1.4733, 9: 1.5799999999999996, 10: 1.4367, 11: 1.4867, 12: 1.6766999999999999, 13: 1.5133}, 'crise': {4: 0, 5: 0, 6: 0, 7: 1, 8: 1, 9: 1, 10: 1, 11: 0, 12: 0, 13: 0}, 'prev': {4: 0.187232436694447, 5: 0.3105689948109355, 6: 0.2539228791308063, 7: 0.13916924600664243, 8: 0.11171277377207611, 9: 0.09915690331613247, 10: 0.11627115062260285, 11: 0.1100762567460945, 12: 0.08869839928964451, 13: 0.10687875779138178}}  And here is the code of the plot : import plotly.express as px fig = px.line(test,x='sasdate', y="prev", labels = 'ligne') fig.update_layout(legend=dict( yanchor="top", y=0.99, xanchor="left", x=0.01 )) fig.update_layout(showlegend =True) fig.show()  It seems the plots doesn't consider all the observations of the series. My 'date' features is on a datetime format, the problem is the same with matplotlib. Do you have suggestions ?

Answer: You dates are in the wrong format. They need to be YYYY-MM-DD rather than YYYY-DD-MM, e.g., swap '1959-01-09 00:00:00' to be '1959-09-01 00:00:00' as so on, and the plot will look as expected. If you don't want to do the conversion manually, you could use the datetime package and tell it the format of the date/time that you have, by doing something like: from datetime import datetime test["sasdate"] = { i: Timestamp(datetime.strptime(str(test["sasdate"][i]), "%Y-%d-%m %H:%M:%S")) for i in test["sasdate"] }  If test is already a DataFrame, then you could do: from pandas import Series test["sasdate"] = Series( [ Timestamp(datetime.strptime(str(date), "%Y-%d-%m %H:%M:%S")) for date in test["sasdate"] ], index=test["sasdate"].index )
Question: Let's consider I have the following TypedDict: class A(TypedDict): a: int b: int  What is the best practice for setting default values for this class? I tried to add a constructor but it doesn't seem to work. class A(TypedDict): a: int b: int def __init__(self): TypedDict.__init__(self) a = 0 b = 1  EDIT: I don't want to use dataclass because I need to serialize and deserialize to JSON files and dataclasses have some problem with it. What do you think?

Answer: TypedDict is only for specifying that a dict follows a certain layout, not an actual class. You can of course use a TypedDict to create an instance of that specific layout but it doesn't come with defaults. One possible solution is to add a factory method to the class. You could use this factory method instead to set defaults. from typing import TypedDict class A(TypedDict): a: int b: int @classmethod def create(cls, a: int = 0, b: int = 1) -> A: return A(a=a, b=b) a = A.create(a=4) # {"a": 4, "b": 1}  If having a dict is not a strict requirement then @dataclass is good having small objects with defaults. from dataclasses import dataclass @dataclass class A: a: int = 0 b: int = 1  If you need to create a dictionary from them, you can use asdict from dataclasses import asdict a = A(a=4) asdict(a) # {"a": 4, "b": 1}
Question: I am logging the model using  mlflow.sklearn.log_model(model, "my-model")  and I want to set tags to the model during logging, I checked that this method does not allow to set tags, there is a mlflow.set_tags() method but it is tagging the run not the model. Does anyone know how to tag the model during logging? Thank you!

Answer: When using mlflow.sklearn.log_model you work with the experiment registry which is run-focused so only experiments and runs can be described and tagged. If you want to set tags on models, you need to work with the model registry. The solution I would recommend is to register the model when logging using registered_model_name (there are more fine-grained ways, too) and use MLFlowClient API to set custom properties (like tags) of the already registered model. Here is a working example: import mlflow from mlflow.client import MlflowClient mlflow.set_tracking_uri('http://0.0.0.0:5000') experiment_name = 'test_mlflow' try: experiment_id = mlflow.create_experiment(experiment_name) except: experiment_id = mlflow.get_experiment_by_name(experiment_name).experiment_id from sklearn.linear_model import LogisticRegression from sklearn.datasets import load_iris from sklearn.metrics import accuracy_score with mlflow.start_run(experiment_id = experiment_id): # log performance and register the model X, y = load_iris(return_X_y=True) params = {"C": 0.1, "random_state": 42} mlflow.log_params(params) lr = LogisticRegression(**params).fit(X, y) y_pred = lr.predict(X) mlflow.log_metric("accuracy", accuracy_score(y, y_pred)) mlflow.sklearn.log_model(lr, artifact_path="models", registered_model_name='test-model' ) # set extra tags on the model client = MlflowClient(mlflow.get_tracking_uri()) model_info = client.get_latest_versions('test-model')[0] client.set_model_version_tag( name='test-model', version=model_info.version, key='task', value='regression' )  Here is the illustration  See also this excellent documentation of MLFlow Client.
Question: I've been experimenting with C modules in python and I've had no problems with compilation. However, when it comes to importing the module I receive this error message: >>> import _strrev Traceback (most recent call last): File "<stdin>", line 1, in <module> ImportError: /path/to/my/module.so: undefined symbol: strrev  The contents of the file are _strrev.c #include <Python.h> #include <string.h> static PyObject* _strrev(PyObject* self, PyObject *args) { PyObject* name; if (!PyArg_ParseTuple(args, "U", &name)) { return NULL; } PyObject *ret = strrev(*name); return ret; } static struct PyMethodDef methods[] = { {"strrev", (PyCFunction)_strrev, METH_VARARGS}, {NULL, NULL} }; static struct PyModuleDef module = { PyModuleDef_HEAD_INIT, "_strrev", NULL, -1, methods }; PyMODINIT_FUNC PyInit__strrev(void) { return PyModule_Create(&module); }  setup.py from distutils.core import setup, Extension setup( name='strrev_lib', version='1', ext_modules=[Extension('_strrev', ['_strrev.c'])], )  When running nm -D on this object file, I get this output $ nm -D _strrev.cpython-39-x86_64-linux-gnu.so w __cxa_finalize@GLIBC_2.2.5 w __gmon_start__ w _ITM_deregisterTMCloneTable w _ITM_registerTMCloneTable U PyArg_ParseTuple 00000000000011a0 T PyInit__strrev U PyModule_Create2 U __stack_chk_fail@GLIBC_2.4 U strrev  I can see that strrev is here, so I don't understand why it says that the symbol in undefined. I've also tried using a different module and that works perfectly fine when attempting to import it. $ python3 Python 3.9.2 (default, Feb 28 2021, 17:03:44) [GCC 10.2.1 20210110] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import _hello >>> _hello.hello("Jared") 'hello Jared'  The contents of the files are: _hello.c #include <Python.h> static PyObject* _hello(PyObject* self, PyObject *args) { PyObject* name; if (!PyArg_ParseTuple(args, "U", &name)) { return NULL; } PyObject *ret = PyUnicode_FromFormat("hello %U", name); return ret; } static struct PyMethodDef methods[] = { {"hello", (PyCFunction)_hello, METH_VARARGS}, {NULL, NULL} }; static struct PyModuleDef module = { PyModuleDef_HEAD_INIT, "_hello", NULL, -1, methods }; PyMODINIT_FUNC PyInit__hello(void) { return PyModule_Create(&module); }  setup.py from setuptools import setup from setuptools import Extension setup( name='hello-lib', version='1', ext_modules=[Extension('_hello', ['_hello.c'])], )  Can anyone give me pointers as to what I'm doing wrong? Is it something with the compilation or with the code that I've written? Any help would be appreciated!

Answer:  I can see that strrev is here, so I don't understand why it says that the symbol in undefined.  Au contraire: you can see that strrev is not there -- U in nm output means undefined. The strrev appears to be a Windows thing, and you are not on Windows. There is no such symbol in GLIBC: $ nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep strrev $  To fix this, you would need to implement strrev() yourself.
Question: I have some scraped data that needs some cleaning. After the cleaning, I want to create a "numerical and categorical pipelines" inside a ColumnTransformer such as: categorical_cols = df.select_dtypes(include='object').columns numerical_cols = df.select_dtypes(exclude='object').columns num_pipeline = Pipeline( steps=[ ('scaler', StandardScaler()) ] ) cat_pipeline = Pipeline( steps=[ ('onehotencoder', OneHotEncoder(handle_unknown='ignore')) ] ) preprocessor = ColumnTransformer([ ('num_pipeline', num_pipeline, numerical_cols), ('cat_pipeline', cat_pipeline, categorical_cols) ])  My idea was to create a transformer class Transformer(BaseEstimator, TransformerMixin): and create a pipeline with it. That transformer would include all the cleaning steps. My problem is that some of the steps change the dtype from object to integer mostly so I'm thinking that instead of defining the categorical_cols and numerical_cols with dtypes, instead, do it with column names. Would that be the correct approach? The idea would be to automate the process so I can train the model with new data every time.

Answer: Instead of making a list of columns beforehand you can use scikit-learn's make_column_selector to dynamically specify the columns that each transformer will be applied to. In your example: from sklearn.compose import make_column_selector as selector preprocessor = ColumnTransformer([ ('num_pipeline', num_pipeline, selector(dtype_exclude=object)), ('cat_pipeline', cat_pipeline, selector(dtype_include=object)) ])  Under the hood it uses pandas' select_dtypes for the type selection. You can pass a regex and select based on column name as well. I also recommend you checking out make_column_transformer for more control over the pipeline.
Question: I would like to match either "direction" (">" or "<") from a string like "->", "<==", "...". When the string contains no direction, I want to match "". More precisely, the equivalent Python expression would be: ">" if ">" in s else ("<" if "<" in s else "")  I first came up with this simple regular expression: re.search(r"[<>]|", s)[0]  ... but it evaluates to "" as soon as the direction is not in the first position. How would you do that?

Answer: A simpler version of Andrej Kesely's solution: re.search('<|>|$', s)[0]
Question: In AWS, created a docker image with a python script to print a string(basicprint.py) docker file: FROM python COPY ./basicprint.py ./ CMD ["python", "basicprint.py"]  It works fine then saved docker image as .tgz file. copy that .tgz file in to my local. I converted docker image(.tgz) into singularity image by using singularity build sing_image.sif docker-archive://filename.tgz  it was successfully created sing_image.sif singularity run sing_image.sif  It throws an error : basicprint.py: No such file or directory Any suggestions on correct method of conversion without missing the file.

Answer: Note that you put your file under the very root. singularity removed your file during conversion and cleaning. Similar issues have been reported and are, in general, expected when working with containers. This slightly modified image FROM python:3.11-slim COPY ./basicprint.py ./ CMD ["ls"]  demonstrates that the file lands under the linux root: (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ docker build . --tag test-sing (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ docker run -it test-sing root@ed8e3545a00b:/# ls basicprint.py bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var  Now, let me demonstrate the working version. Importantly, follow best Docker practices and use the working dir # Dockerfile FROM python:3.11-slim-bullseye RUN mkdir -p /usr/src WORKDIR /usr/src COPY ./basicprint.py /usr/src/ CMD ["python", "/usr/src/basicprint.py"]  where the sample Python script is # basicprint.py print('Welcome to my Docker!')  Then I build and test on Debian GNU/Linux 10 with Docker 20.10.17 and Singularity 3.11: (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ singularity --version singularity-ce version 3.11.0+277-gcd6fa5c0d (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ sudo docker build . --tag test-sing:latest Successfully tagged test-sing:latest (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ docker save test-sing:latest --output test-sing.tgz (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ sudo singularity build test-sing.sif docker-archive:test-sing.tgz ... INFO: Build complete: test-sing.sif (base) maciej.skorski@kaggle-cpu-maciej:~/docker-debug$ sudo singularity run test-sing.sif Welcome to my Docker!  Under the same software, I reproduced your error. So I think it has been solved! If you need any further assistance, we can set up a shared Virtual Machine with GitHub Codespace to make it fully reproducible.
Question: I created a model and deployed using streamlit. I am running in a virtual environement and inspite of running pip install streamlit it is still not working. The following error is shown enter image description here The error that is shown is this Traceback (most recent call last): File "C:\Python39\lib\runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Python39\lib\runpy.py", line 87, in _run_code exec(code, run_globals) File "D:\Machine Learning\sms-spam-classifier\venv\Scripts\streamlit.exe\__main__.py", line 4, in <module> File "d:\machine learning\sms-spam-classifier\venv\lib\site-packages\streamlit\__init__.py", line 70, in <module> from streamlit.delta_generator import DeltaGenerator as _DeltaGenerator File "d:\machine learning\sms-spam-classifier\venv\lib\site-packages\streamlit\delta_generator.py", line 90, in <module> from streamlit.elements.arrow_altair import ArrowAltairMixin File "d:\machine learning\sms-spam-classifier\venv\lib\site-packages\streamlit\elements\arrow_altair.py", line 35, in <module> from altair.vegalite.v4.api import Chart ModuleNotFoundError: No module named 'altair.vegalite.v4' PS D:\Machine Learning\sms-spam-classifier> 

Answer: Did you try pip install altair
Question: I created a bar plot with hatches using seaborn. I was also able to add a legend that included the hatch styles, as shown in the MWE below: import matplotlib.pyplot as plt import seaborn as sns tips = sns.load_dataset("tips") hatches = ['\\\\', '//'] fig, ax = plt.subplots(figsize=(6,3)) sns.barplot(data=tips, x="day", y="total_bill", hue="time") # loop through days for hues, hatch in zip(ax.containers, hatches): # set a different hatch for each time for hue in hues: hue.set_hatch(hatch) # add legend with hatches plt.legend().loc='best' plt.show()   However, when I try to create a histogram on seaborn with a legend, the same code does not work; I get the following error: No artists with labels found to put in legend. Note that artists whose label start with an underscore are ignored when legend() is called with no argument. I've looked for an answer online, but I haven't been successful at finding one. How can I add the hatches to the legend of a histogram for the MWE below? import matplotlib.pyplot as plt import seaborn as sns tips = sns.load_dataset("tips") hatches = ['\\\\', '//'] fig, ax = plt.subplots(figsize=(6,3)) sns.histplot(data=tips, x="total_bill", hue="time", multiple='stack') # loop through days for hues, hatch in zip(ax.containers, hatches): # set a different hatch for each time for hue in hues: hue.set_hatch(hatch) # add legend with hatches plt.legend().loc='best' # this does not work plt.show()  

Answer: The issue is ax1.get_legend_handles_labels() returns empty lists for seaborn.histplot. Refer to this answer. Use the explicit interface by adding ax=ax to seaborn.histplot(...). Use ax.get_legend().legend_handles (.legendHandles is deprecate) to get the handles for the legend, and add hatches with set_hatch(). ax.get_legend().legend_handles returns the handles in the reverse order compared to the container, so the order can be reversed with [::-1]. Tested in python 3.11.2, matplotlib 3.7.1, seaborn 0.12.2 import matplotlib.pyplot as plt import seaborn as sns tips = sns.load_dataset("tips") hatches = ['\\\\', '//'] fig, ax = plt.subplots(figsize=(6, 3)) sns.histplot(data=tips, x="total_bill", hue="time", multiple='stack', ax=ax) # added ax=ax # iterate through each container, hatch, and legend handle for container, hatch, handle in zip(ax.containers, hatches, ax.get_legend().legend_handles[::-1]): # update the hatching in the legend handle handle.set_hatch(hatch) # iterate through each rectangle in the container for rectangle in container: # set the rectangle hatch rectangle.set_hatch(hatch) plt.show()
Question: I have been working with the gymnasium environment for some weeks now and I had no problems with it in Google Colab by using this command in the notebook: pip3 install gymnasium[box2d] However, without changing anything I try to run the command once again and it suddenly stopped installing Box2d properly. I get the following output: Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/ Collecting gymnasium[box2d] Downloading gymnasium-0.28.1-py3-none-any.whl (925 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 925.5/925.5 kB 19.7 MB/s eta 0:00:00 Requirement already satisfied: numpy>=1.21.0 in /usr/local/lib/python3.10/dist-packages (from gymnasium[box2d]) (1.22.4) Collecting jax-jumpy>=1.0.0 (from gymnasium[box2d]) Downloading jax_jumpy-1.0.0-py3-none-any.whl (20 kB) Requirement already satisfied: cloudpickle>=1.2.0 in /usr/local/lib/python3.10/dist-packages (from gymnasium[box2d]) (2.2.1) Requirement already satisfied: typing-extensions>=4.3.0 in /usr/local/lib/python3.10/dist-packages (from gymnasium[box2d]) (4.5.0) Collecting farama-notifications>=0.0.1 (from gymnasium[box2d]) Downloading Farama_Notifications-0.0.4-py3-none-any.whl (2.5 kB) Collecting box2d-py==2.3.5 (from gymnasium[box2d]) Downloading box2d-py-2.3.5.tar.gz (374 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 374.4/374.4 kB 11.0 MB/s eta 0:00:00 Preparing metadata (setup.py) ... done Collecting pygame==2.1.3 (from gymnasium[box2d]) Downloading pygame-2.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.7/13.7 MB 28.2 MB/s eta 0:00:00 Collecting swig==4.* (from gymnasium[box2d]) Downloading swig-4.1.1-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 13.0 MB/s eta 0:00:00 Building wheels for collected packages: box2d-py error: subprocess-exited-with-error × python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip. Building wheel for box2d-py (setup.py) ... error ERROR: Failed building wheel for box2d-py Running setup.py clean for box2d-py Failed to build box2d-py ERROR: Could not build wheels for box2d-py, which is required to install pyproject.toml-based projects  I tried changing versions of gymnasium and even installing simply from gym, but the error persists. I don't know what's going on because literally yesterday I was not having any problems with this.

Answer: I was running into the same problem. After some digging and quite a lot of trial and error, I was able to get it to work by first running pip install swig. I hope that helps.
Question: I would like to merge two dataframes df1 and df2 in order to compare two values info 1 and info 2. The key to merge them is hidden in the name columns. Df1 is 'clean' as it has a first name column and a last name column. Df2, however, is tricky. There is only a name column and the names can be given in different ways. The standard case is first and last name but as shown in the picture below it can contain two names separated by an 'and' or a '&' or it can even be something totally different like a school.  Here is the dummy data in code: data1 = [['Anna','Tessmann',10], ['Ben','Fachmann',20], ['John','Smith',10]] df1 = pd.DataFrame(data1, columns=['FirstName','LastName','Info1']) data2 = [['Ben Fachmann',30], ['School AAA',40], ['John and Melissa Smith',50], ['Bob & Anna Tessmann',20]] df2= pd.DataFrame(data2, columns=['Name','Info2'])  Would anyone know an efficient way to merge these two? Is there the possibility to merge on st like 'df2.Name contains df1.Lastname'? Or I was looking into trying to parse df2.Name, I found nameparser import HumanName but I think it can't deal with 'and' and '&'. I apologize if something is unclear. Thanks a lot for any help in advance!

Answer: You can use a double substring merge: import re pattern1 = '|'.join(map(re.escape, df1['FirstName'])) pattern2 = '|'.join(map(re.escape, df1['LastName'])) match1 = df2['Name'].str.extractall(f'(?P<FirstName>{pattern1})').droplevel(1) match2 = df2['Name'].str.extractall(f'(?P<LastName>{pattern2})').droplevel(1) out = df1.merge(df2.join(match1).join(match2), on=['FirstName', 'LastName'])  Output:  FirstName LastName Info1 Name Info2 0 Anna Tessmann 10 Bob & Anna Tessmann 20 1 Ben Fachmann 20 Ben Fachmann 30 2 John Smith 10 John and Melissa Smith 50
Question: I am using Python's virtual environment 'venv'. My current version is 3.11.2 I need to downgrade it. I have already tried the following steps: pip3 install python==3.10.10  and got the following error: ERROR: Could not find a version that satisfies the requirement python==3.10.10 (from versions: none) ERROR: No matching distribution found for python==3.10.10 I also tried for lower versions like 3.8.0, 3.9.0, ... Allways same error. Thanks

Answer: To answer your question directly Try python.org. Head straight to https://www.python.org/downloads/ to download the distribution you want. If I am to guess what problem you are facing, here are some details When you are running your command prompt, make sure you know which python you are executing. Example: PS C:\Users\jli8\pythonWork\dpr-data-presenter> conda activate p310 (p310) PS C:\Users\jli8\pythonWork\dpr-data-presenter> .\venv\Scripts\activate (venv) (p310) PS C:\Users\jli8\pythonWork\dpr-data-presenter> where.exe python C:\Users\jli8\pythonWork\dpr-data-presenter\venv\Scripts\python.exe C:\Users\jli8\Anaconda3\envs\p310\python.exe C:\Users\jli8\AppData\Local\Programs\Python\Python311\python.exe C:\Users\jli8\AppData\Local\Microsoft\WindowsApps\python.exe (venv) (p310) PS C:\Users\jli8\pythonWork\dpr-data-presenter>  From my terminal above, I actually have 3 pythons.  Python3.11 installed in AppData\Local\Programs\Python\Python311\python.exe Python3.10 installed in Anaconda env "p310" (which I used to create venv), finally the last venv in use with is in my working folder dpr-data-presenter\venv\Scripts\python.exe  From this limited context here, the most straightforward answer to you would be just download python3.10.10 from python.org. Make sure you understand how to execute from that python3.10.10 that you have installed and create your venv from that python3.10.10. Some helpful reference for you: https://realpython.com/python-virtual-environments-a-primer/