eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Is there a way how to display the counted value of the histogram aggregate in the Plotly.Express histogram? px.histogram(pd.DataFrame({"A":[1,1,1,2,2,3,3,3,4,4,4,5]}),x="A")  If I would use regular histogram, I can specify text parameter which direct to the column which contain the value to display. px.bar(pd.DataFrame({"val":[1,2,3,4,5], "height": [3,2,3,3,1]}), x="val", y="height", text="height")  But with histograms, this value is calculated and it's not even part of the fig.to_dict(). Is there a way to add the text labels into histogram? Using the answers below, I've summarized the finding to an article - https://towardsdatascience.com/histograms-with-plotly-express-complete-guide-d483656c5ad7

Answer: As far as I know, plotly histograms do not have a text attribute. It also turns out that it's complicated if at all possible to retrieve the applied x and y values and just throw them into appropriate annotations. Your best option seems to be to take care of the binning using numpy.histogram and the set up your figure using go.Bar. The code snippet below will produce the following plot:  Complete code: import numpy as np import plotly.express as px import plotly.graph_objects as go # sample data df = px.data.tips() # create bins bins = [0, 10, 20, 30, 40, 50] counts, bins = np.histogram(df.total_bill, bins=bins) #bins2 = 0.5 * (bins1[:-1] + bins2[1:]) fig = go.Figure(go.Bar(x=bins, y=counts)) fig.data[0].text = counts fig.update_traces(textposition='inside', textfont_size=8) fig.update_layout(bargap=0) fig.update_traces(marker_color='blue', marker_line_color='blue', marker_line_width=1, opacity=0.4) fig.show()
Question: Assume I have a pydantic model class EventEditRequest(BaseModel): uid: UUID name: str start_dt: datetime end_dt: datetime  I send request with body b'{"uid":"a38a7543-20ca-4a50-ab4e-e6a3ae379d3c","name":"test event2222","start_dt":1600414328,"end_dt":1600450327}' So both start_dt and end_dt are unix timestamps. But in endpoint they become datetimes with timezones. @app.put('...') def edit_event(event_data: EventEditRequest): event_data.start_dt.tzinfo is not None # True  I don't want to manually edit start_dt and end_dt in the endpoint function to get rid of timezones. How can I set up my pydantic model so it will make datetime without timezones?

Answer: You can use own @validator to parse datetime manually: from datetime import datetime from pydantic import BaseModel, validator class Model(BaseModel): dt: datetime = None class ModelNaiveDt(BaseModel): dt: datetime = None @validator("dt", pre=True) def dt_validate(cls, dt): return datetime.fromtimestamp(dt) print(Model(dt=1600414328)) print(ModelNaiveDt(dt=1600414328))  Output: dt=datetime.datetime(2020, 9, 18, 7, 32, 8, tzinfo=datetime.timezone.utc) dt=datetime.datetime(2020, 9, 18, 10, 32, 8)
Question: I'm currently trying the wemake-python-styleguide and found WPS335:  Using lists, dicts, and sets do not make much sense. You can use tuples instead. Using comprehensions implicitly create a two level loops, that are hard to read and deal with.  It gives this example: # Correct: for person in ('Kim', 'Nick'): ... # Wrong: for person in ['Kim', 'Nick']: ...  Is this purely personal preference or is there more that speaks for using a tuple? I can only think about speed, but I cannot imagine that this makes a difference. I think I have seen more people using lists and I wonder if there is a reason to change it.

Answer: Using lists instead of tuples as constants makes no difference in CPython. As of some versions, both are compiled to tuples. >>> dis.dis(""" ... for person in ["Kim", "Nick"]: ... ... ... """) 2 0 SETUP_LOOP 12 (to 14) 2 LOAD_CONST 0 (('Kim', 'Nick')) 4 GET_ITER >> 6 FOR_ITER 4 (to 12) 8 STORE_NAME 0 (person) 3 10 JUMP_ABSOLUTE 6 >> 12 POP_BLOCK >> 14 LOAD_CONST 1 (None) 16 RETURN_VALUE  Note how the list literal was transformed to a LOAD_CONST (('Kim', 'Nick')) instruction of a tuple.  As for preference, CPython prefers tuple. If you have the choice, you should do so as well.
Question: Should I commit and push my Django's project static files into my git repo? I know there is collectstatic command but it's just for prod deployment right? I work on the same project from 2 different computers and then, I have static files in one that I don't have on the other. Am I supposed to collectstatic from one to the other? But I don't understand this command neither how to use it nor what it does. Thank you for helping. Best regards.

Answer: Yes, normally we commit static files. The command collectstatic just copies the static files from the individual app folders into one general folder (normally used only in PROD server). But the static files should already be present (and committed) in the individual app folders, so that each development PC and also the PROD server have the same static files present.
Question: Is there a numpy way (and without for loop) to extract all the indices in a numpy array list_of_numbers, where values are in a list values_of_interest? This is my current solution: list_of_numbers = np.array([11,0,37,0,8,1,39,38,1,0,1,0]) values_of_interest = [0,1,38] indices = [] for value in values_of_interest: this_indices = np.where(list_of_numbers == value)[0] indices = np.concatenate((indices, this_indices)) print(indices) # this shows [ 1. 3. 9. 11. 5. 8. 10. 7.] 

Answer: Use numpy.where with numpy.isin: np.argwhere(np.isin(list_of_numbers, values_of_interest)).ravel()  Output: array([ 1, 3, 5, 7, 8, 9, 10, 11])
Question: I'm trying to update an existing Dockerfile to switch from python3.5 to python3.8, previously it was creating a symlink for python3.5 and pip3 like this: RUN ln -s /usr/bin/pip3 /usr/bin/pip RUN ln -s /usr/bin/python3 /usr/bin/python  I've updated the Dockerfile to install python3.8 from deadsnakes:ppa apt-get install python3-pip python3.8-dev python3.8-distutils python3.8-venv  if I remove python3-pip, it complains about gcc  C compiler or Python headers are not installed on this system. Try to run: sudo apt-get install gcc python3-dev  with these installations in place I'm trying to update existing symlink creation something like this: RUN ln -s /usr/bin/pip3 /usr/local/lib/python3.8/dist-packages/pip RUN ln -s /usr/bin/pip /usr/local/lib/python3.8/dist-packages/pip RUN ln -s /usr/bin/python3.8 /usr/bin/python3  it fails, saying  ln: failed to create symbolic link '/usr/bin/python3': File exists  which I assume fails because python3 points to python3.6. if I try: RUN ln -s /usr/bin/python3.8 /usr/bin/python it doesn't complain about symlink and image gets build successfully, but fails while installing requirements later (we use Makefile targets to install dependencies inside the container using pip and pip-sync):  ERROR: Cannot uninstall 'python-apt'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.  which I assume because python-apt gets installed as part of the default python3.6 installation and python3.8 pip can't uninstall it. PS: my Dockerfile image is based on Ubunut 18.04 which comes with python3.6 as default. How can I properly switch Dockerfile / image from python3.5 to python3.8? so I can later use pip directly and it points to python3.8's pip

Answer: Replacing the system python in this way is usually not a good idea (as it can break operating-system-level programs which depend on those executables) -- I go over that a little bit in this video I made "why not global pip / virtualenv?" A better way is to create a prefix and put that on the PATH earlier (this allows system executables to continue to work, but bare python / python3 / etc. will use your other executable) in the case of deadsnakes which it seems like you're using, something like this should work: FROM ubuntu:bionic RUN : \ && apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ software-properties-common \ && add-apt-repository -y ppa:deadsnakes \ && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \ python3.8-venv \ && apt-get clean \ && rm -rf /var/lib/apt/lists/* \ && : RUN python3.8 -m venv /venv ENV PATH=/venv/bin:$PATH  the ENV line is the key here, that puts the virtualenv on the beginning of the path $ docker build -t test . ... $ docker run --rm -ti test bash -c 'which python && python --version && which pip && pip --version' /venv/bin/python Python 3.8.5 /venv/bin/pip pip 20.1.1 from /venv/lib/python3.8/site-packages/pip (python 3.8)   disclaimer: I'm the maintainer of deadsnakes
Question: I am learning python 3 and programming in general for the first time, but I can't seem to distinguish a parameter and a variable?

Answer: A variable is just something that refers/points to some data you have. x = 5  Here x is a variable. Variables can point to more kinds of data than just numbers, though. They can point to strings, functions, etc. A parameter is something that is passed into a function def my_function(y): print(y)  Here y is a parameter. It doesn't contain a value yet. But if I want to call the function, I need to provide an argument to the function. An argument is the actual value you provide to the function that replaces the parameter. my_function(5)  Here, 5 is the argument. Of course, since x points to the value "5", I can do this too: my_function(x)  which also prints 5
Question: I installed Python on my computer. When I type python in the command prompt I get the following message: 'python' is not recognized as an internal or external command, operable program or batch file.  But when I type py it seems to be working and I get the following: Python 3.7.0 (v3.7.0, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information.  Why is this happening? FYI: I checked the path variable in environmental variables and I don't see any path to python installation. But then how is visual code able to find the path to python.exe and able to run python code? I am confused.

Answer: py is itself located in C:\Windows (which is always part of the PATH), which is why you find it. When you installed Python, you didn't check the box to add it to your PATH, which is why it isn't there. In general, it's best to use the Windows Python Launcher, py.exe anyway, so this is no big deal. Just use py for launching consistently, and stuff will just work. Similarly, if py.exe was associated with the .py extension at installation time, a standard shebang line (details in PEP linked above) will let you run the script without even typing py. I don't know precisely what VSCode uses to find Python (using py.exe directly, using a copy of Python that ships with the editor, performing registry lookup, a config file that just says where to find it, etc.), but that's not really relevant to running your scripts yourself.
Question: I’m building an app using Python and the clean architecture principles, with TDD. Some unit tests require executing some raw SQL queries against an in-memory database. I am trying to switch from sqlite to postgresql inmemory data, using pytest-postgres. Problem  When using sqlite inmemory database, I can both insert and select data. When using Postgresql inmemory database, I can only SELECT (raw INSERT fails).  Insert work in sqlite…  s_tb_name = "tb_customer" ls_cols = ["first_name", "last_name", "email"] ls_vals = ['("John", "Doe", "john.doe@mail.net")', '("Jane", "Doe", "jane.doe@mail.net")', '("Eric", "Dal", "eric.d@home.com")'] s_cols = ', '.join(ls_cols) s_vals = ', '.join(ls_vals) session.execute(f"INSERT INTO {s_tb_name} ({s_cols}) VALUES ({s_vals})")  …but fail in Postgres: E sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column "John" does not exist E LINE 1: ..., email) VALUES (("John"....  From this psycopg documentation page, I understand this is due to pyscopg2. It prevents injecting raw dynamic SQL, and it seems I should add this : tb_sql_id = sql.Identifier(s_tb_name) cols_sql_id = sql.SQL(' ,').join(map(sql.Identifier, ls_cols)) vals_sql_id = sql.SQL(' ,').join(map(sql.Literal, ls_vals)) psycopg2_query = sql.SQL(f"INSERT INTO {tb_sql_id} ({cols_sql_id}) VALUES ({vals_sql_id})")  but logically, sqlalchemy refuses to execute the psycopg2_query : sqlalchemy.exc.ArgumentError: SQL expression object expected, got object of type <class 'psycopg2.sql.SQL'> instead  Question Is there a way to execute raw dynamic insert queries in Postgres using SQL Alchemy?

Answer: As pointed by others, injecting SQL like this is to be avoided in most cases. Here, the SQL is written in the unit test itself. There is no external input leaking to the SQL injection, which alleviates the security risk. Mike Organek’s solution did not fully work for me, but it pointed me to the right direction : I just had to also remove the parens from ls_vals.  s_tb_name = "tb_customer" ls_cols = ["first_name", "last_name", "email"] ls_vals = ["'John', 'Doe', 'john.doe@mail.net'", "'Jane', 'Doe', 'jane.doe@mail.net'", "'Eric', 'Dal', 'eric.d@home.com'"] s_cols = ', '.join(ls_cols) s_vals = '(' + '), ('.join(ls_vals) + ')' session.execute(f"INSERT INTO {s_tb_name} ({s_cols}) VALUES {s_vals}")  This made the insert test pass, both when using the sqlite engine and the postgres engine.
Question: I have the following dataframe:  A B C 0 NaN NaN cat 1 dog NaN NaN 2 NaN cat NaN 3 NaN NaN dog  I would like to add a colunm with the value that doesnt have the NaN value. So that:  A B C D 0 NaN NaN cat cat 1 dog NaN NaN dog 2 NaN cat NaN cat 3 NaN NaN dog dog  would it be using an lambda function? or fillna? Any help would be appreciated! Thanks!

Answer: use combine_first chained df['D'] = df.A.combine_first(df.B).combine_first(df.C)  alternatively, forward fill and pick the last column df['D'] = df.ffill(axis=1).iloc[:,-1] # specifying the columns explicitly: df['D'] = df[['A', 'B', 'C']].ffill(1).iloc[:, -1]
Question: I'm trying to drop a row at certain index in every group inside a GroupBy object. The best I have been able to manage is: import pandas as pd x_train = x_train.groupby('ID') x_train.apply(lambda x: x.drop([0], axis=0))  However, this doesn't work. I have spent a whole day on this to no solution, so have turned to stack. Edit: A solution for any index value is needed as well

Answer: You can do it with cumcount idx= x_train.groupby('ID').cumcount() x_train = x_train[idx!=0]
Question: I am adding abstract classes to my python package like this: class AbstractClass(ABC): @abstractmethod def do_something(self): pass  There will be multiple subclasses that inherit from AbstractClass like this: class SubClass(AbstractClass): def do_something(self): pass  I am wondering if there are any conventions for python packages regarding abstract classes that I am unaware of.  should the abstract classes be separated from the subclasses, or should they all be in the same directory? what about naming of abstract functions, any conventions there?  I realize this is fairly subjective. I am asking for opinions, not what is 'right' or 'wrong' Thanks!

Answer: No convention comes to mind. If this is all for just one project, I'd be looking to put the abstract class in with the concrete subclasses at whatever level they're at. If all the subclasses were in one file, then I'd put the abstract class in that file too. If all the subclasses were in individual files in a dir, I'd put the abstract class right next to them in its own file. The reason I'd put the abstract class somewhere else is if I'm separating off utility code that can be reused with other projects. In short, an abstract class is just a piece of code. Just another class. Treat it like any other. As far as naming, the natural naming of the class given what it is/does is usually enough. If your abstract class were Animal, and your subclasses were Goat, Horse, etc., I'd see no reason to call your abstract class AbstractAnimal, as it's pretty clear already what's going on...that you wouldn't instantiate Animal directly. Also, if you're looking at a class thinking of reusing it, and you see an abstract method in it, then you know it's abstract.
Question: I'm trying to build a docker image on my ubuntu 18.04 machine and I have located requirements.txt in the same building directory but still its showing up this error.  Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt' The command '/bin/sh -c pip3 install -r requirements.txt' returned a non-zero code: 1  requirements.txt is to install python modules using pip3. requirement.txt:  numpy opencv-contrib-python opencv-python scikit-image pillow imutils scikit-learn matplotlib progressbar2 beautifulsoup4 pandas matplotlib re2 regex json argparse pickle  DockerFile: FROM nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04 COPY requirements.txt /home/usr/app/requirements.txt WORKDIR /home/usr/app/ RUN apt-get update && apt-get install -y python3 python3-pip sudo RUN pip3 install -r requirements.txt FROM tensorflow/tensorflow:latest-gpu-jupyter 

Answer: I suspect that you haven't copied over your requirements.txt file to your Docker image. Typically you add the following lines to your Dockerfile to copy your requirements.txt file and install it using pip: COPY requirements.txt /tmp/requirements.txt RUN python3 -m pip install -r /tmp/requirements.txt  If you don't explicitly copy over anything to your Docker image your image has no data save for what is on the base image.
Question: I am seeing that PyYaml, truncates zero's while loading from yaml file, if one uses: yaml.safe_load(stream). It can be fixed, if one uses yaml.load(stream, Loader=yaml.BaseLoader), but is that advisable? It works with yaml.load and zeros are not truncated. I want to understand that would it be safe to switch to yaml.load instead of yaml.safe_load? Example: Test yaml content: $cat test.yml number: 5.10  Code: $python -c 'import yaml, sys; content = yaml.safe_load(sys.stdin); print(content) ' < test.yml {'number': 5.1}  << It truncates the 0 at the end. But that is due to floating point value >> whereas what I want is the exact number as is. $python -c 'import yaml, sys; content = yaml.load(sys.stdin, Loader=yaml.BaseLoader); print(content) ' < test.yml {u'number': u'5.10'}  Is that the correct approach to change it to yaml.load ?

Answer: yaml.safe_load(sys.stdin) just does yaml.load(sys.stdin, Loader=yaml.SafeLoader). The facilities to execute arbitrary Python code (which makes loading unsafe) are implemented in yaml.Loader which is used by default. yaml.BaseLoader does not contain them. Therefore, if you use yaml.BaseLoader, loading will not execute arbitrary Python code (that is, unless you yourself register custom constructors with yaml.BaseLoader).
Question: I'm not able to rotate my xlabels in Seaborn/Matplotlib. I have tried many different solutions but not able to fix it. I have seen many related questions here on stackoverflow, but they have not worked for me. My current plot looks like this, but I want the xlabels to rotate 90.  @staticmethod def plotPrestasjon(plot): sns.set(style="darkgrid") ax = sns.catplot(x="COURSE", y="FINISH", hue="COURSE", col="BIB#", data=plot, s=9, palette="Set2") ax.set(xlabel='COURSES', ylabel='FINISH (sec)') plt.show()  I have tried: ax.set_xticklabels(ax.get_xticklabels(), rotation=90)  But that fails to generate the plot. Any ideas how I can fix it?

Answer:  The correct way to set the xticklabels for sns.catplot, according to the documentation, is with the .set_xticklabels method (.e.g. g.set_xticklabels(rotation=30)). Using a loop to iterate through the Axes, should be used if changes need to be made on a plot by plot basis, within the FacetGrid. Building structured multi-plot grids seaborn.FacetGrid g, or in the case of the OP, ax is a seaborn.axisgrid.FacetGrid  When iterating through ax.axes.flat, axes is a <class 'matplotlib.axes._subplots.AxesSubplot'>, which has a wide array of class methods, including .get_xticklabels().   In cases with many columns, and col_wrap= is used, g.set_xticklabels(rotation=30) may result in removing all the xticklabels.  Instead, labels= must be specified, g.set_xticklabels(g.axes.flat[-1].get_xticklabels(), rotation=30), where g.axes.flat[-1] should be the last facet axes with xticklabels. Alternatively, use g.tick_params(axis='x', rotation=30)   Tested in python 3.12, matplotlib 3.8.1, seaborn 0.13.0  One Row & Multiple Columns import seaborn as sns # load data exercise = sns.load_dataset("exercise") # plot catplot g = sns.catplot(x="time", y="pulse", hue="kind", col="diet", data=exercise) # set rotation g.set_xticklabels(rotation=30)   Multiple Rows & Columns # plot catplot g = sns.catplot(x="time", y="pulse", hue="diet", col="kind", col_wrap=2, data=exercise) # set rotation using one of the following options # g.set_xticklabels(labels=g.axes.flat[-1].get_xticklabels(), rotation=30) g.tick_params(axis='x', rotation=30)     Using g.set_xticklabels(g.get_xticklabels(), rotation=30) results in an AttributeError.  --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) <ipython-input-442-d1d39d8cc4f0> in <module> 1 g = sns.catplot(x="time", y="pulse", hue="kind", col="diet", data=exercise) ----> 2 g.set_xticklabels(g.get_xticklabels(), rotation=30) AttributeError: 'FacetGrid' object has no attribute 'get_xticklabels'
Question: Im trying to convert HEIC to JPG using python. The only other answers about this topic used pyheif. I am on windows and pyheif doesn't support windows. Any suggestions? I am currently trying to use pillow.

Answer: The following code will convert an HEIC file format to a PNG file format from PIL import Image import pillow_heif heif_file = pillow_heif.read_heif("HEIC_file.HEIC") image = Image.frombytes( heif_file.mode, heif_file.size, heif_file.data, "raw", ) image.save("./picture_name.png", format("png"))
Question: Is there any advantage to using the 'type hint' notation in python? import sys def parse(arg_line: int) -> str: print (arg_line) # passing a string, returning None if __name__ == '__main__': parse(' '.join(sys.argv[1:]))  To me it seems like it complicates the syntax without providing any actual benefit (outside of perhaps within a development environment). Based on this:  Are there any plans for python to contain type constraints within the language itself? What is the advantage of having a "type hint" ? Couldn't I just as easily throw that into the docstring or something?  I also don't see this much in the python codebase itself as far as I can tell -- most types are enforced manually, for example: argparse.py and any other files I've glanced at in https://github.com/python/cpython/blob/3.7/Lib/.

Answer:  Are there any plans for python to contain type constraints within the language itself?  Almost certainly not, and definitely not before the next major version (4.x).  What is the advantage of having a "type hint" ? Couldn't I just as easily throw that into the docstring or something?  Off the top of my head, consider the following:  Type hints can be verified with tooling like mypy. Type hints can be used by IDEs and other tooling to give hints and tips. E.g., when you're calling a function and you've just written foo(, the IDE can pick up on the type hints and display a box nearby that shows foo(x: int, y: List[int]). The advantage to you as a developer is that you have exactly the information you need exposed to you and don't have to munge an entire docstring. Type hints can be used by modules like functools.singledispatch or external libraries like multipledispatch to add additional type-related features (in this case, dispatching function calls based on name and type, not just name).