eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have the following code which produces a pandas.io.formats.style.Styler object: import pandas as pd import numpy as np df = pd.DataFrame({'text': ['foo foo', 'bar bar'], 'number': [1, 2]}) df1 = df.style.set_table_styles([dict(selector='th', props=[('text-align', 'center')])]) df2 = df1.set_properties(**{'text-align': 'center'}).hide_index() df2 # df2 is a pandas.io.formats.style.Styler object  How do I print df2 if I have more code running underneath the above script, for eg.: import pandas as pd import numpy as np df = pd.DataFrame({'text': ['foo foo', 'bar bar'], 'number': [1, 2]}) df1 = df.style.set_table_styles([dict(selector='th', props=[('text-align', 'center')])]) df2 = df1.set_properties(**{'text-align': 'center'}).hide_index() df2 np.round(0.536, 2)  I tried using the print statement but it's giving me an output as below: import pandas as pd import numpy as np df = pd.DataFrame({'text': ['foo foo', 'bar bar'], 'number': [1, 2]}) df1 = df.style.set_table_styles([dict(selector='th', props=[('text-align', 'center')])]) df2 = df1.set_properties(**{'text-align': 'center'}).hide_index() print(df2) np.round(0.536, 2)  <pandas.io.formats.style.Styler object at 0x000000000B4FAFC8> 0.54  Any help would really be appreciated. Many thanks in advance.

Answer: I found the answer for this: import pandas as pd from IPython.display import display import numpy as np df = pd.DataFrame({'text': ['foo foo', 'bar bar'], 'number': [1, 2]}) df1 = df.style.set_table_styles([dict(selector='th', props=[('text-align', 'center')])]) df2 = df1.set_properties(**{'text-align': 'center'}).hide_index() display(df2) np.round(0.536, 2)
Question: I want people that know of the old name to be directed to the new name. For the pypi website, it's easy to upload a package with a README linking to the new package. I'm not sure what's the best way to handle people using pip to install it. I assume it might be possible to show an error on pip install old_name, looking around it seems to be possible using cmdclass in setup.py and maybe throwing an exception in the right place but the documentation around it is scarce to put it mildly. So I was wondering if anyone is aware of proper built-in systems for this, or common practices to handle this sort of thing.

Answer: Declare the new package a dependency of the old. See for example how scikit-learn does it: the old package sklearn declares in its setup.py: install_requires=['scikit-learn'],  Thus everyone who does pip install sklearn automatically gets scikit-learn.
Question: Is there any in-built way in pydantic to specify options? For example, let's say I want a string value that must either have the value "foo" or "bar". I know I can use regex validation to do this, but since I use pydantic with FastAPI, the users will only see the required input as a string, but when they enter something, it will give a validation error. All in-built validations of pydantic are displayed in the api interface, so would be great if there was something like  class Input(BaseModel): option: "foo" || "bar" 

Answer: Yes, you can either use an enum: class Choices(Enum): foo = 'foo' bar = 'bar' class Input(BaseModel): option: Choices  see here Or you can use Literal: from typing import Literal class Input(BaseModel): option: Literal['foo', 'bar']  see here
Question: I am currently accessing the parent directory of my file using Pathlib as follows: Path(__file__).parent  When I print it, and this gives me the following output: print('Parent: ', Path(__file__).parent) #output /home/user/EC/main-folder  The main-folder has a .env file which I want to access and for that I want to join the parent path with the .env. Right now, I did: dotenv_path = os.path.join(Path(__file__).parent, ".env")  which works. But I would like to know, if there is a Pathlib alternate to os.path.join()? Something like: dotenv_path = pathlib_alternate_for_join(Path(__file__).parent, ".env") 

Answer: Use pathlib.Path.joinpath: (Path(__file__).parent).joinpath('.env')
Question: Consider the following package structure:  With the following setup.py contents: from setuptools import setup, find_packages setup( name='dfl_client', packages=find_packages(exclude=['*tests*']), include_package_data=True, package_data={"": ['py.typed', '*.pyi']}, )  When I package it using python setup.py sdist bdist_wheel, the resulting wheel:  contains the py.typed file, which is good contains the tests folder, while it should be excluded according to the find_packages doc.  I spent hours trying to understand why with no success. Especially because it seems to work for other projects !

Answer: (I spent so many time trying to understand this stupid issue that I answer my own question hoping that can save time to others facing the same problem) I finally found the culprit: it is a hidden interaction between setuptools_scm and the include_package_data=True flag. By itself, include_package_data=True does not make the tests directory be included in the wheel. However if setuptools_scm is installed and the folder is under version control (and the tests directory is in the list of git-managed files), then the exclude directive does not seem to be taken into account anymore. So the solution was simply to remove the include_package_data=True, that is actually not needed when package_data is present: from setuptools import setup, find_packages setup( name='dfl_client', packages=find_packages(exclude=['*tests*']), package_data={"": ['py.typed', '*.pyi']}, )  See setuptools doc on including files (that is actually very straightforward about include_package_data) and this related issue and workaround (the workaround seems to work for the wheel too, not only the sdist).
Question: I'm trying to consume messages from a Kafka topic. I'm using a wrapper around confluent_kafka consumer. I need to check if connection is established before I start consuming messages. I read that the consumer is lazy, so I need to perform some action for the connection to get established. But I want to check the connection establishment without doing a consume or poll operation. Also, I tried giving some bad configurations to see what the response on a poll would be. The response I got was: b'Broker: No more messages'  So, how do I decide if the connection parameters are faulty, the connection is broken, or there actually are no messages in the topic?

Answer: I am afraid there is no direct approach for testing whether Kafka Brokers are up and running. Also note that if your consumer has already consumed the messages it doesn't mean that this is a bad behaviour and obviously it does not indicate that the Kafka broker is down.  A possible workaround would be to perform some sort of quick operation and see if the broker responds. An example would be listing the topics: Using confluent-kafka-python and AdminClient # Example using confuent_kafka from confluent_kafka.admin import AdminClient kafka_broker = {'bootstrap.servers': 'localhost:9092'} admin_client = AdminClient(kafka_broker) topics = admin_client.list_topics().topics if not topics: raise RuntimeError()  Using kafka-python and KafkaConsumer # example using kafka-python import kafka consumer = kafka.KafkaConsumer(group_id='test', bootstrap_servers=['localhost:9092']) topics = consumer.topics() if not topics: raise RuntimeError()  Use kafka-python at your own risk, library has not been updated in years, might not be compatible with amazon msk or confluent containers
Question: I was following this guide from official jetbrains page, until the step 2 comes in the existence. In the picture mentioned in that page, has so many options like ssh, wsl, vagrant, docker, etc. In my pycharm (latest 2019.3.4) it only shows 4 options - venv, conda, pipenv and system-interpreter.  There is no WSL menu in the add python interpreter dialog.  See the below image:  Searched web for an hour and found no results that show how to fix it. I started plugin search in the PyCharm if there's an external plugin to do so, but there were no plugin named as WSL. I don't know how to setup the WSL interpreter, I have python3.8 installed on my wsl right now. Any help will be appreciated!

Answer: I have solved this by  Uninstalling pycharm with history and cache. Removing folders completely from C:\Users\%USERNAME%\AppData\Local and C:\Users\%USERNAME%\AppData\Roaming\JetBrains and Clean re-install of pycharm WSL interpreter option shows up as normal Ideavim is creating a conflict I guess.
Question: I'm trying to add text labels next to the data points in a Plotly scatter plot in Python but I get an error.  How can I do that? Here is my dataframe:  world_rank university_name country teaching international research citations income total_score num_students student_staff_ratio international_students female_male_ratio year 0 1 Harvard University United States of America 99.7 72.4 98.7 98.8 34.5 96.1 20,152 8.9 25% NaN 2011  Here is my code snippet: citation = go.Scatter( x = "World Rank" + timesData_df_top_50["world_rank"], <--- error y = "Citation" + timesData_df_top_50["citations"], <--- error mode = "lines+markers", name = "citations", marker = dict(color = 'rgba(48, 217, 189, 1)'), text= timesData_df_top_50["university_name"])  The error is shown below. TypeError: ufunc 'add' did not contain a loop with signature matching types dtype('<U32') dtype('<U32') dtype('<U32') 

Answer: You can include the text labels in the text attribute. To make sure that they are displayed on the scatter plot, set mode='lines+markers+text'. See the Plotly documentation on text and annotations. I included an example below based on your code. import plotly.graph_objects as go import pandas as pd df = pd.DataFrame({'world_rank': [1, 2, 3, 4, 5], 'university_name': ['Harvard', 'MIT', 'Stanford', 'Cambridge', 'Oxford'], 'citations': [98.8, 98.7, 97.6, 97.5, 96]}) layout = dict(plot_bgcolor='white', margin=dict(t=20, l=20, r=20, b=20), xaxis=dict(title='World Rank', range=[0.9, 5.5], linecolor='#d9d9d9', showgrid=False, mirror=True), yaxis=dict(title='Citations', range=[95.5, 99.5], linecolor='#d9d9d9', showgrid=False, mirror=True)) data = go.Scatter(x=df['world_rank'], y=df['citations'], text=df['university_name'], textposition='top right', textfont=dict(color='#E58606'), mode='lines+markers+text', marker=dict(color='#5D69B1', size=8), line=dict(color='#52BCA3', width=1, dash='dash'), name='citations') fig = go.Figure(data=data, layout=layout) fig.show()
Question: I'm working on this plot:  I need to write something inside the first plot, between the red and the black lines, I tried with ax1.text() but it shows the text between the two plots and not inside the first one. How can I do that? The plot was set out as such: fig, (ax1,ax2) = plt.subplots(nrows=2, ncols=1, figsize = (12,7), tight_layout = True) 

Answer: Without more code details, it's quite hard to guess what is wrong. The matplotlib.axes.Axes.text works well to show text box on subplots. I encourage you to have a look at the documentation (arguments...) and try by yourself. The text location is based on the 2 followings arguments:  transform=ax.transAxes: indicates that the coordinates are given relative to the axes bounding box, with (0, 0) being the lower left of the axes and (1, 1) the upper right. text(x, y,...): where x, y are the position to place the text. The coordinate system can be changed using the below parameter transform.  Here is an example: # import modules import matplotlib.pyplot as plt import numpy as np # Create random data x = np.arange(0,20) y1 = np.random.randint(0,10, 20) y2 = np.random.randint(0,10, 20) + 15 # Create figure fig, (ax1,ax2) = plt.subplots(nrows=2, ncols=1, figsize = (12,7), tight_layout = True) # Add subplots ax1.plot(x, y1) ax1.plot(x, y2) ax2.plot(x, y1) ax2.plot(x, y2) # Show texts ax1.text(0.1, 0.5, 'Begin text', horizontalalignment='center', verticalalignment='center', transform=ax1.transAxes) ax2.text(0.9, 0.5, 'End text', horizontalalignment='center', verticalalignment='center', transform=ax2.transAxes) plt.show()  output
Question: I am new in using PyCharm but I am loving it gradually. I am getting a red underline on <!DOCTYPE html> and the error is "Unexpected Token". Why PyCharm shows it? I can't understand. 

Answer: It usually happens when you don't enable Django in Pycharm's settings. To resolve the problem:  In Pycharm open Setting in File menu Select and expand Languages & Frameworks Select Django and enable it Select your Django project root Select your project setting.py file Select your project manage.py file Apply setting
Question: Is it possible for a GitLab CI/CD pipeline to commit code changes? I would like to run a stage that uses black to format my code automatically whenever I push my work. gitlab-ci.yml image: python:3.6 stages: - test before_script: - python3 -m pip install -r requirements.txt test:linting: script: - black ./  I made sure to include a file that needs reformatting to test if this works.  Job output  $ black ./ reformatted test.py All done! ✨ 🍰 ✨ 1 file reformatted.  The file in my repository remains unchanged which leads me to believe that this might not be possible.

Answer: Black won't automatically commit corrected python code unless you use pre-commit hook. Best way to run black in CI is to include something like : black . --check --verbose --diff --color  This will fail the test if python code fail to adhere to code format and force user to fix formatting. please checkout black --help for all tags. This is a good reference on Black : https://www.mattlayman.com/blog/2018/python-code-black/ Repo : https://github.com/psf/black
Question: In VS Code with Python, we can run a "cell" (block that starts with #%%) in the "Python Interactive Window" Can we do the same thing on the Integrated Terminal? I know we can do this in Spyder, where the terminal is generally always an IPython terminal Matlab works in the same way with its terminal. Can we do this in VS Code?

Answer: I've opened a issue in GitHub, @AdamAL also opened, but it seems they don't have intention to do this. Here is a workaround to other users. EDIT: I've answered before with a workaround that takes 2 VSCode extensions and not use the smart command jupyter.selectCellContents. @AdamAL shared a better solution using this command, but with the caveat that the cursor loses position and focusing only in Python terminal. @Maxime Beau pointed that out expanding @AdamAL solution to the active terminal (that could be IPython, for instance) Now I'm taking all answers and posting a general solution. This solution don't loose the cursor position when running only one cell. Also there's another command that runs the cell and advance to the next cell (as in Jupyter) and it is general to the active terminal (that could be IPython). 1. Install macros extension: This extension have some additional commands that multi-command didn't have (like the delay command) 2. In settings.json "macros.list": { "runCellinTerminal": [ "jupyter.selectCellContents", "workbench.action.terminal.runSelectedText", "cursorUndo", {"command": "$delay","args": {"delay": 100}}, {"command": "workbench.action.terminal.sendSequence","args": { "text": "\n" }}, ], "runCellinTerminaladvance": [ "jupyter.selectCellContents", "workbench.action.terminal.runSelectedText", "cursorDown", {"command": "$delay","args": {"delay": 100}}, {"command": "workbench.action.terminal.sendSequence","args": { "text": "\n" }}, ], }  OBS: cursorUndo takes the cursor back to the right position. cursorDown takes the cursor to the next cell. The commands delay and sendSequence \n are usefull when the terminal is an IPython terminal. 3. In keybinding.json: { "key": "ctrl+alt+enter", "command": "macros.runCellinTerminal", "when": "editorTextFocus && jupyter.hascodecells" }, { "key": "shift+alt+enter", "command": "macros.runCellinTerminaladvance", "when": "editorTextFocus && jupyter.hascodecells" },
Question: I want to create a modified dataframe with the specified columns. I tried the following but throws the error "Passing list-likes to .loc or [] with any missing labels is no longer supported" # columns to keep filtered_columns = ['text', 'agreeCount', 'disagreeCount', 'id', 'user.firstName', 'user.lastName', 'user.gender', 'user.id'] tips_filtered = tips_df.loc[:, filtered_columns] # display tips tips_filtered  Thank you

Answer: It looks like Pandas has deprecated this method of indexing. According to their docs:  This behavior is deprecated and will show a warning message pointing to this section. The recommended alternative is to use .reindex()  Using the new recommended method, you can filter your columns using: tips_filtered = tips_df.reindex(columns = filtered_columns). NB: To reindex rows, you would use reindex(index = ...) (More information here).
Question: I have a module installed in my Juyter notebook !pip install gensim Requirement already satisfied: gensim in /home/m.gawinecki/virtualenv/la-recoms/lib/python3.7/site-packages (3.8.2)  However, when I try to import it, it fails import gensim --------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) <ipython-input-1-e70e92d32c6e> in <module> ----> 1 import gensim ModuleNotFoundError: No module named 'gensim'  It looks like it has been installed properly: !pip list | grep gensim gensim 3.8.2  How can I fix it?

Answer: Add your virtual environment as Python kernel in this way (Make sure it's activated): (venv) $ ipython kernel install --name "local-venv-kernel" --user  Now, you can select the created kernel "local-venv-kernel" when you start Jupyter notebook or lab. You could check the installed libraries using this code in a notebook cell: !pip freeze
Question: This answer contains a very elegant way of setting all the types of your pandas columns in one line: # convert column "a" to int64 dtype and "b" to complex type df = df.astype({"a": int, "b": complex})  I am starting to think that that unfortunately has limited application and you will have to use various other methods of casting the column types sooner or later, over many lines. I tested 'category' and that worked, so it will take things which are actual python types like int or complex and then pandas terms in quotation marks like 'category'. I have a column of dates which looks like this: 25.07.10 08.08.10 07.01.11  I had a look at this answer about casting date columns but none of them seem to fit into the elegant syntax above. I tried: from datetime import date df = df.astype({"date": date})  but it gave an error: TypeError: dtype '<class 'datetime.date'>' not understood  I also tried pd.Series.dt.date which also didn't work. Is it possible to cast all your columns including the date or datetime column in one line like this?

Answer: This has been answered in the comments where it was noted that the following works: df.astype({'date': 'datetime64[ns]'})  In addition, you can set the dtype when reading in the data: pd.read_csv('path/to/file.csv', parse_dates=['date'])
Question: When I do: >>> import os >>> os.uname()  I get an attribute error which looks like this: Traceback (most recent call last): File "<pyshell#1>", line 1, in <module> os.uname() AttributeError: module 'os' has no attribute 'uname'  How can I fix this is my python broken or something else because in the docs.

Answer: I've run your code the exact same way in IDLE on Windows 10 and got the same result. >>> print(os.uname()) Traceback (most recent call last): File "<pyshell#2>", line 1, in <module> print(os.uname()) AttributeError: module 'os' has no attribute 'uname'  And as @Joran Beasley pointed out, this function is only available in certain operating systems. From an online compiler: posix.uname_result(sysname='Linux', nodename='Check', release='5.4.10-x86_64-linode132', version='#1 SMP PREEMPT Thu Jan 9 21:17:12 UTC 2020', machine='x86_64')  If you want to get current os, I recommend the platform module. >>> import platform >>> platform.platform() 'Windows-10-10.0.18362-SP0'  Some people prefer using os module, but platform is much more readable.
Question: my dataframe looks something like this:  user age gender 0 23 12 male 1 24 13 male 2 25 15 female 3 26 20 male 4 27 21 male  and using px.sunburst(df, path=["gender", "age"])  gives me correct sunburst plot where gender is in middle part of pie chart and for each gender it has associated ages. I want to do this using graph_objects instead of plotly express since I want two sunburst plots to be side by side. from df I have above how can I use it in graph_objects. I do not understand what values to add to lables, parents, ids, etc... fig = go.Figure() fig.add_trace( go.Sunburst( lables = df.age, parents = df.gender, domain=dict(column=0) ) ) fig.show()  I've read the documentation however I cannot understand how it works. If someone knows, please tell me how I can create sunburst plot using graph_object with df I have above.

Answer: The answer: Just build one figure using px, and "steal" all your figure elements from there and use it in a graph_objects figure to get what you need!  The details: If px does in fact give you the desired sunburst chart like this: Plot 1:  Code 1: # imports import pandas as pd import plotly.graph_objects as go import plotly.express as px # data df = pd.DataFrame({'user': [23, 24, 25, 26, 27], 'age': [12, 13,15, 20, 21], 'gender': ['male','male', 'female','male', 'male'] }) # plotly express figure fig = px.sunburst(df, path=["gender", "age"]) fig.show()  Then, to my knowledge, you'll have to restructure your data in order to use graph_objects. Currently, your data has the form  And graph_objects would require label = ['12', '13', '15', '20', '21', 'female', 'male']. So what now? Go through the agonizing pain of finding the correct data structure for each element? No, just build one figure using px, and "steal" all your figure elements from there and use it in a graph_objects figure: Code 2: # imports import pandas as pd import plotly.graph_objects as go import plotly.express as px # data df = pd.DataFrame({'user': [23, 24, 25, 26, 27], 'age': [12, 13,15, 20, 21], 'gender': ['male','male', 'female','male', 'male'] }) # plotly express figure fig = px.sunburst(df, path=["gender", "age"]) # plotly graph_objects figure fig2 =go.Figure(go.Sunburst( labels=fig['data'][0]['labels'].tolist(), parents=fig['data'][0]['parents'].tolist(), ) ) fig2.show()  Plot 2:  Now, if you'd like to display som more features of your dataset in the same figure, just add ids=fig['data'][0]['ids'].tolist() to the mix: Plot 3:  Complete code: # imports import pandas as pd import plotly.graph_objects as go import plotly.express as px # data df = pd.DataFrame({'user': [23, 24, 25, 26, 27], 'age': [12, 13,15, 20, 21], 'gender': ['male','male', 'female','male', 'male'] }) # plotly express figure fig = px.sunburst(df, path=["gender", "age"]) # plotly graph_objects figure fig2 =go.Figure(go.Sunburst( labels=fig['data'][0]['labels'].tolist(), parents=fig['data'][0]['parents'].tolist(), values=fig['data'][0]['values'].tolist(), ids=fig['data'][0]['ids'].tolist(), domain={'x': [0.0, 1.0], 'y': [0.0, 1.0]} )) fig2.show()
Question: I have a problem with this plot:  The y-axis is in unit but I need them to be in millions as such:  Do you know a method to achieve this? Thanks in advance.

Answer: You can use a custom FuncFormatter like this: from matplotlib.ticker import FuncFormatter import matplotlib.pyplot as plt def millions(x, pos): 'The two args are the value and tick position' return '%1.1fM' % (x * 1e-6) formatter = FuncFormatter(millions) fig, ax = plt.subplots() ax.yaxis.set_major_formatter(formatter)  Or you can even replace millions with the following functions to support all magnitudes:  def human_format(num, pos): magnitude = 0 while abs(num) >= 1000: magnitude += 1 num /= 1000.0 # add more suffixes if you need them return '%.2f%s' % (num, ['', 'K', 'M', 'G', 'T', 'P'][magnitude])