eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a dataframe containing positive,negative and zero values. import pandas as pd import numpy as np import matplotlib.pyplot as plt df1 = pd.DataFrame({'A': [-1, 3, 9, 5, 0, -1, -1], 'B': [4, -5, 5, 7, 9, 8, 6], 'C': [7, 5, -3,-1, 5, 9, 3], 'D': [-3, 6, 7, 4, 0, -2, -1], 'date':['01-02-2020', '01-06-2020', '01-03-2021', '01-05-2021', '01-10-2021', '01-03-2022', '01-08-2022']}) # make sure the time column is actually time format df1['date']=pd.to_datetime(df1['date']) # set time as the index df1.set_index('date',inplace=True) df1  I want to show the month and year on the X-axis, there should be 4 scatterplots and I want the markers (i.e. scatter circles) to be 'red' for the negative values in the corresponding month/year,'green' for positive and yellow for zero, which I have tried to do with a list comprehension statement below. # to condition by colour kcolors = ['green' if s>0 else 'red' if s<0 else 'yellow' for s in df1] # this gives a TypeError: '>' not supported between instances of 'str' and 'int' # I have also tried # kcolors = ['#00ff41' if s>0 else '#65000B' if s<0 else '#F5D300' for s in df1] # plot fig, ax = plt.subplots() df1.plot(kind='scatter', marker='o', color=kcolors, ax=ax) ax.figure.autofmt_xdate(rotation=45, ha='center') plt.legend(loc='best') plt.show()  How can I achieve this?

Answer: The original post requested the use of a line plot with markers but was changed because a comment suggested that wasn't possible. The following answer color-codes the markers, and the line from its marker. Tested in python v3.12.3, pandas v2.2.2, matplotlib v3.8.4. import pandas as pd import matplotlib.pyplot as plt # Create the DataFrame df1 = pd.DataFrame({ 'A': [-1, 3, 9, 5, 0, -1, -1], 'B': [4, -5, 5, 7, 9, 8, 6], 'C': [7, 5, -3, -1, 5, 9, 3], 'D': [-3, 6, 7, 4, 0, -2, -1], 'date': ['01-02-2020', '01-06-2020', '01-03-2021', '01-05-2021', '01-10-2021', '01-03-2022', '01-08-2022'] }) # Convert the date column to datetime format df1['date'] = pd.to_datetime(df1['date']) # Set the date column as the index df1.set_index('date', inplace=True) # Create a 2x2 subplot grid fig, axs = plt.subplots(2, 2, figsize=(14, 10)) axs = axs.flatten() # Flatten the 2x2 array of axes for easy iteration # Loop through each column and its corresponding subplot axis for ax, col in zip(axs, df1.columns): # Loop through each pair of consecutive points for i in range(len(df1)-1): x = df1.index[i:i+2] # Get the two consecutive dates y = df1[col].iloc[i:i+2] # Get the two consecutive values for the current column # Determine the color based on the value if y.iloc[0] > 0: color = 'green' elif y.iloc[0] < 0: color = 'red' else: color = 'yellow' # Plot the segment with the determined color ax.plot(x, y, marker='o', color=color) # Set the title of the subplot to the column name ax.set_title(f'{col}') # Rotate and format the x-axis dates ax.figure.autofmt_xdate(rotation=45, ha='center') # Adjust layout to prevent overlap plt.tight_layout() # Display the plot plt.show()   Similar Questions:  color line by "value" How to plot one line in different colors Different color for line depending on corresponding values
Question: I'm trying to do a simple log plot using the matplotlib library, but I can't seem to get the x-axis ticks to have the same fontsize. My example code is: import numpy as np import matplotlib.pyplot as plt fontsize = 8 x = np.linspace(5.6e-5,6.5e-5, 120) y = np.logspace(-10, 10, 120) fig = plt.figure(figsize=(3.5, 2.65), constrained_layout=True) ax = fig.gca() ax.plot(x,y) plt.xlabel('X label', fontsize=fontsize) plt.ylabel('Y label', fontsize=fontsize) ax.tick_params(axis='both', labelsize=fontsize) plt.gca().set_yscale('log') plt.gca().set_xscale('log') plt.show()  And the resulting plot: 

Answer: It seems the ticks on the x axis are minor ones and are not affected the way you tried. If you also provide 'both' for the which parameter as in plt.tick_params(axis='both', which='both', labelsize=fontsize)  it should work.
Question: I was trying to install "causal_conv1d" using: pip install --no-cache-dir -t /scratch/ahmed/lib causal_conv1d==1.0.0  The error I got is: Collecting causal_conv1d==1.0.0 Downloading causal_conv1d-1.0.0.tar.gz (6.4 kB) Preparing metadata (setup.py) ... error error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [9 lines of output] Traceback (most recent call last): File "<string>", line 2, in <module> File "<pip-setuptools-caller>", line 34, in <module> File "/tmp/pip-install-9i0wsv2k/causal-conv1d_fc0a21267f664102adca1aa336c93106/setup.py", line 19, in <module> from torch.utils.cpp_extension import ( File "/scratch/ahmed/lib/torch/utils/cpp_extension.py", line 28, in <module> from pkg_resources import packaging # type: ignore[attr-defined] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ImportError: cannot import name 'packaging' from 'pkg_resources' (/scratch/ahmed/lib/pkg_resources/__init__.py) [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. 

Answer: I don't know the exact problem, but it seems this problem happened when I used two directories for Python "lib": one was the default Anaconda lib, and I had another separate one. The problem disappeared when I used only the default Anaconda lib. It works fine now.
Question: I am using regex to allow alphanumeric, underscore, hyphen and square brackets in a text box. regex i am using to validate input is  r'^[a-zA-Z0-9_\-\[\] ]*$.  I need to modify the regex such that if empty brackets are given it should return false. Sample cases "Your message here" - Valid "Your [text] message here" - Valid "your_text_message [text]" - valid "Your [] message here" - Invalid "[] message" - Invalid

Answer: Can it go like this? ^(?!.*\[\s*\].*)[a-zA-Z0-9_\-\[\] ]*$
Question: Consider the following multi-level column index dataframe: import numpy as np import pandas as pd arrays = [ ["A", "A", "B", "B"], ["one", "two", "one", "two"], ["1", "2", "1", "pd.NA"], ] idx = pd.MultiIndex.from_arrays(arrays, names=["level_0", "level_1", "level_2"]) data = np.random.randn(3, 4) df = pd.DataFrame(data, columns=idx) print(df) level_0 A B level_1 one two one two level_2 1 2 1 pd.NA 0 -1.249285 0.314225 0.011139 0.675274 1 -0.654808 -0.492350 0.596338 -0.087334 2 0.113570 0.566687 -0.361334 0.085368  level_2 holds values of type object (str really). df.columns.get_level_values(2) Index(['1', '2', '1', 'pd.NA'], dtype='object', name='level_2')  I need to parse it to the correct data type and change this particular column level. new_level_2 = [ pd.NA if x == "pd.NA" else int(x) for x in df.columns.get_level_values(2) ]  I am looking for a pythonic way to replace the old level_2 with new_level_2.

Answer: You could convert the MultiIndex.to_frame then back to MultiIndex.from_frame, change the values with replace and as_type: df.columns = pd.MultiIndex.from_frame(df.columns.to_frame() .replace({'level_2': {'pd.NA': pd.NA}}) .astype({'level_2': 'Int64'}))  Output: level_0 A B level_1 one two one two level_2 1 2 3 <NA> 0 0.144044 1.454274 0.761038 0.121675 1 0.443863 0.333674 1.494079 -0.205158 2 0.313068 -0.854096 -2.552990 0.653619
Question: This is a followup question to What does asyncio.create_task() do? There, as well in other places, asyncio.create_task(c) is described as "immediately" running the coroutine c, when compared to simply calling the coroutine, which is then only executed when it is awaited. It makes sense if you interpret "immediately" as "without having to await it", but in fact a created task is not executed until we run some await (possibly for other coroutines) (in the original question, slow_coro started being executed only when we await fast_coro). However, if we do not run any await at all, the tasks are still executed (only one step, not to completion) at the end of the program: import asyncio async def counter_loop(x, n): for i in range(1, n + 1): print(f"Counter {x}: {i}") await asyncio.sleep(0.5) return f"Finished {x} in {n}" async def main(): slow_task = asyncio.create_task(counter_loop("Slow", 4)) fast_coro = asyncio.create_task(counter_loop("Fast", 2)) print("Created tasks") for _ in range(1000): pass print("main ended") asyncio.run(main()) print("program ended")  the output is Created tasks main ended Counter Slow: 1 Counter Fast: 1 program ended  I am curious: why are the two created tasks executed at all if there was no await being run anywhere?

Answer: Let's try to put it in these words: the event loop enters execution with one task to be performed: the main(). When "main()" is complete, there are other two tasks ready to be processed - so before returning to main() caller, those are executed up the next await statement inside each one. (either an await or an async for or async with). For the next loop iteration, as the main task is "over", the loop cancels the remaining tasks, and shuts down. This happens because what signals the loop that the "main" task is over is a callback that is set when loop.run_until_complete (called by asyncio.run) is done: it is this callback that signals that the loop should stop. But the callback itself will be executed only on the next loop iteration, after the main co-routine is done. And the loop iteration, even though gets a mark that the asyncio loop should stop there, will only actually shutdown after running once over all pending tasks - this implies in advancing each created task to the next await point. This is done by throwing asyncio.CancelledError into the tasks code, at the await statement. So if you have a try/except/finally clause encompassing the await, you can still clean-up your task before the loop ends. All of that is not documented in "English" - it is rather the current code behavior in the asyncio implementation. One have to follow the code at the asyncio/base_events.py file to understand it.
Question: I'm using SQLModel for an API. I want the object to be like this: class Post(SQLModel): id: int categories: list[str] # category.name  instead of nested objects: class Post(SQLModel): id: int categories: list[Category]  Do I have to change the serialization function or is there a way to do this automatically?

Answer: As far as I understood Post is your db model. I would highly recommend to separate you db model (should reflect the database structure) from dto (should be used for comfortable data transfer between application components) class PostDTO(pydantic.BaseModel): id: int categories: list[str] @classmethod def from_db_model(cls, model: Post) -> "PostDTO": """Build Post DTO based on db model.""" return cls( id=model.id, categories=[category.name for category in model.categories] ) post = Post(...) post_dto = PostDTO.from_db_model(model=post)  Don't be afraid to create an extra object if it makes your code more scalable, readable and logical!
Question: I am currently working with Snowflake to view the query history. However, when I attempt to execute the same command in Python, following its syntax rules, I encounter an error stating ‘INVALID IDENTIFIER’ for ‘information_schema’. select * from table(information_schema.query_history()) 

Answer: Usually these things can be resolved by simply advising the documentation, and reviewing the query profile from within Snowflake. In this case, please try using the following code, with snowflake specified instead of directly targeting the information_schema. Other than that, also ensure your user has the correct and specified access. select * from table(snowflake.information_schema.query_history()) order by start_time  If your issue persists, please add as an comment what the query looks like from snowflake point of view, as what you're running in python might not be what is actually running depending on your lib.
Question: I can create a sample data frame with the following code and save it as parquet. When I try to read it throws "TypeError: unhashable type: 'numpy.ndarray'". Is it possible to save an index comprised of tuples or do I have to reset the index before saving to parquet? Thanks import pandas as pd # Creating sample data data = { 'A': [1, 2, 3], 'B': [6, 7, 8], 'C': [11, 12, 13], } # Creating multi-index index = pd.MultiIndex.from_tuples( [ ((10, 30), (0.75, 1.0)), ((10, 30), (0.75, 1.25)), ((10, 30), (1.0, 1.25)) ], names=['level_0', 'level_1'] ) # Creating DataFrame with multi-index df = pd.DataFrame(data, index=index) print(df) df.to_parquet(path="test.parquet") pd.read_parquet("test.parquet") 

Answer: You must specify the levels: import pandas as pd data = { 'A': [1, 2, 3], 'B': [6, 7, 8], 'C': [11, 12, 13], } index = pd.MultiIndex.from_tuples( [ (str((10, 30)), str((0.75, 1.0))), (str((10, 30)), str((0.75, 1.25))), (str((10, 30)), str((1.0, 1.25))) ], names=['level_0', 'level_1'] ) df = pd.DataFrame(data, index=index) print("Original DataFrame:") print(df) df_reset = df.reset_index() df_reset.to_parquet(path="test.parquet") df_read = pd.read_parquet("test.parquet") df_read.set_index(['level_0', 'level_1'], inplace=True) print("DataFrame read from Parquet:") print(df_read)  which returns Original DataFrame: A B C level_0 level_1 (10, 30) (0.75, 1.0) 1 6 11 (0.75, 1.25) 2 7 12 (1.0, 1.25) 3 8 13 DataFrame read from Parquet: A B C level_0 level_1 (10, 30) (0.75, 1.0) 1 6 11 (0.75, 1.25) 2 7 12 (1.0, 1.25) 3 8 13
Question: I'm new to Docker and want to build a Dockerfile with the following snippet: RUN \ cur=`pwd` && \ wget http://www.coppeliarobotics.com/files/CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz && \ tar -xf CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz && \ export COPPELIASIM_ROOT="$cur/CoppeliaSim_Edu_V4_1_0_Ubuntu20_04" && \ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$COPPELIASIM_ROOT:$COPPELIASIM_ROOT/platforms && \ export QT_QPA_PLATFORM_PLUGIN_PATH=$COPPELIASIM_ROOT && \ git clone https://github.com/stepjam/PyRep.git && \ cd PyRep && \ pip3 install -r requirements.txt && \ pip3 install setuptools && \ pip3 install .  Because of the network issue, the build process is unstable and interrupted on the git clone... line. However, when I try to build again, CoppeliaSim_Edu_V4_1_0_Ubuntu20_04 which has already been saved is downloaded again, which consumes a lot of time. Is there any method to skip the download process if the .tar file has already been saved during the last build process?

Answer: Put the download command of .tar files on a separate command. RUN wget ttp://www.coppeliarobotics.com/files/CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz && \ tar -xf CoppeliaSim_Edu_V4_1_0_Ubuntu20_04.tar.xz RUN # rest of your code   Each instruction in the Dockerfile translates to a layer in your final image. You can think of image layers as a stack, with each layer adding more content on top of the layers that came before it:    Whenever a layer changes, that layer will need to be re-built. If a layer changes, all other layers that come after it are also affected.  source : docker docs What it means is: After the first built, Docker stores each layer in Cache, in the future builds, if the dockerfile content doesn't change, docker uses the cached layer instead of rebuilding them. To your question: "Is there any method to skip the download process if the .tar file has already been saved during the last build process?" Put the download .tar file command in a separate RUN Command that precedes your other commands to take advantage of docker caching.
Question: Here's simple example of code: class meta(type): pass class Test(str, metaclass = meta): pass  When I run mypy on it (just from cli, without any flags or other additional setup) I'm seeing next output: test.py:2: error: Metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases [misc]  I know that similar error appears at runtime when metaclass of derived class is not a subtype of all parents metaclasses. But type(str) is type, and my class meta derived from type, so I cannot understand what's incorrect here. Provided example isn't working only with str parent class. Mypy says everything is correct when inheriting from int or float. So, how to get rid of such error message? Or it is just a mypy bug?

Answer: str is statically-typed as a subclass of typing.Protocol, whose metaclass is special-cased by mypy as typing._ProtocolMeta (typing.Protocol is actually an instance of typing._ProtocolMeta at runtime, but this isn't reflected in the typing stubs, so normally mypy wouldn't know about this). Therefore, to accurately reflect both static typing and runtime in your example, you can do this (mypy Playground, Pyright Playground): import typing as t if t.TYPE_CHECKING: from typing import _ProtocolMeta else: _ProtocolMeta = type(str) class meta(_ProtocolMeta): pass class Test(str, metaclass=meta): pass  Since typing._ProtocolMeta has a bare minimum typing interface, this leaves you relatively free to implement meta without too much maintenance if type-checkers and/or the Python runtime decides to change the metaclass of str in the future. Note that keyword arguments in a class declaration do not currently work properly in mypy for user-defined metaclasses, so you'll run into some type-safety issues for those. Instead of using metaclasses, consider using a base class defining __init_subclass__ instead.
Question: From what I've read, the built-in types such as int, float, dict, etc have been implemented in C, and they used to be different from user-defined classes before Python 2.2 However, the built-in types and user defined classes have long been unified now, and the following happens >>> int.__class__ <class 'type'>  Why does this happen? I'm aware that type is a metaclass in Python, and all the user defined classes (by default) are (meta) instances of the same, and it gives me the impression that any object that is an instance of type has been instantiated using the type metaclass. But built-in classes, such as int, have been implemented in C, and haven't been created using the type metaclass, right? Then, why does int.__class__, for example, give <class 'type'>? Is it that, "manually", the __class__ attribute of built-in types such as int, has been set to type, to enable the unification?

Answer: Classes created in C are still classes and share the same core structure in memory as classes created in pure Python code. The same mechanisms used from Python to create a class can be used to create a class from C code. The only thing is that they don't "have to": a class in C can have all its inner fields hard coded, and its class members manually assigned: unlike Python code there are no guards for assigning the __class__ field of the class object. As I said: regardless of that, these built-in classes are still "real" instances of type, as the memory layout to the special slots, both public and private have the layout used by type. And - on top of all that - even for Python defined classes, it is possible to change the __class__ attribute of a class (but with restrictions). yes, as simple as that: just assign a new value to the __class__ attribute of an existing instance. This will effectivelly "transform" an object into another, carrying its instance namespace. It is not the same thing as cast does in static typed languages: with cast in Java/C++ an object is unchanged, and the compiler just tried to apply the next operation as if the casted instance where from the cast-target class. Assigning to __class__ is a permanent change (until a reassignment, of course) to the underlying instance: See this in action: In [51]: class A: ...: def a(self): ...: ... ...: In [52]: class B: ...: def b(self): ...: print("at method B") ...: In [53]: a = A() In [54]: a.__class__ = B In [55]: a.b() at method B  It won't work if both classes have diffing "memory layouts" - that is, data structures defined in native code, such as lists or tuples, or using the __slots__ mechanism. But for "ordinary" classes which keep the instance namespace in the __dict__ attribute, it just works.
Question: I tried various methods found online but can't add an image at the correct location in the top left corner of my figure. Here's one on my attempts, where I try to set exact the size of the generated figure in pixels, and then add a 48x48 px image in top left corner. Issues:  The figures are not created with the chosen pixel size On the X axis, where the origin is left, I need to offset by 34 pixels to get the image near to left border, which makes no sense to me. On the Y axis, the image is also badly placed, mostly because the size is not as expected.  Why is the code below failing? How can I achieve what I want? import matplotlib.pylab as plt # Plot from PIL import Image # Add background image from matplotlib.offsetbox import AnnotationBbox, OffsetImage # Add background image def plot_image(w, h): px = 1/plt.rcParams['figure.dpi'] fig,ax = plt.subplots(figsize=(w*px, h*px)) image_path = '......../The Simpsons 48x48.png' image_data = Image.open(image_path) image_box = OffsetImage(image_data, zoom=1) anno_box = AnnotationBbox(image_box, xy=(34,h-48), xycoords='figure pixels', frameon=False) ax.add_artist(anno_box) plt.text(0.5, 0.5, f'Requested: {w}x{h}') plt.text(0.5, 0.4, 'Obtained: 579x455' if w==640 else 'Obtained: 759x561' if w==800 else 'Obtained: 960x711' ) plt.show() plot_image( 640, 480) plot_image( 800, 600) plot_image(1024, 768)  Results:    Image used: 

Answer: Are you using a notebook? When you show in a notebook the figure is automatically resized to fit around the artists on it. This is equivalent to passing bbox_inches='tight' when you call savefig. It is possible to turn that off in a notebook, but I just tried replacing plt.show() in your example with plt.savefig(f'simpson_test{w}x{h}.png') and they all came out the right size. I have not understood why the positioning is strange with AnnotationBbox, but a simpler way is figimage: import matplotlib.pyplot as plt # Plot from PIL import Image # Add background image def plot_image(w, h): px = 1/plt.rcParams['figure.dpi'] fig,ax = plt.subplots(figsize=(w*px, h*px)) image_path = 'simpson.png' image_data = Image.open(image_path) fig.figimage(image_data, yo=h-48) plt.text(0.5, 0.5, f'Requested: {w}x{h}') plt.savefig(f'simpson_test{w}x{h}.png') plot_image( 640, 480) plot_image( 800, 600) plot_image(1024, 768)