eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: when I try to reload firewalld, it tells me Error: COMMAND_FAILED: 'python-nftables' failed: internal:0:0-0: Error: Could not process rule: Numerical result out of range JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_policy_allow-host-ipv6"}}}]}  I don't know why this is, after Google, it still hasn't been resolved

Answer: I had the same error message. I enabled verbose debugs on firewalld and tailed the logs to file for a deeper dive. In my case the exception was originally happening in "nftables.py" on line "361". Exception:  2022-01-23 14:00:23 DEBUG3: <class 'firewall.core.nftables.nftables'>: calling python-nftables with JSON blob: {"nftables": [{"metainfo": {"json_schema_version": 1}}, {"add": {"chain": {"family": "inet", "table": "firewalld", "name": "filter_IN_policy_allow-host-ipv6"}}}]} 2022-01-23 14:00:23 DEBUG1: Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/firewall/core/fw.py", line 888, in rules backend.set_rule(rule, self._log_denied) File "/usr/lib/python3.6/site-packages/firewall/core/nftables.py", line 390, in set_rule self.set_rules([rule], log_denied) File "/usr/lib/python3.6/site-packages/firewall/core/nftables.py", line 361, in set_rules raise ValueError("'%s' failed: %s\nJSON blob:\n%s" % ("python-nftables", error, json.dumps(json_blob))) ValueError: 'python-nftables' failed: internal:0:0-0: Error: Could not process rule: Numerical result out of range  Line 361 in "nftables.py":  self._loader(config.FIREWALLD_POLICIES, "policy")  Why this is a problem: Basically nftables is a backend service and firewalld is a frontend service. They are dependent on each other to function. Each time you restart firewalld it has to reconcile the backend, in this case nftables. At some point during the reconciliation a conflict is occurring in the python code. That is unfortunate as the only real solution will likely have to come from code improvements from nftables in how it is able to populate policies into chains and tables. A work-around: The good news is, if you are like me, you don't use ipv6, in which case we simply disable the policy rather than solve for the issue. I'll put the work-around steps below. Work-around Steps: The proper way to remove the policy is to use the command "firewall-cmd --delete-policy=allow-host-ipv6 --permanent" but I encountered other errors and exceptions in python when attempting to do that. Since I don't care about ipv6 I manually deleted the XML from configuration and restarted the firewalld service.  rm /usr/lib/firewalld/policies/allow-host-ipv6.xml   rm /etc/firewalld/policies/allow-host-ipv6.xml   systemctl restart firewalld  Side Note: Once I fixed this conflict, I also had some additional conflicts between nftables/iptables/fail2ban that had to be cleared up. For that I just used the command "fail2ban-client unban --all" to make fail2ban wipe clean all of the chains it added to iptables.
Question: Since this morning, I'm no longer able to run projects in PyCharm. When generating a new virtual environment, I get an "Invalid Python SDK" error. Cannot set up a python SDK at Python 3.11... The SDK seems invalid. What I noticed: No matter what base interpreter I select (3.8, 3.9, 3.10) Pycharm always generates a Python 3.11 interpreter. I did completely uninstall PyCharm, as well as all my python installations and reinstalled everything. I also went through the "Repair IDE" option in PyCharm. I also removed and recreated all virtual environments. When I run "cmd" and type 'python' then python 3.10.1 opens without a problem. This morning, I installed a new antivirus software that did some checks and deleted some "unnecessary files" - maybe it is related (antivirus software is uninstalled again).

Answer: Dealt with the same issue despite using python and pycharm without issue for months. Recently kept giving me the error despite changing the PATH variable of my system and even manually pathing within pycharm. After hours of reinstalling pycharm, python and even jumping around versions with no success it turned out it was because my python directory had a space in it that it just randomly decided to break. For anyone who has tried what seems like everything to no avail ensure that NO part of the path to your python directory contains spaces
Question: I have a logger function from logging package that after I call it, I can send the message through logging level. I would like to send this message also to another function, which is a Telegram function called SendTelegramMsg(). How can I get the message after I call the funcion setup_logger send a message through logger.info("Start") for example, and then send this exatcly same message to SendTelegramMsg() function which is inside setup_logger function? My currently setup_logger function: # Define the logging level and the file name def setup_logger(telegram_integration=False): """To setup as many loggers as you want""" filename = os.path.join(os.path.sep, pathlib.Path(__file__).parent.resolve(), 'logs', str(dt.date.today()) + '.log') formatter = logging.Formatter('%(levelname)s: %(asctime)s: %(message)s', datefmt='%m/%d/%Y %H:%M:%S') level = logging.DEBUG handler = logging.FileHandler(filename, 'a') handler.setFormatter(formatter) consolehandler = logging.StreamHandler() consolehandler.setFormatter(formatter) logger = logging.getLogger('logs') if logger.hasHandlers(): # Logger is already configured, remove all handlers logger.handlers = [] else: logger.setLevel(level) logger.addHandler(handler) logger.addHandler(consolehandler) #if telegram_integration == True: #SendTelegramMsg(message goes here) return logger  After I call the function setup_logger(): logger = setup_logger() logger.info("Start")  The output: INFO: 01/06/2022 11:07:12: Start  How am I able to get this message and send to SendTelegramMsg() if I enable the integration to True?

Answer: Implement a custom logging.Handler: class TelegramHandler(logging.Handler): def emit(self, record): message = self.format(record) SendTelegramMsg(message) # SendTelegramMsg(message, record.levelno) # Passing level # SendTelegramMsg(message, record.levelname) # Passing level name  Add the handler: def setup_logger(telegram_integration=False): # ... if telegram_integration: telegram_handler = TelegramHandler() logger.addHandler(telegram_handler) return logger  Usage, no change: logger = setup_logger() logger.info("Start")
Question: Python 3.9 on Mac running OS 11.6.1. My application involves placing a plot on a frame inside my root window, and I'm struggling to get the plot to take up a larger portion of the window. I thought rcParams in matplotlib.pyplot would take care of this, but I must be overlooking something. Here's what I have so far: import numpy as np from tkinter import Tk,Frame,TOP,BOTH import matplotlib from matplotlib import pyplot as plt from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg plt.rcParams["figure.figsize"] = [18,10] root=Tk() root.wm_title("Root Window") root.geometry('1500x1000') x = np.linspace(0, 2 * np.pi, 400) y = np.sin(x ** 2) fig, ax = plt.subplots() ax.plot(x, y) canvas_frame=Frame(root) # also tried adjusting size of frame but that didn't help canvas_frame.pack(side=TOP,expand=True) canvas = FigureCanvasTkAgg(fig, master=canvas_frame) canvas.draw() canvas.get_tk_widget().pack(side=TOP,fill=BOTH,expand=True) root.mainloop()  For my actual application, I need for canvas to have a frame as its parent and not simply root, which is why canvas_frame is introduced above.

Answer: try something like this: fig.subplots_adjust(left=0.05, bottom=0.07, right=0.95, top=0.95, wspace=0, hspace=0)  this is output, figure now takes more screen area % [
Question: How do I fund the nearest LINESTRING near a point? First I have a list of LINESTRING and point value. How do I have the nearest LINESTRING to the POINT (5.41 3.9) and maybee the distance? from shapely.geometry import Point, LineString line_string = [LINESTRING (-1.15.12 9.9, -1.15.13 9.93), LINESTRING (-2.15.12 8.9, -2.15.13 8.93)] point = POINT (5.41 3.9) #distance line_string [0].distance(point)  So far I think I got the distance value by doing line_string [0].distance(point) for the first LINESTRING so far but I just want to make sure I am going about it the right way.

Answer:  your sample geometry is invalid for line strings, have modified it's simple to achieve with sjoin_nearest()  import geopandas as gpd import shapely.wkt import shapely.geometry line_string = ["LINESTRING (-1.15.12 9.9, -1.15.13 9.93)", "LINESTRING (-2.15.12 8.9, -2.15.13 8.93)"] # fix invalid wkt string... line_string = ["LINESTRING (-1.15 9.9, -1.15 9.93)", "LINESTRING (-2.15 8.9, -2.15 8.93)"] point = "POINT (5.41 3.9)" gdf_p = gpd.GeoDataFrame(geometry=[shapely.wkt.loads(point)]) gdf_l = gpd.GeoDataFrame(geometry=pd.Series(line_string).apply(shapely.wkt.loads)) df_n = gpd.sjoin_nearest(gdf_p, gdf_l).merge(gdf_l, left_on="index_right", right_index=True) df_n["distance"] = df_n.apply(lambda r: r["geometry_x"].distance(r["geometry_y"]), axis=1) df_n       geometry_x index_right geometry_y distance     0 POINT (5.41 3.9) 0 LINESTRING (-1.15 9.9, -1.15 9.93) 8.89008    distance in meters  use a CRS that is in meters. UTM has it's limitations if all points are not in same zone  import geopandas as gpd import shapely.wkt import shapely.geometry line_string = ["LINESTRING (-1.15.12 9.9, -1.15.13 9.93)", "LINESTRING (-2.15.12 8.9, -2.15.13 8.93)"] # fix invalid wkt string... line_string = ["LINESTRING (-1.15 9.9, -1.15 9.93)", "LINESTRING (-2.15 8.9, -2.15 8.93)"] point = "POINT (5.41 3.9)" gdf_p = gpd.GeoDataFrame(geometry=[shapely.wkt.loads(point)], crs="epsg:4326") gdf_l = gpd.GeoDataFrame(geometry=pd.Series(line_string).apply(shapely.wkt.loads), crs="epsg:4326") gdf_p = gdf_p.to_crs(gdf_p.estimate_utm_crs()) gdf_l = gdf_l.to_crs(gdf_p.crs) df_n = gpd.sjoin_nearest(gdf_p, gdf_l).merge(gdf_l, left_on="index_right", right_index=True) df_n["distance"] = df_n.apply(lambda r: r["geometry_x"].distance(r["geometry_y"]), axis=1) df_n
Question: I am wondering why the official documentation of conda does not mention anything about the command conda env? That makes me wonder if it would be possible to do every operation of conda env with the commands listed here and which one is recommended to use in practice. Right now I would assume that conda env creates an easy way to manipulate and work with conda environments.

Answer: The reason why the conda env commands are not similarly documented is historical. Namely, after conda was developed, others then developed an add-on package called conda-env that provided some convenience methods for operating on whole environments rather than package operations within environments. Eventually, the conda-env package was integrated directly into the conda package, but apparently there was never any systematic effort to unify the documentation. Instead, most of the high-level documentation on using conda env commands is found under the "Managing environments" section of the Conda documentation. As an end user, I typically use conda env for creating (from YAML), archiving/serializing (to YAML), and deleting whole environments. More directly, the documentation for conda env is consulted with $ conda env --help usage: conda-env [-h] {create,export,list,remove,update,config} ... positional arguments: {create,export,list,remove,update,config} create Create an environment based on an environment file export Export a given environment list List the Conda environments remove Remove an environment update Update the current environment based on environment file config Configure a conda environment optional arguments: -h, --help Show this help message and exit.  and documentation of individual subcommands can be similarly consulted with conda env <subcommand> --help.
Question: I'm trying to read some files with pandas using the s3Hook to get the keys. I'm able to get the keys, however I'm not sure how to get pandas to find the files, when I run the below I get:  No such file or directory:  Here is my code: def transform_pages(company, **context): ds = context.get("execution_date").strftime('%Y-%m-%d') s3 = S3Hook('aws_default') s3_conn = s3.get_conn() keys = s3.list_keys(bucket_name=Variable.get('s3_bucket'), prefix=f'S/{company}/pages/date={ds}/', delimiter="/") prefix = f'S/{company}/pages/date={ds}/' logging.info(f'keys from function: {keys}') """ transforming pages and loading data back to S3 """ for file in keys: df = pd.read_csv(file, sep='\t', skiprows=1, header=None) 

Answer: The format you are looking for is the following: filepath = f"s3://{bucket_name}/{key}"  So in your specific case, something like: for file in keys: filepath = f"s3://s3_bucket/{file}" df = pd.read_csv(filepath, sep='\t', skiprows=1, header=None)  Just make sure you have s3fs installed though (pip install s3fs).
Question: I would like to get the indexes of the elements in each rolling window of a Pandas Series. A solution that works for me is from this answer to an existing question: I get the window.index for each window obtained from the rolling function described in the answer. I am only interested in step=1 for the aforementioned function. But this function is not specific for DataFrames and Series, it would work on basic Python lists. Isn't there some functionality that takes advantage of Pandas rolling operations? I tried the Rolling.apply method: s = pd.Series([1, 2, 3, 4, 5, 6, 7]) rolling = s.rolling(window=3) indexes = rolling.apply(lambda x: x.index)  But it result in a TypeError: must be real number, not RangeIndex. Apparently, the Rolling.apply method only accepts functions that return a number based on each window. The functions cannot return other kinds of objects. Are there other methods of the Pandas Rolling class I could use? Even private methods. Or are there any other Pandas-specific functionalities to get the indexes of overlapping rolling windows? Expected output As output, I expect some kind of list-of-lists object. Each inner list should countain the index values of each window. The original s Series has [0, 1, 2, 3, 4, 5, 6] as index. So, rolling with a window=3, I expect as outcome something like: [ [0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5, 6], ] 

Answer: The apply function after rolling must return a numeric value for each window. One possible workaround is to use a list comprehension to iterate over each window and apply the custom transformation as required: [[*l.index] for l in s.rolling(3) if len(l) == 3]  Alternatively you can also use sliding_window_view to accomplish the same: np.lib.stride_tricks.sliding_window_view(s.index, 3)  Or even an list comprehension would do the job just fine: w = 3 [[*s.index[i : i + w]] for i in range(len(s) - w + 1)]  Result [[0, 1, 2], [1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5, 6]]
Question: How do I make the bar appear when one of the value of y is zero? It just leaves a gap by default. Is there a way I can enable it to plot for zero values? I am able to see a line on the x-axis at y=0 for the same if just plotted using go.Box. I would like to see this in the Bar plot as well. So far, I set the base to zero. But that doesn't plot for y=0 either. Here is my sample code. My actual code contains multiple traces, that's why I would like to see the plot for y=0 Here is the sample python code:  import plotly.graph_objects as go fig = go.Figure() fig.add_trace(go.Bar(x=[1, 2, 3], y=[0, 3, 2])) fig.show() 

Answer: Bar charts come with a line around the bars that by default are set to the same color as the background. In your case '#E5ECF6'. If you change that, the line will appear as a border around each bar that will remain visible even when y = 0 for any given x. fig.update_traces(marker_line_color = 'blue', marker_line_width = 12)  If you set the line color to match that of the bar itself, you'll get this: Plot 1: Bars with identical fill and line colors  If I understand correctly, this should be pretty close to what you're trying to achieve. At least visually. I would perhaps consider adjusting the yaxis range a bit to make it a bit clearer that the y value displayed is in fact 0. Plot 2: Adjusted y axis and separate colors  Complete code for Plot 1: import plotly.graph_objects as go fig = go.Figure() fig.add_trace(go.Bar(x=[1, 2, 3], y=[0, 3, 2], marker_color = 'blue')) fig.update_traces(marker_line_color = 'blue', marker_line_width = 12) fig.show()  Complete code for Plot 2: import plotly.graph_objects as go fig = go.Figure() fig.add_trace(go.Bar(x=[1, 2, 3], y=[0, 3, 2], marker_color = '#00CC96')) f = fig.full_figure_for_development(warn=False) fig.update_traces(marker_line_color = '#636EFA', marker_line_width = 4) fig.update_yaxes(range=[-1, 4]) fig.show()   Edit after comments Just to verify that the line color is the same as the background color using plotly version 5.4.0 Plot 1:  Plot 2: Zoomed in
Question: I'd like to start by saying that I'm very new to Python, and I started this project for fun. Specifically, it’s simply a program which sends compliments to you as notifications periodically throughout the day. This is not for school, and I was actually just trying to make it for my girlfriend while introducing myself to Python. With that in mind, here's my problem. I started this project by writing the simplest version of it: one you have to start each time your computer loads, and runs while you're actively using the computer. This portion works perfectly; however, I can't seem to figure out how to do the next step: have the program carry on as normal after reboot and save its progress. I know how to get it to start up again after reboot. Still, I'm not sure how to save its progress. Particularly, since I'm pulling the compliments out of a text file, I'm not sure how to have the program save what line it's on before rebooting. This is needed as I don't want the program to start from the first compliment each time, as there are over 300 unique ones as of now. In order to help you understand where my code currently is as for the best advice, I've shown it below: import datetime import time from plyer import notification Compliment = None try: with open('C:/Users/conno/Documents/compliments.txt') as f: lines = f.readlines() except: print("I'm sorry, I can't give you a new compliment today because I can't find the file.") for compliment in lines: notification.notify( title = "Your New Compliment for {}".format(datetime.date.today()), message = compliment, app_icon = "C:/Users/conno/Downloads/Paomedia-Small-N-Flat-Bell.ico", timeout = 10 ) time.sleep(60*30)  I know I could easily have a variable count which line it is on, but how do I save that value?

Answer: You can simply save the count (which is the index of the last compliment line) as an integer in a pickle file, or easier in a text file and read from it every time your script starts after reboot. import datetime import time from plyer import notification Compliment = None compliment_index = 0 try: with open('C:/Users/conno/Documents/compliments.txt') as f: lines = f.readlines() except: print("I'm sorry, I can't give you a new compliment today because I can't find the file.") try: with open('C:/Users/conno/Documents/compliments_counter.txt') as f: compliment_index = int(f.readlines()[0]) except: with open('C:/Users/conno/Documents/compliments_counter.txt', "w") as f: f.write(str(compliment_index)) for index in range(compliment_index, len(lines)): compliment = lines[index] notification.notify( title = "Your New Compliment for {}".format(datetime.date.today()), message = compliment, app_icon = "C:/Users/conno/Downloads/Paomedia-Small-N-Flat-Bell.ico", timeout = 10) with open('C:/Users/conno/Documents/compliments_counter.txt', "w") as f: f.write(str(index)) time.sleep(60*30)
Question: I have a User model, a Contact model, and a Group model. I'm looking to find all of the 2nd-order User's Groups given a particular user in a single query. That is, I'd like to:  Use all contacts of a particular user... ... to get all the users who are also a contact of the given user, and use that to... ... get all groups of those (2nd-order) users  Right now I've got something like this (where user.id is the particular user whose contacts-of-contacts I'd like to find): from sqlalchemy.orm import aliased SecondOrderUser = aliased(User) # This returns the phone number of all contacts who're a contact of the particular user subquery = User.query \ .join(Contact, Contact.user_id == User.id) \ .filter(User.id == user.id) \ .with_entities(Contact.contact_phone) \ .subquery() # This filters all users by the phone numbers in the above query, gets their contacts, and gets the group # IDs of those contacts who are themselves users contacts = User.query \ .filter(User.phone.in_(subquery)) \ .join(UserContact, UserContact.user_id == User.id) \ .join(SecondOrderUser, SecondOrderUser.phone == UserContact.phone) \ .join(Group, Group.user_id == SecondOrderUser.id) \ .with_entities(Group.id) \ .all()  The only thing that Contact and User share (to link them together—that is, to find contacts that are themselves users) is a common phone number. I [think I] could also do it with four join statements and aliases, but this gives me the same error. Namely: sqlalchemy.exc.InvalidRequestError: Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explicit ON clause if not present already to help resolve the ambiguity.  What am I doing incorrectly here? Where/how to join feels clear to me, which indicates that I'm totally missing something.

Answer: The problem here was two-fold. We've got to alias a table every time we use it (not just the 2nd time onward) and, when using with_entities, we've got to use all columns that we compare on—even if we don't intend on using their data in the end. My final code looked something like this: from sqlalchemy.orm import aliased User1 = aliased(User) User2 = aliased(User) User3 = aliased(User) Contact1 = aliased(Contact) Contact2 = aliased(Contact) contacts = User1.query \ .join(Contact1, Contact1.user_id == User1.id) \ .join(User2, User2.phone == Contact1.phone) \ .join(Contact2, Contact2.user_id == User2.id) \ .join(User3, User3.phone == Contact2.phone) \ .join(Group, Group.user_id == User3.id) \ .with_entities( Contact1.phone, Contact1.user_id, Contact2.phone, Contact2.user_id, Group.id Group.user_id, User1.id, User2.id, User2.phone, User3.id, User3.phone, ) \ .all()
Question: I have a few pytest fixtures I use from third-party libraries and sometimes their names are overly long and cumbersome. Is there a way to create a short alias for them? For example: the django_assert_max_num_queries fixture from pytest-django. I would like to call this max_queries in my tests.

Answer: You cannot just add an alias in the form of max_queries = django_assert_max_num_queries  because fixtures are looked up by name at run-time and not imported (and even if they can be imported in some cases, this is not recommended). But you can always write your own fixture that just yields another fixture: @pytest.fixture def max_queries(django_assert_max_num_queries): yield django_assert_max_num_queries  Done this way, max_queries will behave exactly the same as django_assert_max_num_queries. Note that you should use yield and not return, to make sure that the control is returned to the fixture.