eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I would like to extract the column names of a resulting table directly from the SQL statement:  query = """ select sales.order_id as id, p.product_name, sum(p.price) as sales_volume from sales right join products as p on sales.product_id=p.product_id group by id, p.product_name; """ column_names = parse_sql(query) # column_names: # ['id', 'product_name', 'sales_volume']  Any idea what to do in parse_sql()? The resulting function should be able to recognize aliases and remove the table aliases/identifiers (e.g. "sales." or "p."). Thanks in advance!

Answer: I've done something like this using the library sqlparse. Basically, this library takes your SQL query and tokenizes it. Once that is done, you can search for the select query token and parse the underlying tokens. In code, that reads like import sqlparse def find_selected_columns(query) -> list[str]: tokens = sqlparse.parse(query)[0].tokens found_select = False for token in tokens: if found_select: if isinstance(token, sqlparse.sql.IdentifierList): return [ col.value.split(" ")[-1].strip("`").rpartition('.')[-1] for col in token.tokens if isinstance(col, sqlparse.sql.Identifier) ] else: found_select = token.match(sqlparse.tokens.Keyword.DML, ["select", "SELECT"]) raise Exception("Could not find a select statement. Weired query :)")  This code should also work for queries with Common table expressions, i.e. it only return the final select columns. Depending on the SQL dialect and the quote chars you are using, you might to have to adapt the line col.value.split(" ")[-1].strip("`").rpartition('.')[-1]
Question: I'm wondering about code like this: def all_lines(filename): with open(filename) as infile: yield from infile  The point of a context manager is to have explicit control over the lifetime of some form of state, e.g. a file handle. A generator, on the other hand, keeps its state until it is exhausted or deleted. I do know that both cases work in practice. But I'm worried about whether it is a good idea. Consider for example this: def all_first_lines(filenames): return [next(all_lines(filename), None) for filename in filenames]  I never exhaust the generators. Instead, their state is destroyed when the generator object is deleted. This works fine in reference-counted implementations like CPython, but what about garbage-collected implementations? I'm practically relying on the reference counter for managing state, something that context managers were explicitly designed to avoid! And even in CPython it shouldn't be too hard to construct cases were a generator is part of a reference cycle and needs the garbage collector to be destroyed. To summarize: Would you consider it prudent to avoid context managers in generators, for example by refactoring the above code into something like this? def all_lines(filename): with open(filename) as infile: return infile.readlines() def first_line(filename): with open(filename) as infile: return next(infile, None) def all_first_lines(filenames): return [first_line(filename) for filename in filenames] 

Answer: While it does indeed extend the lifetime of the object until the generator exits or is destroyed, it also can make the generators clearer to work with. Consider creating the generators under an outer with and passing the file as an argument instead of them opening it. Now the file is invalid for use after the context manager is exited, even though the generators can still be seen as usable. If limiting the time for how long the handles are held is important, you can explicitly close the generators using the close method after you are done with them. This is a similar problem to what trio tries to solve with its nurseries for asynchronous tasks, where the nursery context manager waits for every task spawned from that nursery to exit before proceeding, the tutorial example illustrates this. This blog post by the author can provide some reasoning for the way it's done in trio which can be an interesting read that's somewhat related to the problem.
Question: I am working with Sphinx and I want to set the html_static_path variable in the config.py file. The default was: html_static_path = ['_static'] My project setup is: docs/ build/ doctrees html/ _static/ source/ conf.py  The sphinx documentation says that I need to set the path relative to this directory, i.e. relative path from conf.py. Thus, I tried: html_static_path = ['..\build\html\source\_static']  AND I tried to set the absolute path. But I still get the warning:  WARNING: html_static_path entry 'build\html\source\_static' does not exist  Can you help me? Thank you!

Answer: The build directory is created after you build the docs, which is why you get that error. When you make your docs, Sphinx will copy the static directory from your source location as defined by html_static_path to the build location. Create a new directory source/_static and place any static assets inside of it. Change the value in conf.py to this: html_static_path = ["_static"]
Question: While serializing Enum object to JSON using json.dumps(...), python will throw following error: >>> class E(enum.Enum): ... A=0 ... B=1 >>> import json >>> json.dumps(E.A) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\guptaaman\Miniconda3\lib\json\__init__.py", line 231, in dumps return _default_encoder.encode(obj) File "C:\Users\guptaaman\Miniconda3\lib\json\encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "C:\Users\guptaaman\Miniconda3\lib\json\encoder.py", line 257, in iterencode return _iterencode(o, 0) File "C:\Users\guptaaman\Miniconda3\lib\json\encoder.py", line 179, in default raise TypeError(f'Object of type {o.__class__.__name__} ' TypeError: Object of type E is not JSON serializable  How to make Enum object JSON serializable?

Answer: We can make the class which is inheriting Enum to also inherit str class as following: >>> class E(str, enum.Enum): ... A="0" ... B="1" ... >>> json.dumps(E.A) '"0"'  Reference: https://hultner.se/quickbits/2018-03-12-python-json-serializable-enum.html
Question: whenever I try to import matplotlib or matplotlib.pyplot in VS Code I get the error in the title: Import "matplotlib" could not be resolved from source Pylance(reportMissingModuleSource)  or Import "matplotlib.pyplot" could not be resolved from source Pylance(reportMissingModuleSource)  The hyperlink of the reportMissingModuleSource sends me to https://github.com/microsoft/pylance-release/blob/main/DIAGNOSTIC_SEVERITY_RULES.md#diagnostic-severity-rules, where it says: "Diagnostics for imports that have no corresponding source file. This happens when a type stub is found, but the module source file was not found, indicating that the code may fail at runtime when using this execution environment. Type checking will be done using the type stub." However, from the explanation I don't understand exactly what's wrong and what I should do to fix this, can someone help me with this?

Answer: I can reproduce your question when I select a python interpreter where doesn't exist matplotlib:  So, the solution is opening an integrated Terminal then run pip install matplotlib. After it's installed successfully, please reload window, then the warning should go away.
Question: I can use the normal F2 rename variable functionality in regular python files in vscode. But not when editing python in a jupyter notebook. When I press F2 on a variable in a jupyter notebook in vscode I get the familiar change variable window but when I press enter the variable is not changed and I get this error message:  No result. No result.  Is there a way to get the F2 change variable functionality to work in jupyter notebooks? Here's my system info: jupyter module version (adventofcode) C:\git\leetcode>pip show jupyter Name: jupyter Version: 1.0.0 Summary: Jupyter metapackage. Install all the Jupyter components in one go. Home-page: http://jupyter.org Author: Jupyter Development Team Author-email: jupyter@googlegroups.org License: BSD Location: c:\users\johan\anaconda3\envs\adventofcode\lib\site-packages Requires: ipykernel, qtconsole, nbconvert, jupyter-console, notebook, ipywidgets Required-by:  Python version: (adventofcode) C:\git\leetcode>python --version Python 3.10.0  vscode version: 1.63.2 (user setup)  vscode Jupyter extension version (from the changelog in the extensions window): 2021.11.100 (November Release on 8 December 2021) 

Answer: Notice that you put up a bug report in GitHub and see this issue: Renaming variables didn't work, the programmer replied:  Some language features are currently not supported in notebooks, but we are making plans now to hopefully bring more of those online soon.  So please wait for this feature.
Question: Here are my Parent and Child classes: class Parent(Base): id = Column(...) ... children = relationship("Child", backref="parent", lazy="select") class Child(Base): id = Column(...) parent_id = Column(...) active = Column(Boolean(), ...)  The reason behind the loading technique of children of Parent being lazy is that there can be a very large number of children associated with a parent. Now, I would like to get the number of active children of a parent as a hybrid-property. Here is how I tried to do it: class Parent(Base): ... @hybrid_property def active_count(self): return len([child for child in self.children if child.active]) @active_count.expression def active_count(cls): return ( select(func.count(Child.id)) .where(Child.parent_id == cls.id) .where(Child.active == True) )  But the problem with this method is that when I call parent.active_count, it fires a query to get all of the children. How can I get only the count (of active children) without reading whole the children?

Answer: I think you unnecessary iterate over children within active_count hybrid_property definition. This should works for you: class Parent(Base): ... children = relationship("Child", backref="parent", lazy="dynamic") @hybrid_property def active_count(self): return self.children.filter_by(active=True).with_entities(func.count('*')).scalar() # or # return self.children.filter_by(active=True).count() # but it would have worse performance
Question: I want to draw a circle in the console using characters instead of pixels, for this I need to know how many pixels are in each row. The diameter is given as input, you need to output a list with the width in pixels of each line of the picture For example: input: 7 output: [3, 5, 7, 7, 7, 5, 3]  input: 12 output: [4, 8, 10, 10, 12, 12, 12, 12, 10, 10, 8, 4]  How can this be implemented?

Answer: This was a good reminder for me to be careful when mixing zero-based and one-based computations. In this case, I had to account for the for loops being zero-based, but the quotient of the diameter divided by 2 being one-based. Otherwise, the plots would have been over or under by 1. By the way, while I matched your answer for 7, I didn't come up with the same exact plot for 12:  NOTE - Tested using Python 3.9.6  pixels_in_line = 0 pixels_per_line = [] diameter = int(input('Enter the diameter of the circle: ')) # You must account for the loops being zero-based, but the quotient of the diameter / 2 being # one-based. If you use the exact radius, you will be short one column and one row. offset_radius = (diameter / 2) - 0.5 for i in range(diameter): for j in range(diameter): x = i - offset_radius y = j - offset_radius if x * x + y * y <= offset_radius * offset_radius + 1: print('*', end=' ') pixels_in_line += 1 else: print(' ', end=' ') pixels_per_line.append(pixels_in_line) pixels_in_line = 0 print() print('The pixels per line are {0}.'.format(pixels_per_line))  Output for 7: Enter the diameter of the circle: 7 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * The pixels per line are [3, 5, 7, 7, 7, 5, 3].  Output for 12: Enter the diameter of the circle: 12 * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * The pixels per line are [2, 6, 8, 10, 10, 12, 12, 10, 10, 8, 6, 2].
Question: I know there's a ton resources online for outlier removal, but I haven't yet managed to obtain what I exactly want, so posting here, I have an array (or DF) of 4 columns. Now I want to remove the rows from the DF based on a column's outlier values. The following is what I have tried, but they are not perfect. def outliers2(data2, m = 4.5): c=[] data = data2[:,1] # Choosing the column d = np.abs(data - np.median(data)) # deviation comoutation mdev = np.median(d) # mean deviation for i in range(len(data)): if (abs(data[i] - mdev) < m * np.std(data)): c.append(data2[i]) return c x = pd.DataFrame(outliers2(np.array(b))) column = ['t','orig_w','filt_w','smt_w'] x.columns = column #Plot plt.rcParams['figure.figsize'] = [10,8] plt.plot(b.t,b.orig_w,'o',label='Original',alpha=0.8) # Original plt.plot(x.t,x.orig_w,'.',c='r',label='Outlier removed',alpha=0.8) # After outlier removal plt.legend()  the plot illustrates how the results looks, red points after the outlier treatment over the blue original points. I would really like to get rid of those vertical group of points around the x~0 mark. What to do ? A link to the data file is provided here : Full data  The green circles show typically the points i would like to get rid of 

Answer: You could use scipy's median_filter: import pandas as pd from matplotlib import pyplot as plt from scipy.ndimage import median_filter b = pd.read_csv("test.csv") x = b.copy() x.orig_w = median_filter(b.orig_w, size=15) #Plot plt.rcParams['figure.figsize'] = [10,8] #Original plt.plot(b.t,b.orig_w,'o',label='Original',alpha=0.8) # After outlier removal plt.plot(x.t,x.orig_w,'.',c='r',label='Outlier removed',alpha=0.8) plt.legend() plt.show()  Sample output:
Question: Is there a simple way to loop variables through all possible combinations of true/false? For example, say we have a function: def f(a, b, c): return not (a and (b or c))  Is there a way to loop a,b,c through 000, 001, 010, 011, 100, etc.? My first thought was to loop through the integers and get the bit that corresponds to the variable position, like so: for i in range (8): a = i & 4 b = i & 2 c = i & 1 print(f(a,b,c), " ", a,b,c)  It seems to work, but a, b, and c when printed out are all integers, and integers interact with logical operators differently than booleans. For example, 4 and 2 equals 2, and 9 or 3 equals 9. I don't mind it too much, it just took some thinking through to convince myself this doesn't matter. But the question still stands, is there a still simpler way to loop through all possible true/false or 0/1 values?

Answer: Use itertools.product: from itertools import product for a, b, c in product([True, False], repeat=3): print(f(a,b,c), " ", a,b,c)  If you really want integers 1 and 0, just replace [True, False] with [1, 0]. There's little reason to treat integers as bit arrays here.
Question: Lambda functions support the environment parameter and make it easy to define a key-value pair. But what about getting an object (defined by a module variable eg) into the function's environment? Quick example of what I'm trying to accomplish in python 3.7: Terraform: # variable definition variable foo { type = map(any) default = { a = "b" c = "d" } } resource "aws_lambda_function" "lambda" { . . . environment { foo = jsonencode(foo) } }  and then in my function: def bar: for k in os.environ["foo"]: print(k)  Thanks !

Answer: In python, you will have to get json string and convert it to dict: import json def bar: for k in json.loads(os.environ["foo"]): print(k)
Question: The following class has a class method create(), which as a type hint for return type, for creating an instance of the class. class X: @classmethod def create(cls) -> X: pass  However, it got the following error?  NameError: name 'X' is not defined 

Answer: The name X doesn't exist until the class is fully defined. You can fix this by importing a __future__ feature called annotations. Just put this at the top of your file. from __future__ import annotations  This wraps all annotations in quotation marks, to suppress errors like this. It's the same as doing this class X: @classmethod def create(cls) -> 'X': # <-- Note the quotes pass  but automatically. This will be the default behavior in some future Python version (originally, it was going to be 3.10, but it's been pushed back due to compatibility issues), but for now the import will make it behave the way you want. The future import was added in Python 3.7. If you're on an older version of Python, you'll have to manually wrap the types in strings, as I did in the example above.
Question: I have Kubernetes cluster set up and managed by AKS, and I have access to it with the python client. Thing is that when I'm trying to send patch scale request, I'm getting an error. I've found information about scaling namespaced deployments from python client in the GitHub docs, but it was not clear what is the body needed in order to make the request work: # Enter a context with an instance of the API kubernetes.client with kubernetes.client.ApiClient(configuration) as api_client: # Create an instance of the API class api_instance = kubernetes.client.AppsV1Api(api_client) name = 'name_example' # str | name of the Scale namespace = 'namespace_example' # str | object name and auth scope, such as for teams and projects body = None # object | pretty = 'pretty_example' # str | If 'true', then the output is pretty printed. (optional) dry_run = 'dry_run_example' # str | When present, indicates that modifications should not be persisted. An invalid or unrecognized dryRun directive will result in an error response and no further processing of the request. Valid values are: - All: all dry run stages will be processed (optional) field_manager = 'field_manager_example' # str | fieldManager is a name associated with the actor or entity that is making these changes. The value must be less than or 128 characters long, and only contain printable characters, as defined by https://golang.org/pkg/unicode/#IsPrint. This field is required for apply requests (application/apply-patch) but optional for non-apply patch types (JsonPatch, MergePatch, StrategicMergePatch). (optional) force = True # bool | Force is going to \"force\" Apply requests. It means user will re-acquire conflicting fields owned by other people. Force flag must be unset for non-apply patch requests. (optional) try: api_response = api_instance.patch_namespaced_deployment_scale(name, namespace, body, pretty=pretty, dry_run=dry_run, field_manager=field_manager, force=force) pprint(api_response) except ApiException as e: print("Exception when calling AppsV1Api->patch_namespaced_deployment_scale: %s\n" % e)  So when running the code I'm getting Reason: Unprocessable Entity Does anyone have any idea in what format should the body be? For example if I want to scale the deployment to 2 replicas how can it be done?

Answer: The body argument to the patch_namespaced_deployment_scale can be a JSONPatch document, as @RakeshGupta shows in the comment, but it can also be a partial resource manifest. For example, this works: >>> api_response = api_instance.patch_namespaced_deployment_scale( ... name, namespace, ... [{'op': 'replace', 'path': '/spec/replicas', 'value': 2}])  (Note that the value needs to be an integer, not a string as in the comment.) But this also works: >>> api_response = api_instance.patch_namespaced_deployment_scale( ... name, namespace, ... {'spec': {'replicas': 2}})
Question: Is there a correct way to have two walrus operators in 1 if statement? if (three:= i%3==0) and (five:= i%5 ==0): arr.append("FizzBuzz") elif three: arr.append("Fizz") elif five: arr.append("Buzz") else: arr.append(str(i-1))  This example works for three but five will be "not defined".

Answer: The logical operator and evaluates its second operand only conditionally. There is no correct way to have a conditional assignment that is unconditionally needed. Instead use the "binary" operator &, which evaluates its second operand unconditionally. arr = [] for i in range(1, 25): # v force evaluation of both operands if (three := i % 3 == 0) & (five := i % 5 == 0): arr.append("FizzBuzz") elif three: arr.append("Fizz") elif five: arr.append("Buzz") else: arr.append(str(i)) print(arr) # ['1', '2', 'Fizz', '4', 'Buzz', 'Fizz', '7', '8', 'Fizz', 'Buzz', '11', ...]  Correspondingly, one can use | as an unconditional variant of or. In addition, the "xor" operator ^ has no equivalent with conditional evaluation at all. Notably, the binary operators evaluate booleans as purely boolean - for example, False | True is True not 1 – but may work differently for other types. To evaluate arbitrary values such as lists in a boolean context with binary operators, convert them to bool after assignment: # |~~~ force list to boolean ~~| | force evaluation of both operands # v v~ walrus-assign list ~vv v if bool(lines := list(some_file)) & ((today := datetime.today()) == 0): ...  Since assignment expressions require parentheses for proper precedence, the common problem of different precedence between logical (and, or) and binary (&, |, ^) operators is irrelevant here.
Question: I have these two scripts: clock.py #!/usr/bin/env python3 import time while True: print("True", flush=True) time.sleep(1)  continuous_wc.py #!/usr/bin/env python3 import sys def main(): count = 0 for line in sys.stdin: sys.stdout.write(str(count)) count += 1 if __name__=='__main__': main()  And I run them like so: ./clock.py | ./continuous_wc.py  I'm hoping that it prints: 1 2 3 4 5 ...  Every second is like a clock, because it's counting the lines in the file basically. But it doesn't output anything. Why not?

Answer: In addition to print(x, flush=True) you must also flush after sys.stdout.write. Note that the programs would technically work without flush, but they would print values very infrequently, in very large chunks, as the Python IO buffer is many kilobytes. Flushing is there to make it work more real-time. sys.stdout.write(str(count)) sys.stdout.flush()
Question: After updating from Django 2 to Django 4.0.1 I am getting CSRF errors on all POST requests. The logs show: "WARNING:django.security.csrf:Forbidden (Origin checking failed - https://127.0.0.1 does not match any trusted origins.): /activate/" I can't figure out how to set a wildcard for CSRF_TRUSTED_ORIGINS? I have a server shipped to customers who host it on their own domain so there is no way for me to no the origin before hand. I have tried the following with no luck: CSRF_TRUSTED_ORIGINS = ["https://*", "http://*"]  and CSRF_TRUSTED_ORIGINS = ["*"]  Explicitly setting "https://127.0.0.1" in the CSRF_TRUSTED_ORIGINS works but won't work in my customer's production deployment which will get another hostname.

Answer: The Django app is running using Gunicorn behind NGINX. Because SSL is terminated after NGINX request.is_secure() returns false which results in Origin header not matching the host here: https://github.com/django/django/blob/3ff7f6cf07a722635d690785c31ac89484134bee/django/middleware/csrf.py#L276 I resolved the issue by adding the following in Django: SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')  And ensured that NGINX is forwarding the http scheme with the following in my NGINX conf: proxy_set_header X-Forwarded-Proto $scheme;
Question: I want to add some annotations and arrows to my plots using plt.annotate() and I am not sure how to change the arrow parameters. import pandas as pd import matplotlib.pyplot as plt df = pd.DataFrame({'weight': [10, 10, 20, 5, 10, 15]}) print(df)  Dummy Data:  weight 0 10 1 10 2 20 3 5 4 10 5 15  Plot Line: ax = df.plot(figsize=(8, 6)) ax.annotate("Maximum", xy=(2, 20), xytext=(2, 10), arrowprops=dict(arrowstyle="->"), color='red')   How do I change the arrow color and other parameters to also match the Text? In this case I changed the text color to red. Thanks for the help!

Answer: In arrowprops you should be able to set arrow color and other properties , something like this:  arrowprops=dict(arrowstyle= '->', color='red', lw=3, ls='--')