eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I want to serve my React frontend using FastAPI. The goal being 0 Javascript dependency for the user. The user can simply download the Python code, start server, and view the website on localhost. My folder structure is: - my-fullstack-app - frontend/ - build/ - public/ - ... - package.json - backend/ - main.py - static/  I ran npm run build to generate the frontend/build/ folder which contains: build/ ├── asset-manifest.json ├── favicon.ico ├── index.html ├── logo192.png ├── logo512.png ├── manifest.json ├── robots.txt └── static ├── css │ ├── main.073c9b0a.css │ └── main.073c9b0a.css.map ├── js │ ├── 787.cda612ba.chunk.js │ ├── 787.cda612ba.chunk.js.map │ ├── main.af955102.js │ ├── main.af955102.js.LICENSE.txt │ └── main.af955102.js.map └── media └── logo.6ce24c58023cc2f8fd88fe9d219db6c6.svg  I copied the contents of the frontend/build/ folder inside backend/static/. Now, I want to serve this backend/static/ folder via FastAPI as opposed to running another server. In my FastAPI's main.py I have: from fastapi import FastAPI from fastapi.staticfiles import StaticFiles app = FastAPI() app.mount("/", StaticFiles(directory="static/"), name="static")  I then start the server using - uvicorn main:app --reload. But it doesn't work. When I open http://127.0.0.1:8000/ in the browser, the output is a JSON file which says {"detail":"Not Found"} and console has Content Security Policy: The page's settings blocked the loading of a resource at http://127.0.0.1:8000/favicon.ico ("default-src").. How do I get this to work? I've seen examples for similar functionality with React and Express.

Answer: The answer mentioned by @flakes is exactly what I was looking for. Check it out for more details. For completeness, Without html=True flag. app.mount("/", StaticFiles(directory="static/"), name="static")  Navigate to http://127.0.0.1:8000/index.html to view the html file.  With html=True flag. app.mount("/", StaticFiles(directory="static/", html=True), name="static")  Navigate to http://127.0.0.1:8000/ to view the html file.
Question: Need help translating this SQL query into SQLAlchemy: select COALESCE(DATE_1,DATE_2) as DATE_COMPLETE, QUESTIONS_CNT, ANSWERS_CNT from ( (select DATE as DATE_1, count(distinct QUESTIONS) as QUESTIONS_CNT from GUEST_USERS where LOCATION like '%TEXAS%' and DATE = '2021-08-08' group by DATE ) temp1 full join (select DATE as DATE_2, count(distinct ANSWERS) as ANSWERS_CNT from USERS where LOCATION like '%TEXAS%' and DATE = '2021-08-08' group by DATE ) temp2 on temp1.DATE_1=temp2.DATE_2 )  Mainly struggling with the join of the two subqueries. I've tried this (just for the join part of the SQL): query1 = db.session.query( GUEST_USERS.DATE_WEEK_START.label("DATE_1"), func.count(GUEST_USERS.QUESTIONS).label("QUESTIONS_CNT") ).filter( GUEST_USERS.LOCATION.like("%TEXAS%"), GUEST_USERS.DATE == "2021-08-08" ).group_by(GUEST_USERS.DATE) query2 = db_session_stg.query( USERS.DATE.label("DATE_2"), func.count(USERS.ANSWERS).label("ANSWERS_CNT") ).filter( USERS.LOCATION.like("%TEXAS%"), USERS.DATE == "2021-08-08" ).group_by(USERS.DATE) sq2 = query2.subquery() query1_results = query1.join( sq2, sq2.c.DATE_2 == GUEST_USERS.DATE) ).all()  In this output I receive only the DATE_1 column and the QUESTIONS_CNT columns. Any idea why the selected output from the subquery is not being returned in the result?

Answer: Not sure if this is the best solution but this is how I got it to work. Using 3 subqueries essentially. query1 = db.session.query( GUEST_USERS.DATE_WEEK_START.label("DATE_1"), func.count(GUEST_USERS.QUESTIONS).label("QUESTIONS_CNT") ).filter( GUEST_USERS.LOCATION.like("%TEXAS%"), GUEST_USERS.DATE == "2021-08-08" ).group_by(GUEST_USERS.DATE) query2 = db_session_stg.query( USERS.DATE.label("DATE_2"), func.count(USERS.ANSWERS).label("ANSWERS_CNT") ).filter( USERS.LOCATION.like("%TEXAS%"), USERS.DATE == "2021-08-08" ).group_by(USERS.DATE) sq1 = query1.subquery() sq2 = query2.subquery() query3 = db.session.query(sq1, sq2).join( sq2, sq2.c.DATE_2 == sq1.c.DATE_1) sq3 = query3.subquery() query4 = db.session.query( func.coalesce( sq3.c.DATE_1, sq3.c.DATE_2), sq3.c.QUESTIONS_CNT, sq3.c.ANSWERS_CNT ) results = query4.all()
Question: Qt5Agg is necessary to use the mayavi 3D visualization package. I have installed PyQt5 and mayavi using pip in a separate copied conda environment. The default backend then changes from TkAgg to QtAgg. This is a bit weird because in an earlier installation in a different PC the default changed directly to Qt5Agg. I always check the backend using the following commands from the python console : import matplotlib matplotlib.get_backend()  Even with the backend being 'QtAgg', I am able to use mayavi from the terminal without any issue but not when I do so in Pycharm. Here I get a non-responsive empty window (image below) : Image of the non-responsive window I have been able to get rid of this issue by explicitly using Qt5Agg instead of QtAgg before the plt call : import matplotlib matplotlib.use('Qt5Agg') import matplotlib.pyplot as plt  But I would prefer a better way than using the above in every script that I write. As I had mentioned earlier, I already have mayavi installed and have used it successfully it in Pycharm in a different PC and there the default backend is 'Qt5Agg' and hence there is no need to change the backend explicitly. Is there anything obvious that I'm overlooking ? Can you please let me know of a way to change the default backend for matplotlib from QtAgg to Qt5Agg after PyQt5 installation using pip ? Thanks in advance !!

Answer: Thanks to @PaulH's comment, I was able to solve the issue. Owing to @mx0's suggestion, I shall now explicitly mention the fix below so that others can also benefit from it. In a particular conda environment, if matplotlib package is installed, then there will be a 'matplotlibrc' file stored somewhere that defines what the default backend will be whenever matplotlib is imported from that conda environment. The location of this 'matplotlibrc' can be found using the following commands : import matplotlib matplotlib.matplotlib_fname()  Please look into the following link if there's any deprecation issue with the above commands : https://matplotlib.org/stable/tutorials/introductory/customizing.html#customizing-with-matplotlibrc-files Once the location of the 'matplotlibrc' file is known, open it and simply uncomment one line inside this file. Just change the backend from : ##backend: Agg  to : backend: Qt5Agg  And that's it. All the plot window troubles in PyCharm will be solved as far as the mayavi 3D visualization package is concerned. For any other use, where a specific backend is necessary, you can also set the default to any other backend of choice.
Question: I'm used to typescript, in which one can use a ! to tell the type-checker to assume a value won't be null. Is there something analogous when using type annotations in python? A (contrived) example: When executing the expression m.maybe_num + 3 in the code below, the enclosing if guarantees that maybe_num won't be None. But the type-checker doesn't know that, and returns an error. (Verified in https://mypy-play.net/?mypy=latest&python=3.10.) How can I tell the type-checker that I know better? from typing import Optional class MyClass: def __init__(self, maybe_num: Optional[int]): self.maybe_num = maybe_num def has_a_num(self) -> bool: return self.maybe_num is not None def three_more(self) -> Optional[int]: if self.has_a_num: # mypy error: Unsupported operand types for + ("None" and "int") return self.maybe_num + 3 else: return None 

Answer: Sadly there's no clean way to infer the type of something from a function call like this, but you can work some magic with TypeGuard annotations for the has_a_num() method, although the benefit from those annotations won't really be felt unless the difference is significantly more major than the type of a single int. If it's just a single value, you should just use a standard is not None check. if self.maybe_num is not None: ...  You can define a subclass of your primary subclass, where the types of any parameters whose types are affected are explicitly redeclared. class MyIntClass(MyClass): maybe_num: int  From there, your checker function should still return a boolean, but the annotated return type tells MyPy that it should use it for type narrowing to the listed type. Sadly it will only do this for proper function parameters, rather than the implicit self argument, but this can be fixed easily enough by providing self explicitly as follows: if MyClass.has_a_num(self): ...  That syntax is yucky, but it works with MyPy. This makes the full solution be as follows # Parse type annotations as strings to avoid # circular class references from __future__ import annotations from typing import Optional, TypeGuard class MyClass: def __init__(self, maybe_num: Optional[int]): self.maybe_num = maybe_num def has_a_num(self) -> TypeGuard[_MyClass_Int]: # This annotation defines a type-narrowing operation, # such that if the return value is True, then self # is (from MyPy's perspective) _MyClass_Int, and # otherwise it isn't return self.maybe_num is not None def three_more(self) -> Optional[int]: if MyClass.has_a_num(self): # No more mypy error return self.maybe_num + 3 else: return None class _MyClass_Int(MyClass): maybe_num: int  TypeGuard was added in Python 3.10, but can be used in earlier versions using the typing_extensions module from pip.
Question: I'm trying to concatenate two dataframes with these conditions :  for an existing header, append to the column ; otherwise add a new column.  The code is working but the columns names are lost in case 2. Why? It doesn't seem to be mentioned in Pandas doc. Or I missed something? How to keep the column names? The code : # Testing # Merge, join, concatenate # Pandas documentation : https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html df1 = pd.DataFrame( { "A": ["A0", "A1", "A2", "A3"], "B": ["B0", "B1", "B2", "B3"], "C": ["C0", "C1", "C2", "C3"], "D": ["D0", "D1", "D2", "D3"], }, #index=[0, 1, 2, 3], ) df2 = pd.DataFrame( { "A": ["A4", "A5", "A6", "A7"], "B": ["B4", "B5", "B6", "B7"], "C": ["C4", "C5", "C6", "C7"], "D": ["D4", "D5", "D6", "D7"], }, #index=[4, 5, 6, 7], ) df3 = pd.DataFrame( { "E": ["E0", "E1", "E2", "E3", "E4", "E5"], }, #index=[0, 1, 2, 3, 4 , 5], ) frames = [df1, df2] result_1 = pd.concat(frames, ignore_index=True) print(result_1) frames = [result_1, df3] if "E" in df3.columns : result_2 = pd.concat(frames, axis=1, ignore_index=True) print(result_2) 

Answer: You requested to drop the index with ignore_index=True. As you are concatenating on axis=1 the index is the columns! frames = [result_1, df3] if "E" in df3.columns : result_2 = pd.concat(frames, axis=1) print(result_2)  Output:  A B C D E 0 A0 B0 C0 D0 E0 1 A1 B1 C1 D1 E1 2 A2 B2 C2 D2 E2 3 A3 B3 C3 D3 E3 4 A4 B4 C4 D4 E4 5 A5 B5 C5 D5 E5 6 A6 B6 C6 D6 NaN 7 A7 B7 C7 D7 NaN
Question: From the pyspark docs, I Can do: gdf = df.groupBy(df.name) sorted(gdf.agg({"*": "first"}).collect())  In my actual use case I have maaaany variables, so I like that I can simply create a dictionary, which is why: gdf = df.groupBy(df.name) sorted(gdf.agg(F.first(col, ignorenulls=True)).collect())  @lemon's suggestion won't work for me. How can I pass a parameter for first (i.e. ignorenulls=True), see here.

Answer: You can use list comprehension. gdf.agg(*[F.first(x, ignorenulls=True).alias(x) for x in df.columns]).collect()
Question: I have an f-string like this: f"foo-{i:03d}"  This is producing a filename for a file that I read in. The problem is sometimes there are hundreds of files, so the files are named like foo-001, foo-002, etc., but other times there are thousands, so the files are foo-0001 and 04d would be needed. Is there a way I can do this dynamically? For example, if I have the total number of files as a variable n, then I could do: pad = len(str(n))  but then how do I use that pad variable in the f-string?

Answer: Nested f-string: >>> pad = 4 >>> i = 1 >>> f"foo-{i:>0{pad}}" 'foo-0001'
Question: I want to make a search filter which searches in multiple fields with multiple conditions, using only one search field. I have this filters.py file: import django_filters from .models import Product class ProductFilter(django_filters.FilterSet): q = django_filters.CharFilter(method='search_filter', label='Cerca') class Meta: model = Product fields = ['q'] def search_filter(self, queryset, name, value): return queryset.filter(name__icontains=value, sku__iexact=value)  But return queryset.filter(name__icontains=value, sku__iexact=value) doesn't work, neither return queryset.filter(Product(name__icontains=value) | Product(sku__iexact=value)) How can I do this?

Answer: You can filter with Q objects [Django-doc]: import django_filters from django.db.models import Q from .models import Product class ProductFilter(django_filters.FilterSet): q = django_filters.CharFilter(method='search_filter', label='Cerca') class Meta: model = Product fields = ['q'] def search_filter(self, queryset, name, value): return queryset.filter(Q(name__icontains=value) | Q(sku__iexact=value))
Question: Consider this: >>> '{x[1]}'.format(x="asd") 's' >>> '{x[1:3]}'.format(x="asd") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: string indices must be integers  What could be the cause for this behavior?

Answer: An experiment based on your comment, checking what value the object's __getitem__ method actually receives: class C: def __getitem__(self, index): print(repr(index)) '{c[4]}'.format(c=C()) '{c[4:6]}'.format(c=C()) '{c[anything goes!@#$%^&]}'.format(c=C()) C()[4:6]  Output (Try it online!): 4 '4:6' 'anything goes!@#$%^&' slice(4, 6, None)  So while the 4 gets converted to an int, the 4:6 isn't converted to slice(4, 6, None) as in usual slicing. Instead, it remains simply the string '4:6'. And that's not a valid type for indexing/slicing a string, hence the TypeError: string indices must be integers you got. Update: Is that documented? Well... I don't see something really clear, but @GACy20 pointed out something subtle. The grammar has these rules field_name ::= arg_name ("." attribute_name | "[" element_index "]")* element_index ::= digit+ | index_string index_string ::= <any source character except "]"> +  Our c[4:6] is the field_name, and we're interested in the element_index part 4:6. I think it would be clearer if digit+ had its own rule with meaningful name: field_name ::= arg_name ("." attribute_name | "[" element_index "]")* element_index ::= index_integer | index_string index_integer ::= digit+ index_string ::= <any source character except "]"> +  I'd say having index_integer and index_string would more clearly indicate that digit+ is converted to an integer (instead of staying a digit string), while <any source character except "]"> + would stay a string. That said, looking at the rules as they are, perhaps we should think "what would be the point of separating the digits case out of the any-characters case which would match it as well?" and think that the point is to treat pure digits differently, presumably to convert them to an integer. Or maybe some other part of the documentation even states that digit or digits+ in general gets converted to an integer.
Question: I am testing/attempting to learn flask, and flast_restful. This issue I get is: code 400, message Bad request syntax ('name=testitem') main.py: from flask import Flask,request from flask_restful import Api, Resource, reqparse app = Flask(__name__) api = Api(app) product_put_args = reqparse.RequestParser() product_put_args.add_argument("name", type = str, help = "Name of the product") product_put_args.add_argument("quantity", type = int, help = "Quantity of the item") products = {} class Product(Resource): def get(self, barcode): return products[barcode] def put(self, barcode): args = product_put_args.parse_args() return {barcode: args} api.add_resource(Product, "/product/<int:barcode>") if(__name__) == "__main__": app.run(debug = True)  and my test.py import requests base = "http://127.0.0.1:5000/" response = requests.put(base + "product/1", {"name": "testitem"}) print(response.json())  I have attempted to reform mat and change around both files to figure out what is sending the issue, I feel like it is something simple, but if you can help me, I bet this will help me and many others that are trying to start creating a rest API.

Answer: You need to add the location information to the RequestParser by default it tries to parse values from flask.Request.values, and flask.Request.json, but in your case, the values need to be parsed from a flask.request.form. Below code fixes your error from flask import Flask,request from flask_restful import Api, Resource, reqparse app = Flask(__name__) api = Api(app) product_put_args = reqparse.RequestParser() product_put_args.add_argument("name", type = str, help = "Name of the product", location='form') product_put_args.add_argument("quantity", type = int, help = "Quantity of the item", location='form') products = {} class Product(Resource): def get(self, barcode): return products[barcode] def put(self, barcode): args = product_put_args.parse_args() products[barcode] = args['name'] return {barcode: args} api.add_resource(Product, "/product/<int:barcode>") if(__name__) == "__main__": app.run(debug = True)
Question: I am new to unit testing and I require to perform some simple unit tests for an object storage class. I have a class named OSBucket as follows: def __initBucket(self): ecs_session = boto3.Session( aws_access_key_id="OSKEY", aws_secret_access_key="SECRETKEY" ) OS_resource = ecs_session.resource('s3', verify=cert, endpoint_url=endpoint) self.mybucket = OS_resource.Bucket(OS_BUCKET) def get_mybucket(self): return self.mybucket def download_file(self, fileName,filepath): self.mybucket.download_file(fileName, filepath) def upload_file(self, filepath,dest_file_name): self.mybucket.upload_file(filepath, '%s%s' % ("/",dest_file_name))  The method __initBucket is called in the constructor of the class. How could I start creating a unit test class to test, for example, the download_file method? UPDATE 1  moto_fake.start() conn = boto3.resource('s3', aws_access_key_id="fake_id", aws_secret_access_key="fake_secret") conn.create_bucket(Bucket="OS_BUCKET") os_bucket = OSBucket.OSBucket(thisRun) sourcefile = "testingMoto.txt" filePath = os.path.join("/", sourcefile) os_bucket.upload_file(filePath, sourcefile)  which executes the moto_fake.start() before creating the os_bucket object does not work for me. UPDATE 2 Using patch.object to change the endpoint variable to None, makes the test pass

Answer: First Approach: using python mocks You can mock the s3 bucket using standard python mocks and then check that you are calling the methods with the arguments you expect. However, this approach won't actually guarantee that your implementation is correct since you won't be connecting to s3. For example, you can call non-existing boto functions if you misspelled their names - the mock won't throw any exception. Second Approach: using moto Moto is the best practice for testing boto. It simulates boto in your local machine, creating buckets locally so you can have complete end-to-end tests. I used moto to write a couple of tests for your class (and I might have found a bug in your implementation - check the last test line - those are the arguments needed to make it find the file and not throw an exception) import pathlib from tempfile import NamedTemporaryFile import boto3 import moto import pytest from botocore.exceptions import ClientError from os_bucket import OSBucket @pytest.fixture def empty_bucket(): moto_fake = moto.mock_s3() try: moto_fake.start() conn = boto3.resource('s3') conn.create_bucket(Bucket="OS_BUCKET") # or the name of the bucket you use yield conn finally: moto_fake.stop() def test_download_non_existing_path(empty_bucket): os_bucket = OSBucket() os_bucket.initBucket() with pytest.raises(ClientError) as e: os_bucket.download_file("bad_path", "bad_file") assert "Not Found" in str(e) def test_upload_and_download(empty_bucket): os_bucket = OSBucket() os_bucket.initBucket() with NamedTemporaryFile() as tmp: tmp.write(b'Hi') file_name = pathlib.Path(tmp.name).name os_bucket.upload_file(tmp.name, file_name) os_bucket.download_file("/" + file_name, file_name) # this might indicate a bug in the implementation  Final notes:  I actually gave a pycon lecture about this exact topic, watch it here The source code for the lecture can be found here
Question: np.cumsum([1, 2, 3, np.nan, 4, 5, 6]) will return nan for every value after the first np.nan. Moreover, it will do the same for any generator. However, np.cumsum(df['column']) will not. What does np.cumsum(...) do, such that dataframes are treated specially? In [2]: df = pd.DataFrame({'column': [1, 2, 3, np.nan, 4, 5, 6]}) In [3]: np.cumsum(df['column']) Out[3]: 0 1.0 1 3.0 2 6.0 3 NaN 4 10.0 5 15.0 6 21.0 Name: column, dtype: float64 

Answer: When you call np.cumsum(object) with an object that is not a numpy array, it will try calling object.cumsum() See this thread for details . You can also see it in the Numpy source. The pandas method has a default of skipna=True. So np.cumsum(df) gets turned into the equivalent of df.cumsum(axis=None, skipna=True, *args, **kwargs), which, of course skips the NaN values. The Numpy method does not have a skipna option. You can also verify this yourself by overriding the pandas method with your own: class DF(pd.DataFrame): def cumsum(self, axis=None, skipna=True, *args, **kwargs): print('calling pandas cumsum') return super().cumsum(axis=None, skipna=True, *args, **kwargs) df = DF({'column': [1, 2, 3, np.nan, 4, 5, 6]}) # does calling the numpy function call your pandas method? np.cumsum(df)  This will print calling pandas cumsum  and return the expected result:  column 0 1.0 1 3.0 2 6.0 3 NaN 4 10.0 5 15.0 6 21.0  You can then experiment with the result of changing skipna=True.
Question: The Python https://requests-cache.readthedocs.io/ library can be used to cache requests. If I'm using requests-cache, how do I detect whether a response came from the cache, or had to be re-fetched from the network?

Answer: Based on the docs  The following attributes are available on responses:  from_cache: indicates if the response came from the cache cache_key: The unique identifier used to match the request to the response (see Request Matching for details) created_at: datetime of when the cached response was created or last updated expires: datetime after which the cached response will expire (see Expiration for details) is_expired: indicates if the cached response is expired (if, for example, an old response was returned due to a request error)   From their example from requests_cache import CachedSession session = CachedSession(expire_after=timedelta(days=1)) response = session.get('http://httpbin.org/get') print(response.from_cache)
Question: My code is running fine in local, but crashes on Heroku. What can I do to fix it?  Traceback (most recent call last) File "/app/nao_entre_em_panico.py", line 2, in <module> from flask import Flask, jsonify, request File "/app/.heroku/python/lib/python3.10/site-packages/flask/__init__.py", line 17, in <module> from werkzeug.exceptions import abort File "/app/.heroku/python/lib/python3.10/site-packages/werkzeug/__init__.py", line 151, in <module> __import__('werkzeug.exceptions') File "/app/.heroku/python/lib/python3.10/site-packages/werkzeug/exceptions.py", line 71, in <module> from werkzeug.wrappers import Response File "/app/.heroku/python/lib/python3.10/site-packages/werkzeug/wrappers.py", line 27, in <module> from werkzeug.http import HTTP_STATUS_CODES, \ File "/app/.heroku/python/lib/python3.10/site-packages/werkzeug/http.py", line 1148, in <module> from werkzeug.datastructures import Accept, HeaderSet, ETags, Authorization, \ File "/app/.heroku/python/lib/python3.10/site-packages/werkzeug/datastructures.py", line 16, in <module> from collections import Container, Iterable, MutableSet ImportError: cannot import name 'Container' from 'collections' (/app/.heroku/python/lib/python3.10/collections/__init__.py) 

Answer: Container, Iterable, MutableSet, and other abstract base classes are in collections.abc since Python 3.3.