eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have a very basic flask app with dependencies installed from my requirements.txt. All of these dependencies are installed in my virtual environment. requirements.txt given below, aniso8601==6.0.0 Click==7.0 Flask==1.0.3 Flask-Cors==3.0.7 Flask-RESTful==0.3.7 Flask-SQLAlchemy==2.4.0 itsdangerous==1.1.0 Jinja2==2.10.1 MarkupSafe==1.1.1 # psycopg2-binary==2.8.2 pytz==2019.1 six==1.12.0 # SQLAlchemy==1.3.4 Werkzeug==0.15.4 python-dotenv requests authlib  My code in NewTest.py file, from flask import Flask, request, jsonify, abort, url_for app = Flask(__name__) @app.route('/') def index(): return jsonify({ 'success': True, 'index': 'Test Pass' }) if __name__ == '__main__': app.run(debug=True)  When I run the app through, export FLASK_APP=NewTest.py export FLASK_ENV=development export FLASK_DEBUG=true flask run or flask run --reload  I get the following error, 127.0.0.1 - - [09/Feb/2020 12:43:40] "GET / HTTP/1.1" 500 - Traceback (most recent call last): File "/projects/env/lib/python3.8/site-packages/flask/_compat.py", line 36, i n reraise raise value File "/projects/NewTest.py", line 3, in <module> app = Flask(__name__) File "/projects/env/lib/python3.8/site-packages/flask/app.py", line 559, in _ _init__ self.add_url_rule( File "/projects/env/lib/python3.8/site-packages/flask/app.py", line 67, in wr apper_func return f(self, *args, **kwargs) File "/projects/env/lib/python3.8/site-packages/flask/app.py", line 1217, in add_url_rule self.url_map.add(rule) File "/projects/env/lib/python3.8/site-packages/werkzeug/routing.py", line 1388, in add rule.bind(self) File "/projects/env/lib/python3.8/site-packages/werkzeug/routing.py", line 730, in bind self.compile() File "/projects/env/lib/python3.8/site-packages/werkzeug/routing.py", line 794, in compile self._build = self._compile_builder(False).__get__(self, None) File "/projects/env/lib/python3.8/site-packages/werkzeug/routing.py", line 951, in _compile_builder code = compile(module, "<werkzeug routing>", "exec") TypeError: required field "type_ignores" missing from Module  Can anyone please point out what am I missing or doing wrong and how can I fix it? Thanks.

Answer: The bug was fixed in werkzeug 0.15.5. Upgrade from 0.15.4 to to a later version.
Question: I am reading the python doc https://docs.python.org/3/reference/expressions.html about generator.close(). My translation of the documentation is: ##generator.close() Raises a GeneratorExit at the point where the generator function was paused.  If the generator function then exits gracefully :  1.1 is already closed, 1.2 or raises GeneratorExit (by not catching the exception), close returns to its caller.  If the generator yields a value, a RuntimeError is raised.  If the generator raises any other exception, it is propagated to the caller.   close() does nothing if the generator has already exited due to an exception or normal exit.  I don't understand how the close() behavior corresponds to the documentation. >>> def echo(value=None): ... print("Execution starts when 'next()' is called for the first time.") ... try: ... while True: ... try: ... value = (yield value) ... except Exception as e: ... value = e ... finally: ... print("Don't forget to clean up when 'close()' is called.") ... >>> generator = echo(1) >>> next(generator) Execution starts when 'next()' is called for the first time. >>> generator.close() Don't forget to clean up when 'close()' is called.  Which rule applies to generator.close() ? I am confused. My understanding:  generator.close() raise a GeneratorExit exception GeneratorExit is catched by except Exception as e: and loop continues value = (yield value) executes according to rule 2 above, a RuntimeError will be raised.  But that doesn't seem to be the case. Please tell me what's going on inside.

Answer: GeneratorExit does not inherit from Exception, but from the more fundamental BaseException. Thus, it is not caught by your except Exception block. So your assumption 2 is wrong. The generator exits gracefully via case 1.3, since GeneratorExit is not stopped.  The GeneratorExit is thrown at (yield value). The try: except Exception as e: checks whether the current exception is subclass of Exception. Since this is not the case, it unwinds. The while True: unwinds due the current exception. The try: finally: unwinds, running its finally: block. This causes the message to be displayed. The generator exits with the current exception, i.e. GeneratorExit. generator.close detects and suppresses GeneratorExit.
Question: For example, define  df=pd.DataFrame(np.random.randint(0,10,(6,6))) df  Which gives below display in Jupyter notebook  My question is that is it possible to add a column delimiter to the dataframe like   Thank you for all the answers, currently I use below custom functions def css_border(x,pos): return ["border-left: 1px solid red" if i in pos else "border: 0px" for i, col in enumerate(x)] def display_df_with_delimiter(df,pos): return df.style.apply(partial(css_border,pos=pos), axis=1)  and  display_df_with_delimiter(df,[0,1,2,5])  gives 

Answer: This piece of code should add the desired lines to the table.  from IPython.display import display, HTML CSS = """ .rendered_html td:nth-child(even) { border-left: 1px solid red; } """ HTML('<style>{}</style>'.format(CSS))  Note that you can change the style of those linse by simply changing the definition of border-left attribute, i.e border-left: 2px solid green to make the lines thicker and green. Here is a snapshot demonstrating the output.
Question: I am trying to understand asyncio module and spend about one hour with run_coroutine_threadsafe function, I even came to the working example, it works as expected, but works with several limitations. First of all I do not understand how should I properly call asyncio loop in main (any other) thread, in the example I call it with run_until_complete and give it a coroutine to make it busy with something until another thread will not give it a coroutine. What are other options I have? What are situations when I have to mix asyncio and threading (in Python) in real life? Since as far as I understand asyncio is supposed to take place of threading in Python (due to GIL for not IO ops), if I am wrong, do not be angry and share your suggestions. Python version is 3.7/3.8 import asyncio import threading import time async def coro_func(): return await asyncio.sleep(3, 42) def another_thread(_loop): coro = coro_func() # is local thread coroutine which we would like to run in another thread # _loop is a loop which was created in another thread future = asyncio.run_coroutine_threadsafe(coro, _loop) print(f"{threading.current_thread().name}: {future.result()}") time.sleep(15) print(f"{threading.current_thread().name} is Finished") if __name__ == '__main__': loop = asyncio.get_event_loop() main_th_cor = asyncio.sleep(10) # main_th_cor is used to make loop busy with something until another_thread will not send coroutine to it print("START MAIN") x = threading.Thread(target=another_thread, args=(loop, ), name="Some_Thread") x.start() time.sleep(1) loop.run_until_complete(main_th_cor) print("FINISH MAIN") 

Answer:  First of all I do not understand how should I properly call asyncio loop in main (any other) thread, in the example I call it with run_until_complete and give it a coroutine to make it busy with something until another thread will not give it a coroutine. What are other options I have?  This is a good use case for loop.run_forever(). The loop will run and serve the coroutines you submit using run_coroutine_threadsafe. (You can even submit such coroutines from multiple threads in parallel; you never need to instantiate more than one event loop.) You can stop the loop from a different thread by calling loop.call_soon_threadsafe(loop.stop).  What are situations when I have to mix asyncio and threading (in Python) in real life?  Ideally there should be none. But in the real world, they do crop up; for example:  When you are introducing asyncio into an existing large program that uses threads and blocking calls and cannot be converted to asyncio all at once. run_coroutine_threadsafe allows regular blocking code to make use of asyncio. When you are dealing with older "async" APIs which use threads under the hood and call the user-supplied APIs from other threads. There are many examples, such as Python's own multiprocessing. When you need to call blocking functions that have no async equivalent from asyncio - e.g. CPU-bound functions, legacy database drivers, things like that. This is not a use case for run_coroutine_threadsafe, here you'd use run_in_executor, but it is another example of mixing threads and asyncio.
Question: I'm running a FastAPI app in Python using uvicorn on a Windows machine without a frontend (e.g. Next.js, etc.) so there should NOT be any iteraction between a local frontend and backend like there is in this question. Plus the answer(s) to that question would not have solved my issue/question. That question was also asked AFTER this one so THIS QUESTION IS NOT A DUPLICATE! It works fine when I do any one of the following options:  Run the following code on my mac, or When I don't specify the port for uvicorn (remove the host parameter from the uvicorn.run call) When I specify port '127.0.0.1', which is the host it uses when I don't specify a host at all.  from fastapi import FastAPI import uvicorn app = FastAPI() @app.get("/") async def root(): return {"message": "Hello World"} if __name__ == '__main__': uvicorn.run(app, port=8080, host='0.0.0.0')  When I go to 0.0.0.0:8080 on my browser, I get an error that says "This site can’t be reached". I have checked my current active ports to make sure I'm not getting a collision using netstat -ao |find /i "listening" and 0.0.0.0:8080 is not in use. My current file configuration looks like this: working_directory └── app ├── gunicorn_conf.py └── main.py  My gunicorn_conf.py is super simple and just tries to set the host and port: host = "0.0.0.0" port = "8080"  How can I get this to work when I specify host '0.0.0.0'?

Answer: As I was writing the question above, I found the solution and thought I would share in case someone else runs into this. To get it to work put "http://localhost:8080" into the web browser instead of "http://0.0.0.0:8080" and it will work fine. This also works if you're hitting the endpoint via the python requests package, etc.
Question: I am following Tensorflow's tutorial on building a simple neural network, and after importing the necessary libraries (tensorflow, keras, numpy & matplotlib) and datasets (fashion_mnist) I ran this code as per the tutorial:  model = keras.Sequential([ keras.layers.Flatten(input_shape=(28, 28)), keras.layers.Dense(128, activation='relu'), keras.layers.Dense(10, activation='softmax') ])  after running this code i received this warning message:   WARNING:tensorflow:From /Applications/anaconda3/envs/tensorfloe/lib/python3.7/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.init (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version. Instructions for updating: If using Keras pass *_constraint arguments to layers.  How do i fix this? Your help is highly appreciated.

Answer: This is internal TensorFlow message, you can safely ignore it. It will be gone in future versions of TensorFlow, no actions from your side is needed.
Question: I have got this code from Plotly page. I need to make the background transparent and the axis highlighted. And also the legends positioned inside the plot. import plotly.graph_objects as go fig = go.Figure() fig.add_trace(go.Scatter( x=[1, 2, 3, 4, 5], y=[1, 2, 3, 4, 5], name="Increasing" )) fig.add_trace(go.Scatter( x=[1, 2, 3, 4, 5], y=[5, 4, 3, 2, 1], name="Decreasing" )) fig.update_layout(legend_title='<b> Trend </b>') fig.show()  The code above shows the output below:  My expected output:  Hoow can i convert the first image to get the features of the second image?

Answer: To change the background color you need specify it by plot_bgcolor='rgba(0,0,0,0)',, while to move the legend inside the plot, on the left, you need to explicitly define the position: import plotly.graph_objects as go trace0 = go.Scatter( x=[1, 2, 3, 4, 5], y=[1, 2, 3, 4, 5], name="Increasing" ) trace1 = go.Scatter( x=[1, 2, 3, 4, 5], y=[5, 4, 3, 2, 1], name="Decreasing" ) data = [trace0, trace1] layout = go.Layout( plot_bgcolor='rgba(0,0,0,0)', legend=dict( x=0, y=0.7, traceorder='normal', font=dict( size=12,), ), annotations=[ dict( x=0, y=0.75, xref='paper', yref='paper', text='Trend', showarrow=False ) ] ) fig = go.Figure(data = data, layout = layout) fig.update_xaxes(showgrid=True, gridwidth=1, gridcolor='LightGray') fig.update_yaxes(showgrid=True, gridwidth=1, gridcolor='LightGray') fig.show()  and you get:
Question: I have several functions that create plots, which I use in Jupyter notebooks to visualise data. I want to create basic tests for these, checking that they still run without erroring on various inputs if I make changes. However, if I call these functions using pytest, creating the plots causes the program to hang until I manually minimise the plot. import pytest import matplotlib.pyplot as plt def plot_fn(): plt.plot([1,2,3]) plt.show() def test_plot_fn(): plot_fn()  How can I test that functions like 'plot_fn' run without erroring using Pytest? I tried the following, but it doesn't work, I think because plt.show() causes the script to hang, and so not reach plt.close('all'). def test_plot_fn(): plot_fn() plt.close('all')  I'm happy to change the behaviour of my plotting function, for example to return the plt object?

Answer: This works. from unittest.mock import patch import pytest import matplotlib.pyplot as plt def plot_fn(): plt.plot([1,2,3]) plt.show() @patch("matplotlib.pyplot.show") def test_plot_fn(mock_show): plot_fn()  Based on this answer (possible duplicate) Turn off graphs while running unittests
Question: I have the following task to solve:  Files are being sent at irregular times through an endpoint and stored locally. I need to trigger a DAG run for each of these files. For each file the same tasks will be performed  Overall the flows looks as follows: For each file, run tasks A->B->C->D Files are being processed in batch. While this task seemed trivial to me, I have found several ways to do this and I am confused about which one is the "proper" one (if any). First pattern: Use experimental REST API to trigger dag. That is, expose a web service which ingests the request and the file, stores it to a folder, and uses the experimental REST api to trigger the DAG, by passing the file_id as conf Cons: REST apis are still experimental, not sure how Airflow can handle a load test with many requests coming at one point (which shouldn't happen, but, what if it does?) Second pattern: 2 dags. One senses and triggers with TriggerDagOperator, one processes. Always using the same ws as described before, but this time it justs stores the file. Then we have:  First dag: Uses a FileSensor along with the TriggerDagOperator to trigger N dags given N files Second dag: Task A->B->C   Cons: Need to avoid that the same files are being sent to two different DAG runs. Example: Files in folder x.json Sensor finds x, triggers DAG (1) Sensor goes back and scheduled again. If DAG (1) did not process/move the file, the sensor DAG might reschedule a new DAG run with the same file. Which is unwanted. Third pattern: for file in files, task A->B->C As seen in this question. Cons: This could work, however what I dislike is that the UI will probably get messed up because every DAG run will not look the same but it will change with the number of files being processed. Also if there are 1000 files to be processed the run would probably be very difficult to read Fourth pattern: Use subdags I am not yet sure how they completely work as I have seen they are not encouraged (at the end), however it should be possible to spawn a subdag for each file and have it running. Similar to this question. Cons: Seems like subdags can only be used with the sequential executor.  Am I missing something and over-thinking something that should be (in my mind) quite straight-forward? Thanks

Answer: I found this article: https://medium.com/@igorlubimov/dynamic-scheduling-in-airflow-52979b3e6b13 where a new operator, namely TriggerMultiDagRunOperator is used. I think this suits my needs.
Question: I'm trying to run a simple Cartopy example:  import cartopy.crs as ccrs import matplotlib.pyplot as plt ax = plt.axes(projection=ccrs.PlateCarree()) ax.coastlines() plt.show()  But I'm getting this error:  Geometry must be a Point or LineString python: geos_ts_c.cpp:4179: int GEOSCoordSeq_getSize_r(GEOSContextHandle_t, const geos::geom::CoordinateSequence*, unsigned int*): Assertion0 != cs' failed`. I installed Cartopy using miniconda3: conda install -c conda-forge cartopy I also tried to install Cartopy with pip (in a virtual environment), but I get the same error. My OS is Debian Buster. Any idea?

Answer: The problem is a wrong version of shapely, with Cartopy the binary package shoudn't be used, it should be built from source instead. This is explained here and here. So I did: pip uninstall shapely pip install shapely --no-binary shapely
Question: What are the advantages of ViewSet, ModelViewSet and APIView. In the django-rest-framework documents it is not clear, it does not say when to use ViewSet, ModelViewSet and APIView. I want to implement an API that will have a business logic in there, a great business logic with data processing as well, what should be used for this case? I researched a lot and managed to understand a little about routers and urlpatterns but I didn't understand which one about views.

Answer: Summarizing: on one hand you have the APIView, which is the most generic of the three, but also in which you must do almost all business logic 'manually'. You have the class methods mapping http methods (get, post, ...) plus some class attributes to configure things like authentication, rendering, etc. Often you'll be developing endpoints to interact with resources (entities, like Users, Products, Orders, etc.) via CRUD operations, and that is what ViewSet is for: they have more semantic class methods like list, create, retrieve ... that the router can then automatically map to urls and http methods at the expense of making some assumptions: for example, the retrieve assumes the http call to be GET /you_resource/<pk>. It is more rigid than a generic APIView but it takes away from you some boilerplate/manual config that you would have to repeat again and again in most cases. One step further is the ModelViewSet, which is an extension of the ViewSet for when you are working with Django models. Just specifying a serializer_class and a queryset you have all the CRUD operations of the ViewSet ready to go. Obviously, you can also add your own methods to a ViewSet or customize the behavior of its default methods. In my experience, it pays off to use ViewSets. The code looks cleaner and you avoid some boilerplate code. The assumptions it makes are reasonable, and I would even say that you probably will end up with a cleaner API design following them.
Question: I have an API on FastAPI and i need to get the client real IP address when he request my page. I'm ty to use starlette Request. But it returns my server IP, not client remote IP. My code: @app.post('/my-endpoint') async def my_endpoint(stats: Stats, request: Request): ip = request.client.host print(ip) return {'status': 1, 'message': 'ok'}  What i'm doing wrong? How to get real IP (like in Flask request.remote_addr)?

Answer: request.client should work, unless you're running behind a proxy (e.g. nginx) in that case use uvicorn's --proxy-headers flag to accept these incoming headers and make sure the proxy forwards them.
Question: Consider the following files: holy_hand_grenade.py def count(one, two, five='three'): print('boom')  test_holy_hand_grenade.py from unittest import mock import holy_hand_grenade def test_hand_grenade(): mock_count = mock.patch("holy_hand_grenade.count", autospec=True) with mock_count as fake_count: fake_count(1, 2, five=5) # According to https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.call_args # this should work assert fake_count.call_args.kwargs['five'] == 5  According to the docs, call_args should be:  This is either None (if the mock hasn’t been called), or the arguments that the mock was last called with. This will be in the form of a tuple: the first member, which can also be accessed through the args property, is any ordered arguments the mock was called with (or an empty tuple) and the second member, which can also be accessed through the kwargs property, is any keyword arguments (or an empty dictionary).  (emphasis mine) But this blows up in my face, with TypeError: tuple indices must be integers or slices, not str Um. No? The thing I really don't understand, is that if this is a call object, which it is, because assert isinstance(fake_count.call_args, (mock._Call,))  passes, it's supposed to have kwargs and args. And it... well, it sort of does. But they appear to not actually be the correct thing: assert isinstance(fake_count.call_args.kwargs, (mock._Call,)) #this works assert isinstance(fake_count.call_args.kwargs, (dict,)) # doesn't work  What am I doing wrong here?

Answer: This is a feature introduced in Python 3.8 in this issue. The 3.7 documentation does not mention it (while the newest docs do) - so you have to access the arguments by index in Python < 3.8.
Question: In Keras, using the Flatten() layer retains the batch size. For eg, if the input shape to Flatten is (32, 100, 100), in Keras output of Flatten is (32, 10000), but in PyTorch it is 320000. Why is it so?

Answer: As OP already pointed out in their answer, the tensor operations do not default to considering a batch dimension. You can use torch.flatten() or Tensor.flatten() with start_dim=1 to start the flattening operation after the batch dimension. Alternatively since PyTorch 1.2.0 you can define an nn.Flatten() layer in your model which defaults to start_dim=1.
Question: Say I have a batch of images in the form of tensors with dimensions (B x C x W x H) where B is the batch size, C is the number of channels in the image, and W and H are the width and height of the image respectively. I'm looking to use the transforms.Normalize() function to normalize my images with respect to the mean and standard deviation of the dataset across the C image channels, meaning that I want a resulting tensor in the form 1 x C. Is there a straightforward way to do this? I tried torch.view(C, -1).mean(1) and torch.view(C, -1).std(1) but I get the error: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.  Edit After looking into how view() works in PyTorch, I know realize why my approach doesn't work; however, I still can't figure out how to get the per-channel mean and standard deviation.

Answer: You just need to rearrange batch tensor in a right way: from [B, C, W, H] to [B, C, W * H] by: batch = batch.view(batch.size(0), batch.size(1), -1)  Here is complete usage example on random data: Code: import torch from torch.utils.data import TensorDataset, DataLoader data = torch.randn(64, 3, 28, 28) labels = torch.zeros(64, 1) dataset = TensorDataset(data, labels) loader = DataLoader(dataset, batch_size=8) nimages = 0 mean = 0. std = 0. for batch, _ in loader: # Rearrange batch to be the shape of [B, C, W * H] batch = batch.view(batch.size(0), batch.size(1), -1) # Update total number of images nimages += batch.size(0) # Compute mean and std here mean += batch.mean(2).sum(0) std += batch.std(2).sum(0) # Final step mean /= nimages std /= nimages print(mean) print(std)  Output: tensor([-0.0029, -0.0022, -0.0036]) tensor([0.9942, 0.9939, 0.9923])