eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I got below error message when I run model_main_tf2.py on Object Detection API: Traceback (most recent call last): File "/content/models/research/object_detection/model_main_tf2.py", line 32, in <module> from object_detection import model_lib_v2 File "/usr/local/lib/python3.7/dist-packages/object_detection/model_lib_v2.py", line 29, in <module> from object_detection import eval_util File "/usr/local/lib/python3.7/dist-packages/object_detection/eval_util.py", line 36, in <module> from object_detection.metrics import lvis_evaluation File "/usr/local/lib/python3.7/dist-packages/object_detection/metrics/lvis_evaluation.py", line 23, in <module> from lvis import results as lvis_results File "/usr/local/lib/python3.7/dist-packages/lvis/__init__.py", line 5, in <module> from lvis.vis import LVISVis File "/usr/local/lib/python3.7/dist-packages/lvis/vis.py", line 1, in <module> import cv2 File "/usr/local/lib/python3.7/dist-packages/cv2/__init__.py", line 9, in <module> from .cv2 import _registerMatType ImportError: cannot import name '_registerMatType' from 'cv2.cv2' (/usr/local/lib/python3.7/dist-packages/cv2/cv2.cpython-37m-x86_64-linux-gnu.so)  The weird thing is I run the same code before, it worked well but now it gives me an error.

Answer: The same thing occurred to me yesterday when I used Colab. A possible reason may be that the version of opencv-python(4.1.2.30) does not match opencv-python-headless(4.5.5.62). Or the latest version 4.5.5 may have something wrong... I uninstalled opencv-python-headless==4.5.5.62 and installed 4.1.2.30 and it fixed.
Question: I'm exploiting the cachetools @ttl_cache decorator (not @cached). I need to ignore some params in the cache key. E.g,. @ttl_cache(maxsize=1024, ttl=600) def my_func(foo, ignore_bar, ignore_baz): # do stuff  Working that way, I get this: >>> my_func("foo", "ignore_bar", "ignore_baz") # cache miss >>> my_func("foo", "ignore_bar", "ignore_baz") # cache hit >>> my_func("foo", "ignore_bar_bar", "ignore_baz_baz") # cache miss!  What I need: >>> my_func("foo", "ignore_bar", "ignore_baz") # cache miss >>> my_func("foo", "ignore_bar", "ignore_baz") # cache hit >>> my_func("foo", "ignore_bar_bar", "ignore_baz_baz") # cache hit!!!!!  Is there a way to get that using @ttl_cache?

Answer: I haven't used cachetools, but I've looked at the docs out of interest. Apparently, there's no built-in way. If you really need this functionality, I can suggest a hack like the following: class PackedArgs(tuple): def __hash__(self): return hash(self[0]) def __eq__(self, other): if isinstance(other, self.__class__): return self[0] == other[0] else: return NotImplemented def pack_args(func): def inner(packed_args): return func(*packed_args) return inner def unpack_args(func): def inner(*args): return func(PackedArgs(args)) return inner @unpack_args @ttl_cache(maxsize=1024, ttl=600) @pack_args def my_func(foo, ignore_bar, ignore_baz): # do stuff  Essentially: "pack" your function's arguments to a single tuple-like object that hashes as it's 0th element and causes @ttl_cache to behave like you need. Then, "unpack" them to restore the normal interface and avoid actually having to pass this one big argument when calling. Please note that this is just a (very hacky) concept, I haven't tested this code at all. It probably won't work as is. By the way, your requirement sounds interesting. You can submit it to cachetools as a feauture request and link this thread. I can imagine it being implemented as something like a key= kwarg lambda, similar to builtin sort's.
Question: I am doing an assessment that is asking by the given "n" as input which is a length of a stick; how many triangles can you make? (3 < n < 1,000,000) For example: input: N=8 output: 1 explanation: (3,3,2) input: N=12 output: 3 explanation: (4,4,4) (4,5,3) (5,5,2)  Now the codes I wrote are returning 33 % accuracy as the web assessment is throwing time limit error. ans = 0 n = int(input()) for a in range(1, n + 1): for b in range(a, n - a + 1): c = n - a - b if a + b > c >= b: ans += 1 print(ans)  code b: ans = 0 n = int(input()) for i in range(1,n): for j in range(i,n): for c in range(j,n): if(i+j+c==n and i+j>c): ans+=1 print(ans)  How can this be made faster?

Answer: This is an intuitive O(n) algorithm I came up with: def main(): n = int(input()) if n < 3: print(0) return ans = n % 2 for a in range(2, n//2+1): diff = n - a if diff // 2 < a: break if diff % 2 == 0: b = diff // 2 else: b = diff // 2 + 1 b = max(b - a // 2, a) c = n - b - a if abs(b - c) >= a: b += 1 c -= 1 ans += abs(b-c)//2 + 1 print(ans) main()  I find the upper bound and lower bound for b and c and count the values in that range.
Question: I have a DataFrame like name col1 col2 a aa 123 a bb 123 b aa 234  and a list [1, 2, 3]  I want to replace the col2 of every row with col1 = 'aa' with the list like name col1 col2 a aa [1, 2, 3] a bb 123 b aa [1, 2, 3]  I tried something like df.loc[df[col1] == 'aa', col2] = [1, 2, 3]  but it gives me the error: ValueError: could not broadcast input array from shape (xx,) into shape (yy,)  How should I get around this?

Answer: import pandas as pd df = pd.DataFrame({"name":["a","a","b"],"col1":["aa","bb","aa"],"col2":[123,123,234]}) l = [1,2,3] df["col2"] = df.apply(lambda x: l if x.col1 == "aa" else x.col2, axis =1) df
Question: I have a dictionary with a lot of strings. Is it possible to store a formatted string with placeholders and pass in a actual values later? I'm thinking of something like this: d = { "message": f"Hi There, {0}" } print(d["message"].format("Dave"))  The above code obviously doesn't work but I'm looking for something similar.

Answer: You use f-string; it already interpolated 0 in there. You might want to remove f there d = { # no f here "message": "Hi There, {0}" } print(d["message"].format("Dave"))  Hi There, Dave
Question: What I am playing around with right now is to work with PyTorch within a pipeline, where all of the preprocessing will be handled. I am able to make it work. However, the results I am getting are a bit off. The loss function seems to be not decreasing and gets stuck (presumably in local optima?) as the training loop progresses. I follow the standard PyTorch training loop and wrap it inside the fit method as this is what sklearn wants: import torch from sklearn.base import BaseEstimator, TransformerMixin import torch.nn.functional as F from IPython.core.debugger import set_trace # + import pandas as pd import seaborn as sns import numpy as np from tqdm import tqdm import random # - df = sns.load_dataset("tips") df.head() # + class LinearRegressionModel(torch.nn.Module, BaseEstimator, TransformerMixin): def __init__(self, loss_func = torch.nn.MSELoss()): super(LinearRegressionModel, self).__init__() self.linear = torch.nn.Linear(3, 1) # One in and one out self.loss_func = loss_func self.optimizer = torch.optim.SGD(self.parameters(), lr = 0.01) def forward(self, x): y_pred = F.relu(self.linear(x)) return y_pred def fit(self, X, y): # set_trace() X = torch.from_numpy(X.astype(np.float32)) y = torch.from_numpy(y.values.astype(np.float32)) for epoch in tqdm(range(0, 12)): pred_y = self.forward(X) # Compute and print loss loss = self.loss_func(pred_y, X) # Zero gradients, perform a backward pass, # and update the weights. self.optimizer.zero_grad() loss.backward() self.optimizer.step() print('epoch {}, loss {}'.format(epoch, loss.item())) # + from sklearn.pipeline import Pipeline from sklego.preprocessing import PatsyTransformer # - my_model = LinearRegressionModel() pipe = Pipeline([ ("patsy", PatsyTransformer("tip + size")), ("model", my_model) ]) pipe.fit(df, df['total_bill'])  It is not only due to the model being to simple. If I use sklearn linear regression estimated via stochastic gradient descent (SGDRegressor) the results seem nice. Therefore, I am concluding that problem is within my PyTorch class # + from sklearn.linear_model import LinearRegression from sklearn.metrics import mean_squared_error pipe2 = Pipeline([ ("patsy", PatsyTransformer("tip + C(size) + C(time)")), ("model", LinearRegression()) ]) pipe2.fit(df, df['total_bill']) # - mean_squared_error(df['total_bill'], pipe2.predict(df)) 

Answer: The problem in this implementation is in the fit method. We are comparing prediction and design matrix # Compute and print loss loss = self.loss_func(pred_y, X)  Should be prediction and real value y: loss = self.loss_func(pred_y, y)
Question: I am try to run code sh run.sh  and it showed me the error File "/anaconda3/envs/_galaxy_/lib/python3.6/site-packages/filelock/__init__.py", line 8 from __future__ import annotations ^ SyntaxError: future feature annotations is not defined  I saw some solutions indicated that I need to update my python version, but I am in a python verion 3.9.7. (py39) KedeMacBook-Pro:~ ke$ python --version Python 3.9.7  However, in the error code it showed a python veriosn of 3.6. So, I am not sure where went wrong. Why it is not using the python envrionment that I have? Please help, thank you.

Answer: Based on the error, it looks like your code is using Python 3.6 and not Python 3.9. This import is available starting from Python 3.7. Check run.sh to make sure it is referencing the right python interpreter. I'd also recommend using a virtual env using the python version you require and running your script inside that.
Question: I ran the code in VSCode and got a TypeError: Object of type set is not JSON serializable. I just start to learn to code, really don't get it, and googled it, also didn't know what does JSON serializable means. from solcx import compile_standard import json # get the contract content with open("./SimpleStorage.sol", "r") as file: simple_storage_file = file.read() # compile the contract compiled_sol = compile_standard( { "language": "Solidity", "sources": {"SimpleStorage.sol": {"content": simple_storage_file}}, "settings": { "outputSelection": { "*": { "*": {"abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"} } } }, }, solc_version="0.6.0", ) # creat json file dump the comiled code in it to make it more readable. with open("compiled_code.json", "w") as file: json.dump(compiled_sol, file) print(compiled_sol)  The full error information is below: (env) (base) liwei@liweideMacBook-Pro practice % python3 deploy.py Traceback (most recent call last): File "deploy.py", line 10, in <module> compiled_sol = compile_standard( File "/Users/liwei/Desktop/demos/practice/env/lib/python3.8/site-packages/solcx/main.py", line 375, in compile_standard stdin=json.dumps(input_data), File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 231, in dumps return _default_encoder.encode(obj) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 199, in encode chunks = self.iterencode(o, _one_shot=True) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 257, in iterencode return _iterencode(o, 0) File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.__class__.__name__} ' TypeError: Object of type set is not JSON serializable 

Answer: Instead of this: {"abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"}  you should use this: ["abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"]  Sets in Python aren't JSON serializable.
Question: Reading this documentation https://docs.djangoproject.com/en/4.0/topics/db/transactions/#django.db.transaction.on_commit This is the use case for on_commit with transaction.atomic(): # Outer atomic, start a new transaction transaction.on_commit(foo) # Do things... with transaction.atomic(): # Inner atomic block, create a savepoint transaction.on_commit(bar) # Do more things... # foo() and then bar() will be called when leaving the outermost block  But why not just write the code like normal without on_commit hooks? Like this: with transaction.atomic(): # Outer atomic, start a new transaction # Do things... with transaction.atomic(): # Inner atomic block, create a savepoint # Do more things... foo() bar() # foo() and then bar() will be called when leaving the outermost block  It's easier to read since it doesn't require more knowledge of the Django APIs and the statements are put in the order of when they are executed. It's easier to test since you don't have to use any special test classes for Django. So what is the use-case for the on_commit hook?

Answer: Django documentation:  Django provides the on_commit() function to register callback functions that should be executed after a transaction is successfully committed  It is the main purpose. A transaction is a unit of work that you want to treat atomically. It either happens completely or not at all. The same applies to your code. If something went wrong during DB operations you might not need to do some things. Let's consider some business logic flow:  User sends their registration data to our endpoint, we validate it, etc. We save the new user to our DB. We send them a "hello" letter to email with a link for confirming his account.  If something goes wrong during step 2, we shouldn't go to step 3. We can think that, well, I'll get an exception and wouldn't execute that code as well. Why do we still need it? Sometimes you take actions in your code based on an assumption of the transaction being successful before potentially dangerous DB operations. For example, you want firstly to check if can send an email to your user, because you know that your emailing 3rd-party often gives you 500. In that case, you want to raise a 500 for the user and ask them to register later (a very bad idea, btw, but it's just a synthetic example). When your function (e.g. with @atomic decorator) contains a lot of DB operations you surely don't want to memorize all the variables states in order to use them after all DB-related code. Like this:  Validation of user's order. Checking at DB if it could be completed. If it could be done we need to send a request to 3rd-party CRM with the order's details. If it couldn't, then we should create a support ticket in another 3rd-party. Saving user's order to DB, updating user's model. Sending a messenger notification to the employee who is responsible for the order. Saving information, that notification for employee was sent successfully to the DB.  You can imagine what a mess would we have if we hadn't on_commit in this situation and we had a really big try-catch on this.
Question: I am trying to use a Jupyter notebook for some Pandas in VS Code. I set up a virtual environment venv where I installed Pandas and jupyter. I always did it like this and it worked fine. But suddenly it does not work anymore. 

Answer: Could you try to reinstall the pyzmq module? pip uninstall pyzmq pip install pyzmq==19.0.2  The version number may be different depending on the jupyter-client version.
Question: I'm writing a pytest file to check if my machine learning libraries use the GPU. For Tensorflow I can check this with tf.config.list_physical_devices(). For XGBoost I've so far checked it by looking at GPU utilization (nvdidia-smi) while running my software. But how can I check this in a simple test? Something similar to the test I have for Tensorflow would do. import pytest import tensorflow as tf import xgboost # Marking all tests to be GPU dependent pytestmark = pytest.mark.gpu def test_tf_finds_gpu(): """Check if Tensorflow finds the GPU.""" assert tf.config.list_physical_devices("GPU") def test_xgb_finds_gpu(): """Check if XGBoost finds the GPU.""" ... # What can I write here? 

Answer: Note that tree_method="gpu_hist" is deprecated and will stop / has stopped working since xgboost==2.0.0. Histogram type and device are currently split into two parameters: tree_method (an unfortunate overwriting of the existing parameter, but with a different set of permitted levels) and a new one called device: import numpy as np import xgboost as xgb xgb_model = xgb.XGBRegressor( # tree_method="gpu_hist" # deprecated tree_method="hist", device="cuda" ) X = np.random.rand(50, 2) y = np.random.randint(2, size=50) xgb_model.fit(X, y) xgb_model  Output when GPU access works correctly (no warnings): XGBRegressor(base_score=None, booster=None, callbacks=None, colsample_bylevel=None, colsample_bynode=None, colsample_bytree=None, device='cuda', early_stopping_rounds=None, enable_categorical=False, eval_metric=None, feature_types=None, gamma=None, grow_policy=None, importance_type=None, interaction_constraints=None, learning_rate=None, max_bin=None, max_cat_threshold=None, max_cat_to_onehot=None, max_delta_step=None, max_depth=None, max_leaves=None, min_child_weight=None, missing=nan, monotone_constraints=None, multi_strategy=None, n_estimators=None, n_jobs=None, num_parallel_tree=None, random_state=None, ...)  vs. No GPU access - a warning that the device argument has not been used: [11:43:35] WARNING: ../src/learner.cc:767: Parameters: { "device" } are not used.
Question: I have a simple route as below that written in FastAPI, from fastapi import FastAPI app = FastAPI() @app.get("/foo/bar/{rand_int}/foo-bar/") async def main(rand_int: int): return {"path": f"https://some-domain.com/foo/bar/{rand_int}/foo-bar/?somethig=foo"}  How can I get the current path "programmatically" with,  domain (some-domain.com) path (/foo/bar/{rand_int}/foo-bar/) and query parameters (?somethig=foo) 

Answer: We can use the Request.url-(starlette doc) API to get the various URL properties. To get the absolute URL, we need to use the Request.url._url private API ( or str(Request.url)), as below from fastapi import FastAPI, Request app = FastAPI() @app.get("/foo/bar/{rand_int}/foo-bar/") async def main(rand_int: int, request: Request): return {"raw_url": str(request.url)}
Question: I want to generate a toy example to illustrate a convex piecewise linear function in python, but I couldn't figure out the best way to do this. What I want to do is to indicate the number of lines and generate the function randomly. A convex piecewise-linear function is defined as:  For instance, if I want to have four linear lines, then I want to generate something as shown below.  Since there are four lines. I need to generate four increasing random integers to determine the intervals in x-axis. import random import numpy as np random.seed(1) x_points = np.array(random.sample(range(1, 20), 4)) x_points.sort() x_points = np.append(0, x_points) x_points [0 3 4 5 9]  I can now use the first two points and create a random linear function, but I don't know how I should continue from there to preserve the convexity. Note that a function is called convex if the line segment between any two points on the graph of the function does not lie below the graph between the two points.

Answer: The slope increases monotonically by a random value from the range [0,1), starting from 0. The first y value is also zero, see the comments. import numpy as np np.random.seed(0) x_points = np.random.randint(low=1, high=20, size=4) x_points.sort() x_points = np.append(0, x_points) # the first 0 point is 0 slopes = np.add.accumulate(np.random.random(size=3)) slopes = np.append(0,slopes) # the first slope is 0 y_incr = np.ediff1d(x_points)*slopes y_points = np.add.accumulate(y_incr) y_points = np.append(0,y_points) # the first y values is 0  A possible output looks like this: print(x_points) print(y_points) # [ 0 1 4 13 16] # [ 0. 0. 2.57383685 17.92061306 24.90689622]   To print this figure: import matplotlib.pyplot as plt fig, ax = plt.subplots() ax.plot(x_points,y_points, '-o', label="convex piecewise-linear function") ax.legend() fig.patch.set_facecolor('white') plt.show()
Question: I have an async FastApi application with async sqlalchemy, source code (will not provide schemas.py because it is not necessary): database.py from sqlalchemy import ( Column, String, ) from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine from sqlalchemy.ext.declarative import declarative_base from sqlalchemy.orm import sessionmaker from sqlalchemy.orm.decl_api import DeclarativeMeta from app.config import settings engine = create_async_engine(settings.DATABASE_URL) Base: DeclarativeMeta = declarative_base() async_session = sessionmaker(engine, class_=AsyncSession, expire_on_commit=False) class Titles(Base): __tablename__ = "titles" id = Column(String(100), primary_key=True) title = Column(String(100), unique=True) async def get_session() -> AsyncSession: async with async_session() as session: yield session  routers.py import .database from fastapi_utils.cbv import cbv from fastapi_utils.inferring_router import InferringRouter router = InferringRouter() async def get_titles(session: AsyncSession): results = await session.execute(select(database.Titles))) return results.scalars().all() @cbv(router) class TitlesView: session: AsyncSession = Depends(database.get_session) @router.get("/titles", status_code=HTTP_200_OK) async def get(self) -> List[TitlesSchema]: results = await get_titles(self.session) return [TitlesSchema.from_orm(result) for result in results]  main.py from fastapi import FastAPI from app.routers import router def create_app() -> FastAPI: app = FastAPI() app .include_router(routers, prefix="/", tags=["Titles"]) return printer_app app = create_app()  It runs with docker: CMD ["uvicorn", "main:app", "--reload", "--host", "0.0.0.0", "--port", "8000", "--limit-max-requests", "10000"]  And it has Postgres database with default settings in docker too. It all runs at docker-swarm. Works fine at first, accepts all requests. But if you leave it for 15-30 minutes (I did not count), and then make a request, it will not work: <class 'asyncpg.exceptions.ConnectionDoesNotExistError'>: connection was closed in the middle of operation  And right after that I send the next request and it doesn't throw an error. What could it be? How do I get rid of the ConnectionDoesNotExistError?

Answer: I solve that using pool_pre_ping setting like that: engine = create_async_engine(DB_URL, pool_pre_ping=True)  https://docs.sqlalchemy.org/en/14/core/pooling.html