eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am using the Binance Python API (Python 3.x) When one uses the “create_order” functionality, it creates an order on the SPOT exchange with a STATUS of NEW. When it gets filled, the STATUS goes to FILLED. Also, when it is FILLED, my understanding is that a POSITION is being created (Long or Short) My question is as follows: What endpoint can I use to get a list of the Open Positions. Why do I want this? If a Position is on the SELL side, I would like to execute a BUY to close it. If a Position is on the BUY side, I would like to execute a SELL to close it. Can this be done? Any help, hints or advice would be ~greatly~ appreciated. TIA @michaeldel ETA: I am using this here: https://python-binance.readthedocs.io/en/latest/ For the Orders, I have been following: https://python-binance.readthedocs.io/en/latest/account.html?highlight=orders#orders Can you note what the equivalent would be the equivalent under this (Python) API? I have been using: "get_all_orders" with a focus of the "STATUS" being "FILLED". https://python-binance.readthedocs.io/en/latest/binance.html#binance.client.Client.get_all_orders I was looking for Open Positions (not Orders) If a BTCUSDT SELL Position that has status=FILLED with a origQty =.20, I want to be able to reverse it with a BUY and a Quantity of .20 If a BTCUSDT BUY Position has status=FILLED and a origQty=.30, I want to be able to reverse it with a SELL and a Quantity of .30 Does this make sense? Is there a better way to do it? Am I missing something? Thanks for the input!

Answer:  Also, when it is FILLED, my understanding is that a POSITION is being created (Long or Short)  As far as I know, Binance does not provide semantics for position (in terms of trading). Such abstractions are usually implemented for derivatives (e.g. futures) when it comes to currency markets, since currencies buying-and-selling-to-make-profit is not their only use. On Binance, and most other cryptocurrency exchanges, you are making spot transactions, i.e. giving some amount of a currency to receive some amount of another currency. Plain and simple. You may abstract positions yourself though, but that may involve much more work especially considering heterogeneous chains of transaction (e.g. BTC -> ETH -> USDT -> BTC), partial fills, etc.
Question: Why we are using the MinMaxScaler() and what does it do? scaler = MinMaxScaler() scaler.fit(X_train) X_train = scaler.transform(X_train) X_test = scaler.transform(X_test) model = LogisticRegression() model.fit(X_train, y_train) y_pred = model.predict(X_test) 

Answer: Core of the method A way to normalize the input features/variables is the Min-Max scaler. By doing so, all features will be transformed into the range [0,1] meaning that the minimum and maximum value of a feature/variable is going to be 0 and 1, respectively.  Why to normalize prior to model fitting? The main idea behind normalization/standardization is always the same. Variables that are measured at different scales do not contribute equally to the model fitting & model learned function and might end up creating a bias. Thus, to deal with this potential problem feature-wise normalization such as MinMax Scaling is usually used prior to model fitting. More here: https://towardsdatascience.com/everything-you-need-to-know-about-min-max-normalization-in-python-b79592732b79
Question: I want to create a FastAPI endpoint that just accepts an arbitrary post request body and returns it. If I send {"foo" : "bar"} , I want to get {"foo" : "bar"} back. But I also want to be able to send {"foo1" : "bar1", "foo2" : "bar2"} and get that back. I tried:  from fastapi import FastAPI app = FastAPI() app.post("/") async def handle(request: BaseModel): return request  But that returns an empty dictionary, no matter what I send it. Any ideas?

Answer: The accepted answer works as long as the input is wrapped in a dictionary. That is: started with a { and ends with a }. However, that does not cover all valid JSON inputs. For example, the following valid JSON inputs would fail:  true / false 1.2 null "text" [1,2,3]  In order to have a truly generic JSON input accepted by the endpoint, the following would work: from typing import Any, Dict, List, Union from fastapi import FastAPI app = FastAPI() @app.post("/") async def handle(request: Union[List,Dict,Any]=None): return request  Just using Any did not work for some reason. When I used it, FastApi was expecting the input from the query arguments, not from the request body. The =None makes it accept null and also an empty body. You can leave that part off and then the request body will be required to be not empty/null. If you are using Python3.10 then you can get rid of the Union and write the definition as: async def handle(request: List | Dict | Any = None):
Question: I am writing a Fast API server that accepts requests, checks if users are authorized and then redirects them to another URL if successful. I need to carry over URL parameters, e.g. http://localhost:80/data/?param1=val1&param2=val2 should redirect to http://some.other.api/?param1=val1&param2=val2, thus keeping previously allotted parameters. The parameters are not controlled by me and could change at any moment. How can I achieve this? Code: from fastapi import FastAPI from starlette.responses import RedirectResponse app = FastAPI() @app.get("/data/") async def api_data(): params = '' # I need this value url = f'http://some.other.api/{params}' response = RedirectResponse(url=url) return response 

Answer: In the docs they talk about using the Request directly, which then lead me to this: from fastapi import FastAPI, Request from starlette.responses import RedirectResponse app = FastAPI() @app.get("/data/") async def api_data(request: Request): params = request.query_params url = f'http://some.other.api/?{params}' response = RedirectResponse(url=url) return response
Question: I had to switch my public Github repository to private and cannot access files, not with access tokens that I was able to with the public Github repo.  I can access my private repo's CSV with curl: ''' curl -s https://{token}@raw.githubusercontent.com/username/repo/master/file.csv ''' However, I want to access this information in my python file. When the repo was public I could simply use: ''' url = 'https://raw.githubusercontent.com/username/repo/master/file.csv' df = pd.read_csv(url, error_bad_lines=False) ''' This no longer works now that the repo is private, and I cannot find a work around to download this CSV in python instead of pulling from terminal. If I try: ''' requests.get(https://{token}@raw.githubusercontent.com/username/repo/master/file.csv) ''' I get a 404 response, which is basically the same thing that is happening with the pd.read_csv(). If I click on the raw file I see that a temporary token is created and the URL is: ''' https://raw.githubusercontent.com/username/repo/master/file.csv?token=TEMPTOKEN ''' Is there a way to attach my permanent private access token so that I can always pull this data from github?

Answer: This is what ended up working for me - leaving it here if anyone runs into the same issue. Thanks for the help!  import json, requests, urllib, io user='my_github_username' pao='my_pao' github_session = requests.Session() github_session.auth = (user, pao) # providing raw url to download csv from github csv_url = 'https://raw.githubusercontent.com/user/repo/master/csv_name.csv' download = github_session.get(csv_url).content downloaded_csv = pandas.read_csv(io.StringIO(download.decode('utf-8')), error_bad_lines=False)
Question: I'd like to resample a pandas object using a specific date (or month) as the edge of the first bin. For instance, in the following snippet I'd like my first index value to be 2020-02-29 and I'd be happy specifying start=2 or start="2020-02-29". >>> dates = pd.date_range("2020-01-29", "2021-07-04") >>> s = pd.Series(range(len(dates)), index=dates) >>> s.resample('4M').count() 2020-01-31 3 2020-05-31 121 2020-09-30 122 2021-01-31 123 2021-05-31 120 2021-09-30 34 Freq: 4M, dtype: int64  So far this is the cleanest I can come up with uses pd.cut and groupby: >>> rule = "4M" >>> start = pd.Timestamp("2020-02-29") - pd.tseries.frequencies.to_offset(rule) >>> end = s.index.max() + pd.tseries.frequencies.to_offset(rule) >>> bins = pd.date_range(start, end, freq=rule) >>> gb = s.groupby(pd.cut(s.index, bins)).count() >>> gb.index = gb.index.categories.right >>> gb 2020-02-29 32 2020-06-30 122 2020-10-31 123 2021-02-28 120 2021-06-30 122 2021-10-31 4 dtype: int64 

Answer: My answer feels a little hacky, but uses resample and gives the desired output. Find the date one bin length (e.g. 4 months, or month ends specifically) before the specified date, append it to s, and then resample: rule = '4M' date = '02-29-2020' base_date = pd.to_datetime(date) - pd.tseries.frequencies.to_offset(rule) s.loc[base_date] = np.nan output = s.resample(rule=rule).count() output=output[output.index >= date]  Result: 2020-02-29 32 2020-06-30 122 2020-10-31 123 2021-02-28 120 2021-06-30 122 2021-10-31 4 Freq: 4M, dtype: int64  I added output=output[output.index >= date] b/c otherwise you get an additional empty bin: 2019-10-31 0 2020-02-29 32 2020-06-30 122 2020-10-31 123 2021-02-28 120 2021-06-30 122 2021-10-31 4 Freq: 4M, dtype: int64
Question: I am trying to capture the console log of Firefox using Selenium but I am getting "HTTP method not allowed" error This is how I am doing it currently: from selenium import webdriver from selenium.webdriver.common.desired_capabilities import DesiredCapabilities # enable browser logging d = DesiredCapabilities.FIREFOX d['loggingPrefs'] = {'browser': 'ALL'} driver = webdriver.Firefox(capabilities=d) # load some site driver.get('url') # print messages for entry in driver.get_log('browser'): print entry  The error I am getting is selenium.common.exceptions.WebDriverException: Message: HTTP method not allowed

Answer: get_log is not implemented by Firefox driver. See https://github.com/mozilla/geckodriver/issues/330
Question: Python 3.8 (or CPython 3.8?) added the warning SyntaxWarning: "is" with a literal. Did you mean "=="?  for the code 0 is 0. I understand the warning, and I know the difference between is and ==. However, I also know that CPython caches the object for small integers and shares it in other cases as well. (Out of curiosity, I just checked the code (header) again. Small ints are cached in tstate->interp->small_ints. 0 and 1 are even more special and are stored globally in _PyLong_Zero and _PyLong_One. All new creations of ints are via PyLong_FromLong and that one first checks if it is a small integer and cached.) Given this background, if you know you have an int object, you could say that the check x is 0 should be safe, right? Also, you could derive that 0 is 0 should always be True, right? Or is this an implementation detail of CPython and other interpreters do not follow this? Which interpreter does not follow this? Despite this more generic question (which I'm just curious about), consider this more specific (example) code: def sum1a(*args): y = 0 for x in args: if y is 0: y = x else: y = y + x return y  Vs: def sum1b(*args): y = 0 for x in args: if y == 0: y = x else: y = y + x return y  Vs: def sum1c(*args): y = None for x in args: if y is None: y = x else: y = y + x if y is None: return 0 return y  Vs: def sum2(*args): y = 0 for x in args: y = y + x return y  The reason I would sometimes prefer sum1* over sum2 is that depending on the library, sum1* can really be more efficient. E.g. if the argument is a Numpy/TensorFlow/PyTorch array, you really would save a (potentially costly) operation here. The reason I would prefer sum1a over sum1b is that sum1b would break on certain inputs. E.g. if the input is a Numpy array, this would not work. Of course, you could use sum1c instead of sum1a. However, sum1a is shorter. So this is nicer? If the answer to the original question is that this should always work, and if you agree that sum1a is the best option then, how would you get rid of the warning? Is there a simple workaround? In general, I can see that the warning can be useful. So I would not want to disable it completely. I just want to disable it for this specific statement. Maybe I could wrap it up in a function: def is_(a, b): return a is b  And then just use if is_(y, 0): .... Does this work? Is that a good idea?

Answer: No, it isn't. Case in point the Rust implementation for Python returns False: >>>>> 0 is 0 False  and this is not wrong, though I expect this to change in future versions (it has!). is calls id who's only stipulation is that the id returned is unique and constant for a given object. Whether the source code representation for a number (0 here) maps to a distinct object or not is up for the implementation to define.
Question: So if I run this code in Pytorch: x = torch.ones(2,2, requires_grad=True) x.add_(1)  I will get the error:  RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.  I understand that Pytorch does not allow inplace operations on leaf variables and I also know that there are ways to get around this restrictions. What I don't understand is the philosophy behind this rule. Why is it wrong to change a leaf variable with inplace operations?

Answer: As I understand it, any time you do a non-traditional operation on a tensor that was initialized with requires_grad=True, Pytorch throws an error to make sure it was intentional. For example, you normally would only update a weight tensor using optimizer.step(). For another example, I ran into this issue when trying to update the values in a backprop-able tensor during network initialization. self.weight_layer = nn.Parameter(data=torch.zeros(seq_length), requires_grad=True) self.weight_layer[true_ids == 1] = -1.2  RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.  The problem is that, because requires_grad=True, the network doesn't know that I'm still initializing the values. If this is what you are trying to do, wrapping the update in a torch.no_grad block is one solution: with torch.no_grad() self.weight_layer = nn.Parameter(data=torch.zeros(seq_length), requires_grad=True) self.weight_layer[true_ids == 1] = -1.2  Otherwise, you could just set requires_grad=True after you finish initializing the Tensor: self.weight_layer = nn.Parameter(data=torch.zeros(seq_length)) self.weight_layer[true_ids == 1] = -1.2 self.weight_layer.requires_grad = True
Question: I've been tasked with handling the update from Mypy 0.770 to 0.870 in our FastAPI project, and this has produced an error that I can't quite wrap my head around. My endpoint can return two different models based on some condition, and this was denoted as follows the endpont decorator: @router.get("/", response_model=Union[Model1, Model2])  Mypy 0.870 now complains about this, stating that Argument "response_model" to "get" of "APIRouter" has incompatible type "object"; expected "Optional[Type[Any]]"  Setting it to single types, such as Model1 or even str removes the error. Any however, does not work. Now, looking into the get method, I see that the response_model argument is typed as Type[Any], which I assume must be a pointer. How I can define non-simple return models for my API, and make Mypy happy? edit: I tried to reproduce the problem in a smaller frame, but couldn't. The following code works fine: from typing import Any, Type, Union def test1(var, response_model: Type[Any]): print(f"Accepted Type[Any], {var}") def test2(var, response_model: Union[dict, set]): print(f"Accepted Union, {var}") def main(): test1('test1', response_model=Union[dict, set]) test2('test2', response_model=Union[dict, set]) if __name__ == '__main__': main() 

Answer: This is a compatibility issue introduced in newer versions of mypy. There is an open issue on Github about this topic: https://github.com/tiangolo/fastapi/issues/2279 In the discussion they provide the following workarounds:  Using a different approach to create a type alias: NewModel = TypeVar('NewModel',Model1,Model2)  Creating a new Pydantic Model : class NewModel(BaseModel): __root__: Union[Model1, Model2]
Question: I have a dataframe that looks like this df [output]: date time 2020-02-28 00:30:45 2020-02-28 00:30:45 2020-03-09 00:21:06 2020-03-09 00:21:06 2020-03-09 00:21:06  with df.time.dtype [output]: dtype('<m8[ns]')  I want to extract the minutes in the time variable with the following command df.time.dt.minute  but instead, I have this error  AttributeError: 'TimedeltaProperties' object has no attribute 'minute'  Does someone know how to fix this problem?

Answer: your column 'time' is of dtype timedelta as the error tells you; you could use the total_seconds() method to convert to seconds and divide by 60 to get the minutes. If you want a full-featured datetime column, combine 'date' and 'time'. Then you can use .dt.minute. Ex: import pandas as pd df = pd.DataFrame({'time': pd.to_timedelta(['00:30:45','00:30:45','00:21:06','00:21:06','00:21:06']), 'date': pd.to_datetime(['2020-02-28','2020-02-28','2020-03-09','2020-03-09','2020-03-09'])}) # to get the "total minutes": df['minutes'] = df['time'].dt.total_seconds()/60 df['minutes'] # 0 30.75 # 1 30.75 # 2 21.10 # 3 21.10 # 4 21.10 # Name: minutes, dtype: float64  [pd.Timedelta docs] # to get a column of dtype datetime: df['DateTime'] = df['date'] + df['time'] # now you can do: df['DateTime'].dt.minute # 0 30 # 1 30 # 2 21 # 3 21 # 4 21 # Name: DateTime, dtype: int64
Question: While maintaining a SQLAlchemy data model and utilizing alembic for version control, the following code change I made resulted in an empty revision: some_column = Column(Boolean, nullable=False, default=False)  While previously it was: some_column = Column(Boolean, nullable=False)  So adding a default value produces no changes in alembic, i.e. generates an empty revision. I tried other values offered by SQLAlchemy like false() and expression.false() instead of False, but the result is the same (empty alembic revision). Also tried server_default instead of default. The database in question is PostgreSQL. By empty revision, of course I mean that alembic doesn't recognize any change being made in SQLAlchemy: def upgrade(): # ### commands auto generated by Alembic - please adjust! ### pass # ### end Alembic commands ### def downgrade(): # ### commands auto generated by Alembic - please adjust! ### pass # ### end Alembic commands ###  Appreciate any help in this regard.

Answer: To do this automatically you have to turn on a setting to detect server default changes. In your env.py, for the context.configure calls (online and offline migrations, so in 2 places), add a compare_server_default=True kwarg. It is probably safer to just put in the alter_column yourself as well as definitely use server_default because default is just for python-side setting of the default(which is ok but sounds like not what you want). Quoted from https://alembic.sqlalchemy.org/en/latest/autogenerate.html#what-does-autogenerate-detect-and-what-does-it-not-detect  Autogenerate can optionally detect: ... Change of server default. This will occur if you set the EnvironmentContext.configure.compare_server_default parameter to True, or to a custom callable function. This feature works well for simple cases but cannot always produce accurate results. ...
Question: I have installed coc.nvim and extension coc-python(:CocInstall coc-python)  When I opened file I refused of linting and then get error: [coc.nvim] Jedi error: Traceback (most recent call last): File "completion.py", line 694, in <module> [coc.nvim] Jedi error: Traceback (most recent call last): [coc.nvim] Jedi error: import jedi ModuleNotFoundError: No module named 'jedi'  I tried to reinstall extension and plugin but It doesn't help.

Answer: It's recommended to use https://github.com/fannheyward/coc-pyright if you're using Python 3, or use https://github.com/pappasam/coc-jedi if you're using Jedi.
Question: I'm trying to do a simple Instagram python bot in order to upload images in my Instagram profile. I've already tried the most common libraries (InstagramAPI, instapy, insta-cly). While I was searching I found out that Instagram has changed something making those libraries useless. Is there any library I can use? I know thatI can use Selenium in order to make it go but I'm wondering if there is any shortcut. Trank you!

Answer: try this library: instabot https://pypi.org/project/instabot/ example of code for uploading an image: from instabot import Bot bot = Bot() bot.login(username="instagram_username", password="your_password") file = open('path_to_your_image', 'r') bot.upload_photo(file, caption="your post caption")
Question: first time posting here ! If my question is lacking anything please tell me and I'll fix it ! Facebook recently released DETR, an object detection model using transformers ! The model is implemented with Pytorch and I'm trying to implement the loss function where Hungarian algorithm is involved but with Keras and Tensorflow as a custom loss function for Keras model. In the original implementation from Facebook, it's line 81-82 in https://github.com/facebookresearch/detr/blob/master/models/matcher.py  In order to use numpy and classic python function, I used:  def hungarian_loss(losses): row_ind, col_ind = linear_sum_assignment(losses) idx = [[i, j] for i, j in zip(row_ind, col_ind)] return idx # dist loss is a 5x5 matrix, and idx is 5x2 indexes idx = tf.py_function(func=hungarian_loss, inp=[dist_loss], Tout=tf.int32) min_val = tf.gather_nd(dist_loss, idx) return K.mean(min_val)  But I got : tensorflow.python.framework.errors_impl.InvalidArgumentError: Inner dimensions of output shape must match inner dimensions of updates shape. Output: [5,5] updates: [5]  Is it because I'm trying to use something that wasn't a tf.Tensor as loss ? 

Answer: Does this work for you? See: https://www.tensorflow.org/api_docs/python/tf/numpy_function @tf.function def tf_linear_sum_assignment(cost_matrix): return tf.numpy_function(func=linear_sum_assignment,inp=[cost_matrix],Tout=[tf.int64,tf.int64])
Question: By default, click adds a --help option that outputs a standardised usage text based on the structure of the click commands: Usage: ... Options: ... Commands: ... ...  How to override this behaviour to have a custom help output ? What I am trying to do is to output a custom message using rich library.

Answer: The trick is to create a click.Group class and override format_help method class RichGroup(click.Group): def format_help(self, ctx, formatter): sio = io.StringIO() console = rich.Console(file=sio, force_terminal=True) console.print("Hello, [bold magenta]World[/bold magenta]!", ":vampire:") formatter.write(sio.getvalue()) @click.group(cls=RichGroup) def cli(): pass