eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm trying to Type Hint the function bar, but I got the Too few arguments error when I run mypy. from typing import Callable, Optional def foo(arg: int = 123) -> float: return arg+0.1 def bar(foo: Callable[[int], float], arg: Optional[int] = None) -> float: if arg: return foo(arg) return foo() print(bar(foo)) print(bar(foo, 90))  I have also tried:  Callable[[], float] (got Too many arguments error) Callable[[Optional[int]], float] (got another error)  So, how should I do the Type Hinting of the bar function?

Answer: Define this: class Foo(Protocol): def __call__(self, x: int = ..., /) -> float: ...  then type hint foo as Foo instead of Callable[[int], float]. Callback protocols allow you to:  define flexible callback types that are hard (or even impossible) to express using the Callable[...] syntax  and optional arguments are one of those impossible things to express with a normal Callable. The / at the end of __call__'s signature makes x a positional-only parameter, which allows any passed function to bar to have a parameter name that is not x (your specific example of foo calls it arg instead). If you removed /, then not only would the types have to line up as expected, but the names would have to line up too because you would be implying that Foo could be called with a keyword argument. Because bar doesn't call foo with keyword arguments, opting into that behavior by omitting the / imposes inflexibility on the user of bar (and would make your current example still fail because "arg" != "x").
Question: Since a few days ago, zappa deploy fails with the following error (zappa version 0.50.0): Traceback (most recent call last): File "/root/repo/venv/lib/python3.6/site-packages/zappa/cli.py", line 2785, in handle sys.exit(cli.handle()) File "/root/repo/venv/lib/python3.6/site-packages/zappa/cli.py", line 510, in handle self.dispatch_command(self.command, stage) File "/root/repo/venv/lib/python3.6/site-packages/zappa/cli.py", line 557, in dispatch_command self.update(self.vargs['zip'], self.vargs['no_upload']) File "/root/repo/venv/lib/python3.6/site-packages/zappa/cli.py", line 993, in update endpoint_configuration=self.endpoint_configuration File "/root/repo/venv/lib/python3.6/site-packages/zappa/core.py", line 2106, in create_stack_template self.cf_template.add_description('Automatically generated with Zappa') AttributeError: 'Template' object has no attribute 'add_description' 

Answer: Since version 3.0.0, the package troposphere removed the deprecated Template methods (see the changelog).  Breaking changes: * Python 3.6+ (Python 2.x and earlier Python 3.x support is now deprecated due to Python EOL) * Remove previously deprecated Template methods.  The above issue can be fixed by adding troposphere<3 in the requirements file.
Question: There are many similar questions on SO, but this specific error message did not turn up in any of my searches: AttributeError: 'WhereNode' object has no attribute 'select_format'  This was raised when trying to annotate() a Django queryset with the (boolean) result of a comparison, such as the gt lookup in the following simplified example: Score.objects.annotate(positive=Q(value__gt=0))  The model looks like this: class Score(models.Model): value = models.FloatField() ...  How to fix this?

Answer: This case can be fixed using the ExpressionWrapper() Score.objects.annotate( positive=ExpressionWrapper(Q(value__gt=0), output_field=BooleanField()))  From the docs:  ExpressionWrapper is necessary when using arithmetic on F() expressions with different types ...  The same apparently holds for Q objects, although I could not find any explicit reference in the docs.
Question: I'm trying to embed a video on my local drive in Jupyter Notebook. The file name is "openaigym.video.6.7524.video000000.mp4" and it is in a folder "gym-results". Using the following code produces nothing whatsoever: from IPython.display import Video Video("./gym-results/openaigym.video.4.7524.video000000.mp4",embed =True)  If I try to use HTML directly (which I got from here), it produces an unplayable video: from base64 import b64encode def video(fname, mimetype): from IPython.display import HTML video_encoded = b64encode(open(fname, "rb").read()) video_tag = '<video controls alt="test" src="data:video/{0};base64,{1}">'.format(mimetype, video_encoded) return HTML(data=video_tag) path= f"./gym-results/openaigym.video.6.7524.video000000.mp4" video(path, "mp4")  That is, it produces the following:  Which cannot be started. How do I solve this?

Answer: First method: it worked with me!  Second method: You could try this as well: from ipywidgets import Video Video.from_file("./play_video_test.mp4", width=320, height=320)   Third method: you should change the type of the cell from code to Markdown <video controls src="./play_video_test.mp4">animation</video>  If all solutions do not work with you, I recommend you to update your jupyter notebook by conda update jupyter or pip install -U jupyter and go through each solution one more again.
Question: Is np.uint9 possible? Why use it? red_lower = np.array([136, 87, 111], np.uint9) 

Answer: https://numpy.org/doc/stable/reference/arrays.scalars.html#unsigned-integer-types class numpy.ubyte[source] Unsigned integer type, compatible with C unsigned char. Character code 'B' Alias on this platform (Linux x86_64) numpy.uint8: 8-bit unsigned integer (0 to 255).  Most often this is used for arrays representing images, with the 3 color channels having small integer values (0 to 255).
Question: I have a DataFrame like this: df = pd.DataFrame({ 'val_a': [3, 3, 3, 2, 2, 2, 1, 1, 1], 'val_b': [3, np.nan, 2, 2, 2, 0, 1, np.nan, 0], 'quantity': [1, 4, 2, 8, 5, 7, 1, 4, 2] })  It looks like this: | | val_a | val_b | quantity | |---:|--------:|--------:|-----------:| | 0 | 3 | 3 | 1 | | 1 | 3 | nan | 4 | | 2 | 3 | 2 | 2 | | 3 | 2 | 2 | 8 | | 4 | 2 | 2 | 5 | | 5 | 2 | 0 | 7 | | 6 | 1 | 1 | 1 | | 7 | 1 | nan | 4 | | 8 | 1 | 0 | 2 |  It is ordered by val_a. I'd like to take a cumulative sum for the total quantity for each val_a. So: df.groupby('val_a', sort=False).sum().cumsum().drop(columns='val_b')  which gives | val_a | quantity | |--------:|-----------:| | 3 | 7 | | 2 | 27 | | 1 | 34 |  However, here's the tricky part. I'd like to exclude rows such that the value of val_b is greater than the key val_a. I'll clarify with an example:  when calculating the total for when val_a is 3, none of the rows have val_b greater than val_a. So the cumulative total for when val_a is 3 is 7; when calculating the total for when val_a is 2, then row 0 has val_b greater than 2. That row has quantity 1. So, excluding that row, the cumulative total for when val_a is 2 is 27 - 1, i.e. 26; when calculating the total for when val_a is 1, then rows 0, 2, 3, 4 have val_b greater than 1,. That row has quantity 1. So, excluding that row, the cumulative total for when val_a is 1 is 34 - 1 - 2 - 8 - 5, i.e. 18;  Here's the desired output: | val_a | quantity | |--------:|-----------:| | 3 | 7 | | 2 | 26 | | 1 | 18 | 

Answer: With the help of NumPy: # sum without conditions raw_sum = df.groupby("val_a", sort=False).quantity.sum().cumsum() # comparing each `val_b` against each unique `val_a` via `gt.outer` sub_mask = np.greater.outer(df.val_b.to_numpy(), df.val_a.unique()) # selecting values to subtract from `quantity` and summing per `val_a` to_sub = (sub_mask * df.quantity.to_numpy()[:, np.newaxis]).sum(axis=0) # subtracting from the raw sum result = raw_sum - to_sub  to get >>> result.reset_index() val_a quantity 0 3 7 1 2 26 2 1 18
Question: Long story short, I want to cythonize my python code and build .so files to hide it from the customer. Take this simple function: def one_positional_argument(a): print(a)  and my setup.py which builds the .so file from setuptools import setup, find_packages from Cython.Build import cythonize setup( name='tmp', version='1.0.0', packages=find_packages(), nthreads=3, ext_modules=cythonize( ["a.py"], compiler_directives={'language_level': 3}, build_dir="output", ), )  When I import the .so file and try to execute my function here is what happens: one_positional_argument(1) # this prints 1 and works fine one_positional_argument(a=1) # throws TypeError: one_positional_argument() takes no keyword arguments  There are multiple workarounds to this but I would like to know if I am doing anything wrong Additional info: If I have a function with 2 positional arguments, or one positional and one with default value, everything works fine. The issue is present only with 1 positional argument.

Answer: You need the always_allow_keywords compiler directive (https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html#compiler-directives). Not allowing them by default is a deliberate compatibility/speed trade-off. However, in the forthcoming Cython v3 (the alpha version is usable now...) that will change: https://github.com/cython/cython/issues/3090
Question: this is probably a very simple question but I haven't been able to find a good answer to it yet. I've found answers for converting QByteArrays into python strings, but not vice versa. Is there a pyqt5 method that allows me to simply convert a python string into a QByteArray (so that it can be sent over a serial connection using QSerialPort.write()). I reckon it's likely that there is a nice built-in feature in pyqt5 to do this without manually extracting the bytes from the string and building a QByteArray from them?

Answer: You have to convert the string to bytes: >>> from PyQt5.QtCore import QByteArray >>> s = "hello world" >>> ba = QByteArray(s.encode()) >>> print(ba) b'hello world'
Question: Is it possible to count how many documents a collection in Firestore has, using python? I just found this code  functions.firestore.document('collection/{documentUid}') .onWrite((change, context) => { if (!change.before.exists) { // New document Created : add one to count db.doc(docRef).update({numberOfDocs: FieldValue.increment(1)}); } else if (change.before.exists && change.after.exists) { // Updating existing document : Do nothing } else if (!change.after.exists) { // Deleting document : subtract one from count db.doc(docRef).update({numberOfDocs: FieldValue.increment(-1)}); } return; });  How I can do this with python?

Answer: Firestore now has no limited support for aggregation queries. Previous answer below, as this still applies for cases that are not supported.  Outside of the built-in count operation, if you want to determine the number of documents, you have two main options:  Read all documents, and then count them in the client. Keep the document count in the database itself, and then update it with every add/delete operation.  While the firsts option is simpler, it is less scalable as you'll end up with clients reading all documents just to determine the count. That's why you'll find most questions/articles about counting documents focusing on the second approach. For more on this, see:  How should I handle aggregated values in Firestore The Firestore documentation on aggregation queries. How fast is counting documents in Cloud Firestore? The Firestore documentation on distributed counters, which you'll need to consider if your count changes more frequently than about once per second.
Question: I am looking for some solution or maybe some script that can help me to replace($ref) or expand its definitions within the YML file with schema validation. (For detail please find below example) **Example: Input with $ref **  /pets/{petId}: get: summary: Info for a specific pet operationId: showPetById tags: - pets parameters: - name: petId in: path required: true description: The id of the pet to retrieve schema: type: string responses: '200': description: Expected response to a valid request content: application/json: schema: $ref: "#/components/schemas/Pet" default: description: unexpected error content: application/json: schema: $ref: "#/components/schemas/Error" components: schemas: Pet: type: object required: - id - name properties: id: type: integer format: int64 name: type: string tag: type: string Pets: type: array items: $ref: "#/components/schemas/Pet" Error: type: object required: - code - message properties: code: type: integer format: int32 message: type: string  Output: All $ref replace or expanded to its definition (with schema validation)  /pets/{petId}: get: summary: Info for a specific pet operationId: showPetById tags: - pets parameters: - name: petId in: path required: true description: The id of the pet to retrieve schema: type: string responses: '200': description: Expected response to a valid request content: application/json: schema: type: object required: - id - name properties: id: type: integer format: int64 name: type: string tag: type: string default: description: unexpected error content: application/json: schema: type: object required: - code - message properties: code: type: integer format: int32 message: type: string components: schemas: Pet: type: object required: - id - name properties: id: type: integer format: int64 name: type: string tag: type: string Pets: type: array items: type: object required: - id - name properties: id: type: integer format: int64 name: type: string tag: type: string Error: type: object required: - code - message properties: code: type: integer format: int32 message: type: string  Can you please suggest?

Answer: Here are some tools that can claim to be able dereference internal $refs in addition to external ones. Be aware of potential issues with circular $refs. CLI:  https://github.com/APIDevTools/swagger-cli swagger-cli bundle --dereference <file>   Redocly OpenAPI CLI redocly bundle --dereferenced --output <outputName> --ext <ext> [entrypoints...]    Libraries:  Java: Swagger Parser with the resolvefully option Node.js: oas-resolver with the resolveInternal option
Question: I have two tables class User(models.Model): id = fields.BigIntField(pk=True) name = CharField(max_length=100) tags: fields.ManyToManyRelation["Tag"] = fields.ManyToManyField( "models.Tag", related_name="users", through="user_tags" ) class Tag(models.Model): id = fields.BigIntField(pk=True) name = fields.CharField(max_length=100) value = fields.CharField(max_length=100) users: fields.ManyToManyRelation[User]  Let's assume this dummy data #users bob = await User.create(name="bob") alice = await User.create(name="alice") #tags foo = await Tag.create(name="t1", value="foo") bar = await Tag.create(name="t2", value="bar") #m2m await bob.tags.add(foo) await alice.tags.add(foo, bar)  Now I want to count users who have both tags foo and bar, which is alice in this case, so it should be 1. The below query will give me a single level of filtering, but how do I specify that the user should have both foo and bar in their tags ? u = await User.filter(tags__name="t1", tags__value="foo").count() 

Answer: Tortoise-ORM provides Q objects for complicated queries with logical operators like |(or) and &(and). Your query could be made like this:  u = await User.filter(Q(tags__name="t1") & (Q(tags__value="foo") | Q(tags__value="bar"))).count()
Question: I have a problem where I have a set of warehouses with a given production capacity that send some product to a list of customers at a given cost. I'm trying to minimize the total cost of sending the products so that each customer's demand is satisfied. That part is sorted. Now I need to add a new objective (or constraint) where I try to satisfy all the clients demand at a minimum cost but also using the minimum number of warehouses possible. Say start with 5 warehouses, if the problem is impossible then try 6, 7, 8 etc. until a solution is found were I satisfy all the demand using the minimum number of warehouses possible. How could I go about this using or-tool constraint programming module? Is it even possible? I've had a good look at the documentation but couldn't find any constraint or function that seemed to cater for this idea.

Answer: Solve with the first objective, constraint the objective with the solution, hint and solve with the new objective. from ortools.sat.python import cp_model model = cp_model.CpModel() solver = cp_model.CpSolver() x = model.NewIntVar(0, 10, "x") y = model.NewIntVar(0, 10, "y") # Maximize x model.Maximize(x) solver.Solve(model) print("x", solver.Value(x)) print("y", solver.Value(y)) print() # Hint (speed up solving) model.AddHint(x, solver.Value(x)) model.AddHint(y, solver.Value(y)) # Maximize y (and constraint prev objective) model.Add(x == round(solver.ObjectiveValue())) # use <= or >= if not optimal model.Maximize(y) solver.Solve(model) print("x", solver.Value(x)) print("y", solver.Value(y))  Reference (github issue)
Question: I have an odd behavior for one of my endpoints in my Flask application which accepts boolean query parameters. No matter what I pass to it, such as asfsdfd or true or false, it is considered true. Only by leaving it empty does it become false. full_info = request.args.get("fullInfo", default=False, type=bool) if full_info: # do stuff  It seems to be that either any input is considered to be true. Is there any way to make this work with the Flask intended way of defining the type, or do I need to accept a string and compare it?

Answer: The type parameter of request.args.get is not for specifying the value's type, but for specifying a callable:   type – A callable that is used to cast the value in the MultiDict. If a ValueError is raised by this callable the default value is returned.   It accepts a callable (ex. a function), applies that callable to the query parameter value, and returns the result. So the code request.args.get("fullInfo", default=False, type=bool)  calls bool(value) where value is the query parameter value. In Flask, the query parameter values are always stored as strings. And calling bool() on a non-empty string will always be True: In [10]: bool('true') Out[10]: True In [11]: bool('false') Out[11]: True In [12]: bool('any non-empty will be true') Out[12]: True In [13]: bool('') Out[13]: False  Instead of bool, you can pass a function that explicitly checks if the string is equal to the literal string true (or whichever value your API rules consider as true-ish): def is_it_true(value): return value.lower() == 'true' @app.route("/test") def test(): full_info = request.args.get('fullInfo', default=False, type=is_it_true) return jsonify({'full_info': full_info})  $ curl -XGET http://localhost:5000/test?fullInfo=false {"full_info":false} $ curl -XGET http://localhost:5000/test?fullInfo=adasdasd {"full_info":false} $ curl -XGET http://localhost:5000/test?fullInfo=11431423 {"full_info":false} $ curl -XGET http://localhost:5000/test?fullInfo= {"full_info":false} $ curl -XGET http://localhost:5000/test?fullInfo=true {"full_info":true} $ curl -XGET http://localhost:5000/test?fullInfo=TRUE {"full_info":true} $ curl -XGET http://localhost:5000/test {"full_info":false}
Question: I would like a diverging colormap that has another colour than white (preferably black) as it center color. Neither matplotlib or cmocean seems to have such a colormap. Is my best option to create an own colormap, or are there existing ones?

Answer: @JohanC put an obvious answer in their comment and I didn't see it because of the accepted answer which requires a less-known package: Seaborn supports arbitrary diverging palettes with black center hue_neg, hue_pos = 250, 15 cmap = sns.diverging_palette(hue_neg, hue_pos, center="dark", as_cmap=True)
Question: How can I generate an authorization code/client secret in python for apple sign in and device check?

Answer:  First of all we need to generate a app specific p8 file (pem formatted private key) do the following for this:   go to your apple developer portal, under certificates identifiers & profiles apple => keys click the + sign and create a key with the services you want to use it for then download the p8 file (be cautious not to lose it you cannot download it again) also copy the key id you will need it later   in python install pyjwt and do the following:   create a payload dict:   data = { "iss": "team_id", # team id of your developer account this can be found in your apple developer portal => identifier of your app => "App ID prefix" "iat": timestamp_now, # creation timestamp in seconds "exp": timestamp_exp, # expiration timestamp in seconds (max 20 mins) see "aud": "https://appleid.apple.com", "sub": client_id # your bundle }   open and read the private key (you downloaded in step 1) into a variable  with open("filename.p8", "r") as f: private_key = f.read()   generate your signed jwt token:  token = jwt.encode(payload=data, key=private_key, algorithm="ES256", headers={ "kid":key_id # the key id is the id u saved in step 1 }).decode()   jwt.encode returns bytes if you want it as a string you need to decode it as I did  the complete code will look like this import jwt def generate_token(): with open("filename.p8", "r") as f: private_key = f.read() team_id = "teamid" client_id = "bundle.id" key_id = "keyid" validity_minutes = 20 timestamp_now = int(utils.time_stamp_seconds()) timestamp_exp = timestamp_now + (60 * validity_minutes) cls.last_token_expiration = timestamp_exp data = { "iss": team_id, "iat": timestamp_now, "exp": timestamp_exp, "aud": "https://appleid.apple.com", "sub": client_id } token = jwt.encode(payload=data, key=private_key, algorithm="ES256", headers={"kid": key_id}).decode()
Question: I'm trying to understand the walrus assignment operator. Classic while loop breaks when condition is reassigned to False within the loop. x = True while x: print('hello') x = False  Why doesn't this work using the walrus operator? It ignores the reassignment of x producing an infinite loop. while x := True: print('hello') x = False 

Answer: You seem to be under the impression that that assignment happens once before the loop is entered, but that isn't the case. The reassignment happens before the condition is checked, and that happens on every iteration. x := True will always be true, regardless of any other code, which means the condition will always evaluate to true.
Question: I have a list of strings: lst = ["orange", "yellow", "green"]  and I want to randomly repeat the values of strings for a given length. This is my code: import itertools lst = ["orange", "yellow", "green"] list(itertools.chain.from_iterable(itertools.repeat(x, 2) for x in lst))  This implementation repeats but not randomly and also it repeats equally, whereas it should be random as well with the given length.

Answer: You can use a list comprehension: import random lst = ["orange", "yellow", "green"] [lst[random.randrange(len(lst))] for i in range(100)]  Explanation:  random.randrange(n) returns an integer in the range 0 to n-1 included. the list comprehension repeatedly adds a random element from lst 100 times. change 100 to whatever number of elements you wish to obtain.