eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I want to initialize a dataclass dictionary with type hint(key should be string, the value should be another dataclass's instances) called usr_orders that does NOT require user init upon initializing instances. Based on my understanding Optional is the union(.., None), so I went with: @dataclass class User: usr_orders: Optional[Dict[str, BookOrder]] = Dict  But when I try to modify it using key-value I got the following error: TypeError: '_GenericAlias' object does not support item assignment  def update_usr_orders(self, book_order:BookOrder): self.usr_orders[book_order.order_id] = book_order # book_order: BookOrder is a dataclass instance  This error seems to be caused by the optional Dict definition. But I don't know how to solve this. Does anyone know why I cannot update the usr_orders? Thanks!

Answer: You'll need to import a field and use a default_factory: from dataclasses import dataclass, field from typing import Optional, Dict @dataclass class BookOrder(): id: str @dataclass class User: usr_orders: Optional[Dict[str, BookOrder]] = field(default_factory=dict) def update_usr_orders(self, book_order:BookOrder): self.usr_orders[book_order.id] = book_order a = BookOrder('1') u = User() u.update_usr_orders(a) print(u.usr_orders)  Part of the issue is that you were using Dict as a default field for the data class. Dict is a type hint; I think you were meaning to use dict() or {} - however, this would not work. Try it and see for yourself! You'll end up with this error: ValueError: mutable default <class 'dict'> for field usr_orders is not allowed: use default_factory. This is a dataclass thing where it prevents you from using 1 dictionary for multiple instances. Further reading
Question: I am using Python 3 and I know all about hex, int, chr, ord, '\uxxxx' escape and '\U00xxxxxx' escape and Unicode has 1114111 codepoints... How can I check if a Unicode codepoint is valid? That is, it is unambiguously mapped to a authoritatively defined character. For example, codepoint 720 is valid; it is 0x2d0 in hex, and U+02D0 points to ː: In [135]: hex(720) Out[135]: '0x2d0' In [136]: '\u02d0' Out[136]: 'ː'  And 888 is not valid: In [137]: hex(888) Out[137]: '0x378' In [138]: '\u0378' Out[138]: '\u0378'  And 127744 is valid: In [139]: chr(127744) Out[139]: '🌀'  And 0xe0000 is invalid: In [140]: '\U000e0000' Out[140]: '\U000e0000'  I have come up with a rather hacky solution: if a codepoint is valid, trying to convert it to a character will either result in the decoded character or the '\xhh' escape sequence, else it will return the undecoded escape sequence exactly same as original, I can check the return value of chr and check if it starts with '\u' or '\U'... Now is the hacky part, chr doesn't decode invalid codepoints but it doesn't raise exceptions either, and the escape sequences will have length of 1 since they are treated as a single character, I have to repr the return value and check the results... I have used this method to identify all invalid codepoints: In [130]: invalid = [] In [131]: for i in range(1114112): ...: if any(f'{chr(i)!r}'.startswith(j) for j in ("'\\U", "'\\u")): ...: invalid.append(i) In [132]: from pathlib import Path In [133]: invalid = [(hex(i).removeprefix('0x'), i) for i in invalid] In [134]: Path('D:/invalid_unicode.txt').write_text(',\n'.join(map(repr, invalid))) Out[134]: 18574537  Can anyone offer a better solution?

Answer: I believe the most straight-forward approach is to use unicodedata.category(). The examples from the OP are unassigned codepoints, which have a category of Cn ("Other, not assigned"). >>> import unicodedata as ud >>> ud.category('\u02d0') 'Lm' >>> ud.category('\u0378') # unassigned 'Cn' >>> ud.category(chr(127744)) 'So' >>> ud.category('\U000e0000') # unassigned 'Cn'  It also works for the control characters in the ASCII range: >>> ud.category('\x00') 'Cc'  Further categories for invalid codepoints (according to comments) are Cs ("Other, surrogate") and Co ("Other, private use"): >>> ud.category('\ud800') # lower surrogate 'Cs' >>> ud.category('\uf8ff') # private use 'Co'  So a function for codepoint validity (as per the OP's definition) could look like this: def is_valid(char): return ud.category(char) not in ('Cn', 'Cs', 'Co')  Important caveat: Python's unicodedata module embeds a certain version of Unicode, so the information is potentially out of date. For example, in my installation of Python 3.8, the Unicode version is 12.1.0, so it doesn't know about codepoints assigned in later versions of Unicode: >>> ud.unidata_version '12.1.0' >>> ud.category('\U0001fae0') # melting face emoji added in Unicode v14 'Cn'  If you need a more recent version of Unicode than the one of your Python version, you probably need to fetch an appropriate table directly from Unicode.
Question: I am currently trying to find the value of a CI/CD variable in a VM. I tried to output it but I find out the variable’s value is masked in job logs. This is the code I used in my .gitlab-ci.yml. image: python:3 stages: - deploy deploy: stage: deploy script: - echo "List all CI/CD variables" - export  The line in question is ... declare -x Secret_variable ="[MASKED]" ...  Is there a way for me to get the find value without modifying the checkbox in the Variables section of Gitlab?

Answer: You can reveal the values of CI/CD variables in the settings page for the project (or group, if it is a group variable) by clicking the "reveal values" button.  You must have maintainer permission or higher to do this. Alternatively, you can expose secrets in the job log if you transform the value such that it won't be masked in the job log. This is a bad idea because you'd be exposing sensitive values in plaintext, but it can be done nevertheless. Unlike other CI platforms, GitLab only masks the exact value. For example, you can print the values base64 encoded, print the characters in reverse order, mangle them in any way, etc. and they will not be masked in the job log. It's also good to know this can happen accidentally, for example, if you use secrets in curl as basic auth requests with verbosity enabled, since the verbose logs will show the parameters base64 encoded. curl -vvv --user "${USERNAME}" --password "${SECRET_PASSWORD}" will expose your CI/CD variables in the job log, for example. You can, of course, do this intentionally as well... expose_secrets: script: - echo $SUPER_SECRET | base64  GitLab's security model works around trusted developers. You should obviously not do this unless you have legitimate reason to do this. Doing this can probably get you in trouble a lot of places, unless it's your job to pentest systems, for example. When secrets are exposed in the job log like this, even in base64 form, they should be treated as compromised and be rotated immediately.
Question: Let's say that I have a list of points from a torunament points = [0, 12, 9]  And I want to have a ranking of the players, so the expected outputs would be [1, 2, 0]  Because the index 1 is first in the ranking, followed by index 2 and then index 0. My idea was to use a for to iterate through all the numbers, getting the biggest value, find the index of the biggest value and then assign the value in the ranking, but it seems unnecessary long and complicated. Any tips?

Answer: Use sorted: points = [0, 12, 9] res = sorted(range(len(points)), key=lambda x: points[x], reverse=True) print(res)  Output [1, 2, 0]  The idea is to sort the indexes of the list (range(3)) according to their value on the list, hence the key=lambda x: points[x]. The reverse True is because you want a descending ranking.
Question: I'm trying to execute a Lambda function but I get the following error: { "errorMessage": "Bad handler 'AlertMetricSender': not enough values to unpack (expected 2, got 1)", "errorType": "Runtime.MalformedHandlerName", "stackTrace": [] }  My Lambda handler is specified in AlertMetricSender.py: from modules.ZabbixSender import ZabbixSender def lambda_handler(event, context): sender = ZabbixSender("10.10.10.10", 10051) sender.add("Zabbix server", "lambda.test", 5.65) sender.send() 

Answer: This is normally caused by an incorrect value specified for the "Handler" setting for the Lambda function. It is a reference to the method in your function code that processes events i.e. the entry point.  The value of the handler argument is comprised of the below, separated by a dot:  The name of the file in which the Lambda handler function is located The name of the Python handler function.  Make sure you have not missed the filename. In this case, it looks like the handler should be set to AlertMetricSender.lambda_handler.
Question: Consider this json file named h.json I want to convert this into a python dataclass. { "acc1":{ "email":"acc1@example.com", "password":"acc1", "name":"ACC1", "salary":1 }, "acc2":{ "email":"acc2@example.com", "password":"acc2", "name":"ACC2", "salary":2 } }  I could use an alternative constructor for getting each account, for example: import json from dataclasses import dataclass @dataclass class Account(object): email:str password:str name:str salary:int @classmethod def from_json(cls, json_key): file = json.load(open("h.json")) return cls(**file[json_key])  but this is limited to what arguments (email, name, etc.) were defined in the dataclass. What if I were to modify the json to include another thing, say age? The script would end up returning a TypeError, specifically TypeError: __init__() got an unexpected keyword argument 'age'. Is there a way to dynamically adjust the class attributes based on the keys of the dict (json object), so that I don't have to add attributes each time I add a new key to the json?

Answer: This way you lose some dataclass features.  Such as determining whether it is optional or not Such as auto-completion feature  However, you are more familiar with your project and decide accordingly There must be many methods, but this is one of them: @dataclass class Account(object): email: str password: str name: str salary: int @classmethod def from_json(cls, json_key): file = json.load(open("1.txt")) keys = [f.name for f in fields(cls)] # or: keys = cls.__dataclass_fields__.keys() json_data = file[json_key] normal_json_data = {key: json_data[key] for key in json_data if key in keys} anormal_json_data = {key: json_data[key] for key in json_data if key not in keys} tmp = cls(**normal_json_data) for anormal_key in anormal_json_data: setattr(tmp,anormal_key,anormal_json_data[anormal_key]) return tmp test = Account.from_json("acc1") print(test.age)
Question: Suppose I have two lists list_1 and list_2 list_1 = [1, 5, 10] list_2 = [3, 4, 15] I want to get a list of tuples containing elements from both list_1 and list_2 such that the difference between the numbers in a tuple is under a some constant c. E.g. suppose c is 2 then the tuples I would have would be: [(1, 3), (5, 3), (5, 4)] Of course one can iterate over both lists and check that the difference between 2 elements is less than c, but that has a complexity of n^2 and I would rather reduce that complexity.

Answer: Here is an implementation of the idea of Marat from the comments: import bisect def close_pairs(list1,list2,c): #assumes that list2 is sorted for x in list1: i = bisect.bisect_left(list2,x-c) j = bisect.bisect_right(list2,x+c) yield from ((x,y) for y in list2[i:j]) list_1 = [1, 5, 10] list_2 = [3, 4, 15] print(list(close_pairs(list_1,list_2,2))) #prints [(1, 3), (5, 3), (5, 4)]  To demonstrate the potential improvement of this strategy over what might be thought of as the "naive" approach, let's timeit. import timeit setup_naive = ''' import numpy list_a = numpy.random.randint(0, 2500, 500).tolist() list_b = numpy.random.randint(0, 2500, 500).tolist() c = 2 def close_pairs(list_a, list_b, c): yield from ((x,y) for x in list_a for y in list_b if abs(x-y) <= c) ''' setup_john_coleman = ''' import bisect import numpy list_a = numpy.random.randint(0, 2500, 500).tolist() list_b = numpy.random.randint(0, 2500, 500).tolist() c = 2 def close_pairs(list_a, list_b, c): list_a = sorted(list_a) list_b = sorted(list_b) for x in list_a: i = bisect.bisect_left(list_b,x-c) j = bisect.bisect_right(list_b,x+c) yield from ((x,y) for y in list_b[i:j]) ''' print(f"john_coleman: {timeit.timeit('list(close_pairs(list_a, list_b, c))', setup=setup_john_coleman, number=1000):.2f}") print(f"naive: {timeit.timeit('list(close_pairs(list_a, list_b, c))', setup=setup_naive, number=1000):.2f}")  On a handy laptop that gives result like: john_coleman: 0.50 naive: 18.35
Question: I have a list of generators in a function alternate_all(*args) that alternates between each generator in the list to print their first item, second item, ..., etc. until all generators are exhausted. My code works until a generator is exhausted and once the StopIteration occurs, it stops printing, when I want it to continue with the rest of the generators and ignore the exhausted one: def alternate_all(*args): iter_list = [] for iterable in args: iter_list.append(iter(iterable)) try: while True: for iterable in iter_list: val = next(iter_list[0]) iter_list.append(iter_list.pop(0)) yield val except StopIteration: pass if __name__ == '__main__': for i in alternate_all('abcde','fg','hijk'): print(i,end='')  My output is: afhbgic  When it should be: afhbgicjdke  How can I get this to ignore the exhausted generator? I would prefer not to use itertools and keep this same structure.

Answer: This works. I tried to stay close to how your original code works (though I did replace your first loop with a list comprehension for simplicity). def alternate_all(*args): iter_list = [iter(arg) for arg in args] while iter_list: i = iter_list.pop(0) try: val = next(i) except StopIteration: pass else: yield val iter_list.append(i)  The main problem with your code was that your try/except was outside of the loop, meaning the first exhausted iterator would exit from the loop. Instead, you want to catch StopIteration inside the loop so you can keep going, and the loop should keep going while iter_list still has any iterators in it.
Question: In python, I can write something like this: some_list = [(1, 2, 3), (3, 2, 1)] for i, *args in some_list: print(args)  I will get the next output: [2, 3] [2, 1]  When we use *args as function arguments, it is unpacked into a tuple. Why do we receive a list in this situation?

Answer: It is just a design decision. Making it a tuple was debated in the PEP 3132, but rejected on usability grounds:  Make the starred target a tuple instead of a list. This would be consistent with a function's *args, but make further processing of the result harder.  Simlarly, making it of the same type as the iterable on the rhs of the assignment, was rejected:  Try to give the starred target the same type as the source iterable, for example, b in a, *b = 'hello' would be assigned the string 'ello'. This may seem nice, but is impossible to get right consistently with all iterables.  The very example of yours is listed in the same PEP under specification. Some reasoning is found in the mailing list of that debate.  When dealing with an iterator, you don't know the length in advance, so the only way to get a tuple would be to produce a list first and then create a tuple from it.
Question: I am trying to consume data from a callback API that sends the POST request in this format: [ { "key1": "asd", "key2": "123" } ]  However my API currently only works when it is sent like this: { "key1": "asd", "key2": "123" }  serializers.py: class RawIncomingDataSerializer(serializers.ModelSerializer): class Meta: model = RawIncomingData fields = '__all__'  views.py: class RawIncomingDataViewSet(viewsets.ModelViewSet): queryset = RawIncomingData.objects.all() serializer_class = RawIncomingDataSerializer  There will only ever be one object in the post data, so I am looking for a simple work around without having to rewrite my serializer to interpret multiple objects in one post request.

Answer: In that case you can override create and explicitly specify many=True in the get_serializer call: class RawIncomingDataViewSet(viewsets.ModelViewSet): ... def create(self, request, *args, **kwargs): serializer = self.get_serializer(data=request.data, many=True) serializer.is_valid(raise_exception=True) self.perform_create(serializer) headers = self.get_success_headers(serializer.data) return Response(serializer.data, status=status.HTTP_201_CREATED, headers=headers)
Question: I need to write a Python script that's compatible with both Python 2 and Python 3, can be called directly from a shell (meaning it has a shebang), and can run on systems that have either Python version installed, but possibly not both. Normally I can write a shebang for Python 2 like so: #!/usr/bin/env python  and for Python 3: #!/usr/bin/env python3  But both of these will fail if their corresponding Python version is not installed, since as far as I'm aware, systems that have Python 3 but not Python 2 do not alias or symlink python to python3. (Do correct me if I'm wrong.) So is there a way to write a Python script with a shebang that will execute the script using either Python 2 or Python 3 so long as one of them is installed? This is mainly intended to solve the removal of Python 2 from upcoming releases of macOS, (Note: I originally wrote that under the mistaken understanding that Python 2 would be replaced with Python 3 in macOS, but in reality Apple is removing Python completely.) but can apply to Linux as well.

Answer: Realistically I would just specify python3 and make that a prerequisite. However, it's technically possible to trick a Python file into being its own shell script wrapper: #!/bin/sh ''':' for name in python3 python2 python do type "$name" > /dev/null 2>&1 && exec "$name" "$0" "$@" done echo >&2 "Please install python" exit 1 ':''' print("Hello from Python")
Question: I am trying to extract a list of followers from a particular user using Tweepy. However, I ran into an error saying that AttributeError: 'API' object has no attribute 'followers_ids'. This code, however, runs smoothly on another machine but not on mine. I tried to reinstall Tweepy but nothing changed. Please help T.T  import os import sys import json import time import math from tweepy import Cursor import tweepy from tweepy import OAuthHandler import datetime auth = OAuthHandler(consumer_key, consumer_secret) auth.set_access_token(access_token, access_token_secret) api = tweepy.API(auth, wait_on_rate_limit=True) MAX_FRIENDS = 10000 def paginate(items, n): """Generate n-sized chunks from items""" for i in range(0, len(items), n): yield items[i:i+n] def get_followers(screen_name): # get followers for a given user fname = "users/{}/followers.json".format(screen_name) max_pages = math.ceil(MAX_FRIENDS / 5000) with open(fname, 'w') as f: for followers in Cursor(api.followers_ids, screen_name=screen_name).pages(max_pages): for chunk in paginate(followers, 100): users = api.lookup_users(user_ids=chunk) for user in users: f.write(json.dumps([user.id, user.screen_name, user.location, str(user.created_at)], sort_keys=True)+"\n") if len(followers) == 5000: print("More results available. Sleeping for 60 seconds to avoid rate limit") time.sleep(60) print("task completed for " + screen_name) 

Answer: Tweepy v4.0.0 renamed API.followers_ids to API.get_follower_ids. You're likely using an older version of Tweepy on the other machine.
Question: Based on the poetry docs:  Likewise if you have command line tools such as pytest or black you can run them using poetry run pytest  The suggested way to use black is: poetry run black myscript.py  However, I do not notice any difference in behaviour if I just use black myscript.py  What is the difference between these two methods?

Answer: It allows you to run black (or whichever command comes after run) installed in your virtual environment without needing to activate your virtual environment first. The relevant note is in the poetry run docs (emphasis mine):  The run command executes the given command inside the project’s virtualenv.  Let's say you have this poetry-demo project with a main.py, and you installed black: poetry-demo$ ls README.rst main.py poetry.lock poetry_demo pyproject.toml tests poetry-demo$ poetry add black The following packages are already present in the pyproject.toml and will be skipped: • black ...  If you don't activate your virtual environment first (i.e. poetry shell) and if you have no black installed anywhere else on your system, simply doing black file.py would fail: poetry-demo$ which black poetry-demo$ black main.py -bash: black: command not found  But, with poetry run, even without activating your virtual environment, you can run black: poetry-demo$ poetry run black main.py All done! ✨ 🍰 ✨ 1 file left unchanged.  The source of your confusion is probably because you already have your virtual environment activated, so then there really is no difference: poetry-demo$ poetry shell Spawning shell within /path/to/virtualenvs/poetry-demo-hCA44HQ0-py3.8 poetry-demo$ . /path/to/virtualenvs/poetry-demo-hCA44HQ0-py3.8/bin/activate (poetry-demo-hCA44HQ0-py3.8) poetry-demo$ black main.py All done! ✨ 🍰 ✨ 1 file left unchanged. (poetry-demo-hCA44HQ0-py3.8) poetry-demo$ poetry run black main.py All done! ✨ 🍰 ✨ 1 file left unchanged.
Question: I need to translate the following code from PyQt5 (It works there) to PyQt6: self.setWindowFlags(Qt.FramelessWindowHint)  This is the error: AttributeError: type object 'Qt' has no attribute 'FramelessWindowHint'  I've already tried this: self.setWindowFlags(Qt.WindowFlags.FramelessWindowHint)  It says: AttributeError: type object 'Qt' has no attribute 'WindowFlags' 

Answer: That flag now lives here: QtCore.Qt.WindowType.FramelessWindowHint
Question: On a small Flask webserver running on a RaspberryPi with about 10-20 clients, we periodically get this error: Error on request: Traceback (most recent call last): File "/home/pi/3D_printer_control/env/lib/python3.7/site-packages/werkzeug/serving.py", line 270, in run_wsgi execute(self.server.app) File "/home/pi/3D_printer_control/env/lib/python3.7/site-packages/werkzeug/serving.py", line 258, in execute application_iter = app(environ, start_response) File "/home/pi/3D_printer_control/env/lib/python3.7/site-packages/flask/app.py", line 2309, in __call__ return self.wsgi_app(environ, start_response) File "/home/pi/3D_printer_control/env/lib/python3.7/site-packages/flask_socketio/__init__.py", line 43, in __call__ start_response) File "/home/pi/3D_printer_control/env/lib/python3.7/site-packages/engineio/middleware.py", line 47, in __call__ return self.engineio_app.handle_request(environ, start_response) File "/home/pi/3D_printer_control/env/lib/python3.7/site-packages/socketio/server.py", line 360, in handle_request return self.eio.handle_request(environ, start_response) File "/home/pi/3D_printer_control/env/lib/python3.7/site-packages/engineio/server.py", line 291, in handle_request socket = self._get_socket(sid) File "/home/pi/3D_printer_control/env/lib/python3.7/site-packages/engineio/server.py", line 427, in _get_socket raise KeyError('Session is disconnected') KeyError: 'Session is disconnected'  The error is generated automatically from inside python-socketio. What does this error really mean and how can I prevent or suppress it?

Answer: As far as I can tell, this usually means the server can't keep up with supplying data to all of the clients. Some possible mitigation techniques include disconnecting inactive clients, reducing the amount of data sent where possible, sending live data in larger chunks, or upgrading the server. If you need a lot of data throughput, there may be also be a better option than socketIO. I have been able to reproduce it by setting a really high ping rate and low timeout in the socketIO constructor: from flask_socketio import SocketIO socketio = SocketIO(engineio_logger=True, ping_timeout=5, ping_interval=5)  This means the server has to do a lot of messaging to all of the clients and they don't have long to respond. I then open around 10 clients and I start to see the KeyError. Further debugging of our server found a process that was posting lots of live data which ran fine with only a few clients but starts to issue the occasional KeyError once I get up to about a dozen.