eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Goal Make request to http://example.com/page? using requests.get() Problem The question mark ("?") is automatically stripped from the request if it is the last character in the URL (eg. http://example.com/page?1 and http://example.com/page?! work, http://example.com/page? does not) Sample code import requests endpoint = "http://example.com/page?" r = requests.get(endpoint) print(r.url) # -> "http://example.com/page" assert r.url == endpoint # Raises AssertionError  Question Without modifying the library, is it possible to reach the intended endpoint? Both intended solutions (if such exist) and workarounds are welcome. Thanks!

Answer: This is not possible with the requests library. URLs passed into requests are parsed by urllib3.util.url.parse_url() into separate parts: scheme auth host port path query fragment  The logic for getting the query part of a URL assumes that the querystring starts after ?, but since there is nothing after the question mark, it gives a blank query. The URL is then reconstructed as a string when you print r.url. That is why the URL does not have the trailing question mark. I found that the behavior you are looking for is possible with urllib.request, though. Here's an example: import urllib.request, urllib.error try: response = urllib.request.urlopen("http://example.com/page?") print(response.url) # -> http://example.com/page? except urllib.error.HTTPError as e: print(e.url) # -> http://example.com/page? print(e.code) # -> 404  I have surrounded the request in a try/except because if the page you are trying to get gives a 404, urllib will raise an error, where requests will simply put up with it.
Question: I have a list of words, noun-verb phrases and I want to:  Search dependency patterns, words, in a corpus of text identify the paragraph that matches appears in extract the paragraph highlight the matched words in the paragraph create a snip/jpeg of the paragraph with matched words highlighted save the image in an excel.  The MWE below pertains to highlighting the matched words and displaying them using displacy. I have mentioned the rest of my task just to provide the context. The output isn't coloring the custom entities with custom colors. import spacy from spacy.matcher import PhraseMatcher from spacy.tokens import Span good = ['bacon', 'chicken', 'lamb','hot dog'] bad = [ 'apple', 'carrot'] nlp = spacy.load('en_core_web_sm') patterns1 = [nlp(good) for good in good] patterns2 = [nlp(bad) for bad in bad] matcher = PhraseMatcher(nlp.vocab) matcher.add('good', None, *patterns1) matcher.add('bad', None, *patterns2) doc = nlp("I like bacon and chicken but unfortunately I only had an apple and a carrot in the fridge") matches = matcher(doc) for match_id, start, end in matches: span = Span(doc, start, end, label=match_id) doc.ents = list(doc.ents) + [span] # add span to doc.ents print([(ent.text, ent.label_) for ent in doc.ents])  The code above produces this output: [('bacon', 'good'), ('chicken', 'good'), ('apple', 'bad'), ('carrot', 'bad')]  But when I try to custom color the entities, it doesn't seem to be working. from spacy import displacy colors = {'good': "#85C1E9", "bad": "#ff6961"} options = {"ents": ['good', 'bad'], "colors": colors} displacy.serve(doc, style='ent',options=options)  This is the output I get: 

Answer: I just copy/pasted your code and it works fine here. I'm using spaCy v3.1.1.  What does the HTML output source look like?  I was able to reproduce your issue on spaCy 2.3.5. I was able to fix it by making the labels upper-case (GOOD and BAD). I can't find a bug about this but since the models normally only use uppercase labels I guess this is an issue with older versions.
Question: I am trying to create a Pydantic basemodel with a key including a '$' sign. It looks like this: class someModel(BaseModel): $something:Optional[str] = None  Then I get SyntaxError: invalid syntax. But I need to keep the key name $something to use in other parts. Is there a way to allow the dollar sign in this case?

Answer: You can use Field(alias=...) to use a different (valid) variable name. from pydantic import BaseModel, Field class SomeModel(BaseModel): something: Optional[str] = Field(alias="$something", default=None)  I've also added a default value of None since you had that in your code. Here's a working example (EDIT: Updated to show how to get alias for your variable name if you need to use it later): import logging from typing import Optional from fastapi import FastAPI, Request from pydantic import BaseModel, Field logging.basicConfig(level=logging.INFO, format="%(levelname)-9s %(asctime)s - %(name)s - %(message)s") LOGGER = logging.getLogger(__name__) app = FastAPI() class SomeModel(BaseModel): something: Optional[str] = Field(alias="$something", default=None) @app.post("/") async def root(request: Request, parsed_body: SomeModel): # A dict of all the model fields and their properties LOGGER.info(f"SomeModel.__fields__: {SomeModel.__fields__}") # To get the alias of the variable name something_alias = SomeModel.__fields__["something"].alias LOGGER.info(f"something_alias: {something_alias}") # Edit: prefer to use "parsed_body_by_alias" than raw_body. Leaving here to show the difference. raw_body: bytes = await request.body() LOGGER.info(f"raw_body: {raw_body}") # Edit: This is better as you get validated / parsed values, including defaults if applicable. parsed_body_by_alias = parsed_body.dict(by_alias=True) LOGGER.info(f"parsed_body_by_alias: {parsed_body_by_alias}") # If you just want "something" instead of "$something" LOGGER.info(f"parsed_body: {parsed_body}") LOGGER.info(f"parsed_body.something: {parsed_body.something}") return 1 if __name__ == "__main__": import uvicorn uvicorn.run(app, host="127.0.0.1", port=8080)  If you run the code then send a POST with body: {"$something": "bar"} you'll see something like: INFO xxx - __main__ - SomeModel.__fields__: {'something': ModelField(name='something', type=Optional[str], required=False, default=None, alias='$something')} INFO xxx - __main__ - something_alias: $something INFO xxx - __main__ - raw_body: b'{"$something": "bar"}' INFO xxx - __main__ - parsed_body_by_alias: {'$something': 'bar'} INFO xxx - __main__ - parsed_body: something='bar' INFO xxx - __main__ - parsed_body.something: bar INFO: 127.0.0.1:xxxxx - "POST / HTTP/1.1" 200 OK
Question: I know, it's a mouthful. I am using pytest-asyncio, which gives you an async event loop for running async code in your tests. I want to use factory-boy with an async ORM. The only problem is that factory-boy does not support async whatsoever. I want to override the _create function on a factory (which is a synchronous function), however, I need that function to run asynchronous code. It doesn't matter if it blocks, it's just tests. Something like this:  class AsyncModelFactory(factory.alchemy.SQLAlchemyFactory): class Meta: sqlalchemy_session = async_session abstract = True @classmethod def _create(cls, model_class, *args, **kwargs): # run_until_complete only works if there isn't already an event loop running return asyncio.get_event_loop().run_until_complete( cls._async_create(model_class, *args, **kwargs) ) @classmethod async def _async_create(cls, model_class, *args, **kwargs): obj = model_class(*args, **kwargs) session = self._meta.sqlalchemy_session session.add(obj) await session.commit() # needs to be awaited return obj  The above actually works, except in the case where an async event loop is already running. Ironically, that factory works from synchronous tests, but not from asynchronous tests. Is there a way to await _async_create from _create in an asynchronous context? All of factory-boy with factories creating sub-factories, etc, is synchronous, so this can only work if _create remains a synchronous call

Answer: You can't await anything in an ordinary function, of course. But there are ways that ordinary functions and async code can interact. An ordinary function can create tasks and return awaitables, for example. Look at this little program: import asyncio import time async def main(): print("Start", time.asctime()) awaitable1 = f1() awaitable2 = f2() await asyncio.gather(awaitable1, awaitable2) print("Finish", time.asctime()) def f1(): return asyncio.create_task(f3()) def f2(): return asyncio.sleep(2.0) async def f3(): await asyncio.sleep(1.0) print("Task f3", time.asctime()) asyncio.run(main())  Result: Start Thu Aug 12 18:55:20 2021 Task f3 Thu Aug 12 18:55:21 2021 Finish Thu Aug 12 18:55:22 2021  Here are two ordinary functions, one of which creates a task and one which creates a simple awaitable. Those objects are returned up the stack to an async function, which then awaits on them. You asked specifically about the situation where the event loop is already running. When that's the case, except for code that's invoked by callback functions, everything that runs in your program is part of some task or other. Somewhere in the call stack will be an async function. If you figure out how to pass awaitable objects up to that level, I think you can solve your problem.
Question: I've been trying to see whether one can create an abstract class property by mixing the three decorators (in Python 3.9.6, if that matters), and I noticed some strange behaviour. Consider the following code: from abc import ABC, abstractmethod class Foo(ABC): @classmethod @property @abstractmethod def x(cls): print(cls) return None class Bar(Foo): @classmethod @property def x(cls): print("this is executed") return super().x  This outputs this is executed <class '__main__.Bar'>  This means that somehow, Bar.x ends up being called. PyCharm warns me that Property 'self' cannot be deleted. If I reverse the order of @classmethod and @property, Bar.x is not called, but I still get the same warning, and also another one: This decorator will not receive a callable it may expect; the built-in decorator returns a special object (this also appears whenever I put @property above @classmethod). Removing any of the three decorators (with the appropriate changes: adding () when removing @property or changing cls to self when removing @classmethod) also prevents Bar.x from being called. I suppose all of this means that it's probably just a bad idea to directly mix those decorators (as indicated by discussion about class properties in other threads here). Neverthless, I am curious: what is happening here? Why is Bar.x called?

Answer: This looks like a bug in the logic that checks for inherited abstract methods. An object in a class dict is considered abstract if retrieving its __isabstractmethod__ attribute produces True. When Bar subclasses Foo, Python needs to determine whether Bar overrides the abstract Foo.x, and if so, whether the override is itself abstract. It should do this by searching the MRO for an 'x' entry in a class dict, so it can examine __isabstractmethod__ on descriptors directly without invoking the descriptor protocol, but instead, it performs a simple Bar.x attribute access. The Bar.x attribute access invokes the class property. It also returns None instead of the abstract property, and None isn't abstract, so Python gets confused about whether Bar.x is abstract. Python ends up still thinking Bar.x is abstract due to a different check, but if you change the example a bit: >>> from abc import ABC, abstractmethod >>> >>> class Foo(ABC): ... @classmethod ... @property ... @abstractmethod ... def x(cls): ... print(cls) ... return None ... >>> class Bar(Foo): pass ... <class '__main__.Bar'> >>> Bar() <__main__.Bar object at 0x7f46eca8ab80>  Python ends up thinking Bar is a concrete class, even though the changed example doesn't override x at all.
Question: In Perl there is a way to find out what context a function is called in - whether it's a list or scalar context - or even finer granularity. See wantarray and the want module. Is there something similar in Python?

Answer: No, Python doesn't have the scalar/array distinction that Perl does. Values are simply bound to names with no regard for the type of the value. The closest analogy I can think of would be something like x += y  This is desugared as x.__iadd__(y), so the method __iadd__ could examine the type of its argument y if it wanted to. For example, even though list.__iadd__ doesn't do this, it could theoretically have been defined to allow things like x = [] x += 9 # x == [9]; append a non-iterable argument x += [11, 12] # x == [9, 11, 12]; extend with an iterable argument
Question: I'm using data that follows a class inheritance pattern... I'm having trouble getting pydantic to deserialize it correctly for some use cases. Given the code below, it appears that the validators are not called when using the parse_* methods. The type for "fluffy" and "tiger" are Animal, however when deserializing the "bob" the Person, his pet is the correct Dog type. Is there another way to approach this problem? Using pydantic is not a requirement, but the ability to deserialize nested complex types (including List and Dict of objects) is. # modified from the following examples # - https://github.com/samuelcolvin/pydantic/issues/2177#issuecomment-739578307 # - https://github.com/samuelcolvin/pydantic/issues/619#issuecomment-713508861 from pydantic import BaseModel TIGER = """{ "type": "cat", "name": "Tiger the Cat", "color": "tabby" }""" FLUFFY = """{ "type": "dog", "name": "Fluffy the Dog", "color": "brown", "breed": "rottweiler" }""" ALICE = """{ "name": "Alice the Person" }""" BOB = f"""{{ "name": "Bob the Person", "pet": {FLUFFY} }}""" class Animal(BaseModel): type: str name: str color: str = None _subtypes_ = dict() def __init_subclass__(cls, type=None): cls._subtypes_[type or cls.__name__.lower()] = cls @classmethod def __get_validators__(cls): yield cls._convert_to_real_type_ @classmethod def _convert_to_real_type_(cls, data): data_type = data.get("type") if data_type is None: raise ValueError("Missing 'type' in Animal") sub = cls._subtypes_.get(data_type) if sub is None: raise TypeError(f"Unsupport sub-type: {data_type}") return sub(**data) class Cat(Animal, type="cat"): hairless: bool = False class Dog(Animal, type="dog"): breed: str class Person(BaseModel): name: str pet: Animal = None tiger = Animal.parse_raw(TIGER) print(f"tiger [{tiger.name}] => {type(tiger)} [{tiger.type}]") fluffy = Animal.parse_raw(FLUFFY) print(f"fluffy [{fluffy.name}] => {type(fluffy)} [{fluffy.type}]") bob = Person.parse_raw(BOB) pet = bob.pet print(f"bob [{bob.name}] => {type(bob)}") print(f"pet [{pet.name}] => {type(pet)}")  Output: tiger [Tiger the Cat] => <class '__main__.Animal'> fluffy [Fluffy the Dog] => <class '__main__.Animal'> bob [Bob the Person] => <class '__main__.Person'> pet [Fluffy the Dog] => <class '__main__.Dog'> 

Answer: Answer from pydantic project discussion: You can simply add class Animal(BaseModel): ... @classmethod def parse_obj(cls, obj): return cls._convert_to_real_type_(obj)
Question: I have code like below  threads = [] t = threading.Thread(target=Subscribe('username', "password", "topic", "host",port).start) t1 = threading.Thread(target=Subscribe('username2', "password2", "topic2", "host",port).start) threads.append(t) threads.append(t1) for thread in threads: thread.start() for thread in threads: thread.join()  My topics send data in every 5 minutes. When I use the code above,it does not work properly, sometimes it sends data sometimes it does not. But when I use one topic without thread , it works fine and data is coming in every 5 minutes perfectly. How can I solve this problem? I want to subscribe two topics simultaneously. I am using Paho for MQTT My subscribe class is class Subscribe: def __init__(self, username, passowrd, topic, host, port): self.username = username self.password = passowrd self.topic = topic self.host = host self.port = port def start(self): self.mqttc = mqtt.Client(client_id="Python") self.connected = False self.mqtt_message = "" self.mqttc.username_pw_set(self.username, self.password) self.mqttc.on_connect = self.on_connect self.mqttc.on_message = self.on_message self.mqttc.connect(self.host, self.port) self.mqttc.loop_forever() def on_message(self, client, userdata, message): """ Fetch data when data coming to Broker """ topic = message.topic m = json.loads(message.payload.decode("utf-8")) print(m) def on_connect(self, client, userdata, flags, rc): if rc == 0: print("Connected to broker", self.topic) self.mqttc.subscribe(self.topic) self.connected = True else: print("could not connect", self.topic) 

Answer: I had to give two different client_ids for the two instances of Client and it solved the issue.
Question: In a python module, in the docstring of the module I have the following: :meth:`my_method`  and I have the following class in the current module: class GameP: ... def my_method(self): return f"{self._name} {self.selected}"  Sphinx does not create a link for that, whilst in the Sphinx documentation we have:  Normally, names in these roles are searched first without any further qualification, then with the current module name prepended, then with the current module and class name (if any) prepended. If you prefix the name with a dot, this order is reversed. For example, in the documentation of Python’s codecs127 module, :py:func:open always refers to the built-in function, while :py:func:.open refers to codecs.open().  Why does the bolded section not hold for me? :meth: role does not make a link for me.

Answer: The documentation is not crystal clear IMHO, but it works if you use :meth:`.my_method`  (with a dot). The dot makes Sphinx look for a match for my_method anywhere. The dot is not needed if the cross-reference is in the docstring of the GameP class. But in this case the cross-reference is in the module docstring, and on the module level there is no "current class name".
Question: I wanted to give VSC a try for developing some Python programs, where I only used PyCharm before. One of the most helpful features for me in Pycharm was the PyDev Console, where I can quickly try small snippets of code (think 3-10 lines), and adjust it to work the way I want it to. I see VSC has a console, but it's much more like the regular IDLE console, where it's kind of hard to write these snippets of code (fixing something 2 lines prior for example is pretty much impossible). I've been searching for an extension that'll give me a PyCharm-like console experience in VSC, but have been unable to find one. Is is out there? Or is there another way to get to the same result (like setting up a custom console based on the same PyDev console)?

Answer: Have you tried Jupyter Notebook and Interactive? There are provided by the Jupyter Extension which is bound with Python Extension. Open the Command Palette (Ctrl+Shift+P), with the command of: Jupyter: Create New Blank Notebook Jupyter: Create Interactive Window  You can refer to the official docs for more details.
Question: I currently have Pandas with version 1.1.5 and I am trying to install the newest version of Pandas by using the following command $ pip install pandas==1.3.1  However, I get an error as follows: ERROR: Could not find a version that satisfies the requirement pandas==1.3.1 (from versions: 0.1, 0.2b0, 0.2b1, 0.2, 0.3.0b0, 0.3.0b2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0rc1, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.22.0, 0.23.0, 0.23.1, 0.23.2, 0.23.3, 0.23.4, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.25.2, 0.25.3, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5) ERROR: No matching distribution found for pandas==1.3.1  I am using Python 3.6.8 How can I solve this? Thanks!

Answer: Pandas dropped support for Python 3.6 in version 1.2.0 (December, 2020). Either upgrade to Python 3.7+ or make do with Pandas 1.1.5, which is the last version to support Python 3.6.
Question: I don't fully understand why the keyword match can be used as a variable or function name, unlike other keywords if, while, etc.? >>> match "abc": ... case "abc": ... print('Hello!') ... Hello! >>> from re import match >>> match('A', 'A Something A') <re.Match object; span=(0, 1), match='A'> >>> match = '????' >>> match '????' >>> case = 'something' >>> case 'something' 

Answer: Per PEP 622, match and case are being added as "soft keywords", so they will remain valid identifiers:  This PEP is fully backwards compatible: the match and case keywords are proposed to be (and stay!) soft keywords, so their use as variable, function, class, module or attribute names is not impeded at all.
Question: I am having TensorFlow version 2.0.0 in my anaconda environment. I want to upgrade it to an upgraded version. How can I do it?

Answer: conda update <package name>  or conda install <package name>  Note: A second install equals an Override... Specific to your case: conda install -c conda-forge -n <environment> tensorflow==<wanted version>
Question: Lets say I have the following code: try: import bar except ImportError: bar = None @bar.SomeProvidedDecorator def foo(): pass  where bar is an optional dependency. The code above will fail, if bar isn't imported. Is there a recommended way of dealing with this problem? I came up with: try: import bar except ImportError: bar = None def foo(): pass if bar is not None: foo = bar.SomeProvidedDecorator(foo)  but I'm wondering if there are better ways of handling this (i.e. is there a way to keep the decorator syntax) ?

Answer: Provide an identity decorator in case of bar unavailability: try: import bar except ImportError: class bar: SomeProvidedDecorator = lambda f: f
Question: I have the following code that should get the last known action of users and insert it's timestamp in the user object: async def get_last_activity(users, db): user_ids = [user.id for user in users] event_query = select(func.max(EventModel.datetime), EventModel.user_id).\ where(EventModel.user_id.in_(user_ids)).\ group_by(EventModel.user_id) events = (await db.execute(event_query)).scalars() for event in events: print(event) for user in users: if event.user_id == user.id: user.last_activity = event.datetime return users @router.get( '/manageable', response_model=Users, ) async def get_manageable_users( db: Session = Depends(get_db) ): """ Returns a list of users that the current user can manage """ # @TODO: get users based on authorization helper. users = (await db.execute(select(UserModel))).scalars().all() users = await get_last_activity(users, db) return [{ 'id': user.id, 'name': user.name, 'email': user.email, 'last_activity': user.last_activity } for user in users]  This results in the following error: File "./app/api/v1/users/read.py", line 26, in get_last_activity if event.user_id == user.id: AttributeError: 'datetime.datetime' object has no attribute 'user_id  Even though the column is being selected in the generated query: api001_1 | 2021-08-11 09:19:16,514 INFO sqlalchemy.engine.Engine SELECT max(events.datetime) AS max_1, events.user_id api001_1 | FROM events api001_1 | WHERE events.user_id IN (%s, %s) GROUP BY events.user_id api001_1 | 2021-08-11 09:19:16,515 INFO sqlalchemy.engine.Engine [generated in 0.00323s] (1, 2) api001_1 | 2016-10-07 22:00:29 <---------- print(event)  Does anyone know why the user_id column is not showing up in the Event objects? Update: It's scalars() that is causing the issue. Without it this is the output of print(event): (datetime.datetime(2016, 10, 7, 22, 0, 29), 1) After running scalars only the datetime is picked up So scalars seems to completely disregard the last number which is the user_id. Why?

Answer: I was under the impression that scalars() was supposed to map the result to an object. Upon reading the docs more closely, it can only do this with one column. This (optional) parameter defaults to 0 (index of column) and hence only the datetime is picked up. I'm not using .scalars() anymore and instead have to do row[0], row[1]