eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I've currently got a list of tuples (though I control creation of the list and tuples, so they could be altered in type if needed). Each tuple has a start and end integer and a string with an ID for that range's source. What I'm trying to do is identify all of the overlapping ranges within the tuples. Currently I have a = [(0, 98, '122:R'), (100, 210, '124:R'), (180, 398, '125:R'), (200, 298, '123:R')] highNum = 0 highNumItem = '' for item in a: if item[0] < highNum: print(highNumItem + ' overlaps ' + item[2]) if item[1] > highNum: highNum = item[1] highNumItem = item[2] # 124:R overlaps 125:R # 125:R overlaps 123:R  Which outputs enough information that overlaps should be able to be manually review and fixed. But, it misses identifying some sets of overlaps. I can't help thinking there's a relatively obvious solution I'm just missing or not using the right search terms to find examples of. But ideally I'd like the output to actually be 124:R overlaps 125:R & 123:R 125:R overlaps 123:R  But using my comparison method, I can't see a way to catch the rare instance where an overlap spans more than just 2 adjacent ranges. If anyone could point me to a function or comparison method appropriate to this, I'd greatly appreciate. Also, if it matters, I'm currently stuck with python 2.7, but need to be able to port solution to 3.x when 3rd party applications allow it.

Answer: This should work: import operator def get_overlaps(end, remaining): output = [] for r in remaining: if r[0] < end: # starts before the end output.append(r[2]) continue break return output def get_all_overlaps(lst): # thanks @Elan-R for this simplification for i, (start, end, name) in enumerate(lst): overlaps = get_overlaps(end, lst[i+1:]) if overlaps: print(name, "overlaps", " & ".join(overlaps)) a = [(0, 98, '122:R'), (100, 210, '124:R'), (180, 398, '125:R'), (200, 298, '123:R')] # sort by start time a.sort(key=operator.itemgetter(0)) # thanks to @moonGoose get_all_overlaps(a)  Output: 124:R overlaps 125:R & 123:R 125:R overlaps 123:R  This code iterates over each item in the list, and then checks every subsequent item to see if the start time is less than the end time of the current item. If so, it adds the name to the list of overlaps. If not, it stops checking for the current item, as the start times increase, so there will be no further overlaps. (Tested for Python 3.6, but should work with any version)
Question: Hi i'm trying to destroy a class object if the condition of an if statement(inside a while is met) global variablecheck class createobject: def __init__(self,userinput): self.userinput = input self.method() def method(self): while True: if self.userinput == variablecheck print('the object created using this class is still alive') else: print('user object created using class(createobject) is dead') #what code can i put here to delete the object of this class? 

Answer: Think of it that way: you're asking a class to self-destruct using an inner method, which is kind of like trying to eat your own mouth. Luckily for you, Python features garbage collection, meaning your class will be automatically destroyed once all of its references have gone out of scope. If you need to do something specific when the instance is being destroyed, you can still override __del__() which will kinda act like a destructor. Here's a silly example: class SelfDestruct: def __init__(self): print("Hi! I'm being instanciated!") def __del__(self): print("I'm being automatically destroyed. Goodbye!") def do_stuff(self): print("I'm doing some stuff...")  Now, try instanciating this class in a local scope (such as a function): def make_a_suicidal_class(): my_suicidal_class = SelfDestruct() for i in range(5): my_suicidal_class.do_stuff() return None  Here, the lifespan of the object is bound by the function. Meaning it'll be automatically destroyed once the call is completed. Thus the output should look like: >>> make_suicidal_class() "Hi! I'm being instanciated!" "I'm doing some stuff..." "I'm doing some stuff..." "I'm doing some stuff..." "I'm doing some stuff..." "I'm doing some stuff..." "I'm being automatically destroyed. Goodbye!" >>>  If your class was instanciated in a global scope, then it won't be destroyed until your program ends. Also, it should be noted that manually calling the __del__() destructor does NOT actually destroy the object. Doing this: foo = SelfDestruct() foo.__del__() foo.do_stuff()  Results is this output: "Hi! I'm being instanciated!" "I'm being automatically destroyed. Goodbye!" "I'm doing some stuff..."  ergo, the instance still has a pulse... If you really need to prevent the instance from being referenced again in the current scope, you have to call del foo to do so. Though as previously stated, Python actually reference-counts classes and variables. So if your class object is used elsewere, invoking del foo will not actually release it from memory. Here's an exhaustive explanation in the python docs https://docs.python.org/2.5/ref/customization.html  "del x" doesn't directly call x.del() -- the former decrements the reference count for x by one, and the latter is only called when x's reference count reaches zero.  Long story short: Don't think about it! Let python deal with memory management. The whole point of garbage collection is to stop worrying about the lifespan of your variables!
Question: I am trying to create an API that receives arguments from postman. The body of the api contains two arguments: { "db":"EUR", "env":"test" }  I parsed these two arguments in the code as below: parser = reqparse.RequestParser() parser.add_argument('fab', type=str, required=True, help='Fab name must be provided.') parser.add_argument('env', type=str, required=False, help='Env is an optional parameter.')  Lately I was asked to add a token validation in the code. The token is passed from Authorization-> Type(Bearer Token) -> Token value: eeb867bd2bcca05  But I don't know how can I read the bearer token from postman into Python code. Could anyone let me know how to read the token value that is being passed from Postman's bearer token into my Python code ? Any help is much appreciated.

Answer: The Bearer token is sent in the headers of the request as 'Authorization' header, so you can get it in python flask as follows: headers = flask.request.headers bearer = headers.get('Authorization') # Bearer YourTokenHere token = bearer.split()[1] # YourTokenHere
Question: Traceback (most recent call last): File "Chiyo.py", line 1, in <module> import discord File "/home/ubuntu/.local/lib/python3.6/site-packages/discord/__init__.py", line 25, in <module> from .client import Client File "/home/ubuntu/.local/lib/python3.6/site-packages/discord/client.py", line 34, in <module> import aiohttp File "/home/ubuntu/.local/lib/python3.6/site-packages/aiohttp/__init__.py", line 6, in <module> from .client import BaseConnector as BaseConnector File "/home/ubuntu/.local/lib/python3.6/site-packages/aiohttp/client.py", line 30, in <module> from yarl import URL File "/home/ubuntu/.local/lib/python3.6/site-packages/yarl/__init__.py", line 1, in <module> from ._url import URL, cache_clear, cache_configure, cache_info File "/home/ubuntu/.local/lib/python3.6/site-packages/yarl/_url.py", line 56, in <module> @rewrite_module File "/home/ubuntu/.local/lib/python3.6/site-packages/yarl/_url.py", line 132, in URL _QUERY_PART_QUOTER = _Quoter(safe="?/:@", qs=True, requote=False) File "yarl/_quoting.pyx", line 192, in yarl._quoting._Quoter.__init__ TypeError: __init__() got an unexpected keyword argument 'requote'  I got this error after updated discord.py and I don't know how to fix it :c using python3.6

Answer: Install yarl 1.4.2 pip install -U yarl==1.4.2
Question: I am currently analyzing a set of PDF files. I want to know how many of the PDF files fall in those 3 categories:  Digitally Created PDF: The text is there (copyable) and it is guaranteed to be correct as it was created directly e.g. from Word Image-only PDF: A scanned document Searchable PDF: A scanned document, but an OCR engine was used. The OCR engine put text "below" the image so that you can search / copy the content. As OCR is pretty good, this is correct most of the time. But it is not guaranteed to be correct.  It is easy to identify Image-only PDFs in my domain as every PDF contains text. If I cannot extract any text, it is image only. But how do I know if it is "just" a searchable PDF or if it is a digially created PDF? By the way, it is not as simple as just looking at the producer as I have seen scanned documents where the Producer field said "Microsoft Word". Note: As a human, it is easy. I just zoom in on the text. If I see pixels, it's "just" searchable. Here are 3 example PDF files to test solutions:  Digitally Created PDF Scanned PDF: Well.. not really; I used a script to create images and then put them together as a PDF. But that only means that the quality is very good. It should be very similar to a scan. Searchable PDF  What I tried/thought about  Using the creator/producer: I see "Microsoft Word" in scanned documents. Also this would be tedious. Embedded fonts: You can extract embedded fonts. The idea was that a scanned document would not have embedded fonts but just use the default. The idea was wrong, as one can see with the example. 

Answer: With PyMuPDF you can easily remove all text as is required for @ypnos' suggestion. As an alternative, with PyMuPDF you can also check whether text is hidden in a PDF. In PDF's relevant "mini-language" this is triggered by the command 3 Tr ("text render mode", e.g. see page 402 of https://www.adobe.com/content/dam/acom/en/devnet/acrobat/pdfs/pdf_reference_1-7.pdf). So if all text is under the influence of this command, then none of it will be rendered - allowing the conclusion "this is an OCR'ed page".
Question: Edit: I notice people commenting about how the type hint should not be used with __eq__, and granted, it shouldn't. But that's not the point of my question. My question is why can't the class be used as type hint in the method parameters, but can be used in the method itself?  Python type hinting has proven very useful for me when working with PyCharm. However, I've come across behaviour I find strange, when trying to use a class' own type in its methods. For example: class Foo: def __init__(self, id): self.id = id pass def __eq__(self, other): return self.id == other.id  Here, when typing other., the property id is not offered automatically. I was hoping to solve it by defining __eq__ as follows:  def __eq__(self, other: Foo): return self.id == other.id  However, this gives NameError: name 'Foo' is not defined. But when I use the type within the method, id is offered after writing other.:  def __eq__(self, other): other: Foo return self.id == other.id  My question is, why is it not possible to use the class' own type for type hinting the parameters, while it is possible within the method?

Answer: The name Foo doesn't yet exist, so you need to use 'Foo' instead. (mypy and other type checkers should recognize this as a forward reference.) def __eq__(self, other: 'Foo'): return self.id == other.id  Alternately, you can use from __future__ import annotations  which prevents evaluation of all annotations and simply stores them as strings for later reference. (This will be the default in Python 3.10.) Finally, as also pointed out in the comments, __eq__ should not be hinted this way in the first place. The second argument should be an arbitrary object; you'll return NotImplemented if you don't know how to compare your instance to it. (Who knows, maybe it knows how to compare itself to your instance. If Foo.__eq__(Foo(), Bar()) returns NotImplemented, then Python will try Bar.__eq__(Bar(), Foo()).) from typing import Any def __eq__(self, other: Any) -> bool: if isinstance(other, Foo): return self.id == other.id return NotImplemented  or using duck-typing, def __eq__(self, other: Any) -> bool: # Compare to anything with an `id` attribute try: return self.id == other.id except AttributeError: return NotImplemented  In either case, the Any hint is optional.
Question: I was able to get the response of one API from another but unable to store it somewhere(in a file or something before returning the response) response=RedirectResponse(url="/apiname/") (I want to access a post request with header and body)  I want to store this response content without returning it.  Yes, if I return the function I will get the results but when I print it I don't find results. Also, if I give post request then I get error Entity not found. I read the starlette and fastapi docs but couldn't get the workaround. The callbacks also didn't help.

Answer: I didn't exactly get the way to store response without returning using fastapi/starlette directly. But I found a workaround for completing this task.  For the people trying to implement same thing, Please consider this way.  import requests def test_function(request: Request, path_parameter: path_param): request_example = {"test" : "in"} host = request.client.host data_source_id = path_parameter.id get_test_url= f"http://{host}/test/{id}/" get_inp_url = f"http://{host}/test/{id}/inp" test_get_response = requests.get(get_test_url) inp_post_response = requests.post(get_inp_url , json=request_example) if inp_post_response .status_code == 200: print(json.loads(test_get_response.content.decode('utf-8')))   Please let me know if there are better approaches.
Question: I am having some trouble with the ccf() method in the (Python) statsmodels library. The equivalent operation works fine in R. ccf produces a cross-correlation function between two variables, A and B in my example. I am interested to understand the extent to which A is a leading indicator for B. I am using the following: import pandas as pd import numpy as np import statsmodels.tsa.stattools as smt  I can simulate A and B as follows: np.random.seed(123) test = pd.DataFrame(np.random.randint(0,25,size=(79, 2)), columns=list('AB'))  When I run ccf, I get the following: ccf_output = smt.ccf(test['A'],test['B'], unbiased=False) ccf_output array([ 0.09447372, -0.12810284, 0.15581492, -0.05123683, 0.23403344, 0.0771812 , 0.01434263, 0.00986775, -0.23812752, -0.03996113, -0.14383829, 0.0178347 , 0.23224969, 0.0829421 , 0.14981321, -0.07094772, -0.17713121, 0.15377192, -0.19161986, 0.08006699, -0.01044449, -0.04913098, 0.06682942, -0.02087582, 0.06453489, 0.01995989, -0.08961562, 0.02076603, 0.01085041, -0.01357792, 0.17009109, -0.07586774, -0.0183845 , -0.0327533 , -0.19266634, -0.00433252, -0.00915397, 0.11568826, -0.02069836, -0.03110162, 0.08500599, 0.01171839, -0.04837527, 0.10352341, -0.14512205, -0.00203772, 0.13876788, -0.20846099, 0.30174408, -0.05674962, -0.03824093, 0.04494932, -0.21788683, 0.00113469, 0.07381456, -0.04039815, 0.06661601, -0.04302084, 0.01624429, -0.00399155, -0.0359768 , 0.10264208, -0.09216649, 0.06391548, 0.04904064, -0.05930197, 0.11127125, -0.06346119, -0.08973581, 0.06459495, -0.09600202, 0.02720553, 0.05152299, -0.0220437 , 0.04818264, -0.02235086, -0.05485135, -0.01077366, 0.02566737])  Here is the outcome I am trying to get to (produced in R):  The problem is this: ccf_output is giving me only the correlation values for lag 0 and to the right of Lag 0. Ideally, I would like the full set of lag values (lag -60 to lag 60) so that I can produce something like the above plot. Is there a way to do this?

Answer: The statsmodels ccf function only produces forward lags, i.e. Corr(x_[t+k], y_[t]) for k >= 0. But one way to compute the backwards lags is by reversing the order of the both the input series and the output. backwards = smt.ccf(test['A'][::-1], test['B'][::-1], adjusted=False)[::-1] forwards = smt.ccf(test['A'], test['B'], adjusted=False) ccf_output = np.r_[backwards[:-1], forwards]  Note that both backwards and forwards contained lag 0, so we had to remove that from one of them when combining them. Edit another alternative is to reverse the order of the arguments and the output: backwards = sm.tsa.ccf(test['B'], test['A'], adjusted=False)[::-1]
Question: I was creating a new virtual environment on Ubuntu 20.04: $ virtualenv my_env  But it gave an error: ModuleNotFoundError: No module named 'virtualenv.seed.embed.via_app_data'   Other info: $ virtualenv --version virtualenv 20.0.17 from /usr/lib/python3/dist-packages/virtualenv/__init__.py 

Answer: Try to create the virtual environment using directly venv module python3 -m venv my_env
Question: I want to write in the .env using python code. This is what I tried but it's not working:- os.environ['username'] = 'John' os.environ['email'] = 'abc@gmail.com' 

Answer: os.environ is a Python dictionary containing the environment. In order to change the environment variables in your currently running process, and any children process spawned with fork, you should use os.putenv as follows: import os os.putenv("username", "John") os.putenv("email", "abc@gmail.com")  Do notice, this changes are not permanent, they just affect the process being currently executed. If you want the changes to be permanent, you can write them to a .env file, and read and reset them on startup: with open(".env", "r") as f: for line in f.readlines(): try: key, value = line.split('=') os.putenv(key, value) except ValueError: # syntax error pass  To generate the file you should: with open(".env", "w") as f: f.write("username=John") f.write("email=abc@gmail.com")  If you want to permanently change this environment variables on the whole OS scope, you need an OS specific solution, since each operating system has its way of changing environment variables. This method will set the environment variable globally, which will affect all applications run and not just yours, so be extremely careful on what are you writing to. Unix-like systems like Linux and macOS let you set up these variables in .profile, so you may do something like this (Python 3.5+): from pathlib import Path with open(str(Path.home()) + "/.profile", "a") as f: f.write("export USERNAME=John\nexport EMAIL abc@gmail.com\n")  On Windows on the other hand you should call setx: import subprocess subprocess.call(["setx", "USERNAME", "John"]) subprocess.call(["setx", "EMAIL", "abc@gmail.com"])
Question: I have train data for 50GB. My google drive capacity was 15GB so I upgraded it to 200GB and I uploaded my train data to my google drive I connected to colab, but I can not find my train data in colab session, So I manually uploaded to colab which has 150GB capacity. It says, it will be deleted when my colab connection is off. It is impossible to save train data for colab permanently? And colab is free for 150GB? And I see colab support nvidia P4 that is almost 5000$. can I use it 100% or it is shared to some portion(like 0.1%) to me? (When P4 is assigned to me)

Answer: The way you can do this is to mount your google drive into colab environment. Assume your files are kept under a folder named myfolder in your google drive. This is what I would suggest, do this before you read/write any file: import os from google.colab import drive MOUNTPOINT = '/content/gdrive' DATADIR = os.path.join(MOUNTPOINT, 'My Drive', 'myfolder') drive.mount(MOUNTPOINT)  then, for example, your file bigthing.zip reside under myfolder in your google drive will be available in colab as path=os.path.join(DATADIR, 'bigthing.zip') Similarly, when you save a file to a path like the above, you can find your file in Google Drive under the same directory.
Question: I have a fastapi application and I want to log every request made on it. I'm trying to use loguru and uvicorn for this, but I don't know how to print the headers and request params (if have one) associated with each request. I want something like this: INFO 2020-08-13 13:36:33.494 uvicorn.protocols.http.h11_impl:send - 127.0.0.1:52660 - "GET /url1/url2/ HTTP/1.1" 400 params={"some": value, "some1":value}  Is there a way ? thanks for your help. Here some links: loguru uvicorn fastapi

Answer: A dependency on a router level could be used (thanks to @lsabi in the comments below): import sys import uvicorn from fastapi import FastAPI, Request, APIRouter, Depends from loguru import logger from starlette.routing import Match logger.remove() logger.add(sys.stdout, colorize=True, format="<green>{time:HH:mm:ss}</green> | {level} | <level>{message}</level>") app = FastAPI() router = APIRouter() async def logging_dependency(request: Request): logger.debug(f"{request.method} {request.url}") logger.debug("Params:") for name, value in request.path_params.items(): logger.debug(f"\t{name}: {value}") logger.debug("Headers:") for name, value in request.headers.items(): logger.debug(f"\t{name}: {value}") @router.get("/{param1}/{param2}") async def path_operation(param1: str, param2: str): return {'param1': param1, 'param2': param2} app.include_router(router, dependencies=[Depends(logging_dependency)]) if __name__ == "__main__": uvicorn.run("app:app", host="localhost", port=8001)  More sophisticated approach is using a middleware for logging every request and doing matching manually: import sys import uvicorn from fastapi import FastAPI, Request from loguru import logger from starlette.routing import Match logger.remove() logger.add(sys.stdout, colorize=True, format="<green>{time:HH:mm:ss}</green> | {level} | <level>{message}</level>") app = FastAPI() @app.middleware("http") async def log_middle(request: Request, call_next): logger.debug(f"{request.method} {request.url}") routes = request.app.router.routes logger.debug("Params:") for route in routes: match, scope = route.matches(request) if match == Match.FULL: for name, value in scope["path_params"].items(): logger.debug(f"\t{name}: {value}") logger.debug("Headers:") for name, value in request.headers.items(): logger.debug(f"\t{name}: {value}") response = await call_next(request) return response @app.get("/{param1}/{param2}") async def path_operation(param1: str, param2: str): return {'param1': param1, 'param2': param2} if __name__ == "__main__": uvicorn.run("app:app", host="localhost", port=8001)  curl http://localhost:8001/admin/home Output: 16:06:43 | DEBUG | GET http://localhost:8001/admin/home 16:06:43 | DEBUG | Params: 16:06:43 | DEBUG | param1: admin 16:06:43 | DEBUG | param2: home 16:06:43 | DEBUG | Headers: 16:06:43 | DEBUG | host: localhost:8001 16:06:43 | DEBUG | user-agent: curl/7.64.0 16:06:43 | DEBUG | accept: */*
Question: I'm trying to create a metaclass but when I assign it to another class I receive the error:  TypeError: __init_subclass__() takes no keyword arguments  But I don't implement any __init_subclass__. Why is this function being called? class Meta(type): def __new__(cls, name, bases, dct): return super().__new__(cls, name, bases, dct) class MyClass(meta=Meta): pass 

Answer: Change meta to metaclass. Any keyword arguments passed to the signature of your class are passed to its parent's __init_subclass__ method. Since you entered meta instead of metaclass this meta kwarg is passed to its parent's (object) __init_subclass__ method: >>> object.__init_subclass__(meta=5) TypeError: __init_subclass__() takes no keyword arguments  A similar error would be raised if you actually implemented a __init_subclass__ but made a typo: class Parent: def __init_subclass__(cls, handler=None): super().__init_subclass__() cls.handler = handler class CorrectChild(Parent, handler=5): pass class TypoChild(Parent, typo=5): # TypeError: __init_subclass__() got an unexpected keyword argument 'typo' pass