eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Referencing this link: fastapi-supporting-multiple-authentication-dependencies I think this is the closest to what I need, but somehow I can’t get either of the dependency to work because fastapi is enforcing both dependencies before it grants access to endpoint. Snippet for custom depedency: def basic_logged_user(credentials: Annotated[HTTPBasicCredentials, Depends(security)]): current_username_bytes = credentials.username.encode("utf8") correct_username_bytes = settings.SESSION_LOGIN_USER.encode("utf8") is_correct_username = secrets.compare_digest( current_username_bytes, correct_username_bytes ) current_password_bytes = credentials.password.encode("utf8") correct_password_bytes = settings.SESSION_LOGIN_PASS.encode("utf8") is_correct_password = secrets.compare_digest( current_password_bytes, correct_password_bytes ) if not (is_correct_username and is_correct_password): raise HTTPException( status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid Credentials", headers={"WWW-Authenticate": "Basic"}, ) return credentials.username def jwt_logged_user(token: str = Depends(utils.OAuth2_scheme), db: Session = Depends(db_session)): credential_exception = HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Incorrect username or password", headers={"WWW-Authenticate": "Bearer"}) token = utils.verify_token(token, credential_exception) user = db.query(User).filter(User.username == token.username).first() return user # custom auth def auth_user(jwt_auth: HTTPBearer = Depends(jwt_logged_user), basic_auth: HTTPBasic = Depends(basic_logged_user)): if not (jwt_auth or basic_auth): raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail='Invalid Credentials') #endpoint @router.get("/") async def get_users(db: Session = Depends(db_session), logged_user: str = Depends(auth_user)): query_users = db.query(User).all() return query_users  I expect it to grant me access to endpoint when I provide correct credential for JWT auth or Basic Auth, but it still forces me to put in credential for both. How can I achieve this effect of providing any one of the 2 auth but not both.

Answer: The idea is to make all of this security dependencies not to raise exceptions on user authentication error at the dependency resolving stage. For HTTPBasic pass auto_error=False: security = HTTPBasic(auto_error=False)  And then in basic_logged_user you should check that def basic_logged_user(credentials: Annotated[Optional[HTTPBasicCredentials], Depends(security)]): if credentials is None: return None ... # Do not raise exception, but return None instead  You need to find the way how to do the same with your second authentication scheme (utils.OAuth2_scheme) - not to raise HTTP_401_UNAUTHORIZED, but return None instead. Then your auth_user will work as you expect. It will raise HTTP_401_UNAUTHORIZED only if both schemes return None.
Question: I'm trying to setup a local chatbot demo for testing purpose. I wanted to use LangChain as the framework and LLAMA as the model. Tutorials I found all involve some registration, API key, HuggingFace, etc, which seems unnecessary for my purpose. Is there a way to use a local LLAMA comaptible model file just for testing purpose? And also an example code to use the model with LangChain would be appreciated. Thanks! UPDATE: I wrote a blog post based on the accepted answer.

Answer: No registration is required to utilize on-prem local models within ecosystems like Hugging Face (HF). Similarly, using Langchain does not involve any registration requirements. Various model formats, such as GGUF and GGML, are employed for storing models for inference and can be found on HF. It is crucial to consider these formats when attempting to load and run a model locally. For instance, consider TheBloke's Llama-2-7B-Chat-GGUF model, which is a relatively compact 7-billion-parameter model suitable for execution on a modern CPU/GPU. To run the model, we can use Llama.cpp from Langchain: def llamacpp(): from langchain_community.llms import LlamaCpp from langchain.prompts import PromptTemplate from langchain.chains import LLMChain llm = LlamaCpp( model_path="models/Llama-2-7B-Chat-GGUF/llama-2-7b-chat.Q4_0.gguf", n_gpu_layers=40, n_batch=512, verbose=True, ) template = """Question: {question} Answer: Let's work this out in a step by step way to be sure we have the right answer.""" prompt = PromptTemplate(template=template, input_variables=["question"]) llm_chain = LLMChain(prompt=prompt, llm=llm) question = "Who is Bjarne Stroustrup and how is he related to programming?" print(llm_chain.run(question))  And get output from the LLM:   Bjarne Stroustrup is a Danish computer scientist who created C++. - He was born in Aarhus, Denmark on August 5, 1950 and earned his PhD from Cambridge University in 1983. - In 1979 he began developing the programming language C++, which was initially called "C with Classes". - C++ was first released in 1983 and has since become one of the most popular programming languages in use today.  Bjarne Stroustrup is known for his work on the C programming language and its extension to C++. - He wrote The C Programming Language, a book that helped establish C as a widely used language. - He also wrote The Design and Evolution of C++, a detailed explanation of how he created C++ and why he made certain design choices.   ...  In this instance, I cloned TheBloke's model repository from HF and positioned it in a directory named models/. The final path for the model became models/Llama-2-7B-Chat-GGUF/llama-2-7b-chat.Q4_0.gguf: # Make sure you have git-lfs installed (https://git-lfs.com) git lfs install git clone https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGUF  Although the model can be run on a CPU, this was locally run on my Windows PC equipped with an RTX 4070 card with good performance during inference.
Question: I am having some trouble understanding parts of syntax. Particularly when parentheses () are required for tuples. For instance, this piece of code below: c = {'a':10,'b':1,'c':22,'d':10} tup = a,b = 4,5 print(a) print(b) print(tup) newlist = [(x,y) for y,x in c.items()] print(newlist)  The output for this code is: 4 5 (4, 5) [(10, 'a'), (1, 'b'), (22, 'c'), (10, 'd')]  When trying to take the parentheses out of x, y in the list comprehension statement, I get a traceback. However, every other tuple in this code does not require parenthesis. What am I missing? Why is it that Python understands a,b to be a tuple but not x,y when it is in a list comprehension statement? It seems to me that Python is inconsistent with tuple syntax. I tried taking the parentheses out and putting them back to understand how the syntax works.

Answer: Parentheses are needed to give priority to operators. Consider the two following lines of code, which differ only by the parentheses: tup_a = 3, 4 + 10, 20 tup_b = (3, 4) + (10, 20)  The first line defines a tuple tup_a of 3 elements, (3, 14, 20). The second line defines two tuples (3, 4) and (10, 20), then concatenates them into a tuple of four elements, (3, 4, 10, 20). Wrong parentheses can easily result in an error: tup_c = 3, 4 * 10, 20 tup_d = (3, 4) * (10, 20)  The first line defines a tuple of 3 elements, (3, 40, 20). The second line results in a TypeError because it defines two tuples (3, 4) and (10, 20) and then attempts to multiply them, but the multiplication operator * doesn't know what to do with tuples. You encountered a similar issue. Consider the following lines of codes: x = 42 c = {'a': 10, 'b': 1, 'c': 22, 'd': 10} l_a = [(x, y) for y,x in c.items()] # [(10, 'a'), (1, 'b'), (22, 'c'), (10, 'd')] l_b = [x, (y for y,x in c.items())] # [42, <generator object>] l_c = [x, y for y,x in c.items()] # SyntaxError: invalid syntax  This code correctly defines two lists l_a and l_b, but then it raises a SyntaxError when trying to define l_c. The designers of python decided that the syntax for l_c was ambiguous, and you should add parentheses to explicitly specify whether you meant the same thing as l_a or as l_b. Note that the line of code for l_b would raise a NameError: name 'x' is not defined if I hadn't added x = 42 at the beginning, since the role of x is not the same in l_b as it is in l_a.
Question: Given the following data frame:     a b c d e sd     -100 2 3 60 4 1   7 5 -50 9 130 2     How would I calculate the standard deviation sd column which excludes the minimum and maximum values from each row? The actual data frame is a few million rows long so something vectorised would be great! To replicate: df = pd.DataFrame( {"a": [-100, 7], "b": [2, 5], "c": [3, -50], "d": [60, 9], "e": [4, 130]} ) 

Answer: Excluding the first min/max An approach that should be fast would be to use numpy to sort the values per row, exclude the first and last and compute the std: df['sd'] = np.sort(df, axis=1)[:, 1:-1].std(axis=1, ddof=1)  Handling duplicate min/max (excluding all) If you can have several times the same min/max and want to exclude all, then you could compute a mask: m1 = df.ne(df.min(axis=1), axis=0) m2 = df.ne(df.max(axis=1), axis=0) df['sd'] = df.where(m1&m2).std(axis=1)  Output:  a b c d e sd 0 -100 2 3 60 4 1.0 1 7 5 -50 9 130 2.0
Question: I have two dfs solar_part = pd.DataFrame( {'pool': 1, 'orig': 635.1}, index = [0] ) solar_aod = pd.DataFrame( {'pool': [1,1,1,1], 'MoP': [1,2,3,4], 'prin': [113.1, 115.3, 456.6, 234.1]} )  Which I then merge together via pipeline and try and transform/assign a new variable solar_p = ( solar_aod .merge(solar_part, on = ['pool'], how = 'left') .assign(remn = ['prin'] / ['orig']) )  The assign (have tried transform as well) gives the error of TypeError: unsupported operand type(s) for /: 'list' and 'list' which I'm guessing is caused by the bracketing. Trying only quotes gives the same same error but with str instead of list. Not including an assign/transform function and then doing it "long hand" via solar_p.prin / solar_p.orig * 100 also works but I have several more equations to include, so I'd like it as concise as possible. How to do this tranform/assign after a merge in a pipeline?

Answer: You can use eval to create an additional column as well as assign but the latter need to evaluate the new dataframe before use columns that why you have to use a lambda function here: solar_p = ( solar_aod .merge(solar_part, on='pool', how='left') .eval('remn = prin / orig') )  Output: >>> solar_p pool MoP prin orig remn 0 1 1 113.1 635.1 0.178082 1 1 2 115.3 635.1 0.181546 2 1 3 456.6 635.1 0.718942 3 1 4 234.1 635.1 0.368603
Question: I have a data Dataframe like, data = { "A": [42, 38, 39,23], "B": [45, 30, 15,65], "C": [60, 50, 25,43], "D": [12, 70, 35,76,], "E": [87, 90, 45,43], "F": [40, 48, 55,76], "G": [58, 42, 85,10], } df = pd.DataFrame(data) print(df) A B C D E F G 0 42 45 60 12 87 40 58 1 38 30 50 70 90 48 42 2 39 15 25 35 45 55 85 3 23 65 43 76 43 76 10  Here, I need to subtract all the values of columns C to G from column B, then add column A. Like df['C']=df['C']-df['B']+df['A'] df['D']=df['D']-df['B']+df['A'] df['E']=df['E']-df['B']+df['A']  How it is possible in simple/single commands?

Answer: You can use a single eval with a multiline expression: df = df.eval('''C=C-B+A D=D-B+A E=E-B+A ''')  Or, for this particular operation, since all are +A-B: df[['C', 'D', 'E']] = df[['C', 'D', 'E']].add(df['A'].sub(df['B']), axis=0)  Output:  A B C D E F G 0 42 45 57 9 84 40 58 1 38 30 58 78 98 48 42 2 39 15 49 59 69 55 85 3 23 65 1 34 1 76 10
Question: import pandas as pd print(pd.date_range(start='1999-08-01', end='2000-07-01', freq='D')) print(pd.period_range(start='1999-08-01', end='2000-07-01', freq='D')) print(pd.date_range(start='1999-08', end='2000-07', freq='M')) #This doesn't include anything from 2000-07, ie is not stop-inclusive - length 11 print(pd.period_range(start='1999-08', end='2000-07', freq='M')) # Length 12  Of these, three are stop-inclusive as one might naively define, whereas the date_range with frequency "M" is not. Why is that?

Answer: This is because 2000-07 is understood as 2000-07-01 and M is the end of month (opposite to MS the start of month) so 2000-07-31 is after 2000-07-01. If you want to include "July" change the frequency from M to MS but it will change the day: >>> pd.date_range(start='1999-08', end='2000-07', freq='M')[[0, -1]] DatetimeIndex(['1999-08-31', '2000-06-30'], dtype='datetime64[ns]', freq=None) >>> pd.date_range(start='1999-08', end='2000-07', freq='MS')[[0, -1]] DatetimeIndex(['1999-08-01', '2000-07-01'], dtype='datetime64[ns]', freq=None)
Question: I am reading text from a file that contains flags start and end. I want to replace everything between start and end with the same text except I want to remove any newlines in the matching group. I tried to do it as follows: import re start = '---' end = '===' text = '''\ Some text ---line 1 line 2 line 3=== More text ... Some more text ---line 4 line 5=== and even more text\ ''' modified = re.sub(pattern=rf'{start}(.+){end}', repl=re.sub(r'\n', ' ', r'\1'), string=text, flags=re.DOTALL) print(modified)  This prints: Some text line 1 line 2 line 3=== More text ... Some more text ---line 4 line 5 and even more text  Couple of issues with this, 1. it matches the biggest group (and not the smaller matching groups), and 2. it does not remove the newlines. I am expecting the output to be: Some text line 1 line 2 line 3 More text ... Some more text line 4 line 5 and even more text  Any help will be appreciated. Thank you!

Answer: Use non-greedy modifier (?) in the capturing group. Also, change the replacement function for simple str.replace: import re start = "---" end = "===" text = """\ Some text ---line 1 line 2 line 3=== More text ... Some more text ---line 4 line 5=== and even more text\ """ modified = re.sub( pattern=rf"{start}(.+?){end}", repl=lambda g: g.group(1).replace("\n", " "), string=text, flags=re.DOTALL, ) print(modified)  Prints: Some text line 1 line 2 line 3 More text ... Some more text line 4 line 5 and even more text
Question: I have 2 polars LazyFrames: df1 = pl.DataFrame(data={ 'foo': np.random.uniform(0,127, size= n).astype(np.float64), 'bar': np.random.uniform(1e3,32767, size= n).astype(np.float64), 'baz': np.random.uniform(1e6,2147483, size= n).astype(np.float64) }).lazy() df2 = pl.DataFrame(data={ 'foo': np.random.uniform(0,127, size= n).astype(np.float64), 'bar': np.random.uniform(1e3,32767, size= n).astype(np.float64), 'baz': np.random.uniform(1e6,2147483, size= n).astype(np.float64) }).lazy()  I would like to multiply each column in df1 with its respective column in df2. If I convert these to non-lazy DataFrames I can achieve this: df1.collect() * df2.collect()  foo bar baz f64 f64 f64 3831.295563 6.4637e6 3.3669e12 164.194271 2.9691e8 2.2696e12 3655.918761 1.9444e7 2.3625e12 7191.48868 3.7044e7 3.1687e12 9559.505277 2.6864e8 2.5426e12  However, if I try to perform the same expression on the LazyFrames, I get an exception df1 * df2   TypeError: unsupported operand type(s) for *: 'LazyFrame' and 'LazyFrame'  How can I perform column-wise multiplication across 2 LazyFrames?

Answer: you'll need to join ( df1.with_row_index() .join(df2.with_row_index(), on="index") .select(pl.col(col) * pl.col(f"{col}_right") for col in df1.columns) .collect() )  shape: (10, 3) ┌─────────────┬──────────┬───────────┐ │ foo ┆ bar ┆ baz │ │ --- ┆ --- ┆ --- │ │ f64 ┆ f64 ┆ f64 │ ╞═════════════╪══════════╪═══════════╡ │ 6623.602754 ┆ 2.7173e8 ┆ 3.7654e12 │ │ 2588.499522 ┆ 7.4295e8 ┆ 3.0266e12 │ │ 933.474643 ┆ 3.7090e8 ┆ 4.2794e12 │ │ 7061.625136 ┆ 2.2365e8 ┆ 2.7040e12 │ │ … ┆ … ┆ … │ │ 2717.969236 ┆ 4.9398e7 ┆ 3.0930e12 │ │ 785.760153 ┆ 1.6305e8 ┆ 1.8954e12 │ │ 9534.366291 ┆ 7.3153e8 ┆ 1.9056e12 │ │ 1916.452503 ┆ 1.4976e8 ┆ 3.2704e12 │ └─────────────┴──────────┴───────────┘
Question: I tried to use Django Multiselectfield library but i got this error:  I already installed it properly by command: pip install django-multiselectfield I also added in my settings.py INSTALLED_APPS = [ ..... 'multiselectfield', ]  My models.py: from multiselectfield import MultiSelectField from django.db import models CATEGORY = [ ('CP', 'Computer Programming'), ('LG', 'Language'), ('MA', 'Mathematics'), ('BM', 'Business Management'), ('GK', 'General Knowledge') ] class Quiz(models.Model): ... title = models.CharField(max_length=256) category = MultiSelectField(choices=CATEGORY, max_choices=2, max_length=5) ... def save(self, *args, **kwargs): if not self.category: self.category = ['GK'] super(Quiz, self).save(*args, **kwargs) def __str__(self): return self.title  The error: 'super' object has no attribute '_get_flatchoices' However this is located in the Multiselectfield library code. Because of this error, even my data cannot be saved. I tried to find out what am I missing, but I have no idea. Any idea what's wrong with my code? Appreciate your help ^_^

Answer: The current version of Multiselectfield is incompatible with Django 5.0. There is no _get_flatchoices in django's CharField anymore, instead Django provides flatchoices directly. Therefore, the following code in ../multiselectfield/db/fields.py should be removed. def _get_flatchoices(self): flat_choices = super(MultiSelectField, self)._get_flatchoices() class MSFFlatchoices(list): # Used to trick django.contrib.admin.utils.display_for_field into # not treating the list of values as a dictionary key (which errors # out) def __bool__(self): return False __nonzero__ = __bool__ return MSFFlatchoices(flat_choices) flatchoices = property(_get_flatchoices)  Thanks for aliceni81 For further information, you can check on https://github.com/goinnn/django-multiselectfield/issues/141 Edit: Now they already fixed the issue, you can install the package by using command: pip install https://github.com/goinnn/django-multiselectfield/archive/master.zip
Question: I am trying to create a pydantic class with Immutable class fields (not instance fields). Here is my base code: from pydantic import BaseModel class ImmutableModel(BaseModel): _name: str = "My Name" _age: int = 25 ImmutableModel._age = 26 print("Class variables:") print(f"Name: {ImmutableModel._name}") print(f"Age: {ImmutableModel._age}")  Output: Class variables: Name: My Name Age: 26  I tried using the Config class inside my ImmutableModel to make fields immutable. But it seems like it only works for instance class fields. class Config: allow_mutation = False  FYI, I use Python 3.6.13 and pydantic==1.9.2

Answer: Initially, I tried to achieve creating immutable Class and Instance using pydantic module. Now I were able to manage it using native method itself. Since this was completely defined by me and immutable, its fine to have no validation. class ImmutableMetaclass(type): def __setattr__(cls, name, value): raise AttributeError("Cannot create or set class attribute '{}'".format(name)) class MyImmutableClass(metaclass=ImmutableMetaclass): # Define all allowed class attributes here CONSTANT_1 = 1 def __setattr__(self, name, value): raise AttributeError("Cannot create or set class or instance attribute '{}'".format(name))  immutable_class = MyImmutableClass immutable_class.CONSTANT_1 = 100 immutable_class.CONSTANT_2 = 200 immutable_instance = MyImmutableClass() immutable_instance.CONSTANT_1 = 100 immutable_instance.CONSTANT_3 = 300  All the above code raises AttributeError.
Question: I can no longer initailize my own earth engine project on Google colab using Python. I tried initializing using the ee.Initialize() method on Colab with used to provide with a link that will generate a token to my project but instead i get this error: WARNING:googleapiclient.http:Encountered 403 Forbidden with reason "PERMISSION_DENIED" --------------------------------------------------------------------------- HttpError Traceback (most recent call last) /usr/local/lib/python3.10/dist-packages/ee/data.py in _execute_cloud_call(call, num_retries) 382 try: --> 383 return call.execute(num_retries=num_retries) 384 except googleapiclient.errors.HttpError as e: 7 frames HttpError: <HttpError 403 when requesting https://earthengine.googleapis.com/v1/projects/earthengine-legacy/algorithms?prettyPrint=false&alt=json returned "Google Earth Engine API has not been used in project 522309567947 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/earthengine.googleapis.com/overview?project=522309567947 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.". Details: "[{'@type': 'type.googleapis.com/google.rpc.Help', 'links': [{'description': 'Google developers console API activation', 'url': 'https://console.developers.google.com/apis/api/earthengine.googleapis.com/overview?project=522309567947'}]}, {'@type': 'type.googleapis.com/google.rpc.ErrorInfo', 'reason': 'SERVICE_DISABLED', 'domain': 'googleapis.com', 'metadata': {'service': 'earthengine.googleapis.com', 'consumer': 'projects/522309567947'}}]"> During handling of the above exception, another exception occurred: EEException Traceback (most recent call last) /usr/local/lib/python3.10/dist-packages/ee/data.py in _execute_cloud_call(call, num_retries) 383 return call.execute(num_retries=num_retries) 384 except googleapiclient.errors.HttpError as e: --> 385 raise _translate_cloud_exception(e) # pylint: disable=raise-missing-from 386 387 EEException: Google Earth Engine API has not been used in project 522309567947 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/earthengine.googleapis.com/overview?project=522309567947 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry  .

Answer: Their lack of documentation on this change is incredibly frustrating, but fortunately it's a very easy fix (or at least it was for me). If you go to your earth engine code editor and click on the 'Assets' tab, then copy the name of the Project that all of your EE assets are that you're trying to access.see linked image here: Then, at the initialization step in your colab script, you replace 'ee.Initialize()' with 'ee.Initiliaze(project='NameofYourEarthEngineProject') Hope this helps.