eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm just going to simplify my problem a bit. I have a function like this: def func(a,b): return a+b  I also have a list of tuples which I would like to map to this function. num = [(0,4),(6,3),(2,2),(9,1)]  I want to be able to map the tuples within the list like (a,b) to the function I provided. In javascript you could acheive this by changing the function definition like so: def func((a,b)): return a+b num = [(0,4),(6,3),(2,2),(9,1)] map(func,num)  This obviously doesn't work in Python. I know that I could also just pass the tuple into the function and then return tuple[0]+tuple[1], but I didn't know if there was a cleaner option.

Answer: map won't perform unpacking without writing a wrapper function to do that actual unpacking for you. That's why itertools.starmap exists: from itertools import starmap def func(a,b): return a+b num = [(0,4),(6,3),(2,2),(9,1)] for result in starmap(func, num): print(result)  The name "starmap" is referring to the implicit star-unpacking operation it performs, making it equivalent to map with the iterable assumed to contain pre-packed iterables of arguments that must be *-unpacked, changing from map being the equivalent of (func(x) for x in iterable) to starmap being the equivalent of (func(*x) for x in iterable).
Question: I'm trying to set up Coveralls to work with GitHub Actions for a Python project, and although I've reviewed the documentation multiple times and followed all the instructions to the best of my understanding, I'm still facing the following error:  Bad Response 422 {“message”: “Couldn’t find a repository matching this job”, “error”: true}  Here is a minimal version of my YAML file: name: coveralls on: pull_request: branches: - main jobs: tests: runs-on: ubuntu-latest steps: - name: checkout uses: actions/checkout@v3 - name: setup python uses: actions/setup-python@v4 with: python-version: '3.9' - name: install requirements run: | pip install --upgrade pip pip install pytest pip install pytest-cov pip install -r app/requirements.txt - name: run tests run: | pytest --cov=app coverage report -m coverage lcov - name: upload coveralls uses: coverallsapp/github-action@master with: github-token: ${{ secrets.GH_TOKEN }} path-to-lcov: coverage.lcov 

Answer: The documentation is not clear enough at this point:      Name Requirement Description     github-token required Must be in form github-token: ${{ secrets.GITHUB_TOKEN }}; Coveralls uses this token to verify the posted coverage data on the repo and create a new check based on the results. It is built into Github Actions and does not need to be manually specified in your secrets store. More Info     While it suggests that the GitHub token does not require manual specification in your secrets store, it is presented as a recommendation rather than a strict rule. It would be more appropriate to state that "it must not be manually specified", since using a custom variable like GH_TOKEN instead of the default GITHUB_TOKEN will not function properly. That being said, you need to replace this line: github-token: ${{ secrets.GH_TOKEN }}  with this line: github-token: ${{ secrets.GITHUB_TOKEN }}
Question: I have a Python script to retrieve username and password from Key Vault (Azure). 3 months ago it worked but now it gives me the error No module named 'azure.keyvault.secrets' when I run 'from azure.keyvault.secrets import SecretClient'. Why I get this error? It gives me error also if I try to run pip install azure!

Answer:  Error ModuleNotFoundError: No module named 'azure.keyvault.secrets' although I installed the package:  If you run pip install azure it won't work for azure.keyvaults module to be used in the code. You need to install or update with the latest version using pip command: pip install azure-keyvault-secrets //4.6.0 is the latest one.  Installed successfully:
Question: I have been trying to use selenium on a webpage but this popup is refraining me to do so.  note that the popup is only shown when you are not signed in (means you have to run my code so that selenium opens up a new browser window for you which does not have any accounts) I want to click on the "Not Interested" button through selenium. I don't want to close the popup every time manually, is there a way to automate this? here is my code: # relevant packages & modules from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.webdriver.common.by import By from selenium.webdriver.chrome.options import Options import time # relevant website website = 'https://www.daraz.pk/' # initialize Chrome driver = webdriver.Chrome('C:\webdrivers\chromedriver.exe') # open website driver.get(website) #maximize window driver.maximize_window() # waiting for popup time.sleep(5) # dealing with pop up # with xpath pop_up_deny = driver.find_element(By.XPATH , '/html/body/div[9]//div/div/div[3]/button[1]') pop_up_deny.click()  It raised this error:  My chrome version : 110.0.5481.178 (Official Build) (64-bit) My selenium version : ChromeDriver 110.0.5481.77

Answer: The popup element is inside Shadow-root element you need to reach to shadow-root first then identify the button not interested shadowRoot= driver.execute_script('''return document.querySelector("div.airship-html-prompt-shadow").shadowRoot''') shadowRoot.find_element(By.CSS_SELECTOR,"button.airship-btn.airship-btn-deny").click()
Question: I am new to Tensorflow, and am trying to train a specific deep learning neural network. I am using Tensorflow (2.11.0) to get a deep neural network model which is described below. The data which I use is also given below: Data: Here is some example data. For sake of ease we can consider 10 samples in data. Here, each sample has shape: (128,128). One can consider the below code as example training data. x_train = np.random.rand(10, 128, 128, 1)  Normalization layer: normalizer = tf.keras.layers.Normalization(axis=-1) normalizer.adapt(x_train)  Build model: def build_and_compile_model(norm): model = tf.keras.Sequential([ norm, layers.Conv2D(128, 128, activation='relu'), layers.Conv2D(3, 3, activation='relu'), layers.Flatten(), layers.Dense(units=32, activation='relu'), layers.Dense(units=1) ]) model.compile(loss='mean_absolute_error', optimizer=tf.keras.optimizers.Adam(0.001)) return model  When I do dnn_model = build_and_compile_model(normalizer) dnn_model.summary()  I get the below error: ValueError: The channel dimension of the inputs should be defined. The input_shape received is (None, None, None, None), where axis -1 (0-based) is the channel dimension, which found to be `None`.  What am I doing wrong here? I have tried to get insights from this, this, this and this. But, I have not found a workable solution yet. What should I do to remove the error and get the model to work? I will appreciate any help.

Answer: Define the input shape directly in the normalization layer (or add an Input layer), since it cannot be inferred directly: import numpy as np import tensorflow as tf x_train = np.random.rand(10, 128, 128, 1) normalizer = tf.keras.layers.Normalization(input_shape=[128, 128, 1], axis=-1) normalizer.adapt(x_train) def build_and_compile_model(norm): model = tf.keras.Sequential([ norm, tf.keras.layers.Conv2D(64, 64, activation='relu'), tf.keras.layers.Conv2D(3, 3, activation='relu'), tf.keras.layers.Flatten(), tf.keras.layers.Dense(units=32, activation='relu'), tf.keras.layers.Dense(units=1) ]) model.compile(loss='mean_absolute_error', optimizer=tf.keras.optimizers.Adam(0.001)) return model dnn_model = build_and_compile_model(normalizer) dnn_model.summary()  Also, your model does not work as it is, you are using a kernel size of 128 in your first Conv2D layer and then another Conv2D layer with a kernel size of 3 but your data has the shape (10, 128, 128, 1). I changed it to make your code executable.
Question: I'm new to the Beanie library which is  an asynchronous Python object-document mapper (ODM) for MongoDB. Data models are based on Pydantic.  I was trying this library with fastAPI framework, and made an ODM for some document, let's say it's name is SomeClass and then tried to insert some data in the db using this ODM. Here's the code for ODM and the method to create a document (insomeClass.py): from beanie import Document from pydantic import Field, BaseModel class SomeClassDto(BaseModel): """ A Class for Data Transferring. """ name: str = Field(max_length=maxsize, min_length=1) class SomeClassDao: """ This is a class which holds the 'SomeClass' class (inherited from Beanie Document), and also, the methods which use the 'SomeClass' class. """ class SomeClass(Document): name: str = Field(max_length=20, min_length=1) @classmethod async def create_some_class(cls, body: SomeClassDto): some_class = cls.SomeClass(**body.dict()) return await cls.SomeClass.insert_one(some_class)  I've used and called the create_some_class function, but it throwed this error: beanie.exceptions.CollectionWasNotInitialized However the error is self-explanatory but I didn't understand at first, and couldn't find any relatable question about my problem in SO, so I decided to post this question and answer it, for the sake of future.

Answer: As the error tells us, we should first Initialize the collection. We should initialize the collection via the init_beanie. I’ve used this function like this (in databse.py): from beanie import init_beanie import motor.motor_asyncio from someClass import SomeClassDao async def init_db(cls): MONGO_DB_DATABASE_NAME = "SomeDBName" MOTOR_CLIENT = motor.motor_asyncio.AsyncIOMotorClient() DATABASE = MOTOR_CLIENT[MONGO_DB_DATABASE_NAME] document_models = [SomeClassDao.SomeClass,] await init_beanie(database=cls.DATABASE, document_models=document_models)  And then we should use this function at the startup of the app, so we use it like this (in main.py): from fastapi import FastAPI from database import init_db app = FastAPI() @app.on_event("startup") async def start_db(): await init_db()   Then after creating the init_beanie function and adding the document we want to use to the document_models, it worked without error.  Hope this helps.
Question: I have a 2D numpy object array: aa = np.array([["aaa","05","1","a"], ["ccc","30","2","v"], ["ddd","50","2","v"], ["bbb","10","1","v"]])  and the following list: sample_ids = ["aaa", "bbb", "ccc", "ddd"]  I would like to sort the numpy array based on the list so that I get the following: [["aaa","05","1","a"], ["bbb","10","1","v"], ["ccc","30","2","v"], ["ddd","50","2","v"]]  Edit: If there are keys (in sample_ids) that are not present in the array. The resulting array would not include these missing keys (i.e. no addition of empty rows). So if we have the following: sample_ids = ["aaa", "bbb", "ccc", "ddd", "eee"]  The final array would still be the same. Also, if the array would contain a row (i.e. row key) that is missing from the keys. That row would be left out of the resulting array as well. Edit 2: Starting from Nick's answer, I came up with this to deal with absent keys. sample_ids2 = ["aaa", "bbb", "eee", "ccc", "ddd"] idxs = [] for i,v in enumerate(sample_ids2): if str(list(aa.T[0])).find(v) != -1: k = list(aa.T[0]).index(v) idxs.append(k) else: print(v + " was not found!!!") print(aa[idxs])  Output: [['aaa' '05' '1' 'a'] ['bbb' '10' '1' 'v'] ['ccc' '30' '2' 'v'] ['ddd' '50' '2' 'v']] 

Answer: Here are a couple of possible solutions. Using numpy: subs = list(aa.T[0]) idxs = [subs.index(i) for i in sample_ids if i in subs] res = aa[idxs] # array([['aaa', '05', '1', 'a'], # ['bbb', '10', '1', 'v'], # ['ccc', '30', '2', 'v'], # ['ddd', '50', '2', 'v']], dtype='<U3')  Using pandas: res = np.array(pd.DataFrame(aa).set_index(0).reindex(sample_ids).dropna().reset_index()) # array([['aaa', '05', '1', 'a'], # ['bbb', '10', '1', 'v'], # ['ccc', '30', '2', 'v'], # ['ddd', '50', '2', 'v']], dtype=object)  For both cases, if sample_ids = ["aaa", "bbb", "ccc", "ddd", "eee"], the output will be the same. If sample_ids = ["ddd", "aaa", "bbb"], the output will be: array([['ddd', '50', '2', 'v'], ['aaa', '05', '1', 'a'], ['bbb', '10', '1', 'v']])
Question: I need to take a list, multiply every item by 4 and separate them by coma. My code is: conc = ['0.05 ml : 25 ml', '0.05 ml : 37.5 ml', '0.05 ml : 50 ml', '0.05 ml : 62.5 ml', '0.05 ml : 75 ml'] new_conc = [", ".join(i*4) for i in conc] print(new_conc)  But when I run it, I get every SYMBOL separated by come. What I need is multiplied number of shown EXPRESSIONS separated by coma. So the output should be: ['0.05 ml : 25 ml', '0.05 ml : 25 ml', '0.05 ml : 25 ml', '0.05 ml : 25 ml', '0.05 ml : 37.5 ml', '0.05 ml : 37.5 ml', '0.05 ml : 37.5 ml', '0.05 ml : 37.5 ml', '0.05 ml : 50 ml', '0.05 ml : 50 ml', '0.05 ml : 50 ml', '0.05 ml : 50 ml', '0.05 ml : 62.5 ml', '0.05 ml : 62.5 ml', '0.05 ml : 62.5 ml', '0.05 ml : 62.5 ml', '0.05 ml : 75 ml', '0.05 ml : 75 ml', '0.05 ml : 75 ml', '0.05 ml : 75 ml'] I found this answered question, but as I already mentioned, I get separate symbols, separated by coma.

Answer: You can use a simple for loop. new_conc = [] for item in conc: new_conc.extend([item] * 4)
Question: I expected Snakemake to allow wildcards to be empty strings, alas, this isn't the case. How can I make a wildcard accept an empty string?

Answer: Wildcards by default only match the regex .+ meaning everything but empty strings. This is unfortunately not documented beyond a Google group conversation. To make a wildcard accept empty strings, simply add a custom wildcard constraint wildcard_constraints: foo=".*", either within the scope of a rule or globally: # Option 1 wildcard_constraints: foo=".*" rule a: input: in{foo}.txt output: out{foo}.txt wildcard_constraints: foo=".*" # Option 2 shell: "cp {input} {output}"
Question: I switched from a zip-based deployment to a docker-based deployment of two lambda functions (which are used in an API Gateway). Both functions where in the same zip file and I want to have both functions in the same docker-based container (meaning I can't use the cmd setting in my Dockerfile (or to be precise need to overwrite it anyway). Previously, I used the handler attribute in the cloudformation template for specifying which handler function to call in which module, e.g. ... ConfigLambda: Type: 'AWS::Serverless::Function' Properties: Handler: config.handler ... ... LogLambda: Type: 'AWS::Serverless::Function' Properties: Handler: logs.handler ...  but with a docker-based build one has to define an ImageConfig, i.e. ... LogLambda: Type: 'AWS::Serverless::Function' Properties: PackageType: Image ImageUri: !Ref EcrImageUri FunctionName: !Sub "${AWS::StackName}-Logs" ImageConfig: WorkingDirectory: /var/task Command: ['logs.py'] EntryPoint: ['/var/lang/bin/python3'] ... ConfigLambda: Type: 'AWS::Serverless::Function' Properties: PackageType: Image ImageUri: !Ref EcrImageUri FunctionName: !Sub "${AWS::StackName}-Config" ImageConfig: WorkingDirectory: /var/task Command: ['config.py'] EntryPoint: ['/var/lang/bin/python3']  I'm a bit stuck because this does not work, no matter what combination I pass to the command array. If I fire a test event in the AWS console, I get the following error RequestId: <uuid> Error: Runtime exited without providing a reason Runtime.ExitError  Judging from the full output, the file is loaded and executed, but the handler function is not invoked (there is some output from a logging setup function which is called right after the module imports). The section in the AWS documentation on python3 based lambdas state that naming for handlers should be file_name.function (e.g. function_lambda.lambda_handler), but this doesn't give any clues on how do to this for command array in a ImageConfig. How do I set the Command section correctly for my lambda function in my cloudformation template?

Answer: First, the container you deploy to AWS Lambda has to implement the Lambda Runtime Interface. AWS Lambda isn't a generic docker container runtime, it only supports running containers that implement a specific interface. The easiest way to ensure your container implements this interface is to base it on one of the AWS provided base images. Note how the ENTRYPOINT for the base Python images isn't python it is a custom script. That script expects the COMMAND to be in the same app.handler format it is in for non-container based Lambda functions. See an example Dockerfile here.
Question: I am trying to read multiple parquet files stored as partitions from google cloud storage and read them as 1 single pandas data frame. As an example, here is the folder structure at gs://path/to/storage/folder/  And inside each of the event_date=*, there are multiple parquet files So the directory structure is something like this - --gs://path/to/storage/folder/ ---event_date=2023-01-01/ ---abc.parquet ---def.parquet ---event_date=2023-01-02/ ---ghi.parquet ---jkl.parquet  I want to load this to pandas data frame and I used below code import pandas as pd import gcsfs from pyarrow import parquet url = "gs://path/to/storage/folder/event_date=*/*" fs = gcsfs.GCSFileSystem() files = ["gs://" + path for path in fs.glob(url)] print(files) data = parquet.ParquetDataset(files, filesystem=fs) multiple_dates_df = data.read().to_pandas() print(multiple_dates_df.shape)  But I get below error - OSError: Passed non-file path: gs://path/to/storage/folder/event_date=2023-01-01/abc.parquet  How do I fix this?

Answer: Seems it is not possible for pandas to read multiple parquet files stored under a gcs path,There is a bug raised for this at github, which is still open further progress can be tracked there.
Question: I have a data frame of online game matches including two specific columns: IDs of matches and IDs of players participated in a particular match. For instance:     match_id player_id     0 1   0 2   0 3   0 4   0 5   1 6   1 1   1 7   1 8   1 2     Hence, player_id is a unique identificator of a player. Meanwhile, match_id is an ID of a match played, and it is duplicated fixed number of times (say, 5), since 5 is a maximum number of players that are able to participate in a certain match. So in each row, match_id corresponds player_id meaning that a certain player participated in a particular game. As it can be seen from the table above, two or more players can play together more than one time (or they can have not any co-plays at all). And it's why I'm interested in transforming this initial data frame into a adjacency matrix, in which the intersection of row and a column would give the number of co-played matches. Another option would be to create a data frame like following:     player_1 player_2 coplays_number     1 2 2   1 3 1   1 4 1   1 10 0   1 5 1   ... ... ...     Hereby, my task is to prepare the data for a further analysis of a co-plays network using igraph or networkx. I also want to get a weighted network, that is a weight of an edge would mean a number of co-played matches between two nodes (players). Edge in this case means that two users have played together, i.e. they have participated in the same match once or they have played together as a team in two or more matches (like players' IDs 1 and 2 in the initial data example above). My question is: how can I transform my initial data frame into network data, that igraph or networkx functions would take as an argument, using pandas and numpy? Or maybe I do not need any data manipulations and igraph or networkx functions are able to work with the initial data frame? Thanks in advance for your answers and recommendations!

Answer: I think you don't need networkx if you use permutations from itertools and pd.crosstab: from itertools import permutations pairs = (df.groupby('match_id')['player_id'] .apply(lambda x: list(permutations(x, r=2))) .explode()) adj = pd.crosstab(pairs.str[0], pairs.str[1], rownames=['Player 1'], colnames=['Player 2'])  Output: >>> adj Player 2 1 2 3 4 5 6 7 8 Player 1 1 0 2 1 1 1 1 1 1 2 2 0 1 1 1 1 1 1 3 1 1 0 1 1 0 0 0 4 1 1 1 0 1 0 0 0 5 1 1 1 1 0 0 0 0 6 1 1 0 0 0 0 1 1 7 1 1 0 0 0 1 0 1 8 1 1 0 0 0 1 1 0  If you want a flat list (not an adjacency matrix), use combinations: from itertools import combinations pairs = (df.groupby('match_id')['player_id'] .apply(lambda x: frozenset(combinations(x, r=2))) .explode().value_counts()) coplays = pd.DataFrame({'Player 1': pairs.index.str[0], 'Player 2': pairs.index.str[1], 'coplays_number': pairs.tolist()})  Output: >>> coplays Player 1 Player 2 coplays_number 0 1 2 2 1 2 4 1 2 6 2 1 3 8 2 1 4 7 2 1 5 1 7 1 6 6 7 1 7 1 8 1 8 6 8 1 9 6 1 1 10 3 5 1 11 1 3 1 12 2 5 1 13 4 5 1 14 2 3 1 15 1 4 1 16 1 5 1 17 3 4 1 18 7 8 1
Question: I’m trying to create a very basic mock password verification program to get more comfortable with meta characters. The program is supposed to take an input, use regex to verify it has at least one capital letter and at least one number, then return either “Password created” if it does, or “Wrong format” if it doesn’t. I’m trying to use an AND statement inside of my conditional statement and I know it’s not optimal, I just don’t understand why it doesn’t work at all. Here’s the code: import re password = input() #check input for at least one cap letter and at least one number if re.match(r"[A-Z]*", password) and re.match(r"[0-9]*", password): print("Password created") else: print("Wrong format")  Edit: To everyone helping and asking for clarification, I’d like to apologize. The original code did not have the asterisks because I’m new to StackOverflow and did not use the correct formatting. I’m also new to asking coding questions so I’ll give some more context as requested. I’ve since changed the code to this: import re password = input() #check input for at least one cap letter and at least one number if re.search(r"[A-Z]*", password) and re.search(r"[0-9]*", password): print("Password created") else: print("Wrong format")  Here are some example inputs and their expected vs actual outputs: In: “Greatness” Expect: “Wrong format” Actual: “Password created” In: “12335” Expect: “Wrong format” Actual: “Password created” In: “Gm16gs” Expect: “Password created” Actual: “Password created” If I’m missing any more context please let me know as I am still new to this. Update: I’m a moron. It wasn’t the and, it was the asterisks. Thank you so much everyone. I’ve marked the first answer as correct because the comments show me that I should’ve been using “+” and not “*”

Answer: Use re.search to find a match anywhere in the string. re.match will only return a match if the match starts from the beginning of the string. if re.search("[A-Z]", password) and re.search("[0-9]", password):
Question: I'm using Psycopg3 (not 2!) and I can't figure out how can I connect to a remote Postgres server psycopg.connect(connection_string)  https://www.psycopg.org/psycopg3/docs/ Thanks!

Answer: Psycopg3 uses the postgresql connection string which can either be a string of keyword=value elements (separated by spaces) with psycopg.connect("host=your_server_hostname port=5432 dbname=your_db_name") as conn:  or a URI with psycopg.connect("postgresql://user:user_password@db_server_hostname:5432") as conn:  So, put all together with their example (mixed with one example from above connection strings) from their docs: # Note: the module name is psycopg, not psycopg3 import psycopg # Connect to an existing database with psycopg.connect("postgresql://user:user_password@db_server_hostname:5432") as conn: # Open a cursor to perform database operations with conn.cursor() as cur: # Execute a command: this creates a new table cur.execute(""" CREATE TABLE test ( id serial PRIMARY KEY, num integer, data text) """) # Pass data to fill a query placeholders and let Psycopg perform # the correct conversion (no SQL injections!) cur.execute( "INSERT INTO test (num, data) VALUES (%s, %s)", (100, "abc'def")) # Query the database and obtain data as Python objects. cur.execute("SELECT * FROM test") cur.fetchone() # will return (1, 100, "abc'def") # You can use `cur.fetchmany()`, `cur.fetchall()` to return a list # of several records, or even iterate on the cursor for record in cur: print(record) # Make the changes to the database persistent conn.commit()