eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I am trying to implement a classification model with ResNet50. I know, CNN or transfer learning models extract the features themselves. How can I get the extracted feature vector from the image dataset in python? Code Snippet of model training: base_model = ResNet152V2( input_shape = (height, width, 3), include_top=False, weights='imagenet' ) from tensorflow.keras.layers import MaxPool2D, BatchNormalization, GlobalAveragePooling2D top_model = base_model.output top_model = GlobalAveragePooling2D()(top_model) top_model = Dense(1072, activation='relu')(top_model) top_model = Dropout(0.6)(top_model) top_model = Dense(256, activation='relu')(top_model) top_model = Dropout(0.6)(top_model) prediction = Dense(len(categories), activation='softmax')(top_model) model = Model(inputs=base_model.input, outputs=prediction) for layer in model.layers: layer_trainable=False from tensorflow.keras.optimizers import Adam model.compile( optimizer = Adam(learning_rate=0.00001), loss='categorical_crossentropy', metrics=['accuracy'] ) import keras from keras.callbacks import EarlyStopping es_callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=3) datagen = ImageDataGenerator(rotation_range= 30, width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, zoom_range=0.3, horizontal_flip=True ) resnet152v2 = model.fit(datagen.flow(X_train, y_train), validation_data = (X_val, y_val), epochs = 100, callbacks=[es_callback] )  I followed a solution from here. Code snippet of extracting features from the trained model: extract = Model(model.inputs, model.layers[-3].output) #top_model = Dropout(0.6)(top_model) features = extract.predict(X_test)  The shape of X_test is (120, 224, 224, 3)

Answer: Once you have trained your model, you can save it (or just directly use it in the same file) then cut off the top layer. model = load_model("xxx.h5") #Or use model directly #Use functional model from tensorflow.keras.models import Model featuresModel = Model(inputs=model.input,outputs=model.layers[-3].output) #Get rids of dropout too #Inference directly featureVector = featuresModel.predict(yourBatchData, batch_size=yourBatchLength)  If inferenced in batches, you would need to separate the output results to see the result for each input in the batch.
Question: I'm stuck and can't figure out a workable way to connect asynchronously to an Azure SQL database using Python. I've tried asyncio, pyodbc and asyncpg to no avail. I think this is close... import asyncio import pyodbc async def query_azure_sql_db(): connection_string = 'Driver={ODBC Driver 17 for SQL Server};Server=tcp:<mySERVER>.database.windows.net,1433;Database=sqldbstockdata;Uid=<myUN>;Pwd=<myPW>;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;' async with pyodbc.connect(connection_string) as conn: async with conn.cursor() as cursor: query = 'SELECT * FROM dbo.<myTABLE>' await cursor.execute(query) results = cursor.fetchall() return results loop = asyncio.get_event_loop() results = loop.run_until_complete(query_azure_sql_db()) print(results)  But results in this cryptic error: AttributeError: __aenter__ I'm open to other libraries. Any help is appreciated.

Answer: The problem is that pyodbc.connect is a function and does not implement async context manager, so why you are getting the AttributeError. I recommend using aioodbc instead of pyodbc as it provides the same functionality as async methods. You can also make it work with async with by implementing the __aenter__ and __aexit__ dunder methods. import asyncio import aioodbc class AsyncPyodbc: def __init__(self, dsn, autocommit=False, ansi=False, timeout=0, loop=None, executor=None, echo=False, after_created=None, **kwargs): self._connection: aioodbc.Connection = aioodbc.connect(dsn=dsn, autocommit=autocommit, ansi=ansi, loop=loop, executor=executor, echo=echo, timeout=timeout, after_created=after_created, **kwargs) async def __aenter__(self) -> aioodbc.Connection: return await self._connection async def __aexit__(self, *_): return await self._connection.close()  This will require an event loop and will be used as follows. async def query_azure_sql_db(): connection_string = ( 'Driver={ODBC Driver 17 for SQL Server};' 'Server=tcp:<mySERVER>.database.windows.net,1433;' 'Database=sqldbstockdata;Uid=<myUN>;Pwd=<myPW>;' 'Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;' ) loop = asyncio.get_event_loop() async with AsyncPyodbc(connection_string, loop=loop) as conn: ...
Question: I have a repository on Github for a private Python package for my organization, and I would like to publish this package in a private Azure Artifacts feed using Github actions for automated CI/CD. It seems like none of the documentation around publishing to Azure Artifacts is for this exact scenario; the documentation from ADO is solely for using Azure Pipelines, or seems to require manual action (for instance, trying to use artifacts-keyring). Is there some way to provide twine with credentials in a Github Actions YAML file that would enable me to publish to my private ADO?

Answer: You can save the authentication information in the project file of Repository, and then run Publish Python packages from the command line in your Github Actions.
Question: I have this simple cache decorator demo here @functools.cache def cached_fib(n): assert n > 0 if n <= 2: return 1 return cached_fib(n - 1) + cached_fib(n - 2) t1 = time.perf_counter() cached_fib(400) t2 = time.perf_counter() print(f"cached_fib: {t2 - t1}") # 0.0004117000003134308  I want to access the actual cache dictionary object inside this cached_fib, but when I try to access through cached_fib.cache, it gives me an error saying that AttributeError: 'functools._lru_cache_wrapper' object has no attribute 'cache' though the cache attribute is used in python and c version alike. Thank you!

Answer: The internals of the cache are encapsulated for thread safety and to allow the underlying implementation details to change. The three public attributes are, some statistics in cache_info: >>> cached_fib.cache_info() CacheInfo(hits=0, misses=0, maxsize=None, currsize=0) >>> cached_fib(123) 22698374052006863956975682 >>> cached_fib.cache_info() CacheInfo(hits=120, misses=123, maxsize=None, currsize=123)  Details about the init mode in cache_parameters >>> cached_fib.cache_parameters() {'maxsize': None, 'typed': False}  And cache_clear which purges the cache. >>> cached_fib.cache_info() CacheInfo(hits=120, misses=123, maxsize=None, currsize=123) >>> cached_fib.cache_clear() >>> cached_fib.cache_info() CacheInfo(hits=0, misses=0, maxsize=None, currsize=0)  Additionally, the "uncached" function version is publicly available via __wrapped__, but this is not something specific to functools.cache: >>> cached_fib <functools._lru_cache_wrapper object at 0x10b7a3270> >>> cached_fib.__wrapped__ <function cached_fib at 0x10bb781f0>  If you want a cache implementation where the underlying dict is exposed to the user, I recommend the third-party cachetools library.
Question: When running ipynbs in VS Code, I've started noticing Pylance warnings on standard library imports. I am using a conda virtual environment, and I believe the warning is related to that. An example using the glob library reads:  "env\Lib\glob.py" is overriding the stdlib "glob" modulePylance(reportShadowedImports) So far my notebooks run as expected, but I am curious if this warning is indicative of poor layout or is just stating the obvious more of an "FYI you are not using the base install of python". I have turned off linting and the problem stills persists. And almost nothing returns from my searches of the error "reportShadowedImports".

Answer: The reason you find nothing by searching is because this check has just been implemented recently (see Github). I ran into the same problem as you because code.py from Micropython/Circuitpython also overrides the module "code" in stdlib. The solution is simple, though you then loose out on this specific check. Just add reportShadowedImports to your pyright config. For VS Code, that would be adding it to .vscode/settings.json: { "python.languageServer": "Pylance", [...] "python.analysis.diagnosticSeverityOverrides": { "reportShadowedImports": "none" }, [...] }
Question: I was in class section of python programming and I am confused here. I have learned that super is used to call the method of parent class but here Employee is not a parent of Programmer yet it's called (showing the result of getLanguage method). What I am missing? This is the code, class Employee: company= "Google" language = "java" def showDetails(self): print("This is an employee"); def getLanguage(self): print(f"1. The language is {self.language}"); class Programmer: language= "Python" company = "Youtubeeee" def getLanguage(self): super().getLanguage(); print(f"2. The language is {self.language}") def showDetails(self): print("This is an programmer") class Programmer2(Programmer , Employee): language= "C++" def getLanguage(self): super().getLanguage(); print(f"3. The language is {self.language}") p2 = Programmer2(); p2.getLanguage();  This is the output, 1. The language is C++ 2. The language is C++ 3. The language is C++ 

Answer: You've bumped into one of the reasons why super exists. From the docs, super delegates method calls to a parent or sibling class of type. Python bases class inheritance on a dynamic Method Resolution Order (MRO). When you created a class with multiple inheritance, those two parent classes became siblings. The left most is first in MRO and the right one is next. This isn't a property of the Programmer class, Its a property of the Programmer2 class that decided to do multiple inheritance. If you use Programmer differently, as in, p3 = Programmer() p3.getLanguage()  You get the error AttributeError: 'super' object has no attribute 'getLanguage' because its MRO only goes to the base object which doesn't have the method. You can view the MRO of the class with its __mro__ attribute. Programmer.__mro__: (<class '__main__.Programmer'>, <class 'object'>) Programmer2.__mro__: (<class '__main__.Programmer2'>, <class '__main__.Programmer'>, <class '__main__.Employee'>, <class 'object'>)
Question: I'm trying to get full article in Google Sheet using Openai API. In column A I just mention the topic and want to get full article in column B. Here is what I'm trying  /** * Use GPT-3 to generate an article * * @param {string} topic - the topic for the article * @return {string} the generated article * @customfunction */ function getArticle(topic) { // specify the API endpoint and API key const api_endpoint = 'https://api.openai.com/v1/completions'; const api_key = 'YOUR_API_KEY'; // specify the API parameters const api_params = { prompt: topic, max_tokens: 1024, temperature: 0.7, model: 'text-davinci-003', }; // make the API request using UrlFetchApp const response = UrlFetchApp.fetch(api_endpoint, { method: 'post', headers: { Authorization: 'Bearer ' + api_key, 'Content-Type': 'application/json', }, payload: JSON.stringify(api_params), }); // retrieve the article from the API response const json = JSON.parse(response.getContentText()); if (json.data && json.data.length > 0) { const article = json.data[0].text; return article; } else { return 'No article found for the given topic.'; } }  How can I get the article?

Answer: Modification points:  When I saw the official document of OpenAI API, in your endpoint of https://api.openai.com/v1/completions, it seems that the following value is returned. Ref  { "id": "cmpl-uqkvlQyYK7bGYrRHQ0eXlWi7", "object": "text_completion", "created": 1589478378, "model": "text-davinci-003", "choices": [ { "text": "\n\nThis is indeed a test", "index": 0, "logprobs": null, "finish_reason": "length" } ], "usage": { "prompt_tokens": 5, "completion_tokens": 7, "total_tokens": 12 } }   In the case of json.data, it seems that the endpoint of https://api.openai.com/v1/models might be required to be used. Ref And, there is no property of json.data[0].text.   I thought that this might be the reason for your current issue. If you want to retrieve the values of text from the endpoint of https://api.openai.com/v1/completions, how about the following modification? From: if (json.data && json.data.length > 0) { const article = json.data[0].text; return article; } else { return 'No article found for the given topic.'; }  To: if (json.choices && json.choices.length > 0) { const article = json.choices[0].text; return article; } else { return 'No article found for the given topic.'; }  Note:  If the value of response.getContentText() is not your expected values, this modification might not be able to be used. Please be careful about this.  Reference:  Completions of OpenAI API
Question: I'm having a hard time figuring out how to pass a function's return as a parameter to another function. I've searched a lot of threads that are deviations of this problem but I can't think of a solution from them. My code isn't good yet, but I just need help on the line where the error is occurring to start with. Instructions:  create a function that asks the user to enter their birthday and returns a date object. Validate user input as well. This function must NOT take any parameters. create another function that takes the date object as a parameter. Calculate the age of the user using their birth year and the current year.  def func1(): bd = input("When is your birthday? ") try: dt.datetime.strptime(bd, "%m/%d/%Y") except ValueError as e: print("There is a ValueError. Please format as MM/DD/YYY") except Exception as e: print(e) return bd def func2(bd): today = dt.datetime.today() age = today.year - bd.year return age  This is the Error I get: TypeError: func2() missing 1 required positional argument: 'bday'  So far, I've tried:  assigning the func1 to a variable and passing the variable as func2 parameter calling func1 inside func2 defining func1 inside func2 

Answer: You're almost there, a few subtleties to consider:  The datetime object must be assigned to a variable and returned. Your code was not assigning the datetime object, but returning a str object for input into func2. Which would have thrown an attribute error as a str has no year attribute. Simply subtracting the years will not always give the age. What if the individual's date of birth has not yet come? In this case, 1 must be subtracted. (Notice the code update below).  For example: from datetime import datetime as dt def func1(): bday = input("When is your birthday? Enter as MM/DD/YYYY: ") try: # Assign the datetime object. dte = dt.strptime(bday, "%m/%d/%Y") except ValueError as e: print("There is a ValueError. Please format as MM/DD/YYYY") except Exception as e: print(e) return dte # <-- Return the datetime, not a string. def func2(bdate): today = dt.today() # Account for the date of birth not yet arriving. age = today.year - bdate.year - ((today.month, today.day) < (bdate.month, bdate.day)) return age  Can be called using: func2(bdate=func1())
Question: I am getting started with tatsu and I am trying to implement a grammar for the miniML language. Once my grammar successfully parsed, I tried to parse some little expressions to check that it was working ; however I discovered Tatsu was unable to recognize some of the expected patterns. Here is the code : ` grammar=""" @@grammar::CALC start = expression $ ; expression = |integer |addition |soustraction |multiplication |division |Fst |Snd |pair |varname |assign |function |application |parentheses ; integer = /\d+/ ; addition = left:'+' right:pair ; soustraction = '-' pair ; multiplication = '*' pair ; division = '/' pair ; Fst = 'Fst' pair ; Snd = 'Snd' pair ; pair = '(' expression ',' expression ')' ; varname = /[a-z]+/ ; assign = varname '=' expression ';' expression ; function = 'Lambda' varname ':' expression ; application = ' '<{expression}+ ; parentheses = '(' expression ')' ; """  ` then parsed : parser = tatsu.compile(grammar) All of those expression are successfully recognized, except the "assign" and the "application" ones. If i try something like this : parser.parse("x=3;x+1")  I get that error message : FailedExpectingEndOfText: (1:2) Expecting end of text : x=3;x+1 ^ start  and same goes for an expression of the type "expression expression". What could be the syntax error I made here ? I have no clue and I can't find any in the documentation. Thanks in advance !

Answer:  It seems the failure of assign comes from a conflict with the varname rule; to solve it, simply place |assign BEFORE |variable in your expression rule.  A now obsolete workaround, that I'll leave anyway: # I added a negative lookahead for '=' so it will not conflict with the assign rule varname = /[a-z]+/!'=' ; assign = /[a-z]+/ '=' expression ';' expression ;  Example: parser.parse("x=1;+(x,1)") # ['x', '=', '1', ';', AST({'left': '+', 'right': ['(', 'x', ',', '1', ')']})]   About 'application' : replacing ' ' with / / at the start of the rule, and placing |application at the start of the expression rule solves the problem:  parser.parse("1 2 (x=1;3) *(4,5)") Out[207]: (' ', '1', (' ', '2', (' ', ['(', ['x', '=', '1', ';', '3'], ')'], ['*', ['(', '4', ',', '5', ')']])))
Question: When computing A @ a where A is a random N by N matrix and a is a vector with N random elements using numpy the computation time jumps by an order of magnitude at N=100. Is there any particular reason for this? As a comparison the same operation using torch on the cpu has a more gradual increase Tried it with python3.10 and 3.9 and 3.7 with the same behavior Code used for generating numpy part of the plot: import numpy as np from tqdm.notebook import tqdm import pandas as pd import time import sys def sym(A): return .5 * (A + A.T) results = [] for n in tqdm(range(2, 500)): for trial_idx in range(10): A = sym(np.random.randn(n, n)) a = np.random.randn(n) t = time.time() for i in range(1000): A @ a t = time.time() - t results.append({ 'n': n, 'time': t, 'method': 'numpy', }) results = pd.DataFrame(results) from matplotlib import pyplot as plt fig, ax = plt.subplots(1, 1) ax.semilogy(results.n.unique(), results.groupby('n').time.mean(), label="numpy") ax.set_title(f'A @ a timimgs (1000 times)\nPython {sys.version.split(" ")[0]}') ax.legend() ax.set_xlabel('n') ax.set_ylabel('avg. time')  Update Adding import os os.environ["MKL_NUM_THREADS"] = "1" os.environ["NUMEXPR_NUM_THREADS"] = "1" os.environ["OMP_NUM_THREADS"] = "1"  before ìmport numpy gives a more expected output, see this answer for details: https://stackoverflow.com/a/74662135/5043576 

Answer: numpy tries to use threads when multiplying matricies of size 100 or larger, and the default CBLAS implementation of threaded multiplication is ... sub optimal, as opposed to other backends like intel-MKL or ATLAS. if you force it to use only 1 thread using the answers in this post you will get a continuous line for numpy performance.
Question: Is there a way to assess whether a case statement variable is inside a particular list? Consider the following scenario. We have three lists: a = [1, 2, 3] b = [4, 5, 6] c = [7, 8, 9]  Then I want to check whether x is in each list. Something like this (of course this is a Syntax Error but I hope you get the point): match x: case in a: return "132" case in b: return "564" case in c: return "798"  This can be easy with an if-else scenario. Nonetheless, focusing on the match-case, if one has many lists. And big lists, it would be a mundane task to write them like that: match x: case 1 | 2 | 3: return "132" case 4 | 5 | 6: return "564" case 7 | 8 | 9: return "762"  Is there an easy way to check for multiple conditions for each case, without having to write them down? I checked for duplicates, but I couldn't find them, I hope I don't miss something. Please be kind and let me know if there is a duplicate question.

Answer: As it seems cases accept a "guard" clause starting with Python 3.10, which you can use for this purpose: match x: case w if w in a: # this was the "case in a" in the question case w if w in b: # this was the "case in b" in the question ...  the w here actually captures the value of x, part of the syntax here too, but it's more useful in some other fancy cases listed on the linked whatsnew page.
Question: I want to be able to mock a function that mutates an argument, and that it's mutation is relevant in order for the code to continue executing correctly. Consider the following code: def mutate_my_dict(mutable_dict): if os.path.exists("a.txt"): mutable_dict["new_key"] = "new_value" return True def function_under_test(): my_dict = {"key": "value"} if mutate_my_dict(my_dict): return my_dict["new_key"] return "No Key" def test_function_under_test(): with patch("stack_over_flow.mutate_my_dict") as mutate_my_dict_mock: mutate_my_dict_mock.return_value = True result = function_under_test() assert result == "new_value"  **Please understand i know i can just mock os.path.exists in this case but this is just an example. I intentionally want to mock the function and not the external module. ** I also read the docs here: https://docs.python.org/3/library/unittest.mock-examples.html#coping-with-mutable-arguments But it doesn't seem to fit in my case. This is the test i've written so far, but it obviously doesn't work since the key changes: def test_function_under_test(): with patch("stack_over_flow.mutate_my_dict") as mutate_my_dict_mock: mutate_my_dict_mock.return_value = True result = function_under_test() assert result == "new_value"  Thanks in advance for all of your time :)

Answer: With the help of Peter i managed to come up with this final test: def mock_mutate_my_dict(my_dict): my_dict["new_key"] = "new_value" return True def test_function_under_test(): with patch("stack_over_flow.mutate_my_dict") as mutate_my_dict_mock: mutate_my_dict_mock.side_effect = mock_mutate_my_dict result = function_under_test() assert result == "new_value"  How it works is that with a side effect you can run a function instead of the intended function. In this function you need to both change all of the mutating arguments and return the value returned.
Question: I've been looking all over the web for the current standards for Python Docstrings and I've come across different answers for different scenarios. What is the currently most-accepted and wide-spread docstring format that I should use? These are the ones that I've found so far: Sphinx format (1): :param type name: description Sphinx format (2): :py:param type name: description NumPy format: Parameters: __________ param: description Other formats: Args: param (type): description Parameters: param (type): description I just want to document my code in a standard way that is accepted by almost every IDE (including VS Code and PyCharm) that also conforms to PEP and readthedocs, so I can also enable hover-over with mouse over the code to see description of the arguments. I'm looking for current standards that are at least backwards compatible with Python 3.6 a since that's the base of the projects I work on.

Answer: The most widely accepted and standardized format for Python docstrings is the one defined in the PEP 257 - Docstring Conventions. This format is supported by most IDEs, including VS Code and PyCharm, and is also used by the Sphinx and NumPy documentation tools. The PEP 257 format for documenting function parameters is as follows: def function_name(param1: type, param2: type) -> return_type: """ Description of the function and its arguments. Parameters: param1 (type): Description of the first parameter. param2 (type): Description of the second parameter. Returns: return_type: Description of the return value. """  This format is compatible with Python 3.6 and later versions, and is also backward compatible with older versions of Python. It is recommended to follow this format for documenting function arguments in your code to ensure consistency and compatibility with different tools and IDEs.