eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I have instructed Docker to inject an environment variable called "INJECT_THIS" using two different methodologies (env.list file via 'docker run', and Dockerfile). My Dockerfile's RUN command invokes a Python file (run.py), but the environment variable is not seen by the Python code's os.environ object: environ({'OLDPWD': '/', 'PATH': '/command:/lsiopy/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'PWD': '/app', 'LC_CTYPE': 'C.UTF-8'})  I've created a reproducible example on Github, but will also paste file contents in this question. Dockerfile FROM linuxserver/blender WORKDIR /app ENV INJECT_THIS = "inject this" COPY run.py ./ RUN apt-get update && apt-get -y install python3-pip CMD ["python3", "-u", "run.py"]  run.py import os # Notice that the injected environment variable "INJECT_THIS" is not present. print('os.environ', os.environ)  env.list INJECT_THIS="inject this" 

Answer: When you do docker inspect <your image name> you will see, there's ENTRYPOINT specified. This entrypoint messes with environment variables. You can specify your ENTRYPOINT instead of CMD (note I removed the spaces in ENV): FROM linuxserver/blender WORKDIR /app ENV INJECT_THIS="inject this" COPY run.py ./ RUN apt-get update && apt-get -y install python3-pip ENTRYPOINT ["python3", "-u", "run.py"]  Then run the docker image: $ docker run -t <your image name>  Prints: os.environ environ({'PATH': '/lsiopy/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': 'bb98d241500d', 'TERM': 'xterm', 'HOME': '/config', 'LANGUAGE': 'en_US.UTF-8', 'LANG': 'en_US.UTF-8', 'S6_CMD_WAIT_FOR_SERVICES_MAXTIME': '0', 'S6_VERBOSITY': '1', 'S6_STAGE2_HOOK': '/docker-mods', 'VIRTUAL_ENV': '/lsiopy', 'DISPLAY': ':1', 'PERL5LIB': '/usr/local/bin', 'OMP_WAIT_POLICY': 'PASSIVE', 'GOMP_SPINCOUNT': '0', 'START_DOCKER': 'true', 'PULSE_RUNTIME_PATH': '/defaults', 'NVIDIA_DRIVER_CAPABILITIES': 'all', 'LSIO_FIRST_PARTY': 'true', 'TITLE': 'Blender', 'INJECT_THIS': 'inject this'})
Question: I'm using hunspell with the pyhunspell wrapper. I'm calling: hunspell.suggest("Yokk")  But this is returning only ["Yolk", "Yoke"]. I saw that "York" is in the dictionary but is not being returned. Is there a way to return more than 2 suggestions, either by increasing the distance threshold or the number of top suggestions? The text I'm trying to correct is "New York" and I have my own ranker that ranks the suggestions downstream. I just need more suggestions. I tried aspell and by default its returning 10 suggestions, one of which is in fact "York". Note: The documentation doesn't mention any other arguments for method suggest. Even using the CLI I only get two suggestions: hunspell -d en_US Hunspell 1.7.2 yokk & yokk 2 0: yolk, yoke  I've checked the default dictionaries are properly loaded using: hunspell -D SEARCH PATH: ... AVAILABLE DICTIONARIES (path is not mandatory for -d option): /Library/Spelling/en_US LOADED DICTIONARY: /Library/Spelling/en_US.aff /Library/Spelling/en_US.dic ➜ 2 subl /Library/Spelling/en_US.dic  And I've also checked that the expected "York" is in the dictionary: cat /Library/Spelling/en_US.dic | grep York York/M  I wonder if there is some other configuration I can set somewhere, I can't see anything evident in either the wrapper or the CLI documentation: https://github.com/pyhunspell/pyhunspell/wiki/Documentation https://github.com/hunspell/hunspell

Answer: I have installed 2020 dictionaries from this link: http://wordlist.aspell.net/dicts/ the newest dictionaries available here: https://github.com/LibreOffice/dictionaries Then tested the code: import hunspell #0.5.5 hobj = hunspell.HunSpell('./en_US.dic', './en_US.aff') tt = hobj.suggest("Yokk") print(tt)  And got the output that is different from yours: ['York', 'Yoko', 'Yolk', 'Yoke']
Question: I have a function that accepts certain literals for a specific argument: from typing import Literal def fn(x: Literal["foo", "bar", "foo|bar"]) -> None: reveal_type(x)  The third contains a pipe symbol (|), "foo|bar". This is interpreted by mypy as an error, as the name foo is not defined. I guess this happens due to how forward references are evaluated? I use Python 3.8 with: from __future__ import annotations  Is there a way to make this work? I can not change the string due to breaking backward compatibility, but currently, the whole annotation is revealed as Any, i.e. it holds no value.

Answer: This bug is now fixed. The change hasn't been released yet, however.
Question: I would like to know the steps involved in adding custom labels to a Vertex AI pipeline’s PipelineJobSchedule. Can anyone please provide me with the necessary guidance as it's not working when I am adding inside the Pipelinejob parameters? # https://cloud.google.com/vertex-ai/docs/pipelines/schedule-pipeline-run#create-a-schedule pipeline_job = aiplatform.PipelineJob( template_path="COMPILED_PIPELINE_PATH", pipeline_root="PIPELINE_ROOT_PATH", display_name="DISPLAY_NAME", labels="{"name":"test_xx"}" ) pipeline_job_schedule = aiplatform.PipelineJobSchedule( pipeline_job=pipeline_job, display_name="SCHEDULE_NAME" ) pipeline_job_schedule.create( cron="TZ=CRON", max_concurrent_run_count=MAX_CONCURRENT_RUN_COUNT, max_run_count=MAX_RUN_COUNT, ) 

Answer: There was a bug in the Vertex AI platform SDK which has been discussed in the Github issue ticket below. It has been fixed in SDK versions 1.37.0 (released on December 5th, 2023). GitHub Issue ticket  https://github.com/googleapis/python-aiplatform/issues/2929
Question: Newbie here. I have been trying to installen the openai library into python, but I keep running into problems. I have already installed C++ libraries. It seems to have problems specific with aio http, and I get the error below. I a running a Windows 11 laptop without admin restrictions. Error "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -IC:\Users\sande\AppData\Local\Programs\Python\Python312\include -IC:\Users\sande\AppData\Local\Programs\Python\Python312\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.37.32822\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" /Tcaiohttp/_websocket.c /Fobuild\temp.win-amd64-cpython-312\Release\aiohttp/_websocket.obj _websocket.c aiohttp/_websocket.c(1475): warning C4996: 'Py_OptimizeFlag': deprecated in 3.12 aiohttp/_websocket.c(3042): error C2039: 'ob_digit': is not a member of '_longobject' [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for aiohttp Failed to build aiohttp ERROR: Could not build wheels for aiohttp, which is required to install pyproject.toml-based projects 

Answer: Either use python 3.11 or pip install aiohttp==3.9.0b0   installs their current beta release that supports python 3.12.x  then try openai installation  Link to git :https://github.com/KillianLucas/open-interpreter/issues/581
Question: When installing a Python package locally in editable mode, both 'pip3 install -e .', and 'python3 setup.py develop', will both install the package locally in editable mode. I know that 'pip3 install -e .' is recommended over 'python3 setup.py develop', but I am unclear as to why and what the differences are between the two. This answer for a separate question (https://stackoverflow.com/a/19048754/8903959) seems to indicate that it's because it's more difficult to uninstall when using python3 setup.py develop. Why is that the case? Additionally, the answer (https://stackoverflow.com/a/19048754/8903959) says that dependencies are managed differently and incorrectly. What are the specific differences surrounding dependencies and why do they exist, especially if 'python3 setup.py develop' manages them incorrectly? Apologies if this is a trivial question, but try as I might I could not find the answer or documentation for this reasoning elsewhere.

Answer: At this point in time, trying to get an accurate description of the differences between python setup.py install and python -m pip install seems rather pointless. python setup.py develop and python setup.py install originate from a time before pip existed. The behavior of these commands has not been kept up to date with all the latest standards while pip's behavior has. setuptools' place and role in the Python packaging ecosystem has shifted, it is now simply a "build back-end". setup.py should now be considered as a configuration file only, not as an executable script. I recommend reading this article for a lot of historical and technical background on the topic: Why you shouldn't invoke setup.py directly. Also this article on the Python packaging user guide: Is setup.py deprecated?
Question: Let's say we have two files: to_patch.py from unittest.mock import patch def patch_a_function(): print("Patching!") patcher = patch("to_be_patched.function") patcher.start() print("Done patching!")  to_be_patched.py from to_patch import patch_a_function def function(): pass patch_a_function() function()  And we run python -m to_be_patched. This will output: Patching! Patching!   Why isn't Done patching! ever printed? Why is Patching! printed twice?  I've narrowed the answer to (2) down; the call to patch.start seems to trigger patch_a_function again. I suspect this is because it's imported in to_be_patched.py, but am not sure why the function itself would run for a second time. Similarly, I'm not sure why the Done patching! line is not reached in either of the calls to patch_a_function. patcher.start() can't be blocking, because the program exits nicely instead of hanging there... right? Edit: Huh. It looks like no one can reproduce Done patching! not being printed (which was honestly the main difficulty)—so I guess that's just a my-side problem

Answer:   Why isn't Done patching! ever printed?   Can not reproduce. $ python -m to_be_patched Patching! Patching! Done patching! Done patching!    Why is Patching! printed twice?   Your module gets imported twice. If you add print(__name__) into the file to_be_patched.py it will be clear: from to_patch import patch_a_function print(f"{__name__=}") def function(): pass patch_a_function() function() # note: this line doesn't actually do anything, and could be commented out  Result: $ python -m to_be_patched __name__='__main__' Patching! __name__='to_be_patched' Patching! Done patching! Done patching!  When you use python -m to_be_patched your module to_be_patched will be loaded as top-level code, i.e. the module __name__ will be "__main__". When mock.patch is used, mock will first import the patch target. When given a patch target as a string like "to_be_patched.function" mock will use importlib, via pkgutil.resolve_name, to find the correct namespace in which to patch. This method loads the target module with __name__ as "to_be_patched", it's not the top-level code environment. Although it's the same underlying .py file being loaded, there is a cache miss in sys.modules, because of the name mismatch: "__main__" != "to_be_patched". The function patch_a_function now has dual identities and exists in the module __main__ as well as the module to_be_patched, so what you're seeing is each one getting called. The first call triggers the second call, by the double-import mechanism described.
Question: Currently I am trying to concat three strings in my Ansible playbook. Two of them can be unset (None/null) and I need them to be separated by underscores. Here is an example: type_mode="A", level_mode=None, and what_to_run="C" combines to A_C. If both where None it would just be C, if all where set it would be A_B_C. My idea was to do this bit in Python: - name: "Set Name" set_fact: name: "{{ ('' if type_mode is None else type_mode + '_') + ('' if level_mode is None else level_mode + '_') + what_to_run }}"  But then I get this error: { "msg": "template error while templating string: Could not load \"None\": 'None'. String: {{ ('' if type_mode is None else type_mode + '_') + ('' if level_mode is None else level_mode + '_') + what_to_run }}. Could not load \"None\": 'None'", "_ansible_no_log": false }  Do you have a idea how I can concat the (possible missing) values with an underscore separator?

Answer:  Make a list out of your strings: [type_mode, level_mode, what_to_run]   Exclude the blank parts with the help of the select filter: [type_mode, level_mode, what_to_run] | select   Join the remaining chunks [type_mode, level_mode, what_to_run] | select | join('_')    Your set_fact task ends up being: - set_fact: name: "{{ [type_mode, level_mode, what_to_run] | select | join('_') }}"    If you also happen to have undefined variables, then you can add defaults to your strings: - set_fact: name: >- {{ [ type_mode | default(''), level_mode | default(''), what_to_run ] | select | join('_') }}    Given a playbook containing. only this task, running it with:  Only what_to_run ansible-playbook play.ym --verbose \ --extra-vars "what_to_run=foo" \ --extra-vars "level_mode=" \ --extra-vars "type_mode="  would give TASK [set_fact] ************************************************* ok: [localhost] => changed=false ansible_facts: var: foo   Both what_to_run and level_mode ansible-playbook play.ym --verbose \ --extra-vars "what_to_run=foo" \ --extra-vars "level_mode=bar" \ --extra-vars "type_mode="  would give TASK [set_fact] ************************************************* ok: [localhost] => changed=false ansible_facts: var: bar_foo   The three variables filled in ansible-playbook play.ym --verbose \ --extra-vars "what_to_run=foo" \ --extra-vars "level_mode=bar" \ --extra-vars "type_mode=baz"  would give TASK [set_fact] ************************************************* ok: [localhost] => changed=false ansible_facts: var: baz_bar_foo
Question: Code: from seleniumbase import Driver driver = Driver(uc=True) driver.get("https://example.com") driver.click("a") p_text = driver.find_element("p").text print(p_text)  this code works fine but i want to add a user profile but when i try from seleniumbase import Driver ud = r"C:\Users\USER\AppData\Local\Google\Chrome\User Data\Profile 9" driver = Driver(uc=True, user_data_dir=ud) driver.get("https://example.com") driver.click("a") p_text = driver.find_element("p").text print(p_text)  this makes a profile called person 1 that works like a normal user and has everything saved but what if i want to access a specific profile? edit: it goes to the path i give it but appends a \Default so it goes to the default profile of that path so: C:\Users\USER\AppData\Local\Google\Chrome\User Data\Profile 9 would be C:\Users\USER\AppData\Local\Google\Chrome\User Data\Profile 9\Default Command Line "C:\Program Files\Google\Chrome\Application\chrome.exe" --window-size=1280,840 --disable-dev-shm-usage --disable-application-cache --disable-browser-side-navigation --disable-save-password-bubble --disable-single-click-autofill --allow-file-access-from-files --disable-prompt-on-repost --dns-prefetch-disable --disable-translate --disable-renderer-backgrounding --disable-backgrounding-occluded-windows --disable-features=OptimizationHintsFetching,OptimizationTargetPrediction --disable-popup-blocking --homepage=chrome://new-tab-page/ --remote-debugging-host=127.0.0.1 --remote-debugging-port=53654 --user-data-dir="C:\Users\fun64\AppData\Local\Google\Chrome\User Data\Profile 9" --lang=en-US --no-default-browser-check --no-first-run --no-service-autorun --password-store=basic --log-level=0 --flag-switches-begin --flag-switches-end --origin-trial-disabled-features=WebGPU Executable Path C:\Program Files\Google\Chrome\Application\chrome.exe Profile Path C:\Users\fun64\AppData\Local\Google\Chrome\User Data\Profile 9\Default

Answer: Partially due to reasons outlined in https://stackoverflow.com/a/67960202/7058266, the best way to handle multiple profiles is to have a unique user-data-dir for each one. Also, since you're using SeleniumBase UC Mode, for reasons outlined in https://www.youtube.com/watch?v=5dMFI3e85ig at the 21-minute mark, don't mix a non-UC-Mode user-data-dir with a UC Mode user-data-dir, because you could get detected that way. Use a user-data-dir that gets created by UC Mode: from seleniumbase import Driver driver_1 = Driver(uc=True, user_data_dir="my_user_dir_1") driver_2 = Driver(uc=True, user_data_dir="my_user_dir_2") # ... driver_1.quit() driver_2.quit()  Longer example: from seleniumbase import Driver driver = Driver(uc=True, user_data_dir="my_user_dir_1") try: driver.get("https://example.com") driver.click("a") print(driver.get_text("p")) finally: driver.quit()
Question: df = pd.DataFrame({ 'subsegment': ['corp', np.nan, 'terr'], 'region': ['japan', np.nan, np.nan], 'subregion': [np.nan, 'se', 'ne'], 'segment': [np.nan,'ent','comm'] })  I am trying to iterate through the above dataframe and if the value is not NaN than adding the column header as the value or part of the value (depending on how many NaNs) in the new column "Mode". Original DF     subsegment region subregion segment     corp japan NaN NaN   NaN NaN se ent   terr NaN ne comm     Desired Output DF     subsegment region subregion segment mode     corp japan NaN NaN subsegment-region   NaN NaN se ent subregion-segment   terr NaN ne comm subsegment-subregion-segment     I have tried to create separate smaller dfs with all the combinations of the columns to which are not null and then concatenating those dfs together but this seems extremely inefficient. df1 = df.loc[~(df['subsegment'].isna()) & (~df['region'].isna()) & (~df['region'].isna())] df2 = df.loc[~(df['region'].isna()) & (~df['subregion'].isna()) & (~df['segment'].isna())] df3 = df.loc[~(df['subsegment'].isna()) & (~df['subregion'].isna()) & (~df['segment'].isna())] pd.concat(df1,df2,df3.....) 

Answer: You can use a dot product: df['mode'] = (df.notna() @ (df.columns+'-')).str[:-1]  Output:  subsegment region subregion segment mode 0 corp japan NaN NaN subsegment-region 1 NaN NaN se ent subregion-segment 2 terr NaN ne comm subsegment-subregion-segment  Alternatively, with a classical groupby.agg: s = df.notna().stack() df['mode'] = s[s].reset_index().groupby('level_0')['level_1'].agg('-'.join)  Or a custom aggregation: df['mode'] = df.notna().mul(df.columns).agg(lambda x: '-'.join(x[x.ne('')]), axis=1)
Question: Hi I want to select those cols of a polars df that are of the dtype list. Selecting by dtypes works ususally fine with df.select(pl.col(pl.Utf8)). However for the type list this does not seem to work... MRE import polars as pl df = pl.DataFrame({"foo": [[c] for c in ["100CT pen", "pencils 250CT", "what 125CT soever", "this is a thing"]]} ) df  Output: foo list[str] ["100CT pen"] ["pencils 250CT"] ["what 125CT soever"] ["this is a thing"]   df.select(pl.col(pl.List))  Output: shape: (0, 0) 

Answer: You need to provide the type of the items in the List unlike primitive types (where print(df.select(pl.col(pl.Int64))) would work in the below example). import polars as pl df = pl.DataFrame({ "foo": [[c] for c in ["100CT pen", "pencils 250CT", "what 125CT soever", "this is a thing"]], "bar": [1, 2, 3, 4] } ) print(df.select(pl.col(pl.List(str))))  I can't seem to find anything that's generic across types that the List contains. There is a NESTED_DTYPES here and this answer suggests that you might be able to use it in a more "catch-all" manner, but it doesn't seem to work if you want to grab columns that contain a nested type regardless of the type of data it contains.  Thanks to @jqurious for pointing out that this seems to be a requested feature in an open ticket. This has an interesting use-case for me in that, the only reason I've switched dfs back to pandas recently is that polars refuses to write List to CSV so I either filter out all such columns by name or, if this is implemented, I could drop them in one go. I didn't create those columns and I don't want them in the output.
Question: I am using ParentDocumentRetriever of langchain. Using mostly the code from their webpage I managed to create an instance of ParentDocumentRetriever using bge_large embeddings, NLTK text splitter and chromadb. I added documents to it, so that I c embedding_function = HuggingFaceEmbeddings(model_name='BAAI/bge-large-en-v1.5', cache_folder=hf_embed_path) # This text splitter is used to create the child documents child_splitter = NLTKTextSplitter(chunk_size=400) # The vectorstore to use to index the child chunks vectorstore = Chroma( collection_name="full_documents", embedding_function=embedding_function, persist_directory="./chroma_db_child" ) # The storage layer for the parent documents store = InMemoryStore() retriever = ParentDocumentRetriever( vectorstore=vectorstore, docstore=store, child_splitter=child_splitter, ) retriever.add_documents(docs, ids=None)  I added documents to it, so that I can query using the small chunks to match but to return the full document: matching_docs = retriever.get_relevant_documents(query_text) Chromadb collection 'full_documents' was stored in /chroma_db_child. I can read the collection and query it. I get back the chunks, which is what is expected: vector_db = Chroma( collection_name="full_documents", embedding_function=embedding_function, persist_directory="./chroma_db_child" ) matching_doc = vector_db.max_marginal_relevance_search('whatever', 3) len(matching_doc) >>3  One thing I can't figure out is how to persist the whole structure. This code uses store = InMemoryStore(), which means that once I stopped execution, it goes away. Is there a way, perhaps using something else instead of InMemoryStore(), to create ParentDocumentRetriever and persist both full documents and the chunks, so that I can restore them later without having to go through retriever.add_documents(docs, ids=None)  step?

Answer: I had the same problem and found the solution here: https://github.com/langchain-ai/langchain/issues/9345 You need to use the create_kv_docstore() function like this: from langchain.storage._lc_store import create_kv_docstore fs = LocalFileStore("./store_location") store = create_kv_docstore(fs) parent_splitter = RecursiveCharacterTextSplitter(chunk_size=2000) child_splitter = RecursiveCharacterTextSplitter(chunk_size=400) vectorstore = Chroma(collection_name="split_parents", embedding_function=embeddings, persist_directory="./db") retriever = ParentDocumentRetriever( vectorstore=vectorstore, docstore=store, child_splitter=child_splitter, parent_splitter=parent_splitter, ) retriever.add_documents(documents, ids=None)  You will end up with 2 folders: the chroma db "db" with the child chunks and the "data" folder with the parents documents. I think there is also a possibility of saving the documents in a Redis db or Azure blobstorage (https://python.langchain.com/docs/integrations/document_loaders/azure_blob_storage_container) but I am not sure.