eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: Suppose for a moment that I have 4 variables a,b,x,y and one constraint min(a,b) > min(x,y). how can I represent this program in pulp python?

Answer: Ok. So, the first answer I posted (deleted) was a bit hasty and the logic was faulty for the relationship described. This is (hopefully) correct! ;) max() and min() are nonlinear, so we need to linearize them somehow (with helper variable) and some logic to relate the 2 minima, which (below) can use a binary helper variable and a Big-M constraint. in pseudocode: a, b, x, y : real-valued variables ab_min : real-valued variable x_lt_y : binary variable, 1 implies x <= y, 0 else M = some suitably large constant, depending on the max range of a, b, x, y  new constraints: ab_min <= a ab_min <= b ab_min >= x - (1 - x_lt_y) * M ab_min >= y - (x_lt_y) * M  Logic:  We find the minimum of a, b with ab_min. We need "upward pressure" from the min(x, y)... So we know that ab_min must be greater than either x or y, or possibly both. For the "or" constraint, we use the binary logic above and multiply it by the "large constant" to make the other constraint trivial.
Question: def func1(): return 5 def func2(param1, param2): var1 = func1() return param1 + param2 + var1  I want to use pytest to test the second function by mocking the first, but I am not sure how to do this. @pytest.fixture(autouse=True) def patch_func1(self): with mock.patch( "func1", return_value= 5, ) as self.mock_func1: yield  I think it can be done with a dependency injection and fixture as above but that would mean changing func1 which I would prefer not to do.

Answer: You don't need to change anything. You can use mocker fixture with pytest (requires installation of pytest-mock). don't worry about the mocker argument, it will magically work. def test_func2(mocker): mocked_value = 4 first = 1 second = 2 func1_mock = mocker.patch("func1") func1_mock.return_value = mocked_value actual_value = func2(first, second) assert actual_value == first + second + mocked_value
Question: I'm working with enum lately and I dont really get the utility of them in some cases. I hope my question is not too trivial or too stupid, and I would really love to better understand the logic behind this python structure. One common use I found online or in some pieces of code I have been working on lately is the use of values that are strings like for example: from enum import Enum class Days(Enum): MONDAY = 'monday' TUESDAY = 'tuesday' ... SUNDAY = 'sunday'  And here from my humble prospective, the values seems redundant: if I print the values of some member I obtain the following: print(Days.MONDAY.value) >> 'monday'  I totally understand the utility when the values are numbers and they represent a gerarchic structure like for example class Levels(Enum): HIGH = 10 MID = 5 LOW = 0  In which you can do stuff like: HIGH > LOW > True  But in a lot of example and actual code use, I see the first approach, the one with MONDAY = 'monday', i.e. when the values are string instead of numerical values, and this case I really dont understand the utility of having a key that is pretty much equal to the value. If anyone can help me understand, or show some utilities I would really love to understand new stuff.

Answer: The members (should) always be named in all uppercase (per the very first note on the enum docs), so if you want them to have "names" that are in some other casing, you can assign strings with arbitrary case, which may be more human-friendly when it comes time to display the values to the user. You can also convert from said values to the enum constant easily, with the Enum's constructor, e.g.: Days('monday') is Days.MONDAY # This is True  so if the data from the user (or database or whatever) has specific values, you can easily convert them to their logical Enum equivalents this way. If the values really aren't meaningful, you can just assign auto() to all of them and not think about the values. Just in case you're asking "why not use the strings themselves?", the general advantage to enums is guaranteed uniqueness and exhaustiveness for efficient checks and self-documenting code. If you use the strings directly, you have to use == checks (str has no guarantee that equal values are the same object unless you explicitly intern them, so is checks can't be used), and people can pass in strings that don't actually come from the expected set of strings. With Enums there is a central definition of all possible values (and therefore all other values are not possible), and since the values are all guaranteed singletons, when you have a member of that Enum, you can use is/is not testing for cheap identity testing (in CPython at least, it's literally just a pointer comparison) without relying on value equality tests, ==/!=, that invoke the more expensive rich comparison machinery. This even works when you make aliases for the same enum member, e.g.: class Foo(Enum): SPAM = 1 EGGS = 2 ALSO_SPAM = 1  which seamlessly makes Foo.ALSO_SPAM the same object as Foo.SPAM so Foo.SPAM is Foo.ALSO_SPAM is true, allowing two aliases with different names to be used interchangably.
Question: I have a big dataframe (120000x40) and I try to find duplicates in every row and display them. Thats what I tried: create dataframe import pandas as pd df = pd.DataFrame({'col1':['1-233','2-766g','6-455','4-356','5-253','2-122','5-531','8-345','1-505','3-127','3-622'], 'col2':['6-998','2-766g','5-955','7-236','5-253','7-258','8-987t','7-567','1-505','6-876','NaN'], 'col3':['3-957','NaN','NaN','3-602m','1-266','2-122','7-834','8-345','2-858','7-984g', 'NaN']}) ## code df["duplicate"] = df.apply(lambda x: len(set(x[x.notna()])) != len(x[x.notna()]), axis=1) print(df)  #output:  But what I wanted was the folling:  To see what exactly is doubled here.

Answer: To keep the function readable and general, so it works for more or less than three cols, I'd just rely on writing a dedicated function that uses pandas built in functionality for finding duplicates, and applying that to the dataframe rows: import numpy as np import pandas as pd df = pd.DataFrame({'col1':['1-233','2-766g','6-455','4-356','5-253','2-122','5-531','8- 345','1-505','3-127','3-622'], 'col2':['6-998','2-766g','5-955','7-236','5-253','7-258','8-987t','7-567','1-505','6-876','NaN'], 'col3':['3-957','NaN','NaN','3-602m','1-266','2-122','7-834','8-345','2-858','7-984g', 'NaN']}) def get_duplicate_value(row): """If row has duplicates, return that value, else NaN.""" duplicate_locations = row.duplicated() if duplicate_locations.any(): dup_index = duplicate_locations.idxmax() return row[dup_index] return np.NaN df["solution"] = df.apply(get_duplicate_value, axis=1)  Check out the docs of pd.Dataframe.apply, pd.Series.duplicated, pd.Series.any and pd.Series.idxmax to figure out how this works exactly. Output:  col1 col2 col3 solution 0 1-233 6-998 3-957 NaN 1 2-766g 2-766g NaN 2-766g 2 6-455 5-955 NaN NaN 3 4-356 7-236 3-602m NaN 4 5-253 5-253 1-266 5-253 5 2-122 7-258 2-122 2-122 6 5-531 8-987t 7-834 NaN 7 8- 345 7-567 8-345 NaN 8 1-505 1-505 2-858 1-505 9 3-127 6-876 7-984g NaN 10 3-622 NaN NaN NaN
Question: I have created a simple c-function (using the guide here Create Numpy ufunc) and I'm now trying to distribute my package on pypi. For it to work, it needs to compile the c-file(s) into a .so -file, which then can be imported from python and everything is good. To compile it needs the header file numpy/ndarraytypes.h from Numpy. When building and installing locally, it works fine. This since I know where the header files are located. However when distributing it, where can we find the Numpy folder? It is obvious from the logs that numpy gets installed before my package is built and installed so I just need to include the correct Numpy folder. from setuptools import setup from setuptools import Extension if __name__ == "__main__": setup( name="myPack", install_requires=[ "numpy>=1.22.3", # <- where is this one located after it gets installed? ], ext_modules=[ Extension( 'npufunc', sources=['npufunc.c'], include_dirs=[ # what to put here for it to find the "numpy/ndarraytypes.h" header file when installing? "/usr/local/Cellar/numpy/1.22.3_1/lib/python3.9/site-packages/numpy/core/include/" # <- this one I have locally and when added, installtion works fine ] ), ] ) 

Answer: You can query numpy for the include directory from python via import numpy numpy.get_include()  This should return a string (/usr/lib/python3.10/site-packages/numpy/core/include on my system) which you can add to the include_dirs. See here for the docs. As for you question: numpy is a build dependency for you project. The old setup.py method is kind of bad at handling those, which is why it has been superseded by the more modern pyproject.toml approach, which (among other things) makes such specifications possible. A fairly minimal pyproject.toml for your setting (setuptools + numpy) would look like this: [build-system] requires = ["setuptools", "wheel", "oldest-supported-numpy"] build-backend = "setuptools.build_meta"  Given such a pyproject.toml you can build the extension using the python build module by calling python -m build  which should produce a wheel with a compiled so inside it.
Question: I ran into this problem where I have a dataframe that looks like the following (the values in the last 3 columns are usually 4-5 alphanumeric codes). import pandas as pd data = {'ID':['P39','S32'], 'Name':['Pipe','Screw'], 'Col3':['Test1, Test2, Test3','Test6, Test7'], 'Col4':['','Test8, Test9'], 'Col5':['Test4, Test5','Test10, Test11, Test12, Test13'] } df = pd.DataFrame(data)       ID Name Col3 Col4 Col5     0 P39 Pipe Test1, Test2, Test3  Test4, Test5   1 S32 Screw Test6, Test7 Test8, Test9 Test10, Test11, Test12, Test13     I want to expand this dataframe or create a new one based on the values in the last 3 columns in each row. I want to create more rows based on the maximum amount of values separated by commas in one of the last 3 rows. I then want to keep the first 2 columns the same in all of the expanded rows. But I want to fill the last 3 columns in the expanded rows with only one value each from the original column. In the above example, the first row would indicate I need 3 total rows (Col3 has the most at 3 values), and the second row would indicate I need 4 total rows (Col5 has the most at 4 values). A desired output would be along the lines of:      ID Name Col3 Col4 Col5     0 P39 Pipe Test1  Test4   1 P39 Pipe Test2  Test5   2 P39 Pipe Test3     3 S32 Screw Test6 Test8 Test10   4 S32 Screw Test7 Test9 Test11   5 S32 Screw   Test12   6 S32 Screw   Test13     I first found a way to figure out the number of rows needed. I also had the idea to append the values to a new dataframe in the same loop. Although, I'm not sure how to separate the values in the last 3 columns and append them one by one in the rows. I know the str.split() is useful to put the values into a list. My only idea would be if I need to loop through each column separately and append it to the correct row, but I'm not sure how to do that. output1 = pd.DataFrame( columns = ['ID', 'Name', 'Col3', 'Col4', 'Col5']) for index, row in df.iterrows(): output2 = pd.DataFrame( columns = ['ID', 'Name', 'Col3', 'Col4', 'Col5']) col3counter = df.iloc[index, 2].count(',') col4counter = df.iloc[index, 3].count(',') col5counter = df.iloc[index, 4].count(',') numofnewcols = max(col3counter, col4counter, col5counter) + 1 iter1 = df.iloc[index, 2].split(', ') iter2 = df.iloc[index, 3].split(', ') iter3 = df.iloc[index, 4].split(', ') #for q in iter1 #output2.iloc[ , 2] = output1 = pd.concat([output1, output2], ignore_index=True) del output2 

Answer: A bit tricky but it should work with melt to flat your dataframe then pivot_table to reshape it: out = (df.reset_index().melt(['ID', 'Name', 'index'], var_name='col', value_name='val') .assign(val=lambda x: x['val'].str.split(', ')).explode('val') .assign(row=lambda x: x.groupby(['index', 'col']).cumcount()) .pivot_table('val', ['index', 'row', 'ID', 'Name'], 'col', aggfunc='first') .droplevel(['index', 'row']).reset_index().rename_axis(columns=None).fillna(''))  Output:      ID Name Col3 Col4 Col5     0 P39 Pipe Test1  Test4   1 P39 Pipe Test2  Test5   2 P39 Pipe Test3     3 S32 Screw Test6 Test8 Test10   4 S32 Screw Test7 Test9 Test11   5 S32 Screw   Test12   6 S32 Screw   Test13
Question: I'm trying to read a specific region on a PDF file. How to do it? I've tried:  Using PyPDF2, cropped the PDF page and read only that. It doesn't work because PyPDF2's cropbox only shrinks the "view", but keeps all the items outside the specified cropbox. So on reading the cropped pdf text with extract_text(), it reads all the "invisible" contents, not only the cropped part. Converting the PDF page to PNG, cropping it and using Pytesseract to read the PNG. Py tesseract doesn't work properly, don't know why. 

Answer: PyMuPDF can probably do this. I just answered another question regarding getting the "highlighted text" from a page, but the solution uses the same relevant parts of the PyMuPDF API you want:  figure out a rectangle that defines the area of interest extract text based on that rectangle  and I say "probably" because I haven't actually tried it on your PDF, so I cannot say for certain that the text is amenable to this process. import os.path import fitz from fitz import Document, Page, Rect # For visualizing the rects that PyMuPDF uses compared to what you see in the PDF VISUALIZE = True input_path = "test.pdf" doc: Document = fitz.open(input_path) for i in range(len(doc)): page: Page = doc[i] page.clean_contents() # https://pymupdf.readthedocs.io/en/latest/faq.html#misplaced-item-insertions-on-pdf-pages # Hard-code the rect you need rect = Rect(0, 0, 100, 100) if VISUALIZE: # Draw a red box to visualize the rect's area (text) page.draw_rect(rect, width=1.5, color=(1, 0, 0)) text = page.get_textbox(rect) print(text) if VISUALIZE: head, tail = os.path.split(input_path) viz_name = os.path.join(head, "viz_" + tail) doc.save(viz_name)  For context, here's the project I just finished where this was working for the highlighted text, https://github.com/zacharysyoung/extract_highlighted_text.
Question: Edit: I also posted this question as an issue on pylance-release github repo, which might be better suited to find an answer. I'm having issues with Visual Studio Code python language server, which cannot find the stubs for a python binding library I am developing (https://github.com/pthom/lg_hello_imgui). I think the issue might be linked to the fact that my library expose several submodules. I made a minimum reproductible example here: https://github.com/pthom/pylance_test When using PyCharm or CLion, I can easily navigate to the stubs I defined for my library, and also in its submodules stubs. However, when using Visual Studio Code, it is less reliable:  when using the Jedi language server I can navigate to the functions definition, although it might fail after a few minutes of usage. when using the Pylance language server, it always fails  I'm using a Mac M1. I've had this failure inside OSX, as well as inside windows ARM, with Parallels. Is it linked to my setup? Here is the installed package structure: .venv/lib/python3.9/site-packages/lg_hello_imgui/ ├── __init__.py ├── _lg_hello_imgui.cpython-39-darwin.so* ├── hello_imgui.pyi ├── imgui.pyi ├── implot.pyi └── py.typed  _lg_hello_imgui is a native library that include three submodules, defined like this via pybind11: PYBIND11_MODULE(_lg_hello_imgui, m) { auto module_imgui = m.def_submodule("imgui"); py_init_module_imgui(module_imgui); auto module_himgui = m.def_submodule("hello_imgui"); py_init_module_hello_imgui(module_himgui); auto module_implot = m.def_submodule("implot"); py_init_module_implot(module_implot); }  and lg_hello_imgui/__init__.py states: from lg_hello_imgui._lg_hello_imgui import imgui from lg_hello_imgui._lg_hello_imgui import hello_imgui from lg_hello_imgui._lg_hello_imgui import implot  Repo for issue reproduction: https://github.com/pthom/pylance_test (Note: the lg_hello_imgui linked library does not exist as wheel yet, so it may require 1 minute or 2 to build)

Answer: I'm answering my own question since I had an answer from the pylance team in the meantime. https://peps.python.org/pep-0484/#stub-files specifies that  Modules and variables imported into the stub are not considered exported from the stub unless the import uses the import ... as ... form or the equivalent from ... import ... as ... form. (UPDATE: To clarify, the intention here is that only names imported using the form X as X will be exported, i.e. the name before and after as must be the same.)  So I had to input this into the init.pyi file: from . import hello_imgui as hello_imgui from . import imgui as imgui from . import implot as implot
Question: I'm using a name entity recognition model to find names in a text string. For hyphenated names like Jane Miller-Smith, the NER model returns the names seperately like this: names = ['Jane','Miller','-','Smith']  What's a simple way to join the items before and after the '-' to one string in this list? So that I have a list of first and last name like name = ['Jane', 'Miller-Smith']? I've so far tried to loop through the list of names based on solutions like this for different hyphenated name versions: name1 = ['Jane', 'Miller', '-','Smith'] name = ['Jane', '-', 'Marie','Miller', '-','Smith'] new_name = [] for cur, nxt in zip (name, name [1:]): print(cur,nxt) if cur == '-': hyph = cur+nxt new_name.append(hyph) print("hyph: ", hyph) else: new_name.append(cur) print("cur: ", cur) print(new_name)  But I can't wrap my head around how to combine only the string before and after the hypen and also keep other non-hyphenated strings in the list in order (so that not the last name is suddenly first).

Answer: Scan from right to left, replacing the three-element slices whenever a hyphen is found: >>> names = ['Jane', '-', 'Marie','Miller', '-','Smith'] >>> for i in reversed(range(len(names))): if names[i] == '-': names[i-1: i+2] = [f'{names[i-1]}-{names[i+1]}'] >>> names ['Jane-Marie', 'Miller-Smith']  An alternative is to loop left-to-right and build a new result list: >>> names = ['Jane', '-', 'Marie', 'Miller', '-','Smith'] >>> result = [] >>> it = iter(names) >>> for tok in it: if tok == '-': tok = result.pop() + '-' + next(it) result.append(tok) >>> result ['Jane-Marie', 'Miller-Smith']
Question: I am able to permute the dimmension of the tensor: I'm able to do this in pytorch! But not in tensorflow! A = torch.rand(1, 2,5) A = A.permute(0,2,1) A.shape   torch.Size([1, 5, 2])  Tensorflow (just a try,I don't know about this): A = tf.random.normal(1, 2,5) A = tf.keras.layers.Permute((0, 2, 1))  Not working

Answer: Use tf.transpose: import tensorflow as tf A = tf.random.normal((1, 2, 5)) A_t = tf.transpose(A, perm=[0, 2, 1]) print(A.shape, A_t.shape) # (1, 2, 5) (1, 5, 2)
Question: When using HTMX framework with Python Flask, you have to be able to:  serve a request as a HTML fragment if it's done by HTMX (via AJAX)  server a request as a full page if it's done by the user (e.g. entered directly in the browser URL bar)   See Single-page-application with fixed header/footer with HTMX, with browsing URL history or Allow Manual Page Reloading  for more details. How to do this with the Flask template system? from flask import Flask, render_template, request app = Flask("") @app.route('/pages/<path>') def main(path): htmx_request = request.headers.get('HX-Request') is not None return render_template(path + '.html', fullpage=not htmx_request) app.run()  What's the standard way to output a full page (based on a parent template pagelayout.html): {% extends "pagelayout.html" %} {% block container %} <button>Click me</button> {% endblock %}  if fullpage is True, and just a HTML fragment: <button>Click me</button>  if it is False?

Answer: This solution based on that we can use a dynamic variable when extending a base template. So depending on the type or the request, we use the full base template or a minimal base template that returns only our fragment's content. Lets call our base template for fragments base-fragments.html: {% block container %} {% endblock %}  It's just returns the main block's content, nothing else. At the view function we have a new template variable baselayout, that contains the name of the base template depending on the request's type (originating from HTMX or not): @app.route('/pages/<path>') def main(path): htmx_request = request.headers.get('HX-Request') is not None baselayout = 'base-fragments.html' if htmx_request else 'pagelayout.html' return render_template(path + '.html', baselayout=baselayout)  And in the page template, we use this baselayout variable at the extends: {% extends baselayout %} {% block container %} <button>Click me</button> {% endblock %}
Question: I would like to have a Python @cache decorator based on identity, not __hash__/__equal. That is to say, I would like the cached value for an argument ka NOT to be used for a different object ka2, even if ka == ka2. Is there a way to do that? In code: from functools import cache class Key: def __init__(self, value): self.value = value def __eq__(self, another): print(f"__eq__ {self.value}, {another.value}") return another.value == self.value def __hash__(self): print(f"__hash__ {self.value}") return hash(self.value) def __repr__(self): return self.value i = 0 @cache def foo(key): global i i += 1 print(f"Computing foo({key}) = {i}") return i ka = Key('a') ka2 = Key('a') print(f"foo(ka): {foo(ka)}") print(f"foo(ka2): {foo(ka2)}") # I would like the cached value for ka NOT to be used even though ka2 == ka. 

Answer: Make a wrapper like Key that compares by the identity of its wrapped object, and wrap your caching function in a helper that uses the wrapper: class Id: __slots__="x", def __init__(self,x): self.x=x def __hash__(self): return id(self.x) def __eq__(self,o): return self.x is o.x def cache_id(f): @functools.cache def id_f(i): return f(i.x) @functools.wraps(f) def call(x): return id_f(Id(x)) return call @cache_id def foo(key): …
Question: How can I set a sort order for the API methods in the FastAPI Swagger autodocs? I would like all my methods grouped by type (GET, POST, PUT, DELETE). This answer shows how to do it in Java. How can I do it in Python? from fastapi import FastAPI app = FastAPI() @app.get("/") def list_all_components(): pass @app.get("/{component_id}") def get_component(component_id: int): pass @app.post("/") def create_component(): pass @app.put("/{component_id}") def update_component(component_id: int): pass @app.delete("/{component_id}") def delete_component(component_id: int): pass  

Answer: You can configure Swagger UI parameters through the FastAPI constructor. app = FastAPI(swagger_ui_parameters={"operationsSorter": "method"})  The full list of parameters can be found in the swagger documentation.
Question: Is it possible to create, by sending the DAG file contents, to Apache Airflow using the API? For example, it is possible to list all DAGs using the API curl -u "admin:admin" http://localhost:8080/api/v1/dags { "dags": [], "total_entries": 0 } 

Answer: You can not create new DAGs via API. You can read a discussion about this request in the project https://github.com/apache/airflow/discussions/24744 which also lists the reasons why Airflow won't have it. In simple words by adding such API it means that the machine(s) where DAGs are deployed to need to have credentials to write those DAG files to all the other components. For such use case you better to use Git sync to add files to the DAG directory.