eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: In my model, I have a normalizing layer for a 1 column feature array. I assume this gives a 1 ndim output: single_feature_model = keras.models.Sequential([ single_feature_normalizer, layers.Dense(1) ])  Normailaztion step: single_feature_normalizer = preprocessing.Normalization(axis=None) single_feature_normalizer.adapt(single_feature)  The error I'm getting is: ValueError Traceback (most recent call last) <ipython-input-98-22191285d676> in <module>() 2 single_feature_model = keras.models.Sequential([ 3 single_feature_normalizer, ----> 4 layers.Dense(1) # Linear Model 5 ]) /usr/local/lib/python3.7/dist-packages/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name) 225 ndim = x.shape.rank 226 if ndim is not None and ndim < spec.min_ndim: --> 227 raise ValueError(f'Input {input_index} of layer "{layer_name}" ' 228 'is incompatible with the layer: ' 229 f'expected min_ndim={spec.min_ndim}, ' ValueError: Input 0 of layer "dense_27" is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (None,)  I seems that the dense layer is looking for a 2 ndim array while the normalization layer outputs a 1 ndim array. Is there anyway to solve this and getting the model working?

Answer: I think you need to explicitly define an input layer with your input shape, since your output layer cannot infer the shape of the tensor coming from the normalization layer: import tensorflow as tf single_feature_normalizer = tf.keras.layers.Normalization(axis=None) feature = tf.random.normal((314, 1)) single_feature_normalizer.adapt(feature) single_feature_model = tf.keras.models.Sequential([ tf.keras.layers.Input(shape=(1,)), single_feature_normalizer, tf.keras.layers.Dense(1) ])  Or define the input shape directly in the normalization layer without using an input layer: single_feature_normalizer = tf.keras.layers.Normalization(input_shape=[1,], axis=None)
Question: I have the below pandas dataframe. d = {'col1': [1, 2,3,4,5,60,0,0,6,3,2,4],'col3': [1, 22,33,44,55,60,1,5,6,3,2,4],'Name': ['2a df a1asd_V1', 'xcd a2asd_V3','23vg aabsd_V1','dfgdf_aabsd_V0','a3as d_V1','aa bsd_V3','aasd_V4','aabsd_V4','aa_adn sd_V15',np.nan,'aasd_V12','aasd120Abs'],'Date': ['2021-06-13', '2021-06-13','2021-06-13','2021-06-14','2021-06-15','2021-06-15','2021-06-13','2021-06-16','2021-06-13','2021-06-13','2021-06-13','2021-06-16']} dff = pd.DataFrame(data=d) dff col1 col3 Name Date 0 1 1 2a df a1asd_V1 2021-06-13 1 2 22 xcd a2asd_V3 2021-06-13 2 3 33 23vg aabsd_V1 2021-06-13 3 4 44 dfgdf_aabsd_V0 2021-06-14 4 5 55 a3as d_V1 2021-06-15 5 60 60 aa bsd_V3 2021-06-15 6 0 1 aasd_V4 2021-06-13 7 0 5 aabsd_V4 2021-06-16 8 6 6 aa_adn sd_V10 2021-06-13 9 3 3 NaN 2021-06-13 10 2 2 aasd_V12 2021-06-13 11 4 4 aasd120Abs 2021-06-16  I want to make two new columns based on the Name column. I want to extract the part of the string in the Name column like V1, V2, V3, V4...V20 like that. Also if there isn't anything like that at end of the Name string or if the Name row is empty, just want to make an empty cell. So I want something like below pandas dataframe.  col1 col3 Name Date Version Version 0 1 1 2a df a1asd_V1 2021-06-13 V1 Version 1 1 2 22 xcd a2asd_V3 2021-06-13 V3 Version 3 2 3 33 23vg aabsd_V1 2021-06-13 V1 Version 1 3 4 44 dfgdf_aabsd_V0 2021-06-14 V0 Version 0 4 5 55 a3as d_V1 2021-06-15 V1 Version 1 5 60 60 aa bsd_V3 2021-06-15 V3 Version 3 6 0 1 aasd_V4 2021-06-13 V4 Version 4 7 0 5 aabsd_V4 2021-06-16 V4 Version 4 8 6 6 aa_adn sd_V10 2021-06-13 V10 Version 10 9 3 3 NaN 2021-06-13 10 2 2 aasd_V12 2021-06-13 V12 Version 12 11 4 4 aasd120Abs 2021-06-16  Is it possible to do that? I know in SQL we can do that using "LIKE" WHEN `Name` LIKE '%V10%' THEN 'Verison 10'.  Is there a similar command or any other way to do that in python? Thanks in advance! Any help is appreciated!

Answer: Use str.extract with a regex and str.replace to rename values: dff['Version_short'] = dff['Name'].str.extract('_(V\d+)$').fillna('') dff['Version_long'] = dff['Version_short'].str.replace('V', 'Version ')  Output: >>> dff col1 col3 Name Date Version_short Version_long 0 1 1 2a df a1asd_V1 2021-06-13 V1 Version 1 1 2 22 xcd a2asd_V3 2021-06-13 V3 Version 3 2 3 33 23vg aabsd_V1 2021-06-13 V1 Version 1 3 4 44 dfgdf_aabsd_V0 2021-06-14 V0 Version 0 4 5 55 a3as d_V1 2021-06-15 V1 Version 1 5 60 60 aa bsd_V3 2021-06-15 V3 Version 3 6 0 1 aasd_V4 2021-06-13 V4 Version 4 7 0 5 aabsd_V4 2021-06-16 V4 Version 4 8 6 6 aa_adn sd_V15 2021-06-13 V15 Version 15 9 3 3 NaN 2021-06-13 10 2 2 aasd_V12 2021-06-13 V12 Version 12 11 4 4 aasd120Abs 2021-06-16
Question: maybe someone met the issue... I use a custom 'chrome driver' for PyTest with 'performance' log: cap = webdriver.DesiredCapabilities.CHROME.copy() cap['goog:loggingPrefs'] = {'performance': 'ALL'} services = Service(executable_path='/usr/local/bin/chromedriver') chrome_driver = webdriver.Chrome(desired_capabilities=cap, options=options, service=services)  My environment:  Selenium 4.0.0 ChromeDriver 95.0.4638.17  And now I have got a warning: =============================== warnings summary =============================== tests/test_name.py::test_main[/] /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py:69: DeprecationWarning: desired_capabilities has been deprecated, please pass in a Service object super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog", -- Docs: https://docs.pytest.org/en/stable/warnings.html ======================== 1 passed, 1 warning in 10.49s =========================  May be someone know how does handle this?

Answer: See below from selenium import webdriver from selenium.webdriver.chrome.service import Service as ChromeService options = webdriver.ChromeOptions() options.set_capability("loggingPrefs", {'performance': 'ALL'}) service = ChromeService(executable_path'/usr/local/bin/chromedriver') driver = webdriver.Chrome(service=service, options=options)  If the above doesn't work, try: options.set_capability("goog:loggingPrefs", {'performance': 'ALL'})
Question: I have a dataframe df:  x y z ... colours marker_size marker_opacity test1 0.118709 0.219099 -0.024387 ... red 100 0.5 test2 -0.344873 -0.401508 0.169995 ... blue 100 0.5 test3 -0.226923 0.021078 0.400358 ... red 100 0.5 test4 0.085421 0.098442 -0.588749 ... purple 100 0.5 test5 0.367666 0.062889 0.042783 ... green 100 0.5  I am trying to plot this with plotly like so: fig = px.scatter_3d(df, x='x', y='y', z = 'z', color='labels', hover_name = df.index, opacity = 0.5, size = 'marker_size') fig.write_html(file_name)  When I open file_name, everything is fine, but my points are too big. When I alter the 'marker_size' column of my df, nothing changes (I have tried 0.1, 1, 10, 100...). Why is this? I have also tried: Param: size = 1: Result: ValueError: Value of 'size' is not the name of a column in 'data_frame'. Expected one of ['x', 'y', 'z', 'labels', 'colours', 'marker_size', 'marker_opacity'] but received: 1 Param: size = [1]*len(df): Result: No difference to using the 'marker_size' df column

Answer: If you're looking to increase the marker size of all traces, just use: fig.update_traces(marker_size = 12)   Details: The size attribute of px.scatter_3d isn't there to let you specify the size of your markers directly. But rather to let you add a fourth dimension to your scatter plot representing the varying size of another variable.  size: str or int or Series or array-like Either a name of a column in `data_frame`, or a pandas Series or array_like object. Values from this column or array_like are used to assign mark sizes.   The reason why changing the value from 1 to 10 or 100 is that you seem to have been changing all the values in df['marker_size'] at the same time:  In order for such an assignment to have effect, you would need to have a variable and not a constant in df['marker_size']. You can take a closer look at how these things work through this snippet: import plotly.express as px df = px.data.iris() fig = px.scatter_3d(df, x='sepal_length', y='sepal_width', z='petal_width', color='species', size = 'petal_length' ) fig.show()   Here you can see that the size attribute works as intended, since your markers will have varying sizes as defined by df['petal_length]:
Question: I know that Python is available for Mac and Linux on ARM because I have Python installed via Homebrew on Mac and it's ARM and I have Python installed via apt on Ubuntu and it's ARM. However, I can't find any download links for Python for Windows on ARM. The Windows download link on the Python website contain amd64 so it is for x86, not ARM, and the list of all links on the Releases page only contains links for 32-bit and 64-bit x86. Is there any way to get Python natively on ARM on Windows? Maybe there's a beta or experimental build, or an unofficial build?

Answer: It will be supported starting with Python 3.11: https://bugs.python.org/issue33125  I'll probably make the ARM64 packages available through the Windows Store for 3.11's prereleases, and possibly as a side-loadable MSIX from python.org.  The same ticket also mentions an unofficial build for testing purposes: https://www.nuget.org/packages/pythonarm64/
Question: My Python package, tinted, finally works. However, when I run dir(tinted), the core.py and sequences.py files exist in the package! I want only the function tint to be included in the package. Current output of dir(tinted): ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', 'core', 'sequences', 'tint']  Desired output of dir(tinted): ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', 'tint']  This is my first time ever uploading a package to PyPi, so please forgive me if I've done anything wrong. Anyone know how to fix this?

Answer: Let's say you have your Python package in a folder named tinted. You have a function named tint defined in module core.py inside that folder. But you want to expose that function at the top level of the package's namespace. Which is good practice for important functions, they should be at the top level. Then in __init__.py you would have this: from .core import tint  (Or from tinted.core import tint if you happen to use absolute imports.) However, as you have noted, this also puts the core module into the package's top-level namespace: >>> import tinted >>> dir(tinted) ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'core', 'tint']  If you want to keep that namespace squeaky-clean, you can remove the internal module after the import in __init__.py. Like so: from .core import tint del core  Then: >>> dir(tinted) ['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'tint']  This is however not commonly done. Even packages in Python's standard library don't usually remove these kind of imports. For example, the logging package imports threading and just leaves it there. Though sometimes they do clean up a bit. Like the copy module, which does del types, weakref at the end. So there's nothing wrong with it either. In this case, the object was a function. Just note that, when it's a class, you cannot completely hide the package structure anyway. Consider this example from the popular Pandas library: >>> import pandas >>> pandas.DataFrame <class 'pandas.core.frame.DataFrame'>
Question: I started a selenium tutorial today and have run into this error when trying to run the code. I've tried other methods but ultimately get the same error. I'm on MacOS using VSC. My Code: from selenium import webdriver PATH = '/Users/blutch/Documents/Chrom Web Driver\chromedriver.exe' driver = webdriver.Chrome(PATH) driver.get("https://www.google.com")  I've also tried inserting C: in front of /Users. Can anyone guide me on why this is happening/how to fix it?

Answer: This error message... DeprecationWarning: executable_path has been deprecated, please pass in a Service object  ...implies that the key executable_path will be deprecated in the upcoming releases. This change is inline with the Selenium 4.0 Beta 1 changelog which mentions:  Deprecate all but Options and Service arguments in driver instantiation. (#9125,#9128)   Solution Once the key executable_path is deprecated you have to use an instance of the Service() class as follows: from selenium import webdriver from selenium.webdriver.chrome.service import Service s = Service('C:/Users/.../chromedriver.exe') driver = webdriver.Chrome(service=s)   TL; DR You can find a couple of relevant detailed discussion in:  Bug Report: deprecate all but Options and Service arguments in driver instantiation Pull Request: deprecate all but Options and Service arguments in driver instantiation
Question: I'm trying to get the column of a Dataframe as Series. df['a']  returns allways a pl.Dataframe. Right now I'm doing it this way pl.Series('GID_1',df['GID_1'].to_numpy().flatten().tolist())  I don't think that's the best way to do it. Does anyone have an idea?

Answer: I don't really understand. This snippet runs, so it returns a pl.Series. df = pl.DataFrame({ "A": [1, 2, 3], "B": [1, 2, 3] }) assert isinstance(df["A"], pl.Series)
Question: I have a dataset that includes video frames partially 1000 real videos and 1000 deep fake videos. each video after preprocessing phase converted to the 300 frames in other worlds I have a dataset with 300000 images with Real(0) label and 300000 images with Fake(1) label. I want to train MesoNet with this data. I used costum DataGenerator class to handle train, validation, test data with 0.8,0.1,0.1 ratios but when I run the project show this message: Filling up shuffle buffer (this may take a while):  What can I do to solve this problem? You can see the DataGenerator class below. class DataGenerator(keras.utils.Sequence): 'Generates data for Keras' def __init__(self, df, labels, batch_size =32, img_size = (224,224), n_classes = 2, shuffle=True): 'Initialization' self.batch_size = batch_size self.labels = labels self.df = df self.img_size = img_size self.n_classes = n_classes self.shuffle = shuffle self.batch_labels = [] self.batch_names = [] self.on_epoch_end() def __len__(self): 'Denotes the number of batches per epoch' return int(np.floor(len(self.df) / self.batch_size)) def __getitem__(self, index): batch_index = self.indexes[index * self.batch_size : (index + 1) * self.batch_size] frame_paths = self.df.iloc[batch_index]["framePath"].values frame_label = self.df.iloc[batch_index]["label"].values imgs = [cv2.imread(frame) for frame in frame_paths] imgs = [cv2.cvtColor(img, cv2.COLOR_BGR2RGB) for img in imgs] imgs = [ cv2.resize(img, self.img_size) for img in imgs if img.shape != self.img_size ] batch_imgs = np.asarray(imgs) labels = list(map(int, frame_label)) y = np.array(labels) self.batch_labels.extend(labels) self.batch_names.extend([str(frame).split("\\")[-1] for frame in frame_paths]) return ( batch_imgs,y ) def on_epoch_end(self): 'Updates indexes after each epoch' self.indexes = np.arange(len(self.df)) if self.shuffle == True: np.random.shuffle(self.indexes) 

Answer: Note that this is not an error, but a log message: https://github.com/tensorflow/tensorflow/blob/42b5da6659a75bfac77fa81e7242ddb5be1a576a/tensorflow/core/kernels/data/shuffle_dataset_op.cc#L138 It seems you may be choosing too large a dataset if it's taking too long: https://github.com/tensorflow/tensorflow/issues/30646 You can address this by lowering your buffer size: https://support.huawei.com/enterprise/en/doc/EDOC1100164821/2610406b/what-do-i-do-if-training-times-out-due-to-too-many-dataset-shuffle-operations
Question: I have a very large pandas.Dataframe and want to create a new Dataframe by selecting all columns where one row has a specific value.  A B C D E Region Nord Süd West Nord Nord value 2.3 1.2 4.2 0.5 1.3 value2 20 400 30 123 200  Now i want to create a new DataFrame with all columns where the row "Region" has the value "Nord". How can it be done? The result should look like this:  A D E Region Nord Nord Nord value 2.3 0.5 1.3 value2 20 123 200  Thanks in advance

Answer: Use first DataFrame.loc for select all rows (:) by mask compred selected row Region by another loc: df = df.loc[:, df.loc['Region'] == 'Nord'] print (df) A D E Region Nord Nord Nord value 2.3 0.5 1.3 value2 20 123 200  Better is crated MultiIndex by first row with original columns, then is possible select by DataFrame.xs: df.columns = [df.columns, df.iloc[0]] df = df.iloc[1:].rename_axis((None, None), axis=1) print (df) A B C D E Nord Süd West Nord Nord value 2.3 1.2 4.2 0.5 1.3 value2 20 400 30 123 200   print (df.xs('Nord', axis=1, level=1)) A D E value 2.3 0.5 1.3 value2 20 123 200 print (df.xs('Nord', axis=1, level=1, drop_level=False)) A D E Nord Nord Nord value 2.3 0.5 1.3 value2 20 123 200
Question: How to I upgrade to the latest version? Specification:  Windows 10, Visual Studio Code, Ubuntu Bash.   Current Version: me@PF2DCSXD:/mnt/c/Users/user/Documents/GitHub/workers-python/workers/composite_key/compositekey/tests$ python3 --version Python 3.8.10  Attempt to update | poetry self update: me@PF2DCSXD:/mnt/c/Users/user/Documents/GitHub/workers-python/workers/composite_key/compositekey/tests$ poetry self update RuntimeError Poetry was not installed with the recommended installer. Cannot update automatically. at ~/.local/lib/python3.8/site-packages/poetry/console/commands/self/update.py:389 in _check_recommended_installation 385│ current = Path(__file__) 386│ try: 387│ current.relative_to(self.home) 388│ except ValueError: → 389│ raise RuntimeError( 390│ "Poetry was not installed with the recommended installer. " 391│ "Cannot update automatically." 392│ ) 393│  Please let me know if there is anything else I can add to post.

Answer: The error message suggests you've probably installed poetry with pip, which does not support automatic poetry updates. You should uninstall the poetry version currently installed, and reinstall it using the recommended method, which uses a custom installation script. On osx/linux, you'll just have to run curl -sSL https://install.python-poetry.org | python3 - to download and run this installation script.
Question: When starting the function def run(driver_path): driver = webdriver.Chrome(executable_path=driver_path) driver.get('https://tproger.ru/quiz/real-programmer/') button = driver.find_element_by_class_name("quiz_button") button.click() run(driver_path)  I'm getting errors like these: <ipython-input-27-c5a7960e105f>:6: DeprecationWarning: executable_path has been deprecated, please pass in a Service object driver = webdriver.Chrome(executable_path=driver_path) <ipython-input-27-c5a7960e105f>:10: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead button = driver.find_element_by_class_name("quiz_button")  but I can't understand why. I'm using WebDriver at the latest version for my Chrome's version. I don't why I get find_element_by_* commands are deprecated  when it's in the documentation that the command exists.

Answer: This error message: DeprecationWarning: find_element_by_* commands are deprecated. Please use find_element() instead  implies that the find_element_by_* commands are deprecated in the latest Selenium Python libraries. As AutomatedTester mentions: This DeprecationWarning was the reflection of the changes made with respect to the decision to simplify the APIs across the languages and this does that.  Solution Instead you have to use find_element(). As an example:  You have to include the following imports from selenium.webdriver.common.by import By    Using class_name: button = driver.find_element_by_class_name("quiz_button")  Needs be replaced with: button = driver.find_element(By.CLASS_NAME, "quiz_button")     Along the lines of, you also have to change the following:  Using id: element = find_element_by_id("element_id")  Needs be replaced with: element = driver.find_element(By.ID, "element_id")   Using name: element = find_element_by_name("element_name")  Needs be replaced with: element = driver.find_element(By.NAME, "element_name")   Using link_text: element = find_element_by_link_text("element_link_text")  Needs be replaced with: element = driver.find_element(By.LINK_TEXT, "element_link_text")   Using partial_link_text: element = find_element_by_partial_link_text("element_partial_link_text")  Needs be replaced with: element = driver.find_element(By.PARTIAL_LINK_TEXT, "element_partial_link_text")   Using tag_name: element = find_element_by_tag_name("element_tag_name")  Needs be replaced with: element = driver.find_element(By.TAG_NAME, "element_tag_name")   Using css_selector: element = find_element_by_css_selector("element_css_selector")  Needs be replaced with: element = driver.find_element(By.CSS_SELECTOR, "element_css_selector")   Using xpath: element = find_element_by_xpath("element_xpath")  Needs be replaced with: element = driver.find_element(By.XPATH, "element_xpath")    Note: If you are searching and replacing to implement the above changes, you will need to do the same thing for find_elements_*, i.e., the plural forms of find_element_*. You may also find this upgrade guide useful as it covers some other unrelated changes you may need to make when upgrading: Upgrade to Selenium 4
Question: This doesn't seem to work: from typing import NewType MyStr = NewType("MyStr", str) x = MyStr("Hello World") isinstance(x, MyStr)  I don't even get False, but TypeError: isinstance() arg 2 must be a type or tuple of types because MyStr is a function and isinstance wants one or more type. Even assert type(x) == MyStr or is MyStr fails. What am I doing wrong?

Answer: The purpose of NewType is purely for static type checking, but for dynamic purposes it produces the wrapped type. It does not make a new type at all, it returns a callable that the static type checker can see, that's all. When you do: x = MyStr("Hello World")  it doesn't produce a new instance of MyStr, it returns "Hello World" entirely unchanged, it's still the original str passed in, even down to identity: >>> s = ' '.join(["a", "b", "c"]) # Make a new string in a way that foils interning, just to rule out any weirdness from caches >>> ms = MyStr(s) # "Convert" it to MyStr >>> type(ms) # It's just a str str >>> s is ms # It's even the *exact* same object you passed in True  The point is, what NewType(...) returns is effectively a callable that:  Acts as the identity function (it returns whatever you give it unchanged); in modern Python, NewType itself is a class that begins with __call__ = _idfunc, that's literally just saying when you make a call with an instance, return the argument unchanged. (In modern Python) Has some useful features like overloading | to produce Unions like other typing-friendly things.  but you can't use it usefully for isinstance, not because it's not producing instances of anything. As other answers have mentioned, if you need runtime, dynamic checking, subclassing is the way to go. The other answers are doing both more (unnecessarily implementing __new__) and less (allowing arbitrary attributes, bloating instances of the subclass for a benefit you won't use) than necessary, so here's what you want for something that:  Is considered a subclass of str for both static and runtime checking purposes Does not bloat the memory usage per-instance any more than absolutely necessary  class MyStr(str): # Inherit all behaviors of str __slots__ = () # Prevent subclass from having __dict__ and __weakref__ slots, saving 16 bytes # per instance on 64 bit CPython builds, and avoiding weirdness like allowing # instance attributes on a logical str  That's it, just two lines (technically, a one-liner like class MyStr(str): __slots__ = () is syntactically legal, but it's bad style, so I avoid it), and you've got what you need.