eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: > import numpy as np > min(50, np.NaN) 50 > min(np.NaN, 50) nan  (Same behaviour occurs with max) I know that I can avoid this behaviour by using numpy.nanmin. But what causes the change when the order is reversed? Is min sensitive to input order?

Answer:  Is min sensitive to input order?  Yes. https://docs.python.org/3/library/functions.html#min "If multiple items are minimal, the function returns the first one encountered." The documentation does not specify exactly how "minimal" is defined in the face of items that don't have a consistent order, but it's likely that min is based on looping over the elements and using the < operator to determine if the new element is smaller than the smallest item found so-far. To confirm this hypothesis we can read the source code (search for builtin_min and min_max in https://github.com/python/cpython/blob/c96d00e88ead8f99bb6aa1357928ac4545d9287c/Python/bltinmodule.c ), it's slightly confusing because the implementations for min and max are combined and the variable names seem to be based on it being a max function but it's not too hard to follow. And it does indeed loop through the elements in order and performs the comparison with a call to PyObject_RichCompareBool with an "opid" of Py_LT which is the C API equivalent of the python < operator. Comparisons between NaN and numbers return false, so in a list containing numbers and NaNs if there is a NaN in the first position it will be considered the minimum as no number will be "less than" it. On the other hand, if the NaN is not in the first position then it will be effectively skipped over as it is not "less than" any number.
Question: I am using Linux Ubuntu 18.04 and python 3. I am trying to build a connection between a maria-db and my python scripts. Therefore I have to install the mariadb package. I have already installed: sudo apt install mariadb-server  But when i try: pip install mariadb  I get following error: Collecting mariadb Using cached mariadb-1.0.0.tar.gz (78 kB) ERROR: Command errored out with exit status 1: command: /home/niklas/Desktop/Stuff/venv/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pycharm-packaging/mariadb/setup.py'"'"'; __file__='"'"'/tmp/pycharm-packaging/mariadb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-wfnscxnz cwd: /tmp/pycharm-packaging/mariadb/ Complete output (12 lines): /bin/sh: 1: mariadb_config: not found Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/pycharm-packaging/mariadb/setup.py", line 26, in <module> cfg = get_config(options) File "/tmp/pycharm-packaging/mariadb/mariadb_posix.py", line 49, in get_config cc_version = mariadb_config(config_prg, "cc_version") File "/tmp/pycharm-packaging/mariadb/mariadb_posix.py", line 27, in mariadb_config "mariadb_config not found.\nPlease make sure, that MariaDB Connector/C is installed on your system, edit the configuration file 'site.cfg' and set the 'mariadb_config'\noption, which should point to the mariadb_config utility.") OSError: mariadb_config not found. Please make sure, that MariaDB Connector/C is installed on your system, edit the configuration file 'site.cfg' and set the 'mariadb_config' option, which should point to the mariadb_config utility. ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.  Can anybody help me ? Edit: I have now been able to connect to the server but not with te mariadb package. (https://linuxhint.com/connect_mariadb_pymysql/)

Answer: To install mariadb python module, you have to install a recent version of MariaDB Connector/C, minimum required version is 3.1.5, afaik Ubuntu 18.04 has 3.0.3. An actual version of Connector/C for bionic is available on the MariaDB Connector/C Download Page. If you want to install it in a special directory, make sure that the PATH and LD_LIBRARY_PATH point to bin and lib directories. I also recommend to use a recent version of MariaDB Server, especially the very fast executemany() method will be much slower on MariaDB Server < 10.2.
Question: Trying to send an email from apache airflow using AWS Simple Email Service (SES), and it's returning errors that are not helping me solve the problem. I believe it's a configuration issue within SES, but I'm not sure what to change. General info:  New SES instance, verified email. Airflow 1.10.10 running on Ubuntu 18.04 (local laptop). Same situation from EC2 instance on a separate AWS account. Running 1 DAG with a python operator that works fine. Sending email works with gmail smtp settings and an app password.  Abbreviated DAG code: ... from airflow.operators.email_operator import EmailOperator ... email_status = EmailOperator( task_id="sending_status_email", to="myverifiedemail@mydomain.com", subject="Test from SES", html_content="Trying to send an email from airflow through SES.", dag=dag ) ...  airflow.cfg SMTP Settings: smtp_host = email-smtp.us-east-1.amazonaws.com smtp_starttls = True smtp_ssl = False smtp_user = AWSUSERKEY smtp_password = PASSWORDFROMAWSSMTP smtp_port = 587 smtp_mail_from = myverifiedemail@mydomain.com  Errors Received while trying various changes to starttls, ssl, and port settings. ERROR - (554, b'Transaction failed: Unsupported encoding us_ascii.') ERROR - STARTTLS extension not supported by server. ERROR - (SSL: WRONG_VERSION_NUMBER) wrong version number (_ssl.c:852) 

Answer: Not sure about the others but we just ran into this error today: ERROR - (554, b'Transaction failed: Unsupported encoding us_ascii.')  This is the default value in the class's __init__ method, which isn't valid: https://github.com/apache/airflow/blob/1.10.10/airflow/operators/email_operator.py#L63 You can fix it by passing in a valid value, like "utf-8": email_status = EmailOperator( mime_charset='utf-8', task_id="sending_status_email", to="myverifiedemail@mydomain.com", subject="Test from SES", html_content="Trying to send an email from airflow through SES.", dag=dag )
Question: I have a decorator that takes a function and returns the same function with some added attributes: import functools from typing import * def decorator(func: Callable) -> Callable: func.attr1 = "spam" func.attr2 = "eggs" return func  How do I type hint the return value of decorator? I want the type hint to convey two pieces of information:  the return value is a Callable the return value has attributes attr1 and attr2  If I write a protocol, class CallableWithAttrs(Protocol): attr1: str attr2: str  then I lose Callable. And apparently I can't make the protocol inherit from Callable; class CallableWithAttrs(Callable, Protocol): attr1: str attr2: str  mypy says: error: Invalid base class "Callable"  On the other hand, if I just use Callable, I lose the information about the added attributes.   This is perhaps even more complicated when introducing type variables, i.e. when the decorator must return the same type of callable as the given function func, as pointed out by MisterMiyagi in the comments. import functools from typing import * C = TypeVar('C', bound=Callable) def decorator(func: C) -> C: func.attr1 = "spam" func.attr2 = "eggs" return func  Now what do I do? I can't inherit from a type variable: class CallableWithAttrs(C, Protocol): attr1: str attr2: str  error: Invalid base class "C" 

Answer: One can parameterise a Protocol by a Callable: from typing import Callable, TypeVar, Protocol C = TypeVar('C', bound=Callable) # placeholder for any Callable class CallableObj(Protocol[C]): # Protocol is parameterised by Callable C ... attr1: str attr2: str __call__: C # ... which defines the signature of the protocol  This creates an intersection of the Protocol itself with an arbitrary Callable.  A function that takes any callable C can thus return CallableObj[C], a callable of the same signature with the desired attributes: def decorator(func: C) -> CallableObj[C]: ...  MyPy properly recognizes both the signature and attributes: def dummy(arg: str) -> int: ... reveal_type(decorator(dummy)) # CallableObj[def (arg: builtins.str) -> builtins.int]' reveal_type(decorator(dummy)('Hello')) # int reveal_type(decorator(dummy).attr1) # str decorator(dummy)(b'Fail') # error: Argument 1 to "dummy" has incompatible type "bytes"; expected "str" decorator(dummy).attr3 # error: "CallableObj[Callable[[str], int]]" has no attribute "attr3"; maybe "attr2"?
Question: I am trying to setup a simple flask server: import envkey import pysher from flask import Flask # from predictor import PythonPredictor app = Flask(__name__) pusher = pysher.Pusher(envkey.get('PUSHER_KEY')) def my_func(*args, **kwargs): print("processing Args:", args) print("processing Kwargs:", kwargs) # We can't subscribe until we've connected, so we use a callback handler # to subscribe when able def connect_handler(data): print('connect habndler') channel = pusher.subscribe('mychannel') channel.bind('myevent', my_func) pusher.connection.bind('pusher:connection_established', connect_handler) @app.route('/') def index(): pusher.connect() return 'Server Works!'  But I Get an error: RuntimeError: cannot join current thread  What am I doing wrong?

Answer: Specifying my Pusher cluster during initalization helped me get rid of that issue: pusher = pysher.Pusher( key=envkey.get('PUSHER_KEY'), # Or however you get the key cluster="eu", # Add cluster! )
Question: I am using the pyo3 rust crate (version 0.11.1) in order to port rust code, into cpython (version 3.8.2) code. I have created a class called my_class and defined the following functions: new, __str__, and __repr__.  TL;DR: The __str__ function exists on a class ported from rust using the pyo3 crate, but doesn't get printed when just using print(obj), and instead having to write print(obj.__str__())  The my_class definition is here: use pyo3::prelude::*; #[pyclass] struct my_class { #[pyo3(get, set)] num: i32, #[pyo3(get, set)] debug: bool, } #[pymethods] impl my_class { #[new] fn new(num: i32, debug: bool) -> Self { my_class {num, debug} } fn __str__(&self) -> PyResult<String> { Ok(format!("[__str__] Num: {}, Debug: {}", self.num, self.debug)) } fn __repr__(&self) -> PyResult<String> { Ok(format!("[__repr__] Num: {}, Debug: {}", self.num, self.debug)) } } #[pymodule] fn pymspdb(_py: Python, m: &PyModule) -> PyResult<()> { m.add_class::<my_class>()?; Ok(()) }  I build this (into release mode), and test the code with the following code: from my_module import my_class def main(): dsa = my_class(1, True) print(dsa) print(dsa.__str__()) if __name__ == "__main__": main()  When running the test python code, I get the following output: <my_class object at 0x7fb7828ae950> [__str__] Num: 1, Debug: true  Now I have thought of possible solutions to this. One solution might be the pyo3 rust crate actually acts as a proxy, and in order to port classes into python might implement some sort of object which transfers all actions over to the ported class. So it might not implement its own __str__ therefore not giving me what I want. The second possible solution I thought of was I might not be overloading the __str__ function properly, therefore when python tries to use the print function it doesn't access the correct function and just does the default behavior. Thanks for reading so far, hope I can find an answer since I didn't find anything online for this.

Answer: I'm pretty sure this is because you need to implement these methods through the PyObjectProtocol trait. Many Python __magic__ methods correspond to C-level function pointer slots in a type object's memory layout. A type implemented in C needs to provide a function pointer in the slot, and Python will automatically generate a method to wrap the pointer for explicit method calls. A type implemented in Python will automatically have function pointers inserted that delegate to the magic methods. The Python internals will usually look for the function pointer rather than the corresponding magic method, and if Python doesn't find the function pointer, it will behave as though the method doesn't exist. That's why, for example, you had to use #[new] to mark your constructor instead of implementing a __new__ static method. __str__ and __repr__ also correspond to function pointers - specifically, tp_str and tp_repr. If you just try to implement them as regular methods, pyo3 won't generate the function pointers needed. PyObjectProtocol is the pyo3 interface to go through for that.
Question: I'm building an app which has a feature for embedding expressions/rules in a config yaml file. So for example user can reference a variable defined in yaml file like ${variables.name == 'John'} or ${is_equal(variables.name, 'John')}. I can probably get by with simple expressions but I want to support complex rules/expressions such ${variables.name == 'John'} and (${variables.age > 18} OR ${variables.adult == true}) I'm looking for a parsing/dsl/rules-engine library that can support these type of expressions and normalize it. I'm open using ruby, javascript, java, or python if anyone knows of a library for that languages. One option I thought of was to just support javascript as conditons/rules and basically pass it through eval with the right context setup with access to variables and other reference-able vars.

Answer: I don't know if you use Golang or not, but if you use it, I recommend this https://github.com/antonmedv/expr. I have used it for parsing bot strategy that (stock options bot). This is from my test unit: func TestPattern(t *testing.T) { a := "pattern('asdas asd 12dasd') && lastdigit(23asd) < sma(50) && sma(14) > sma(12) && ( macd(5,20) > macd_signal(12,26,9) || macd(5,20) <= macd_histogram(12,26,9) )" r, _ := regexp.Compile(`(\w+)(\s+)?[(]['\d.,\s\w]+[)]`) indicator := r.FindAllString(a, -1) t.Logf("%v\n", indicator) t.Logf("%v\n", len(indicator)) for _, i := range indicator { t.Logf("%v\n", i) if strings.HasPrefix(i, "pattern") { r, _ = regexp.Compile(`pattern(\s+)?\('(.+)'\)`) check1 := r.ReplaceAllString(i, "$2") t.Logf("%v\n", check1) r, _ = regexp.Compile(`[^du]`) check2 := r.FindAllString(check1, -1) t.Logf("%v\n", len(check2)) } else if strings.HasPrefix(i, "lastdigit") { r, _ = regexp.Compile(`lastdigit(\s+)?\((.+)\)`) args := r.ReplaceAllString(i, "$2") r, _ = regexp.Compile(`[^\d]`) parameter := r.FindAllString(args, -1) t.Logf("%v\n", parameter) } else { } } }  Combine it with regex and you have good (if not great, string translator). And for Java, I personally use https://github.com/ridencww/expression-evaluator but not for production. It has similar feature with above link. It supports many condition and you don't have to worry about Parentheses and Brackets. Assignment = Operators + - * / DIV MOD % ^ Logical < <= == != >= > AND OR NOT Ternary ? : Shift << >> Property ${<id>} DataSource @<id> Constants NULL PI Functions CLEARGLOBAL, CLEARGLOBALS, DIM, GETGLOBAL, SETGLOBAL NOW PRECISION  Hope it helps.
Question: Swagger documentation says you can do that: https://swagger.io/docs/specification/grouping-operations-with-tags/ But unfortunately drf-yasg not implementing this feature: https://github.com/axnsan12/drf-yasg/issues/454 It is said, that I can add custom generator class, but it is a very general answer. Now I see that drf_yasg.openapi.Swagger gets info block and I have thoughts, that this might be right place to put global tags section as an additional init argument, but it deeper, than customizing generator class and I have lack of knowledge of this module Does anybody have solution to this particular problem, or at least maybe a link to some sort of tutorial, how to properly customize generator class?

Answer: Unfortunately, this is a current issue with drf-yasg. To actually achieve this, you need to create your own schema generator class: from drf_yasg.generators import OpenAPISchemaGenerator class CustomOpenAPISchemaGenerator(OpenAPISchemaGenerator): def get_schema(self, request=None, public=False): """Generate a :class:`.Swagger` object with custom tags""" swagger = super().get_schema(request, public) swagger.tags = [ { "name": "api", "description": "everything about your API" }, { "name": "users", "description": "everything about your users" }, ] return swagger  Make sure to also include it in your Schema View from drf_yasg.views import get_schema_view from drf_yasg import openapi schema_view = get_schema_view( openapi.Info( title="My API", default_version='v1', ), generator_class=CustomOpenAPISchemaGenerator, )  Hope this works for you!
Question: After importing Pandas, when creating a pandas dataframe, Intellisense doesn't show the available attributes/methods of the created object.(Image 2, where I try to use the .head() function). It detects the module pd(pandas) methods without any problem (see Image 1). I don't have this problem when running a Jupyter Notebook or Jupyter Lab on the browser. I'm using:  Windows 7 Python 3.8.3 in a Conda environment. VSCODE 1.46.1 Python extension 2020.6.90262 Microsoft Language Server Visual Studio Intellicode 1.2.8  IMAGE 1: It uses intellisense to detect the module methods/attributes  IMAGE 2: Intellisense doesn't show the pandas object available attributes/methods 

Answer: The detection isn't working because IntelliSense has a hard time with pandas (and pandas.read_csv() especially). It works in Jupyter because it's accessing the live data while IntelliSense has to infer everything from the source code statically. I would advise trying out Pylance as it's the new language server from Microsoft and we have tried to support pandas appropriately. If Pylance doesn't work then try different values for your python.languageServer setting and see which one gives you the best result.
Question: I'm trying to access predictions of intermediate layers of a model during training using custom callback. Following stripped down version of the actual code demonstrates the issue. import tensorflow as tf import numpy as np class Model(tf.keras.Model): def __init__(self, input_shape=None, name="cus_model", **kwargs): super(Model, self).__init__(name=name, **kwargs) def build(self, input_shape): self.dense1 = tf.keras.layers.Dense(input_shape=input_shape, units=32) def call(self, input_tensor): return self.dense1(input_tensor) class CustomCallback(tf.keras.callbacks.Callback): def on_epoch_end(self, epoch, logs=None): get_output = tf.keras.backend.function( inputs = self.model.layers[0].input, outputs = self.model.layers[0].output ) print("Layer output: ",get_output.outputs) X = np.ones((8,16)) y = np.sum(X, axis=1) model = Model() model.compile(optimizer='adam',loss='mean_squared_error', metrics='accuracy') model.fit(X,y, epochs=8, callbacks=[CustomCallback()])  The callback is written as suggested in this answer. Getting following error: <ipython-input-3-635fd53dbffc> in on_epoch_end(self, epoch, logs) 12 def on_epoch_end(self, epoch, logs=None): 13 get_output = tf.keras.backend.function( ---> 14 inputs = self.model.layers[0].input, 15 outputs = self.model.layers[0].output 16 ) . . AttributeError: Layer dense is not connected, no input to return.  What's causing this? How to resolve it?

Answer: I also cannot get the self.layers[0].input because of the same error, but maybe u can directly call function defined in Model like this: class Model(tf.keras.Model): def __init__(self, input_shape=None, name="cus_model", **kwargs): super(Model, self).__init__(name=name, **kwargs) if not input_shape: input_shape = (10,) self.dense1 = tf.keras.layers.Dense(input_shape=input_shape, units=32) self.dev_dataset = np.ones((8,16)) def call(self, input_tensor): return self.dense1(input_tensor) class CustomCallback(tf.keras.callbacks.Callback): def on_epoch_end(self, epoch, logs=None): self.model.call(self.model.dev_dataset) X = np.ones((8,16)) y = np.sum(X, axis=1) model = Model() model.compile(optimizer='adam',loss='mean_squared_error', metrics='accuracy') model.fit(X,y, epochs=1, callbacks=[CustomCallback()])
Question: I have a few seaborn lineplots and I can't figure out how to increase the width of my lines. Here is my code #graph 1 sns.lineplot(x="date", y="nps", data=df_nps, ax=ax1, label="NPS", color='#0550D0') sns.lineplot(x="date", y="ema28", data=df_nps, ax=ax1, label="EMA28", color='#7DF8F3') sns.lineplot(x="date", y="ema7", data=df_nps, ax=ax1, label="EMA7", color='orange') #graph 2 dfz_nps_lineplot = sns.lineplot(x="date", y="nps", data=dfz_nps, ax=ax2, label="NPS", color='#0550D0') dfz_nps_lineplot = sns.lineplot(x="date", y="ema28", data=dfz_nps, ax=ax2, label="EMA28", color='#7DF8F3') dfz_nps_lineplot = sns.lineplot(x="date", y="ema7", data=dfz_nps, ax=ax2, label="EMA7", color='orange') #graph3 dfp_nps_lineplot = sns.lineplot(x="date", y="nps", data=dfp_nps, ax=ax3, label="NPS", color='#0550D0') dfp_nps_lineplot = sns.lineplot(x="date", y="ema28", data=dfp_nps, ax=ax3, label="EMA28", color='#7DF8F3') dfp_nps_lineplot = sns.lineplot(x="date", y="ema7", data=dfp_nps, ax=ax3, label="EMA7", color='orange') # formatting plt.show()  

Answer: As you can see from seaborn.lineplot documentation, the function accepts matplotlib.axes.Axes.plot() arguments, which means you can pass the same arguments you can to matplotlib function in this documentation. If you want to simply adjust the width of your lineplots I find this the easiest: pass an argument linewidth = your_desired_line_width_in_float , for example, linewidth = 1.5 in your sns.lineplot() functions. You can find additional possible arguments in the documentations linked. Example output on random data: seaborn.lineplot() without linewdith argument provided  seaborn.lineplot() with linewidth = 3
Question: I want to download all the python packages mentioned in the requirement.txt to a folder in Linux. I don't want to install them. I just need to download them. python version is 3.6 list of packages in the requirement.txt aiodns==0.3.2 aiohttp==1.1.5 amqp==1.4.7 anyjson==0.3.3 astroid==1.3.2 asyncio==3.4.3 asyncio-redis==0.14.1 billiard==3.3.0.20 blist==1.3.6 boto==2.38.0 celery==3.1 pexpect==4.0 pycryptodomex==3.7.0 pycurl==7.19.5.1 pyinotify==0.9.6 pylint==1.4.0 pyminifier==2.1 pyOpenSSL==0.15.1 pypacker==2.9 pyquery==1.2.9 pysmi==0.3.2 pysnmp==4.4.4 PyStaticConfiguration==0.9.0 python-daemon==2.1.2 python-dateutil==2.4.2 python-ldap==3.2.0 python-libnmap==0.6.2 python-otrs==0.4.3 pytz==2015.4 PyYAML==3.11 query-string==0.0.2 queuelib==1.2.2 redis==2.10.3 requests==2.22.1 requests-aws4auth==0.9 requests-oauthlib==0.5.0 requests-toolbelt==0.5.0 scp==0.10.2 six==1.10.0 South==1.0.1 tlslite==0.4.9 u-msgpack-python==2.1 urllib3==1.14 w3lib==1.12.0 websockets==3.3 Werkzeug==0.10.4 xlrd==1.0.0 XlsxWriter==1.0.5 zope.interface==4.1.2 GitPython==2.1.3 

Answer: The documentation gives what you want : pip download  pip download does the same resolution and downloading as pip install, but instead of installing the dependencies, it collects the downloaded distributions into the directory provided  source So you may try these option with pip download : pip download -r requirement.txt -d your_directory