eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm trying to remove noise with morphology and the kernel is giving me errors: import skimage.io as io import numpy as np import cv2 c=io.imread('circles.png').astype('bool')*1 x=np.random.random_sample(c.shape) c[np.nonzero(x>0.95)]= 0 c[np.nonzero(x<=0.05)] = 1 opening = cv2.morphologyEx(c, cv2.MORPH_OPEN, np.ones((2,2),np.uint8)) io.imshow(opening)  error: error: OpenCV(4.1.2) C:/projects/opencv-python/opencv/modules/imgproc/src/morph.simd.hpp:756: error: (-213:The function/feature is not implemented) Unsupported data type (=4) in function 'cv::opt_AVX2::getMorphologyRowFilter' 

Answer: Your data type (=4) is CV_32SC1, which is 32-bit signed single channel -- you need to convert your data into another data type, I'd recommend using CV_8UC1 because of the smallest memory footprint and ease of use: c = c.astype('uint8') # or c.astype(np.byte)
Question: What's the best way to get a SQLAlchemy engine from an Airflow connection ID? Currently I am creating a hook, retrieving its URI, then using it to create a SQLAlchemy engine. postgres_hook = PostgresHook(self.postgres_conn_id) engine = create_engine(postgres_hook.get_uri())  This works but both commands make a connection to the database. When I have "extra" parameters on the connection a third connection is needed to retrieve those (see Retrieve full connection URI from Airflow Postgres hook) Is there a shorter and more direct method?

Answer: To be clear, indeed your commands will make two database connections, but it's to two separate databases (unless you're trying to connect to your Postgres Airflow database). The first line of initializing the hook should not make any connections. Only the second line first grabs the connection details from the Airflow database (which I don't think you can avoid), then uses that to connect to the Postgres database (which I think is the point). You can make slightly simpler though with: postgres_hook = PostgresHook(self.postgres_conn_id) engine = postgres_hook.get_sqlalchemy_engine()  That seems pretty clean, but if you want to get even more direct without going through PostgresHook, you could fetch it directly by querying Airflow's database. However, that means you're going to end up duplicating the code to build a URI from the connection object. The underlying implementation of get_connection() is a good example if you want to proceed with this. from airflow.settings import Session conn = session.query(Connection).filter(Connection.conn_id == self.postgres_conn_id).one() ... # build uri from connection create_engine(uri)  Additionally, if you want to be able to access the extras without a separate database fetch beyond what get_uri() or get_sqlalchemy_engine() does, is you can override BaseHook.get_connection() to save the connection object to an instance variable for reuse. This would require creating your own hook on top of PostgresHook, so I understand that may not be ideal. class CustomPostgresHook(PostgresHook): @classmethod def get_connection(cls, conn_id): # type: (str) -> Connection conn = super().get_connection(conn_id) self.conn_obj = conn # can't use self.conn because PostgresHook will overriden in https://github.com/apache/airflow/blob/1.10.10/airflow/hooks/postgres_hook.py#L93 by a different type of connection return conn postgres_hook = CustomPostgresHook(self.postgres_conn_id) uri = postgres_hook.get_uri() # do something with postgres_hook.conn_obj.extras_dejson  Some built in Airflow hooks have this behavior already (grpc, samba, tableau), but it's definitely not standardized.
Question: I was just going through the .rect method of pygame in the official docs. We have 2 cases ,  pygame.rect.move(arg1,arg2) which is used to move a .rect object on the screen and  pygame.rect.move_ip(arg1,arg2) which is , according to the docs, also used to move a .rect object on the screen but it moves it in place I didnt quite get what it means. Can anyone explain what move in place means?

Answer: "In place" means the object self. While rect.move_ip changes the pygame.Rect object itself, rect.move does not change the object, but it returns a new object with the same size and "moved" position. Note, the return value of rect.move_ip is None, but the return value of rect.move is a new pygame.Rect object. rect.move_ip(x, y) does the same as rect = rect.move(x, y)
Question: This might sound simple, but I have this problem. I have two docker containers running. One is for my front-end and other is for my backend services. these are the Dockerfiles for both services. front-end Dockerfile : # Use an official node runtime as a parent image FROM node:8 WORKDIR /app # Install dependencies COPY package.json /app RUN npm install --silent # Add rest of the client code COPY . /app EXPOSE 3000 CMD npm start  backend Dockerfile : FROM python:3.7.7 WORKDIR /usr/src/app COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY server.py /usr/src/app COPY . /usr/src/app EXPOSE 8083 # CMD ["python3", "-m", "http.server", "8080"] CMD ["python3", "./server.py"]  I am building images with the docker-compose.yaml as below: version: "3.2" services: frontend: build: ./frontend ports: - 80:3000 depends_on: - backend backend: build: ./backends/banuka ports: - 8080:8083  How can I make this two services Update whenever there is a change to front-end or back-end? I found this repo, which is a booilerplate for reactjs, python-flask and posgresel, which says it has enabled Hot reload for both reactjs frontend and python-flask backend. But I couldn't find anything related to that. Can someone help me? repo link What I want is: after every code change the container should b e up-to-date automatically !

Answer: Try this in your docker-compose.yml  version: "3.2" services: frontend: build: ./frontend environment: CHOKIDAR_USEPOLLING: "true" volumes: - /app/node_modules - ./frontend:/app ports: - 80:3000 depends_on: - backend backend: build: ./backends/banuka environment: CHOKIDAR_USEPOLLING: "true" volumes: - ./backends/banuka:/app ports: - 8080:8083  Basically you need that chokidar environment to enable hot reloading and you need volume bindings to make your code on your machine communicate with code in container. See if this works.
Question: I'm using Python to dig through a pretty big project and dig up information about it. I'm able to create an array of ProjectFiles, however I'm having a hard time figuring out how to filter it. class ProjectFile: def __init__(self, filename: str, number_of_lines: int, language: str, repo: str, size: int): self.filename = filename self.number_of_lines = number_of_lines self.language = language self.repo = repo self.size = size  How would I filter an array of ProjectFile objects for a specific repo? For instance, let's say I wanted to filter for objects whose repo property is SomeCocoapod. I've looked for examples of filter, but everything I've found uses simple examples like lists of str or int.

Answer: You can select attributes of a class using the dot notation.  Suppose arr is an array of ProjectFile objects. Now you filter for SomeCocoapod using.  filter(lambda p: p.repo == "SomeCocoapod", arr)  NB: This returns a filter object, which is a generator. To have a filtered list back you can wrap it in a list constructor.  As a very Pythonic alternative you can use list comprehensions: filtered_arr = [p for p in arr if p.repo == "SomeCocoapod"]
Question: I am trying to build a website that users can add the courses they are taking. I want to know how should I add the ManyToMany relationship. Such that we can get all users in a course based on the course code or instructor or any field. And we can also get the courses user is enrolled in. Currently, my Database structure is:  class Course(models.Model): course_code = models.CharField(max_length=20) course_university = models.CharField(max_length=100) course_instructor = models.CharField(max_length=100) course_year = models.IntegerField(('year'), validators=[MinValueValidator(1984), max_value_current_year]) def __str__(self): return self.course_code  and my user model: class Profile(AbstractUser): bio = models.TextField() image = models.ImageField(default='defaults/user/default_u_i.png', courses = models.ManyToManyField('home.Course',related_name='courses') def __str__(self): return self.username  I was wondering should ManyToMany relationship be in User model or the course model? Or will it make any difference at all? EDIT: For adding course to post object now I am using this view but it seems to not work:  @login_required def course_add(request): if request.method == "POST": form = CourseForm(request.POST or none) if form.is_valid(): course = form.save() request.user.add(course) else: form = CourseForm context = { 'form':form } return render(request,'home/courses/course_add.html', context) 

Answer: For a relational databases, the model where you define the ManyToManyField does not matter. Django will create an extra table with two ForeignKeys to the two models that are linked by the ManyToManyField. The related managers that are added, etc. is all Django logic. Behind the curtains, it will query the table in the middle. You however need to fix the related_name=… parameter [Django-doc]. The related_name specifies the name of the relation in reverse so from Course to Profile in this case. It thus should be something like 'profiles': class Profile(AbstractUser): bio = models.TextField() image = models.ImageField(default='defaults/user/default_u_i.png', courses = models.ManyToManyField('home.Course', related_name='profiles') def __str__(self): return self.username You thus can obtain the people that particiate in a Course object with: mycourse.profiles.all() and you can access the courses in which a Profile is enrolled with: myprofile.courses.all() For more information, see the Many-to-many relationships section of the documentation. You can add a course to the courses of a user with: @login_required def course_add(request): if request.method == 'POST': form = CourseForm(request.POST) if form.is_valid(): course = form.save() request.user.courses.add(course) else: form = CourseForm() context = { 'form': form } return render(request,'home/courses/course_add.html', context)
Question: I am porting some Python 2 code that calls split() on strings, so I need to know its exact behavior. The documentation states that when you do not specify the sep argument, "runs of consecutive whitespace are regarded as a single separator". Unfortunately, it does not specify which characters that would be. There are some obvious contenders (like space, tab, and newline), but Unicode contains plenty of other candidates. Which characters are considered to be whitespace by split()? Since the answer might be implementation-specific, I'm targeting CPython. (Note: I researched the answer to this myself since I couldn't find it anywhere, so I'll be posting it here, hopefully for the benefit of others.)

Answer: Unfortunately, it depends on whether your string is an str or a unicode (at least, in CPython - I don't know whether this behavior is actually mandated by a specification anywhere). If it is an str, the answer is straightforward:  0x09 Tab 0x0a Newline 0x0b Vertical Tab 0x0c Form Feed 0x0d Carriage Return 0x20 Space  Source: these are the characters with PY_CTF_SPACE in Python/pyctype.c, which are used by Py_ISSPACE, which is used by STRINGLIB_ISSPACE, which is used by split_whitespace. If it is a unicode, there are 29 characters, which in addition to the above are:  U+001c through 0x001f: File/Group/Record/Unit Separator U+0085: Next Line U+00a0: Non-Breaking Space U+1680: Ogham Space Mark U+2000 through 0x200a: various fixed-size spaces (e.g. Em Space), but note that Zero-Width Space is not included U+2028: Line Separator U+2029: Paragraph Separator U+202f: Narrow No-Break Space U+205f: Medium Mathematical Space U+3000: Ideographic Space  Note that the first four are also valid ASCII characters, which means that an ASCII-only string might split differently depending on whether it is an str or a unicode! Source: these are the characters listed in _PyUnicode_IsWhitespace, which is used by Py_UNICODE_ISSPACE, which is used by STRINGLIB_ISSPACE (it looks like they use the same function implementations for both str and unicode, but compile it separately for each type, with certain macros implemented differently). The docstring describes this set of characters as follows:  Unicode characters having the bidirectional type 'WS', 'B' or 'S' or the category 'Zs'
Question: In a statement like: g = sns.FacetGrid(df, row="variable", col="state")  I would like to control the order in which the variables and the states appear in the FacetGrid. I can't find where this is even possible. 

Answer: You can use row_order and col_order: from sklearn.datasets import load_iris import pandas as pd import seaborn as sns import numpy as np import matplotlib.pyplot as plt data = load_iris() df = pd.DataFrame(data.data, columns=['sepal.length','sepal.width','petal.length','petal.width']) df['species'] = data.target df['subset'] = np.random.choice(['A','B','C'],150,replace=True) g = sns.FacetGrid(df, col="species", col_order=[0,2,1],row="subset",row_order=['C','B','A']) g = g.map(plt.scatter, "sepal.length", "sepal.width")
Question: from typing import Optional @dataclass class Event: id: str created_at: datetime updated_at: Optional[datetime] #updated_at: datetime = field(default_factory=datetime.now) CASE 1 #updated_at: Optional[datetime] = None CASE 2 @dataclass class NamedEvent(Event): name: str  When creating an event instance I will generally not have an updated_at field. I can either pass the current time as default value or add a value to it when making the insertion in the database and fetch it in subsequent uses of the object. Which is the better way? As per my understanding, I cannot create a NamedEvent instance without passing the updated_at field in case1 and case2 since I do not have a default value in name field. 

Answer: The underlying problem that you have seems to be the same one that is described here. The short version of that post is that in a function signature (including the dataclass-generated __init__ method), obligatory arguments (like NamedEvent's name) can not follow after arguments with default values (which are necessary to define the behavior of Event's updated_at) - a child's fields will always follow after those of its parent. So either you have no default values in your parent class (which doesn't work in this case) or all your child's fields need default values (which is annoying, and sometimes simply not feasible). The post I linked above discusses some patterns that you can apply to solve your problem, but as a nicer alternative you can also use the third part party package pydantic which already solved this problem for you. A sample implementation could look like this: import pydantic from datetime import datetime class Event(pydantic.BaseModel): id: str created_at: datetime = None updated_at: datetime = None @pydantic.validator('created_at', pre=True, always=True) def default_created(cls, v): return v or datetime.now() @pydantic.validator('updated_at', pre=True, always=True) def default_modified(cls, v, values): return v or values['created_at'] class NamedEvent(Event): name: str  The default-value specification through validators is a bit cumbersome, but overall it's a very useful package that fixes lots of the shortcomings that you run into when using dataclasses, plus some more. Using the class definition, an instance of NamedEvent can be created like this: >>> NamedEvent(id='1', name='foo') NamedEvent(id='1', created_at=datetime.datetime(2020, 5, 2, 18, 50, 12, 902732), updated_at=datetime.datetime(2020, 5, 2, 18, 50, 12, 902732), name='foo')
Question: I tried to install pyaudio, but it returns an error like this: Could not build wheels for pyaudio, since package 'wheel' is not installed.   How can I fix this?

Answer: It looks like you just need to install the wheel package. You can do this by running pip install wheel at the terminal.
Question: Whenever I have to deploy a new python function using the gcloud sdk I get this message  Allow unauthenticated invocations of new function [function-name]? (y/N)? WARNING: Function created with limited-access IAM policy. To enable unauthorized access consider "gcloud alpha functions add-iam-policy-binding function-name --region=europe-west1 --member=allUsers --role=roles/cloudfunctions.invoker"  Is there any flag I can add to the command to make it a NO when deploying? This is a sample command I use to deploy one function: gcloud functions deploy function-name --region=europe-west1 --entry-point function-entry-point --trigger-resource "projects/my-project/databases/(default)/documents/user_ids/{user_id}" --trigger-event providers/cloud.firestore/eventTypes/document.create --runtime python37 --timeout 60 --project my-project 

Answer: From https://cloud.google.com/sdk/docs/scripting-gcloud#disabling_prompts:  You can disable prompts from gcloud CLI commands by setting the disable_prompts property in your configuration to True or by using the global --quiet or -q flag.  So for your example, you could run: gcloud functions deploy function-name --quiet --region=europe-west1 --entry-point function-entry-point --trigger-resource "projects/my-project/databases/(default)/documents/user_ids/{user_id}" --trigger-event providers/cloud.firestore/eventTypes/document.create --runtime python37 --timeout 60 --project my-project
Question: I have code which worked in Python 3.6 and fails in Python 3.8. It seems to boil down to calling super in subclass of typing.NamedTuple, as below: <ipython-input-2-fea20b0178f3> in <module> ----> 1 class Test(typing.NamedTuple): 2 a: int 3 b: float 4 def __repr__(self): 5 return super(object, self).__repr__() RuntimeError: __class__ not set defining 'Test' as <class '__main__.Test'>. Was __classcell__ propagated to type.__new__?  In [3]: class Test(typing.NamedTuple): ...: a: int ...: b: float ...: #def __repr__(self): ...: # return super(object, self).__repr__() ...: >>> # works  The purpose of this super(object, self).__repr__ call is to use the standard '<__main__.Test object at 0x7fa109953cf8>' __repr__ instead of printing out all the contents of the tuple elements (which would happen by default). There are some questions on super resulting in similar errors but they:  Refer to the parameter-less version super() Fail already in Python 3.6 (it worked for me before 3.6 -> 3.8 upgrade) I fail to understand how to fix this anyway, given that it's not a custom metaclass I have control over but the stdlib-provided typing.NamedTuple.  My question is how can I fix this while maintaining backwards compatibility with Python 3.6 (otherwise I'd just use @dataclasses.dataclass instead of inheriting from typing.NamedTuple)? A side question is how can this fail at definition time given that the offending super call is inside a method which is not even executed yet. For instance: In [3]: class Test(typing.NamedTuple): ...: a: int ...: b: float ...: def __repr__(self): ...: return foo  works (until we actually call the __repr__) even though foo is an undefined reference. Is super magical in that regard?

Answer: I was slightly wrong in the other question (which I just updated). Apparently, this behavior manifests in both cases of super. In hindsight, I should have tested this. What's happening here is the metaclass NamedTupleMeta indeed doesn't pass __classcell__ over to type.__new__ because it creates a namedtuple on the fly and returns that. Actually, in Python's 3.6 and 3.7 (where this is still a DeprecationWarning), the __classcell__ leaks into the class dictionary since it isn't removed by NamedTupleMeta.__new__. class Test(NamedTuple): a: int b: float def __repr__(self): return super().__repr__() # isn't removed by NamedTupleMeta Test.__classcell__ <cell at 0x7f956562f618: type object at 0x5629b8a2a708>  Using object.__repr__ directly as suggested by Azat does the trick.  how can this fail at definition time  The same way the following also fails: class Foo(metaclass=1): pass  Many checks are performed while the class is being constructed. Among these, is checking if the metaclass has passed the __classcell__ over to type_new.
Question: How do I run my Flask app which uses SSL keys using waitress. The SSL context is specified in my Flask's run() as in app.run(ssl_context=('cert.pem', 'key.pem'))  But app.run() is not used when using waitress as in the code below. So, where do I specify the keys? Thanks for the help. from flask import Flask, request app = Flask(__name__) @app.route("/") def hello(): return "Hello World!" if __name__ == '__main__': # app.run(ssl_context=('../cert.pem', '../key.pem')) from waitress import serve serve(app, host="0.0.0.0", port=5000) 

Answer: At the current version (1.4.3), Waitress does not natively support TLS. See TLS support in https://github.com/Pylons/waitress/blob/36240c88b1c292d293de25fecaae1f1d0ad9cc22/docs/reverse-proxy.rst You either need a reverse proxy in front to handle the tls/ssl part, or use another WSGI server (CherryPy, Tornado...).
Question: I have a python DAG Parent Job and DAG Child Job. The tasks in the Child Job should be triggered on the successful completion of the Parent Job tasks which are run daily. How can add external job trigger ? MY CODE from datetime import datetime, timedelta from airflow import DAG from airflow.operators.postgres_operator import PostgresOperator from utils import FAILURE_EMAILS yesterday = datetime.combine(datetime.today() - timedelta(1), datetime.min.time()) default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': yesterday, 'email': FAILURE_EMAILS, 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, 'retry_delay': timedelta(minutes=5) } dag = DAG('Child Job', default_args=default_args, schedule_interval='@daily') execute_notebook = PostgresOperator( task_id='data_sql', postgres_conn_id='REDSHIFT_CONN', sql="SELECT * FROM athena_rs.shipments limit 5", dag=dag ) 

Answer: Answer is in this thread already. Below is demo code: Parent dag: from datetime import datetime from airflow import DAG from airflow.operators.dummy_operator import DummyOperator default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2020, 4, 29), } dag = DAG('Parent_dag', default_args=default_args, schedule_interval='@daily') leave_work = DummyOperator( task_id='leave_work', dag=dag, ) cook_dinner = DummyOperator( task_id='cook_dinner', dag=dag, ) leave_work >> cook_dinner  Child dag: from datetime import datetime, timedelta from airflow import DAG from airflow.operators.dummy_operator import DummyOperator from airflow.operators.sensors import ExternalTaskSensor default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2020, 4, 29), } dag = DAG('Child_dag', default_args=default_args, schedule_interval='@daily') # Use ExternalTaskSensor to listen to the Parent_dag and cook_dinner task # when cook_dinner is finished, Child_dag will be triggered wait_for_dinner = ExternalTaskSensor( task_id='wait_for_dinner', external_dag_id='Parent_dag', external_task_id='cook_dinner', start_date=datetime(2020, 4, 29), execution_delta=timedelta(hours=1), timeout=3600, ) have_dinner = DummyOperator( task_id='have_dinner', dag=dag, ) play_with_food = DummyOperator( task_id='play_with_food', dag=dag, ) wait_for_dinner >> have_dinner wait_for_dinner >> play_with_food  Images: Dags  Parent_dag  Child_dag