eshangj/stackoverflow_q_and_a_sample huggingface dataset:
Question: I'm trying to the image shown below and I thought python would be a good idea to do this but I'm not sure. I want to randomize lots of football players' stats, make a radar chart for each and save the charts as images.  But the plotly radar charts are not so stylish and I really want to make something stylish. How to turn the below demo code into the reference image and is it possible? Here's a demo code: import plotly.graph_objects as go categories = ['Defending','Speed','Attacking', 'Technical', 'Team play'] fig = go.Figure() fig.add_trace(go.Scatterpolar( r=[1, 5, 2, 2, 3], theta=categories, fill='toself', name='Alice' )) fig.add_trace(go.Scatterpolar( r=[4, 3, 2.5, 1, 2], theta=categories, fill='toself', name='Bob' )) fig.update_layout( polar=dict( radialaxis=dict( visible=True, range=[0, 5] )), showlegend=False ) fig.show() 

Answer: From the documentation for polar layout, it seems that Plotly does not offer much options when it comes the grid shape itself. However, Plotly allows you to create your own templates/themes or examine built-in themes. As a starting point, you should probably analyze the plotly_dark theme as it has some features similar to your picture. simple example with built-in template  dataset.csv categories,player,points Defending,alice,1 Speed,alice,5 Attacking,alice,2 Technical,alice,2 Team play,alice,3 Defending,bob,4 Speed,bob,3 Attacking,bob,2.5 Technical,bob,1 Team play,bob,2  code import plotly.express as px import pandas as pd df = pd.read_csv("dataset.csv") fig = px.line_polar(df, r="points", theta="categories", color="player", line_close=True, color_discrete_sequence=["#00eb93", "#4ed2ff"], template="plotly_dark") fig.update_polars(angularaxis_showgrid=False, radialaxis_gridwidth=0, gridshape='linear', bgcolor="#494b5a", radialaxis_showticklabels=False ) fig.update_layout(paper_bgcolor="#2c2f36") fig.show()  With the above code I don't think it is possible to modify the color of each nested shape. To be able to do so, you will probably have to create your own template and color each nested shape separately. creating grid shape You might have to try something similar the code below to create your desired grid shape. import plotly.graph_objects as go bgcolors = ["#353841", "#3f414d", "#494b5a", "#494b5a", "#58596a"] fig = go.Figure(go.Scatterpolar( r=[42]*8, theta=[0, 45, 90, 135, 180, 225, 270, 315], marker_line_width=2, opacity=0.8, marker=dict(color=bgcolors[0]) )) for i in range(1, 5): fig.add_trace(go.Scatterpolar( r=[44-6*i]*8, theta=[0, 45, 90, 135, 180, 225, 270, 315], marker_line_width=2, marker=dict(color=bgcolors[i]) )) fig.update_polars(angularaxis_dtick='') fig.update_traces(fill='toself') fig.update_polars(angularaxis_showgrid=False, radialaxis_showgrid=False, radialaxis_gridwidth=0, gridshape='linear', radialaxis_showticklabels=False, angularaxis_layer='above traces' ) fig.show()   The colors are off but the general shape is good.
Question: I'm trying to load several types of csv files using Autoloader, it currently merge all csv that I drop into a big parquet table, what I want is to create parquet tables for each type of schema/csv_file Current code does:  #Streaming files/ waiting a file to be dropped spark.readStream.format("cloudFiles") \ .option("cloudFiles.format", "csv") \ .option("delimiter", "~|~") \ .option("cloudFiles.inferColumnTypes","true") \ .option("cloudFiles.schemaLocation", pathCheckpoint) \ .load(sourcePath) \ .writeStream \ .format("delta") \ .option("mergeSchema", "true") \ .option("checkpointLocation", pathCheckpoint) \ .start(pathResult)  What I want 

Answer: You can create different autoloader streams for each file from the same source directory and filter the filenames to consume by using the pathGlobFilter option on Autoloader (databricks documentation). By doing that you will end up having different checkpoints for each table and you can have the different schemas working. This solution is similar to the one pointed here How to filter files in Databricks Autoloader stream.
Question: Can anyone help me to fix this error? FAILURE: Build failed with an exception.  What went wrong: Execution failed for task ':app:compileDebugJavaWithJavac'.   Failed to query the value of task ':app:compileDebugJavaWithJavac' property 'options.generatedSourceOutputDirectory'. Querying the mapped value of map(java.io.File property(org.gradle.api.file.Directory, fixed(class org.gradle.api.internal.file.DefaultFilePropertyFactory$FixedDirectory, C:\Users\any user\Documents\Desktop\Chaquopy 9.1\app\build\generated\ap_generated_sources\debug\out)) org.gradle.api.internal.file.DefaultFilePropertyFactory$ToFileTransformer@7424ad41) before task ':app:compileDebugJavaWithJavac' has completed is not supported   Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.  Get more help at https://help.gradle.org   Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/7.0.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 2s 12 actionable tasks: 12 up-to-date

Answer: It's an AGP related issue: https://youtrack.jetbrains.com/issue/KTIJ-19252 Updating AGP up to 4.2.2 or higher will probably solve this issue.
Question: I'm looking for a way to export a huge Graph generated with pyvis in to a vector graphic .svg or at least .png format. Is there a way to do it? So far I've only found the option to save / export as .html file. Thanks in advance.

Answer: For me this worked:  save the graph to HTML file open the HTML in a new tab right-click -> save image as ...
Question: Empirically it seems that whenever you set_index on a Dask dataframe, Dask will always put rows with equal indexes into a single partition, even if it results in wildly imbalanced partitions. Here is a demonstration: import pandas as pd import dask.dataframe as dd users = [1]*1000 + [2]*1000 + [3]*1000 df = pd.DataFrame({'user': users}) ddf = dd.from_pandas(df, npartitions=1000) ddf = ddf.set_index('user') counts = ddf.map_partitions(lambda x: len(x)).compute() counts.loc[counts > 0] # 500 1000 # 999 2000 # dtype: int64  However, I found no guarantee of this behaviour anywhere. I have tried to sift through the code myself but gave up. I believe one of these inter-related functions probably holds the answer:  set_index set_partitions rearrange_by_column rearrange_by_column_tasks SimpleShuffleLayer  When you set_index, is it the case that a single index can never be in two different partitions? If not, then under what conditions does this property hold?  Bounty: I will award the bounty to an answer that draws from a reputable source. For example, referring to the implementation to show that this property has to hold.

Answer:  is it the case that a single index can never be in two different partitions?  No, it's certainly allowed. Dask will even intend for this to happen. However, because of a bug in set_index, all the data will still end up in one partition. An extreme example (every row is the same value except one): In [1]: import dask.dataframe as dd In [2]: import pandas as pd In [3]: df = pd.DataFrame({"A": [0] + [1] * 20}) In [4]: ddf = dd.from_pandas(df, npartitions=10) In [5]: s = ddf.set_index("A") In [6]: s.divisions Out[6]: (0, 0, 0, 0, 0, 0, 0, 1)  As you can see, Dask intends for the 0s to be split up between multiple partitions. Yet when the shuffle actually happens, all the 0s still end up in one partition: In [7]: import dask In [8]: dask.compute(s.to_delayed()) # easy way to see the partitions separately Out[8]: ([Empty DataFrame Columns: [] Index: [], Empty DataFrame Columns: [] Index: [], Empty DataFrame Columns: [] Index: [], Empty DataFrame Columns: [] Index: [], Empty DataFrame Columns: [] Index: [], Empty DataFrame Columns: [] Index: [], Empty DataFrame Columns: [] Index: [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]],)  This is because the code deciding which output partition a row belongs doesn't consider duplicates in divisions. Treating divisions as a Series, it uses searchsorted with side="right", hence why all the data always ends up in the last partition. I'll update this answer when the issue is fixed.
Question: I've been trying to write a little python script to automate some API testing. I need it to pick up the whole JSON body from a CSV or other format file, but not just a single body per file, rather iterate over all the "bodies" in it. The way I concocted it is, each cell, or value, is an entire body. This comes from how I'm managing various tests in Google Sheets, with the whole JSON bodies in their own cells, and can then be easily exported as CSV files. The issue is that I keep hitting "wrong format" type errors. I think the problem is that, as it's picking it up as a CSV "value", it inputs the data weirdly and that's why it won't work. Sample "problematic" input, i.e. the value that is picked up from the CSV file, as caught through a breakpoint: '{"post":2027,"name":"Test User","email":"email@mail.com","body":"lorem ipsum4"}'  I've already tried a lot of things. This is where I'm at right now. Further below is some sample data and more explanations. Code: from os import read import requests import csv import json filename = 'file.csv' url = "https://gorest.co.in/public/v1/posts/[id]/comments" headers = { "Content-Type": "application/json", "Authorization": "Bearer tricked ya" } with open(filename) as cF: reader = csv.reader(cF) for idx,row in enumerate(reader): for col in row: print("Body: ", col) r = requests.request("POST", url, headers = headers, json = col) print("Status: ", r.status_code) print("Status: ", r.reason) print("Text response: ", r.text) print("\nTest number: ",idx,"\n")  Sample data. In here, each row is a row in a csv file: {"post":2027,"name":"Test User","email":"email@mail.com","body":"lorem ipsum1"} {"post":2027,"name":"Test User","email":"email@mail.com","body":"lorem ipsum2"} {"post":2027,"name":"Test User","email":"email@mail.com","body":"lorem ipsum3"} {"post":2027,"name":"Test User","email":"email@mail.com","body":"lorem ipsum4"}  Sample output: ("Text response" slightly post-formatted for readability) Body: {"post":2027,"name":"Test User","email":"email@mail.com","body":"lorem ipsum4"} Status: 422 Status: Unprocessable Entity Text response: { "meta": null, "data": [{ "field": "name", "message": "can't be blank" }, { "field": "email", "message": "can't be blank" }, { "field": "body", "message": "can't be blank" }] } Test number: 4  The "odd" thing I've noticed, is that I can sometimes (in previous versions) input the body that is printed out (such as in the sample output) back into the JSON, when I'm using breakpoints, and that will work perfectly. So I tried using something to "capture" that "working printed body", but that wasn't really doable, or I didn't do it right.

Answer: csv.reader returns rows of strings, so the strings each need to be converted to a Python object for the json keyword argument in requests.request. We can use json.loads to deserialize a string. req_obj = json.loads(col) r = requests.request("POST", url, headers = headers, json = req_obj)
Question: I am trying to instantiate objects with hydra, I have a class torchio.transforms.RemapLabels that I am using in my config file: _target_: torchio.transforms.RemapLabels  The problem is that torchio.transforms.RemapLabels takes dictionary elements as input, how do I pass those from my hydra config file? (config.yaml)? I am getting error when instantiating: TypeError: Error instantiating 'torchio.transforms.preprocessing.label.remap_labels.RemapLabels' : __init__() missing 1 required positional argument: 'remapping'  example usage of remap label: transform = torchio.RemapLabels({2:1, 4:3, 6:5, 8:7}) 

Answer: There are two options: you can pass the inputs as positional arguments or as named arguments. Using named arguments (a.k.a. keyword arguments) in your yaml file: _target_: torchio.transforms.RemapLabels remapping: 2: 1 4: 3 6: 5 8: 7 masking_method: "Anterior"  or, using json-style maps: _target_: torchio.transforms.RemapLabels remapping: {2: 1, 4: 3, 6: 5, 8: 7} masking_method: "Anterior"  Using positional arguments in your yaml file: _target_: torchio.transforms.RemapLabels _args_: - 2: 1 4: 3 6: 5 8: 7 - "Anterior"  Or, equivalently: _target_: torchio.transforms.RemapLabels _args_: - {2: 1, 4: 3, 6: 5, 8: 7} - "Anterior"  For more information, please refer to the docs on Instantiating objects with Hydra.
Question: I am using pyaudio to record sounds on my Mac BigSur 11.6 (20G165). Specifically, I'm redirecting sound from an application to the input using BlackHole, which works fine. It usually works fine but, sometimes, I get this error in the Terminal:  ||PaMacCore (AUHAL)|| Error on line 2500: err='-10863', msg=Audio Unit: cannot do in current context  Any idea why or how I could prevent it from happening (like, waiting until PaMacCore is ready to record again or something)? I already tried reinstalling but it doesn't help brew install portaudio  or brew install portaudio --HEAD  Here's my code: import pyaudio import wave class Recorder(object): def __init__(self, fname, mode, channels, rate, frames_per_buffer): self.fname = fname self.mode = mode self.channels = channels self.rate = rate self.frames_per_buffer = frames_per_buffer self._pa = pyaudio.PyAudio() self.wavefile = self._prepare_file(self.fname, self.mode) self._stream = None def __enter__(self): return self def __exit__(self, exception, value, traceback): self.close() def record(self, duration): # Use a stream with no callback function in blocking mode self._stream = self._pa.open(format=pyaudio.paInt16, channels=self.channels, rate=self.rate, input=True, frames_per_buffer=self.frames_per_buffer) for _ in range(int(self.rate / self.frames_per_buffer * duration)): audio = self._stream.read(self.frames_per_buffer) self.wavefile.writeframes(audio) return None def start_recording(self): # Use a stream with a callback in non-blocking mode self._stream = self._pa.open(format=pyaudio.paInt16, channels=self.channels, rate=self.rate, input=True, frames_per_buffer=self.frames_per_buffer, stream_callback=self.get_callback()) self._stream.start_stream() return self def stop_recording(self): self._stream.stop_stream() return self def get_callback(self): def callback(in_data, frame_count, time_info, status): self.wavefile.writeframes(in_data) return in_data, pyaudio.paContinue return callback def close(self): self._stream.close() self._pa.terminate() self.wavefile.close() def _prepare_file(self, fname, mode='wb'): wavefile = wave.open(fname, mode) wavefile.setnchannels(self.channels) wavefile.setsampwidth(self._pa.get_sample_size(pyaudio.paInt16)) wavefile.setframerate(self.rate) return wavefile  Usage example: recfile = Recorder(filename, 'wb', 2, 44100, 1024) recfile.start_recording() ... recfile.stop_recording() 

Answer: Apparently the problem were mismatched bitrates in BlackHole's aggregated output device. I was aggregating Blackhole's output (44,1kHz) and the Mac Speakers (48kHz). This did not cause any consistent bad behaviour but sometimes led to these errors.
Question: With sentry_sdk, the sentry documentation explain how to automatically catch exceptions or logging messages. However, how can I catch a python warning, like a DeprecationWarning that would be raised with warnings.warn(DeprecationWarning, "warning message") 

Answer: There's no certain API in sentry for sending warnings, However, you need to ensure you're logging these with the general logging infrastructure that you are using. For example, if you are using Django, you have to change the logging level to be Warning like below in the settings.py file LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'formatters': { 'verbose': { 'format': '%(asctime)s %(levelname)s [%(name)s:%(lineno)s] %(module)s %(process)d %(thread)d %(message)s' } }, 'handlers': { 'console': { 'level': 'WARNING', 'class': 'logging.StreamHandler' }, }, 'loggers': { "": { "level": "WARNING", 'handlers': ['console'], "propagate": True } } }  and no change in sentry config import sentry_sdk from sentry_sdk.integrations.django import DjangoIntegration sentry_config = { 'dsn': os.getenv("SENTRY_DSN", "YOUR CDN"), 'integrations': [DjangoIntegration()], # Set traces_sample_rate to 1.0 to capture 100% # of transactions for performance monitoring. # We recommend adjusting this value in production. 'traces_sample_rate': 1.0, # If you wish to associate users to errors (assuming you are using # django.contrib.auth) you may enable sending PII data. 'send_default_pii': True } sentry_sdk.init(**sentry_config)  In case you don't have Logging infrastructure, you can implement your own, check this Question, it has a lot of examples of how you would create a custom logger. It's all about changing your level to be WARNING and creating a console handler(StreamHandler), then Sentry will take care of the rest Edit: I meant to capture logging.warning(), but for warnings.warn() you have to log them, Python provides a built-in integration between the logging module and the warnings module to let you do this; just call logging.captureWarnings(True) at the start of your script or your custom logger and all warnings emitted by the warnings module will automatically be logged at level WARNING.
Question: I'm working with pytorch now, and I'm missing a layer: tf.keras.layers.StringLookup that helped with the processing of ids. Is there any workaround to do something similar with pytorch? An example of the functionality I'm looking for: vocab = ["a", "b", "c", "d"] data = tf.constant([["a", "c", "d"], ["d", "a", "b"]]) layer = tf.keras.layers.StringLookup(vocabulary=vocab) layer(data) Outputs: <tf.Tensor: shape=(2, 3), dtype=int64, numpy= array([[1, 3, 4], [4, 1, 2]])> 

Answer: Package torchnlp, pip install pytorch-nlp  from torchnlp.encoders import LabelEncoder data = ["a", "c", "d", "e", "d"] encoder = LabelEncoder(data, reserved_labels=['unknown'], unknown_index=0) enl = encoder.batch_encode(data) print(enl)  tensor([1, 2, 3, 4, 3])
Question: I had no problems in the past with the deployment to Heroku via HTTP transport, but recently I am unable to deploy. This is the error I am getting: Enumerating objects: 58668, done. Counting objects: 100% (57434/57434), done. Delta compression using up to 16 threads Compressing objects: 100% (16705/16705), done. Writing objects: 100% (57124/57124), 50.77 MiB | 76.23 MiB/s, done. Total 57124 (delta 44149), reused 52353 (delta 40249) error: RPC failed; HTTP 504 curl 22 The requested URL returned error: 504 fatal: the remote end hung up unexpectedly fatal: the remote end hung up unexpectedly  I've tried switching to SSH transport and it works, but Heroku is retiring SSH transport, so I need to figure out this error. Also I tried to change the postBuffer according to atlassian's page, but I got the error again. git config --global http.postBuffer 157286400  Does anybody have an idea how to solve this? There are very few resources on the web that I found, and none of them are fixing the problem.

Answer: I raised a support ticket to Heroku and the answer was to reset the Git repo heroku plugins:install heroku-repo heroku repo:reset -a <app-name>  After I did this, I had no problems with the deployment
Question: Error Details  Two requests have been generating on button click. What did I search so far? Axios blocked by CORS policy with Django REST Framework CORS issue with react and django-rest-framework but to no avail What am I doing? Submitting POST request from react to DJango API Django side settings file CORS_ORIGIN_ALLOW_ALL = True ALLOWED_HOSTS = [ "http://127.0.0.1:3000", "http://127.0.0.1", "http://localhost:3000", "http://localhost" ] CORS_ORIGIN_WHITELIST = [ "http://127.0.0.1:3000", "http://127.0.0.1", "http://localhost:3000", "http://localhost" ] INSTALLED_APPS = [ ......, "corsheaders" ] MIDDLEWARE = [ ........., 'corsheaders.middleware.CorsMiddleware', 'django.middleware.common.CommonMiddleware', ]  React axios request function authenticate() { let body = { "email": "ac", "password": "def" }; const headers = { 'Access-Control-Allow-Origin': '*', 'Content-Type': 'application/json', } axios.post("http://127.0.0.1:8000/login/", body, { headers: headers }) .then(function(response) { console.log(response.data); }) .catch(function(error) { console.log(error); }); }  Tried another approach using fetch, but to no avail function authenticate() { let body = { "email": "hi", "password": "pass" }; const headers = { 'Content-Type': 'application/json', } fetch("http://127.0.0.1:8000/login", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(body) }) .then(function(response) { console.log(response.data); }) .catch(function(error) { console.log(error); }); }  DJango side method def Login(request): if(request.method == "POST"): return JsonResponse({"message" : "Invalid credentials"}, status=401) 

Answer: Below settings work for me CORS_ORIGIN_ALLOW_ALL = True ALLOWED_HOSTS = [ "127.0.0.1", ] CORS_ALLOWED_ORIGINS = [ "http://127.0.0.1", ] CORS_ALLOW_CREDENTIALS = False INSTALLED_APPS = [ ..... "corsheaders" ] MIDDLEWARE = [ ...... 'corsheaders.middleware.CorsMiddleware', 'django.middleware.common.CommonMiddleware', ]
Question: I have a directory structure as such: airflow_dags ├── dags │ └── hk │ └── hk_dag.py ├── plugins │ └── cse │ └── operators.py │ └── cse_to_bq.py └── test └── dags └── dag_test.py  In the GCS bucket created by Cloud Composer, there's a plugin folder where I upload the cse folder. Now in my hk_dag.py file if I import the plugin like this: from plugins.cse.operators.cse_to_bq import CSEToBQOperator  and run my unit test, it passes, but in cloud composer I get a ModuleNotFoundError: No module named 'plugins' error message. If I import the plugin like this in my hk_dag.py: from cse.operators.cse_to_bq import CSEToBQOperator  My unit test fails with ModuleNotFoundError: No module named 'cse' but it works fine in Cloud Composer. How do I resolve it?

Answer: In Airflow 2.0 to import your plugin you just need to do it directly from the operators module. In your case, has to be something like: from operators.cse_to_bq import CSEToBQOperator  But before that you have to change your folder structure to: airflow_dags ├── dags │ └── hk │ └── hk_dag.py ├── plugins │ └── operators │ └── cse │ └── cse_to_bq.py └── test └── dags └── dag_test.py