Myashka/SO-Python_QA-filtered-2023-tanh_score-after_2023_02 huggingface dataset:
Title: How to display a GStreamer pipeline inside a Gtk window with a specific size?
Tags: python-3.x,gstreamer,drawingarea

Question: I'm working on a Python script that creates a GStreamer pipeline and displays the video output inside a Gtk window. The current code opens two windows: one with the title "Video Window" but with no content, and another one with the GStreamer pipeline inside.
However, I'd like to have the GStreamer pipeline displayed inside the first window as a 200x200 box, without opening a second window. I've tried using a Gtk.DrawingArea widget and the GstVideo.VideoOverlay interface, but I'm not sure how to integrate them into my code.
Here's the current code:
import gi
gi.require_version('Gtk', '3.0')
gi.require_version('Gst', '1.0')
from gi.repository import Gtk, Gst

# Gtk
win = Gtk.Window(title="Video Window")
win.connect("destroy", Gtk.main_quit)
win.set_default_size(600, 400)

# DrawingArea to hold the video
drawingarea = Gtk.DrawingArea()
drawingarea.set_size_request(200, 200)
win.add(drawingarea)

# Gst
Gst.init(None)
pipeline = Gst.parse_launch("videotestsrc ! autovideoconvert ! gtksink")
pipeline.set_state(Gst.State.PLAYING)

# End
win.show_all()
Gtk.main()

Can anyone help me modify this code to achieve my goal? I'm relatively new to Gtk and GStreamer, so a detailed explanation would be appreciated. Thank you in advance!
I may add if the sink is not recommended gtksink please inform me. I'm using Raspberry Pi 4 and need as low latency as possible on this. So the best sink to use would help.

Answer: The gtksink has a property widget where it draws the video to. Get that widget from that property and add this widget to your GTK application window hierachy where you want it to be displayed.
Title: How to calculate percent change of a pandas dataframe across groups with a custom period?
Tags: python,pandas

Question: I am trying to compute the percent change in a a column of a pandas dataframe. I can get it to work when I don't specify periods, but I would like to specify the number of periods to consider. It has to work across groups is the problem, I can get it to work without groups, but for some reason it's not working now?
_df_ty['velocity_7'] = _df_ty.groupby('store')['sales'].apply(pd.Series.pct_change(periods=7)).abs()

It yells at me with:
TypeError: pct_change() missing 1 required positional argument: 'self'

Which I don't understand how that's happening inside of apply? Google and the pandas docs aren't helping here.
Further, all the stack overflow answers I can find are just about calculating the percent change to begin with, I cannot find an example of one with grouping and a non-1 period in use.

Answer: Instead, you should call the pct_change() method on a pd.Series object directly, and then group the resulting series by store to calculate the percent change within each store:
_df_ty['velocity_7'] = _df_ty.groupby('store')['sales'].pct_change(periods=7).abs().groupby('store').fillna(0)
Note that we're calling fillna(0) at the end to replace any missing values (which will occur for the first 7 rows of each store) with 0, since the percent change for those rows is undefined.
Title: How to share an interactive jupyter notebook?
Tags: python,python-3.x,github,jupyter-notebook,file-sharing

Question: i need to send the Jupyter notebook to another person, and enable him to open, enter inputs and running the notebook, to get the results.
please explain in a steps what should i do to share an interactive jupyter notebook to the end user, so he can enter inputs and running the code.

Answer: Follow th following steps:
A. Save your Jupyter notebook file with a .ipynb extension.
B. Make sure that all the necessary libraries and packages are installed in the environment where the notebook will be run. You can provide that person with a requirement.txt file so he/she can create an adequate invironemnt.
C. Export the notebook as a runnable file: File -> Download as -> Runnable .ipynb.
D. Share the exported .ipynb file
E. Instruct the other person to open the .ipynb file in Jupyter Notebookwhatever other envirnoment/platform he/she needs.
Of course, youu have to make sure that the other person has the necessary software installed on their computer to run the notebook, including Python and any necessary packages.
Once they open the notebook, they should be able to run the code cells and input their own data.
Title: Why does ~True = -2 in python?
Tags: python,python-3.x

Question: I am completely perplexed. We came across a bug, which we easily fixed, but we are perplexed as to why the value the bug was generating created the output it did. Specifically:
Why does ~True equal -2 in python?
~True 
>> -2 

Shouldn't the bitwise operator ~ only return binary?
(Python v3.8)

Answer: Shouldn't the bitwise operator ~ only return binary?

Well, technically EVERYTHING in a computer is binary. However, this is only part of the story. There are two important concepts here:

Data types of values
Representations of those values

For the first one True is a boolean and -2 is an integer. In some cases, Python will convert a value from one type to another in order to perform certain operations. In this case, the ~ is the bitwise operator which only works on integers. In this case, an actual conversion isn't necessary since boolean inherits from int, so the boolean value can just be treated as the integer value of 1. Then ~ gives the bitwise inverse of that integer value.
Now if we represent the 1 in binary as 00000001 (yes, technically there are more 0s, but I'm not going to type them out...the concept still holds if we only use 8 bits instead of the actual 32 or 64), we negate this to 11111110 (again, there are more leading 1s because there are really more bits). Note that the way I write the integer value here in binary is merely a representation of the value using characters I can type on a keyboard. This is still 00000001 in binary and 1 in decimal both represent the same underlying value in memory. Similarly 11111110 binary and -2 decimal both represent the same value.
At the end, print() will just print the value in decimal which is why you get -2. If you want to print the value in hex or binary, there are built in methods to get these representations instead.
Title: How can I ensure that GCP Document AI model to output JSON with the same name as the input file?
Tags: python-3.x,google-cloud-platform,cloud-document-ai

Question: I am using Python to BatchProcess PDFs through GCP Document AI ("DocAI"). The PDFs have long file names such as 71.169892_01-2022.10.15-21275188-1111.pdf. Often the only difference between the filenames are the last four digits before .pdf (such as 71.169892_01-2022.10.15-21275188-1111.pdf and 71.169892_01-2022.10.15-21275188-2547.pdf)
When such a PDF is processed through DocAI, it outputs one or more JSON files with a shortened filename such as 71.169892_01-2022.10-0.json, 71.169892_01-2022.10-1.json, and so on. How can I ensure that DocAI does not cut off the filename? Is there an attribute I can add to BatchProcessing Request to ensure that the output preserves the full filename?
This is important because when I process 2 PDFs with nearly identical filenames (e.g. 71.169892_01-2022.10.15-21275188-1111.pdf and 71.169892_01-2022.10.15-21275188-2547.pdf), the resulting JSONs end up with the same filename: 71.169892_01-2022.10-0.json. Which is a problem when such JSONs are moved from the folder where there are automatically stored by DocAI into the same folder (that is--the second JSON simply overwrites the first JSON which has the same name).
The current state is as follows:
Input PDF: 71.169892_01-2022.10.15-21275188-1111.pdf
Output JSON: 71.169892_01-2022.10-0.json
Expecting:
Input PDF: 71.169892_01-2022.10.15-21275188-1111.pdf
Output JSON: 71.169892_01-2022.10.15-21275188-1111.json

Answer: Currently, there isn't a way to specify the output filename from Document AI, other than the output bucket & folder. Batch Processing will always output JSON files with an extra -0 or another number since larger documents can be split up into multiple "shards".
If it's possible, I would recommend sending the files that have nearly identical names in different requests to avoid the overwriting issue, since each request will output into a different folder named for the operation id.
However, this is definitely an edge case that should be handled in the product, so I'll report this issue to the development team.
Update: A fix has been made and it should be rolled out in the next couple of weeks. This should prevent the truncation of the filenames and the overwriting issue, but the output files will still have suffixes like -0.
Title: Check for line in CSV, if not existant - append the line
Tags: python,python-3.x,list,csv

Question: I have the following problem.
I am trying to match all existing rows of a CSV file with the current one.
If the line already exists, the script should only show me it exists.
If the row does not exist, the script should tell me that the row does not exist.
However, the script always tells me that the row does not exist, even though I have checked that the row does exist.
Here is my Code so far:
# Imports from Library(s)
from pathlib import Path
import csv
from windows_tools.installed_software import get_installed_software

# Check if the csv file exists - if not: create it
path = Path('./programms.csv')
existingFile = []
if path.is_file() is not True:
    with open('programms.csv', 'w', newline='') as write1:
        w_object = csv.writer(write1)
        w_object.writerow(["Name", "Version", "Publisher"])
        write1.close()

# Lists all Software on the computer
for software in get_installed_software():
    csv_list = (software['name'], software['version'], software['publisher'])
    
    with open('programms.csv', 'r') as f1:
        existingFile = [line for line in csv.reader(f1, delimiter=',')]
    f1.close()

    #Checks if 
    if csv_list in existingFile:
        print(str(csv_list) + "already is in the list") 
        continue

    if csv_list not in existingFile:
        print("Current Object is not in the Existing lines")

    #     # Open our existing CSV file in append mode
    #     # Create a file object for this file
    #     with open('programms.csv', 'a', newline='') as append1:


    #     # Pass this file object to csv.writer() and create writer_object
    #         writer_object = csv.writer(append1)
        
    #         # Pass the list as an argument intothe writerow() 
    #         writer_object.writerow(csv_list)
    #         # Close the file object
    #         append1.close()
            
print (existingFile)

I already tried to specify the types i want to check it with:
if str(csv_list) in list(existingFile):
Sadly im just starting with python and I'm not quite sure how to tackle this one.

Answer: thanks to you I fixed it!

Try csv_list = [software['name'], software['version'], software['publisher']] –
inspectorG4dget

and

You're setting csv_list to a tuple, not a list because you're using () instead of [] –
Barmar

were right! I just changed the tuple to a list and that was it.
Thanks to all that replied so quickly!
Best Regards
Pr3adus
Title: Mark a task as a success on callback on intentional failure
Tags: python,airflow

Question: In Airflow 2.3.4, I have a task I am intentionally failing and when it fails I want to mark it as a success
in the callback but the below does not work.,
def intentional_failure():
     raise AirflowException("this is a dummy failure")

def handle_failure(context):
     context['task_instance'].state = State.SUCCESS

dummy_failure = PythonOperator(task_id="intentional_failure", python_callable=intentional_failure, on_failure_callback=handle_failure)

How would I programatically mark a task as success on an intentional failure?

Answer: Turns out it is a method call not an attribute. So we need to set context['ti'].set_state(State.SUCCESS) in the on_failure_callback.
Title: Why does Python backspace behave strange?
Tags: python

Question: Python escape character \b (backspace) behaves in a strange way.
Just look at the code below:
print("12\b345")
print("12345\b\b")
print("12345\b\ba")

output:
1345
12345
123a5

I was expecting:
1345
123
123a

Answer: The backspace escape character (\b) in Python is used to move the cursor back one space, without deleting the character at that position.
When combined with other characters in a string, the backspace escape character can be used to create some interesting effects.

In the first line of the code, the string "12\b345" is printed.
The backspace escape character (\b) is used to move the cursor back one space after the "2" in "12", and then the "3" is printed, effectively overwriting the "2". Therefore, the output is "1345".
In the second line of the code, the string "12345\b\b" is printed. The backspace escape character (\b) is used to move the cursor back two spaces after the "5" in "12345", effectively deleting the "5" and leaving the string as "1234". Therefore, the output is "12345".
In the third line of the code, the string "12345\b\ba" is printed. The backspace escape character (\b) is used to move the cursor back one space after the "5" in "12345", effectively deleting the "5" and leaving the string as "1234". Then, the letter "a" is printed, effectively inserting it after the "3". Therefore, the output is "123a5".
Title: I am getting 'numpy.ndarray' object is not callable when I use range() function
Tags: python,numpy,jupyter-notebook

Question: I am getting this error "TypeError: 'numpy.ndarray' object is not callable" in Jupyter notebook when I use the range() function in a very simple for loop. What's the problem?
TypeError                                 Traceback (most recent 
call last)
Input In [60], in <cell line: 1>()
----> 1 for i in range(100):
      2     print(I)

TypeError: 'numpy.ndarray' object is not callable

Answer: I think it is because you had declared some variable with the name "range". You should change the variable name to a different one if you would like to use the function range().
Title: Why do image size differ when vertical vs horizontal?
Tags: python,image,image-processing,python-imaging-library

Question: Tried to create a random image with PIL as per the example:
import numpy
from PIL import image

a = numpy.random.rand(48,84)
img = Image.fromarray(a.astype('uint8')).convert('1')
print(len(img.tobytes()))

This particular code will output 528.
Wen we flip the numbers of the numpy array:
a = numpy.random.rand(84,48)

The output we get is 504.
Why is that?
I was expecting for the byte number to be the same, since the numpy arrays are the same size.

Answer: When you call tobytes() on the boolean array*, the data is likely encoded per row. In your second example, there are 48 booleans in each row of img. So each row can be represented with 6 bytes (48 bits). 6 bytes * 84 rows = 504 bytes in img. However, in your first example, there are 84 pixels per row, which is not divisible by 8. In this case, the encoder represents each row with 11 bytes (88 bits). There are 4 extra bits of padding per row. So now the total size is 11 bytes * 48 rows = 528 bytes.
If you test a bunch of random input shapes for a 2d boolean array to encode, you will find that when the number of elements per row is divisible by 8, the number of total bytes in the encoding is equal to the width * height / 8. However, when the row length is not divisible by 8, the encoding will contain more bytes because it has to pad each row with between 1 and 7 bits.
In summary - ideally, we would want to store eight boolean values per byte, but this is complicated by the fact that the row length isn't always divisible by 8, and the encoder serializes the array by row.
Edit for clarification: *the PIL.Image object in mode "1" (binary or "bilevel" image) effectively represents a boolean array. In mode 1, the original image (in this case, the numpy array a) is thresholded to convert it to a binary image.
Title: In Python, Downloading Png and Jpg images
Tags: python,python-3.x,web-scraping,web-crawler

Question: I am writing a script to download images from a certain website. The website contains jpg and png images.
I was expecting the code to run normally. But the png images are taking a while to download (very slow) while the jpg images are quick.
img_data = requests.get(image_url, headers=headers).content
imagename = ''
if image_url.endswith('.jpg'):
    imagename = str(product) + str(f"{imagevalue:03d}") + '.jpg'
elif image_url.endswith('.png'):
    imagename = str(product) + str(f"{imagevalue:03d}") + '.png'
with open(imagename, 'wb') as file:
    file.write(img_data)

This is the code, working but slowly. Am I Missing something here?

Answer: It is not a problem with your code, rather .png files are much larger than .jpg files.
Title: Is there an efficient way to determine if a sum of floats will be order invariant?
Tags: python,algorithm,math

Question: Due to precision limitations in floating point numbers, the order in which numbers are summed can affect the result.
>>> 0.3 + 0.4 + 2.8
3.5
>>> 2.8 + 0.4 + 0.3
3.4999999999999996


This small error can become a bigger problem if the results are then rounded.
>>> round(0.3 + 0.4 + 2.8)
4
>>> round(2.8 + 0.4 + 0.3)
3

I would like to generate a list of random floats such that their rounded sum does not depend on the order in which the numbers are summed.  My current brute force approach is O(n!).  Is there a more efficient method?
import random
import itertools
import math


def gen_sum_safe_seq(func, length: int, precision: int) -> list[float]:
    """
    Return a list of floats that has the same sum when rounded to the given
    precision regardless of the order in which its values are summed.
    """
    invalid = True
    while invalid:
        invalid = False
        nums = [func() for _ in range(length)]
        first_sum = round(sum(nums), precision)
        for p in itertools.permutations(nums):
            if round(sum(p), precision) != first_sum:
                invalid = True
                print(f"rejected {nums}")
                break

    return nums

for _ in range(3):
    nums = gen_sum_safe_seq(
        func=lambda :round(random.gauss(3, 0.5), 3),
        length=10,
        precision=2,
    )
    print(f"{nums} sum={sum(nums)}")

For context, as part of a programming exercise I'm providing a list of floats that model a measured value over time to ~1000 entry-level programming students.  They will sum them in a variety of ways.  Provided that their code is correct, I'd like for them all to get the same result to simplify checking their code.  I do not want to introduce the complexities of floating point representation to students at this level.

Answer: The easiest way is to create random integers, and then divide (or multiply) them all by the same power of 2.
As long as the sum of the absolute values of the original integers fits into 52 bits, then you can add the resulting floats without any rounding errors.
Title: File size is getting progressively larger as I continue to download data
Tags: python,yahoo-finance

Question: I am downloading financial data using to_hdf and I have noticed that each file gets larger and larger as it keeps downloading. What is happening?
The first file was saved as 223 KB and the most recent where I stopped (67) was saved as 14,609 KB.
The following is the code (some sections that are irrelevant have been removed):
import pandas as pd
import datetime as dt
import yfinance as yf
from pandas.tseries.holiday import USFederalHolidayCalendar
import yahoo_fin.stock_info as si
from pathlib import Path
import os.path


def main():
    end = dt.datetime.now()

    start = end + dt.timedelta(days=-5)

    dr = pd.date_range(start=start, end=end)
    cal = USFederalHolidayCalendar()
    holidays = cal.holidays(start=dr.min(), end=dr.max())
    a = dr[~dr.isin(holidays)] # not US holiday
    b = a[a.weekday != 5]
    b = b[b.weekday != 6]

    for year in set(b.year):
        tmp = b[b.year == year]
        for week in set(pd.Index(tmp.isocalendar().week)):
            temp = tmp[pd.Index(tmp.isocalendar().week) == week]
            start = temp[temp.weekday == temp.weekday.min()][0]  # beginning of week
            end = temp[temp.weekday == temp.weekday.max()][0]  # ending of week

    # get list of all index tickers
    ticker_strings = si.tickers_sp500()

    data_dir = 'data'

    x = 1

    tickers_dir = './tickers'

    Index = '^GSPC'

    # initialize list for the following f(x)
    Df_list = list()

    ticker_data(ticker_strings, start, end, Df_list, data_dir, x)

    print("Complete")


def ticker_data(ticker_strings, start, end, Df_list, data_dir, x):
    # find values for individual stocks
    for ticker in ticker_strings:
        loc_start = start
        while loc_start <= end:
            period_end = loc_start + dt.timedelta(days=1)
            intra_day_data = yf.download(ticker, loc_start, period_end, period="1d", interval="1m")
            extra_day_data = yf.download(ticker, loc_start, period_end, period="1d", interval="1m", prepost=True)
            Df_list.append(intra_day_data)
            Df_list.append(extra_day_data)
            loc_start = loc_start + dt.timedelta(days=1)
        df = pd.concat(Df_list)

        # creates file name
        filename = end.strftime('%F') + " " + ticker + ".h5"
        # saves file name to folder
        df.to_hdf(os.path.join(data_dir, filename), mode='w', key='df')
        #df.to_csv(os.path.join(data_dir, filename))

        print(x, ticker)
        x += 1



if __name__ == "__main__":
    main()

Answer: This could be because it is constantly downloading new data.
Title: File size is getting progressively larger as I continue to download data
Tags: python,yahoo-finance

Question: I am downloading financial data using to_hdf and I have noticed that each file gets larger and larger as it keeps downloading. What is happening?
The first file was saved as 223 KB and the most recent where I stopped (67) was saved as 14,609 KB.
The following is the code (some sections that are irrelevant have been removed):
import pandas as pd
import datetime as dt
import yfinance as yf
from pandas.tseries.holiday import USFederalHolidayCalendar
import yahoo_fin.stock_info as si
from pathlib import Path
import os.path


def main():
    end = dt.datetime.now()

    start = end + dt.timedelta(days=-5)

    dr = pd.date_range(start=start, end=end)
    cal = USFederalHolidayCalendar()
    holidays = cal.holidays(start=dr.min(), end=dr.max())
    a = dr[~dr.isin(holidays)] # not US holiday
    b = a[a.weekday != 5]
    b = b[b.weekday != 6]

    for year in set(b.year):
        tmp = b[b.year == year]
        for week in set(pd.Index(tmp.isocalendar().week)):
            temp = tmp[pd.Index(tmp.isocalendar().week) == week]
            start = temp[temp.weekday == temp.weekday.min()][0]  # beginning of week
            end = temp[temp.weekday == temp.weekday.max()][0]  # ending of week

    # get list of all index tickers
    ticker_strings = si.tickers_sp500()

    data_dir = 'data'

    x = 1

    tickers_dir = './tickers'

    Index = '^GSPC'

    # initialize list for the following f(x)
    Df_list = list()

    ticker_data(ticker_strings, start, end, Df_list, data_dir, x)

    print("Complete")


def ticker_data(ticker_strings, start, end, Df_list, data_dir, x):
    # find values for individual stocks
    for ticker in ticker_strings:
        loc_start = start
        while loc_start <= end:
            period_end = loc_start + dt.timedelta(days=1)
            intra_day_data = yf.download(ticker, loc_start, period_end, period="1d", interval="1m")
            extra_day_data = yf.download(ticker, loc_start, period_end, period="1d", interval="1m", prepost=True)
            Df_list.append(intra_day_data)
            Df_list.append(extra_day_data)
            loc_start = loc_start + dt.timedelta(days=1)
        df = pd.concat(Df_list)

        # creates file name
        filename = end.strftime('%F') + " " + ticker + ".h5"
        # saves file name to folder
        df.to_hdf(os.path.join(data_dir, filename), mode='w', key='df')
        #df.to_csv(os.path.join(data_dir, filename))

        print(x, ticker)
        x += 1



if __name__ == "__main__":
    main()

Answer: You are appending new data to Df_list at every iteration of for ticker in ticker_strings and you are saving all of that every time. Which means that every file will contain also the previous file's data.
You should use a variable local to the for ticker in ticker_strings loop instead of using a list passed in as a parameter.
Title: Pafy module causing error in PythonAnywhere
Tags: python,pythonanywhere,youtube-dl,pafy

Question: I am using pafy in a Flask app. It is working fine on my local machine. But when I am trying to deploy and run on PythonAnywhere, it is throwing an error. Upon execution, it is showing the following ewrror:

youtube_dl.utils.ExtractorError: Unable to download API page: <urlopen error Tunnel connection failed: 403 Forbidden> (caused by URLError(OSError('Tunnel connection failed: 403 Forbidden')))
NO MATCH
During handling of the above exception, another exception occurred:
NO MATCH
Traceback (most recent call last):
File "/home/Philomath/.local/lib/python3.8/site-packages/pafy/backend_youtube_dl.py", line 40, in _fetch_basic
self._ydl_info = ydl.extract_info(self.videoid, download=False)
File "/home/Philomath/.local/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 808, in extract_info
return self.__extract_info(url, ie, download, extra_info, process)
File "/home/Philomath/.local/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 824, in wrapper
self.report_error(compat_str(e), e.format_traceback())
File "/home/Philomath/.local/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 628, in report_error
self.trouble(error_message, tb)
File "/home/Philomath/.local/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 598, in trouble
raise DownloadError(message, exc_info)
youtube_dl.utils.DownloadError: ERROR: Unable to download API page: <urlopen error Tunnel connection failed: 403 Forbidden> (caused by URLError(OSError('Tunnel connection failed: 403 Forbidden')))

I have also removed the lines for dislike count in backend_yoututbe_dl in PythonAnywhere (from bash). Still, no use.

Answer: That will not work on PythonAnywhere. Any youtube video downloads use unofficial methods and so PythonAnywhere will not add youtube to the allowlist for free accounts.
Title: Custom controller very slow for some users, really slow for others
Tags: python,odoo,odoo-13

Question: We have built a custom controller for a customer which passed products data and loads their products in a custom url.
The controllers are basically like this:
class CustomProducts(http.Controller):
    @http.route('/custom', type='http', auth='public', website=True, methods=['GET'])
    def render_custom_products(self, **kw):

        if kw.get('order'):
          if kw.get('order') != "":
            order_by = kw.get('order')
        else:
          order_by = 'list_price desc'

        products = http.request.env['product.product'].sudo().search([('categ_id.name', 'ilike', 'Custom'), ('is_published', '=', 'true'), ('active', '=', 'true')],
        order= order_by,
        limit = 150)
        return http.request.render('custom_module.custom_products', {
          # pass products details to view
          'products': products,
          })

EDIT:
This has worked good for over 2 years but suddenly the page is very slow. The weird thing is that the route is very slow for some visitors and very fast (like before) for some. What could be the issue?
Postgres sometimes gives this error:
Could not serialize access due to concurrent update
SELECT * FROM website_visitor where id = XX FOR NO KEY UPDATE NOAWAIT
The customer only have between 30-50 products for sale at once.

Answer: You should consider adding logging to note the times
http.request.env['product.product'].sudo().search seems to be the only function you are calling, so there can be only 2 (i think) causes of the issue -


Either your search has slowed down
or you have designed things in a way such that your system is not processing things in parallel

You could could try using fastapi/some other framework - to support async or try to diagnose your search issue depending on further finding.
Title: The rest of the python code doesn't work with FastAPI
Tags: python,fastapi

Question: I want to make a discord bot that would work with data that would be sent from a web application via POST request. So for that I need to make my own API containing that POST request with FastAPI.
My question is: how can make my API work with the rest of the code?
Consider this example:
from fastapi import FastAPI, Request
from pydantic import BaseModel

app = FastAPI()

dict = {"name": "","description": ""}

class Item(BaseModel):
    name: str
    description: str

@app.post("/item")
async def create_item(request: Request, item: Item):
    result = await request.json()
    dict["name"] = result["name"]
    print(dict)

print(dict)

When I run the API, type the values in and print dict for the fist time, it outputs something like that: {'name': 'some_name', 'desc': 'some_desc'}
But when I run my file as python code, only {'name': '', 'desc': ''} gets printed out.
I thought that after I type in values in dict on my API page (https://localhost:800/docs), python would output the exact values I typed, but it didn't happen.
What do I do?

Answer: When you call the api, you essentially call the create_item function, which would manipulate your dictionary. When you directly run your python code, you don't call the create_item function, therefore the dictionary isn't manipulated, and you get the original dictionary back, which is {'name': '', 'desc': ''}.
When you run your server, it runs in it's own instance. When you run your file again with python, it's another instance. They're two seperate things that don't know each other's "dict" values.
If you want to know about when your api manipulates something (if you want your data to persist regardless of your python instances), you should use a database and query said database (you can even use a .json file for that purpose)
Title: Convert integer to decimal and recover places after zero
Tags: python

Question: I have array data as below:
a = np.array([1.41607, 2.17922, -14.7047, -1852.51, -2713.39, -165.025])

a is a decimal number and I want to convert a to an integer number such as below:
a = [1, 2, -15, -1853, -2713, -165]

then after I convert a to an integer number, I want to restore the original data. I have tried using the code below, but I can't restore data to the original data.
import numpy as np

a = np.array([1.41607, 2.17922, -14.7047, -1852.51, -2713.39, -165.025])

# Define the number of decimal places to keep
decimal_places = 0

# Round the values to the specified number of decimal places
rounded_a = np.round(a, decimals=decimal_places)

# Convert the rounded values to integers
int_a = (rounded_a * 10**decimal_places).astype(int)

# Convert the integers back to the original decimal values
float_a = int_a / 10**decimal_places

# Print the original values and the recovered values
print("Original values:", a)
print("DecimalToInteger:", int_a)
print("Restored values:", float_a)

Results
Original values: [    1.41607     2.17922   -14.7047  -1852.51    -2713.39     -165.025  ]
DecimalToInteger: [    1     2   -15 -1853 -2713  -165]
Restored values: [    1.     2.   -15. -1853. -2713.  -165.]

Answer: This is impossible. Once you convert a float to an integer, there is no going back. You can convert the integer back into a float, but you won't get back any of the decimal places you had before.
This is because you're using a function that loses information. In your case, you used np.round(). np.round() tells the computer to round the number to the closest integer, then throw away all the information left in the places after the decimal point. Because you threw it away, there's no getting it back.
Title: Attempting to run a FastAPI application on CentOS 7 but getting a module 'asyncio' error
Tags: python,linux,pip,fastapi,centos7

Question: I am attempting to run a simple FastAPI application on CentOS 7 but getting some errors. I will include some more details for context:
Python Version - 3.6.8
pip version - 9.0.3
I am running the application with this command: python3 -m uvicorn main:app
I keep getting this error:
Traceback (most recent call last):
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.6/site-packages/uvicorn/__main__.py", line 4, in <module>
    uvicorn.main()
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/uvicorn/main.py", line 435, in main
    run(app, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/uvicorn/main.py", line 461, in run
    server.run()
  File "/usr/local/lib/python3.6/site-packages/uvicorn/server.py", line 67, in run
    return asyncio.run(self.serve(sockets=sockets))
AttributeError: module 'asyncio' has no attribute 'run'

I was initially getting this error - /home/centos/fast_api/fastapi-tutorial/python3-venv/bin/python3: No module named uvicorn but after installing uvicorn via pip3 install uvicorn, I now get the "module 'asyncio' error"
Any help would be great
I have tried enabling a python virtual environment on the server but I still get the same error.
Could this be an issue with the Python version?

Answer: Update on this. I was able to get this working by upgrading both python and pip.
I upgraded python to 3.9
Upgraded pip using - /usr/local/bin/python3.9 -m pip install --upgrade pip
Thanks for the help.
Title: jupyter notebook can't see GPU while it is available in conda environment
Tags: python,tensorflow,visual-studio-code,jupyter-notebook,anaconda

Question: I'm trying to use tensorflow on GPU with jupyter notebook. I make SSH connection to a remote server with VScode and build up an anaconda environment(called 'myspace') where tensorflow(version 2.6.0) is successfully installed:
(myspace) user@server:~$ python
Python 3.6.13 |Anaconda, Inc.| (default, Jun  4 2021, 14:25:59) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> print("Num GPUs:", len(tf.config.experimental.list_physical_devices('GPU')))
Num GPUs: 1

However, when I open a jupyter notebook and select the same conda environment as its kernel, the GPU can't be seen:
Num GPUs: 0

I suppose that the CUDA toolkit and CUDNN library are successfully installed and I attach the output of tf.sysconfig.get_build_info() here:
OrderedDict([('cpu_compiler', '/usr/bin/gcc-5'),
             ('cuda_compute_capabilities',
              ['sm_35', 'sm_50', 'sm_60', 'sm_70', 'sm_75', 'compute_80']),
             ('cuda_version', '11.2'),
             ('cudnn_version', '8'),
             ('is_cuda_build', True),
             ('is_rocm_build', False),
             ('is_tensorrt_build', True)])

I've read many related posts but none of them worked. I tried to get the os.environ["LD_LIBRARY_PATH"] and got a keyerror. I suppose it might be a problem of system path but I already set the paths in ~/.bashrc:
export PATH=/home/user/cuda-11.2/bin:${PATH}
export LD_LIBRARY_PATH=/home/user/cuda-11.2/lib64:${LD_LIBRARY_PATH}

Answer: The best way I could solve this problem was to change to tensorflow's image as jupyter image did. From here you could update from the comand line.
docker run -d -p 8888:8888 --name jupyter --mount type=bind,source="$(pwd)",target=/tf  --gpus all -e CHOWN_EXTRA="/home/jovyan/work" --user root -e GRANT_SUDO=yes -e NB_GID=100 -e NB_USER:jovyan tensorflow/tensorflow:latest-gpu-jupyter
Title: Regex: How to match a string multiple times while capturing groups differently
Tags: python,regex,regex-group

Question: I have a string that looks something like this:
AB
And I want to capture it using a regex that looks like this:
(?P<GRP1>(A|D))?(?P<GRP2>(A|C))?B
Is there a python function that would produce two matches for the string? The first by capturing A as a part of GRP1 and the second by capturing it as a part of GRP2?
I tried functions like re.fullmatch and re.findall but all of them only capture A as part of GRP1.
The function I'm looking for would produce one match when matching against DB or CB, but two matches for AB, one for each group that A can belong to.

Answer: That is not what regex is designed for. Regex is about matching a span of the input against a pattern. So any span of the input is either a match or it is not. The span is "consumed" in the process, that is, the same span cannot match the pattern twice. On the other hand a pattern can match multiple non-overlapping spans, which is what findall is about.
There are some advanced regex technics that do not consume any characters while matching (lookahead/lookbehind), but those inherently also do not capture any input which makes them useless for you.
Usually this is the point where one should start thinking about not using regex or atleast not using only regex.
I see to possible solutions

Use two regex patterns, one for each GRP. This would make the overall result a tupple representing 0-2 matches representing either none, GPR1, GPR2 or GPR1+GPR2
Use one regex pattern but restructure it like this (?P<GRP1>(D))?(?P<GRP2>(C))?(?P<GRP1_2>(A))?B i.e. have a separate group for each thing that may occure. With GRP1_2 representing the GRP1&GRP2 case. So, for example if your goal is to count the GRP1 and GRP2 matches then you would count GRP1_2 to both.
Title: How to implement consumer for voice chat app on Django?
Tags: python,json,django,sockets,django-channels

Question: I am currently working on a voice chat app as a personal project and this app will be like Discord, where users can hop into a chat room and start talking to people in that room. I am only going to make it voice only for now (I will implement text messaging later). I have been looking at examples on Github and I am having a little trouble with how to make the Consumer.
Here is what I have right now:
import json

from asgiref.sync import async_to_sync
from channels.generic.websocket import WebsocketConsumer

class VoiceConsumer(WebsocketConsumer):

    def connect(self):

        self.room_name = self.scope['url_route']['kwargs']['room_name']

        self.room_group_name = "chat_%s" % self.room_name

        # join the room
        async_to_sync(self.channel_layer.group_add)(
            self.room_group_name, self.channel_name
        )

        self.accept()

    def disconnect(self, closed_code):

        async_to_sync(self.channel_layer.group_discard)(
            self.room_group_name, self.channel_name
        )

    def receive(self, text_data):

        text_data_json = json.loads(text_data)

        # Ask about this, how to receive voice ?
        voice_message = text_data_json['message']

        async_to_sync(self.channel_layer.group_send)(
            self.room_group_name, {'type' : 'chat_message', 'message': voice_message}
        )

As you see, I am confused about the text_data_json['message'] part. This is based off an example I have seen where someone implements messages, but I want to implement voice only. What do I change about this to where I can implement voice only ?
text_data_json = json.loads(text_data)
   
# Ask about this, how to receive voice ?
voice_message = text_data_json['message']
    
async_to_sync(self.channel_layer.group_send)(
              self.room_group_name, {'type' : 'chat_message', 'message': voice_message}
            )

I haven't seen any examples that do this for voice only.

Answer: Send audio bytes as Base64 string
Title: Python program crashes when I assign tags to a Gtk.TextView
Tags: python,gtk,gtk3,pygobject

Question: I am trying to write a program that will take data from a Gtk.TextView and assign tags for it. Basically a markdown formatter. But whenever I add a * * set for itallics the entire program crashes.
This is the code I had. It's not the best looking code because I was experimenting with the idea.
# Imports
import gi
gi.require_version('Gtk', '3.0')
from gi.repository import Gtk, Pango

class MarkdownFormatter:
    def __init__(self, textview):
        self.textview = textview
        self.text_buffer = self.textview.get_buffer()
        self.text_buffer.connect('insert-text', self.on_key_insert)
        self.tag_bold = self.text_buffer.create_tag('bold',weight=Pango.Weight.BOLD)
        self.tag_italic = self.text_buffer.create_tag('italic',style=Pango.Style.ITALIC, )
        self.tag_underline = self.text_buffer.create_tag('underline',underline=Pango.Underline.SINGLE)
        self.bold= False
        self.italic = False
        self.italic_start_iter = Gtk.TextIter()
        self.italic_end_iter = Gtk.TextIter()
        self.underline = False
        
    def on_key_insert(self, buffer, inter, text, iteg):
        end= self.text_buffer.get_end_iter()
        start= self.text_buffer.get_start_iter()
        all_text = self.text_buffer.get_text(end, start, True)
        self.text_buffer.remove_all_tags(start, end)
        count = 0
        self.italic =  False
        for d in all_text:
            count+= 1
            if '*' in d:
                if self.italic == False:
                    self.italic = True
                    self.italic_start_iter.set_offset(count)
                elif self.italic:
                    self.italic = False
                    self.italic_end_iter.set_offset(count)
                    self.text_buffer.apply_tag(self.tag_italic, self.italic_start_iter, self.italic_end_iter)

text = '''
*I is good.*
she is smart.
"Hi" he said.

'''

if __name__ == '__main__':

    def quit(*args):
        Gtk.main_quit()

    window = Gtk.Window.new(Gtk.WindowType.TOPLEVEL)
    window.set_title('Markdown formatter')
    view = Gtk.TextView()
    view.get_buffer().set_text(text)

    formater = MarkdownFormatter(view)

    window.set_default_size(600, 400)
    window.add(view)
    window.show_all()
    window.connect('delete-event', quit)
    Gtk.main()

Run the program and then when you try to type anything it'll crash (It only tries to format the text when you type.

Answer: I found the answer. I have no idea why so if someone can figure out why I'd be very happy.
Basically if you replace the  set_offset(s) with a itername = buffer.get_iter_at_offset(count) It will run fine.
My only guess is that it's something to do with reusing buffers or there's an issue with set_offset.
Title: On Whisper API, when I try to use a python script for transcribing audio files in bulk, I can't get the correct response_format ('srt' or 'vtt') work
Tags: python,syntax-error,srt,webvtt,openai-whisper

Question: I'm using this code for connecting to Whisper API and transcribe in bulk all mp3 in a folder to both srt and vtt:
import requests
import os
import openai

folder_path = "/content/audios/"
def transcribe_and_save(file_path, format):
    url = 'https://api.openai.com/v1/audio/transcriptions'
    headers = {'Authorization': 'Bearer MyToken'}
    files = {'file': open(file_path, 'rb'), 
            'model': (None, 'whisper-1'),
            'response_format': format}
    response = requests.post(url, headers=headers, files=files)
    output_path = os.path.join(folder_path, os.path.splitext(filename)[0] + '.' + format)
    with open(output_path, 'w') as f:
        f.write(response.content.decode('utf-8'))

for filename in os.listdir(folder_path):
    if filename.endswith('.mp3'):
        file_path = os.path.join(folder_path, filename)
        transcribe_and_save(file_path, 'srt')
        transcribe_and_save(file_path, 'vtt')
else:
    print('mp3s not found in folder')


When I use this code, I'm getting the following error:
"error": {
    "message": "1 validation error for Request\nbody -> response_format\n  value is not a valid enumeration member; permitted: 'json', 'text', 'vtt', 'srt', 'verbose_json' (type=type_error.enum; enum_values=[<ResponseFormat.JSON: 'json'>, <ResponseFormat.TEXT: 'text'>, <ResponseFormat.VTT: 'vtt'>, <ResponseFormat.SRT: 'srt'>, <ResponseFormat.VERBOSE_JSON: 'verbose_json'>])",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }

I've tried with different values, but either don't work or I'm only receiving the transcription as a object in plain text, but no srt or vtt. I'm expecting to get srt and vtt files in the same folder as where audios are
Thanks,
Javi

Answer: I am not sure about the whisper api, but you seem to be using an already existing python function as a parameter name.
Perhaps this could be a reason why it is not working, as the function format is being used when calling the endpoint instead of the parameter you passed in.
Try changing the parameter name to something other than format and change the value being used for response_format.
Title: Django render third argument
Tags: python,django,render

Question: I am confused with Django renderring. I usually used context and dictionary in the third argument:
return render(request,'zing_it/home.html', context=my_playlists)

However, in the code below the third argument used list name in strings and list name as a variable. Can you please help me to understand what is my_playlists (in strings) and the second my_playlists (variable). Why so?
from django.shortcuts import render
my_playlists=[
        {"id":1,"name":"Car Playlist","numberOfSongs":4},
        {"id":2,"name":"Coding Playlist","numberOfSongs":2}
    ]

def home(request):
    return render(request,'zing_it/home.html',{"my_playlists":my_playlists})

Answer: context must be a dictionary. It cannot be a list, so the first version won't work assuming my_playlist is the same in both examples. Your second version works because it passes a dictionary correctly.
Title: How can i resolve flake8 "unused import" error for pytest fixture imported from another module
Tags: python,pytest,flake8

Question: I wrote pytest fixture in fixtures.py file and using it in my main_test.py . But I am getting this error in flake8: F401 'utils_test.backup_path' imported but unused for this code:
@pytest.fixture
def backup_path():
    ...

from fixtures import backup_path


def test_filename(backup_path):
    ...

How can I resolve this?

Answer: generally you should not do this
making fixtures available via import side-effects is an unintentional implementation detail of how fixtures work and may break in the future of pytest
if you want to continue doing so, you can use # noqa: F403 on the import, telling flake8 to ignore the unused imports (though the linter is telling you the right thing here!)
the supported way to make reusable fixtures is to place them in a conftest.py which is in a directory above all the tests that need it.  tests will have these fixtures "in scope" automatically
if that file is getting too large, you can write your fixtures in a plugin module and add that plugin module via pytest_plugins = ['module.name'] in your conftest.py

disclaimer: I'm the current flake8 maintainer, I'm also a pytest core dev