satoshi-2000/llms-suitable huggingface dataset:
Question: Sometimes, in a Data JPA environment, when three queries occur in one method, one query does not work normally in DB

Sometimes, in a Data JPA environment, when three queries occur in one method, one query does not work normally in DB.
@Transactional
public void method1(Log log){
        Long userId = log.getUserId();
        User user = userRepository.findById(userId).orElseThrow(RuntimeException::new)

        BigDecimal point = BigDecimal.valueOf(100L);
        user.minusPoint(point);
        log.minusPoint(point);
        logRepository.save(new Log(point));
}

It works normally for one case.
However, I ran it on several occasions, and one did not work properly.
ex) logList.forEach(log-> a.method1(log));
did not work "log.minusPoint(point);" in all cases.
The data in the DB has not changed about log.
But the new log data was inserted and the user data has been changed.
It was confirmed that the update, insert query was created successfully in the test environment.


Answer:
You can enable SQL logging and check what data is actually passed to the database.
One of the reason can be: @Transactional annotation doesn't work. You need to get a service from the Spring Context to allow Spring to wrap method1() with a proxy method to do dirty checking and save data.
@Transactional annotation is a bit messy. It is primary not about transactions, but about dirty checking and Hibernate session control.
When you do
logRepository.save(new Log(point));

data is saved because logRepository.save() has it is own @Transactional annotation and does explicit call to save the data.
log.minusPoint(point) could work, if @Transactional works correctly. A Spring proxy could open Hibernate session (Persistent Context) before method1() call and close it in the end. During closing session Hibernate does dirty checking — checks, if any of objects in session were changed and saves such objects.
Another reason can be that you load log in a method without @Transactional or in a method with own @Transactional. Different @Transactional means different Persistent Context.
One exception — when @Transactional of method1() is wrapped with another @Transactional that is used for a method that calls method1().
It is bad practice to solve all issues with putting @Transactional everywhere. You can just do
logRepository.save(log)
Question: How can I count over multiple fields in mongo?

I am trying to write a mongo query to help produce a report from the data.
The documents look something like this:
[{
    "DeviceType" : "A",
    "DeviceStatus" : "On"
},
{
    "Device Type" : "B",
    "Device Status" : "On"
},
{
    "DeviceType" : "A",
    "DeviceStatus" : "Off"
},
{
    "DeviceType" : "A",
    "DeviceStatus" : "On"
}]

The DeviceType field can take any string value and the DeviceStatus field can be any one of On, Off or InRepair. I want to write a mongo query that displays DeviceStatus count for each one of the devices. The result should be something like this:
[{
    "DeviceType" : "A",
    "DeviceStatusOnCount" : "15",
    "DeviceStatusOffCount" : "13",
    "DeviceStatusInRepairCount" : "12",
},
{
    "DeviceType" : "B",
    "DeviceStatusOnCount" : "6",
    "DeviceStatusOffCount" : "14",
    "DeviceStatusInRepairCount" : "2",
}]

How can I achieve this in mongo?
My current query can only group based on the DeviceType:
db.Collection_Name.aggregate([
    { $group: { _id: "$DeviceType", count: { $sum: 1 } } }
])



Answer:
You can use $cond + $eq to return either 1 or 0 to $sum operator:
db.collection.aggregate([
  {
    $group: {
      _id: "$DeviceType",
      DeviceStatusOnCount: { $sum: { $cond: [ { $eq: [ "$DeviceStatus", "On" ] }, 1, 0 ] } },
      DeviceStatusOffCount: { $sum: { $cond: [ { $eq: [ "$DeviceStatus", "Off" ] }, 1, 0 ] } },
      DeviceStatusInRepairCount: { $sum: { $cond: [ { $eq: [ "$DeviceStatus", "InRepair" ] }, 1, 0 ] } }
    }
  },
  {$addFields: {DeviceType: '$_id'}}, {$project: {_id:0}}
])

Test: mongoplayground
Question: How to remove the "locked" flag (programmatically)

(MacOS Monterey):
I have external hard disks with files written under Windows, using Cygwin rsync.
The files are perfectly readable under MacOS, but when I want to delete/overwrite them, quite some appear "locked" (operation not permitted when doing a rm) on MacOS. Using the Finder, I can remove the lock by getting the context menu on the file, choose "GetInfo", and unchecked the Locked property.
But there are too many of them to do it by hand. I would like to run recursively over the file tree and uncheck the "locked" attribute programmatically. Of course the problem is not the traversal of the directory tree (can done by find for instance), but the change of the attribute from the command line.
chmod does not help here (as the files are 0777). Any other command I can use?


Answer:
I just found the command to do so:
chflags nouchg FILENAME

The command is described here
Question: Updating web redirect uri of Azure AD app registration

I have a scenario where my pipeline should update the app registration with an additional redirectUrl.
I have managed to extract the current web.redirectUris with the following:
 existing_urls=$(az ad app show --id '<client-id>' --query "[web.redirectUris]" --output tsv)
I would like to achieve something like this
existing_urls=$(az ad app show --id '<client-id>' --query "[web.redirectUris]" --output tsv)
az ad app update --id '<client-id>' --web-redirect-uris "$existing_urls https://hostname.com/newCallback"

I have tried updating the web.redirectUris in two ways and both of them have failed when I pass multiple redirect URIs.
Attempt 1
az ad app update --id '<client-id>' --web-redirect-uris "https://hostname.com/callbackx https://hostname.com/callbacky"

One or more properties contains invalid values.

However when having only one uri this worked fine
az ad app update --id '<client-id>' --web-redirect-uris "https://hostname.com/callbackx"

Attempt 2
This one fails regardless of number of redirectUris that are passed
az ad app update --id '<client-id>' --set "web.redirectUris=['https://hostname.com/callbackx', 'https://hostname.com/callbacky']"

Couldn't find 'web' in ''. Available options: []



Answer:
Tried as shown :But got the same error:
az ad app show --id 1e7bxxx7830

existing_urls=$(az ad app show --id 1e7b8fxxxx830 --query "[web.redirectUris]")

az ad app update --id 1e7xxx0a7830 --web-redirect-uris "$existing_urls https://hostname.com/newCallback"

$updated_urls="$existing_urls https://hostname.com/newCallback"

az ad app update --id 1e7b8xxx0a7830  --set "web.redirectUris='$updated_urls'"

az ad app update --id 1e7b8fxxxd0a7830  --set "web.redirectUris='$updated_urls'"

Error:
Couldn't find 'web' in ''. Available options: []


Following command worked foe me in azure cli in updating multiple Redirect Urls:
az ad app update --id '1e7bxxxa7830' --web-redirect-uris "https://hostname.com/callback"  "https://jwt.ms" "https://myexampleapp.com"
here --id is clientId .

So  give the command with required urls as
az ad app update --id '1e7bxxxa7830' --web-redirect-uris "<url1>"  "<url2>" "<url3>"
upon az ad app show --id 1e7b8xxxx830
Question: I'm trying to find the closest value in a reference data frame but it is not outputting the correct row. Where am I going wrong?

I'm trying to find the closest value in a reference data frame but it is not outputting the correct row.
I am using the below data frame which is then used to find the relevant row corresponding to the closest value in column 'P' to a defined variable. For example if p = 0.22222 then the code should output row 2.
DF:
      P   n1      n2      n3      n4      n5      n6      n7      n8      n9   
0   NaN  0.0  0.2000  0.4000  0.6000  0.8000  1.0000  1.2000  1.4000  1.6000
1   0.0  1.0  0.8039  0.6286  0.4855  0.3753  0.2929  0.2318  0.1863  0.1520   
2   0.2  1.0  0.7983  0.6201  0.4771  0.3683  0.2876  0.2279  0.1835  0.1500   
3   0.4  1.0  0.7789  0.5924  0.4508  0.3473  0.2720  0.2167  0.1754  0.1442   
4   0.6  1.0  0.7349  0.5377  0.4043  0.3124  0.2470  0.1989  0.1628  0.1351   
5   0.8  1.0  0.6301  0.4433  0.3368  0.2658  0.2147  0.1762  0.1465  0.1234   
6   1.0  0.5  0.3828  0.3105  0.2559  0.2130  0.1787  0.1510  0.1286  0.1102   
7   1.2  0.0  0.1544  0.1871  0.1795  0.1621  0.1433  0.1257  0.1103  0.0965   
8   1.4  0.0  0.0717  0.1101  0.1216  0.1197  0.1120  0.1024  0.0925  0.0831   
9   1.6  0.0  0.0400  0.0682  0.0829  0.0876  0.0865  0.0824  0.0765  0.0707   
10  1.8  0.0  0.0249  0.0449  0.0580  0.0647  0.0668  0.0659  0.0633  0.0597   
11  2.0  0.0  0.0168  0.0312  0.0418  0.0485  0.0519  0.0528  0.0520  0.0502   
12  3.0  0.0  0.0042  0.0082  0.0118  0.0149  0.0174  0.0193  0.0207  0.0216

The function I am using however outputs the incorrect value:
p = 0.2020202
closest_p = df.iloc[(df['P']-p).abs().argsort()[:1]]

Expected output:
      P   n1      n2      n3      n4      n5      n6      n7      n8      n9   
2   0.2  1.0  0.7983  0.6201  0.4771  0.3683  0.2876  0.2279  0.1835  0.1500

However it is only outputting the last row -
      P   n1      n2      n3      n4      n5      n6      n7      n8      n9   
12  3.0  0.0  0.0042  0.0082  0.0118  0.0149  0.0174  0.0193  0.0207  0.0216

Where am i going wrong????


Answer:
You need to use idxmin:
closest_p = (df['P']-p).abs().idxmin()

Output: 2
For the row: df.loc[(df['P']-p).abs().idxmin()]
A fix of your approach would have been to use sort_values (but it's less efficient):
closest_p = df.loc[(df['P']-p).abs().sort_values().index[0]]
Question: Unused parameter warning in templated function with if constexpr

In this code
#include <type_traits>
#include <iostream>

template <class T>
void func(const T& a)
{
    if constexpr(std::is_same_v<T,int>)
    {
        static_cast<void>(a);
    }
    else if constexpr(std::is_same_v<T,double>)
    {
        // oops forgot to use it here   
    }
    else
    {
        
    }
}

int main() {
    func(4);
    func("this");
}

why doesn't the compiler warn about unused variable in the else-s()? (with -Wall)
My understanding is that logically the instantiations of the method are completely different. If the argument is not being used in one of the instantiations of the template, isn't it an unused variable?
Or does the language/compiler not interpret it like that.


Answer:
We can say that the parameter a is not fully unused, it's conditionally unused.
The C++ standard requires to generate diagnostic messages for ill-formed programs. Your program is not ill-formed. The C++ standard does not require compilers to generate other warnings at all. So it's up to the compiler vendor to decide how to implement that.
While I personally would like to be warned about conditionally unused variables (your example is a good demo why), no compiler vendor has implemented that. Why? They test new version of compilers on large code bases to see whether new C++ features will cause breaking changes. Likely, during those tests, a lot of warnings were generated for code like yours, but the code was considered totally fine by C++ experts. So the warnings were adapted to not occur in such situations. That's to ensure that code with 0 warnings is possible.
Question: Why metadata is written at the end of the file in Apache Parquet?

I wonder why Apache Parquet writes metadata at the end of the file instead of the beginning?
In the official documentation of Apache Parquet, I found that Metadata is written after the data to allow for single pass writing.. Is the metadata written at the end to ensure the integrity of the file? I don't understand what this sentence really means, if someone could explain it to me, I'd appreciate it.


Answer:
I think the main reason is so you can write bigger than memory data to the same file.
The meta data contains information about the schema of the data (type of the columns) and its shape (number of row groups, size of each row groups).
So in order to generate the metadata you need to know what the data is made of. This can be a problem if your data doesn't fit in memory.
In this case, you should still be able to split your data in manageable row groups (that fit in memory) and append them to the file one by one, keeping track of the meta data, and appending the meta data at the end.
import pyarrow as pa
import pyarrow.parquet as pq


schema = pa.schema([pa.field("col1", pa.int32())])

with pq.ParquetWriter("table.parquet", schema=schema) as file:
    for i in range(0, 10):
        file.write(pa.table({"col1": [i] * 10}, schema=schema))

If you're looking for an alternative where the data can be streamed, with the meta data being written at the beginning, you should look at the arrow IPC format.
Question: NewRelic Not working with multiple workers Fast API Uvicorn

We are trying to integrate NewRelic with our Fast API Service. It works fine when we are not providing numbers of worker in uvicorn config
if __name__ == "__main__":
    # newrelic.agent.register_application()
    import newrelic.agent

    newrelic.agent.initialize()
    print("api key ", os.environ.get("NEW_RELIC_LICENSE_KEY", 1))
    print("app name ", os.environ.get("NEW_RELIC_APP_NAME", 1))
    # printing to make sure licence key and app name are defined in env variables.
    uvicorn.run(app, host='0.0.0.0', port=5600)


But when we are defining numbers of workers in the uvicorn config, NewRelic does not show any data in dashboard.
if __name__ == "__main__":
    # newrelic.agent.register_application()
    import newrelic.agent

    newrelic.agent.initialize()
    print("api key ", os.environ.get("NEW_RELIC_LICENSE_KEY", 1))
    print("app name ", os.environ.get("NEW_RELIC_APP_NAME", 1))
    # printing to make sure licence key and app name are defined in env variables.
    uvicorn.run("new_relic_test:app", host='0.0.0.0', port=5600, workers=15)


Is that due to multiple server process being created by uvicorn workers?
I tried removing workers and it worked fine. But with numbers of workers it does not work


Answer:
The reason is the following: when you run uvicorn.run with only one process, it will start the server as a normal Python function. But, when you run with workers=n, uvicorn will start n new processes, and the original Python process will remain as an orchestrator between these. In these new processes, it will not start your code with a different entrypoint, meaning the if __name__ == "__main__" will not run (this is also why you must specify your app as a string instead of the Python instance when running more than one worker, since uvicorn needs to know where to import your app from). So in your case, newrelic.agent.initialize() is not run.
I would suggest moving everything except uvicorn.run out of the if __name__ == "__main__" block and put it in the same file as where you define your app.
Question: How can I open a C-file in Python using 'subprocess' without triggering an Exception?

I am trying to open a C-file in Python by using the "subprocess"-module.
I cannot execute the program without triggering a 'FileNotFoundError', and if I put in the full path of the file, I get an 'Exec format error'.
I don't necessarily need to use the 'subprocess'-module, I just don't know of any other method.
The Python-Skript:
#!/usr/bin/env python
import subprocess

shellcode_file = "shellcode.bin"

try:
    with open(shellcode_file, "rb") as f:
        shellcode = f.read();
    
    subprocess.call(["script.c", shellcode])
except FileNotFoundError as e:
    print(e, "not found.")

I am pretty sure that I am opening the File wrong but I couldn't find a way to fix it.


Answer:
You are passing the C source file itself, not the compiled executable (*.exe).
For further info if needed, C is not an interpreted/scripted language, so it cannot be run through its code file like other scripting languages (e.g. Python, Batch). It's a compiled language, meaning that it has to be run through a compiler to generate a separate file (almost always a .exe file) that you can then run your program with.
Compile your C program first and pass the executable file.
    subprocess.call(["script.exe", shellcode])
Question: Python show output of remote SSH command in web page in Django

I have a simple Django app to show the output of an SSH remote command.
views.py:
from django.http import HttpResponse
from subprocess import *

def index(request):
    with open('/home/python/django/cron/sites.txt', 'r') as file:
        for site in file:
            # out = getoutput(f"ssh -o StrictHostKeyChecking=accept-new -p1994 root@{site} crontab -l")
            out = run(["ssh", "-o StrictHostKeyChecking=accept-new", "-p1994", f"root@{site}".strip(), "crontab", "-l"])
    return HttpResponse(out)

urls.py:
from django.contrib import admin
from django.urls import path
# imported views
from cron import views

urlpatterns = [
    path('admin/', admin.site.urls),
    # configured the url
    path('',views.index, name="homepage")
]

sites.txt:
1.1.1.1
2.2.2.2
3.3.3.3

The issue is when I run localhost:5000, I see this:
CompletedProcess(args=['ssh', '-o StrictHostKeyChecking=accept-new', '-p1994', 'root@3.3.3.3', 'crontab', '-l'], returncode=0)

While I should see something like this:
* * * * * ls
* * * * * date
* * * * * pwd

I tried with both run and getoutput, but they either don't connect or the output is shown in terminal only.
How can I run this and show the output in the webpage?


Answer:
You're returning the CompletedProcess object instead of its output.
Try updating the run command to capture output by adding the optional parameter
out = run([...], capture_output=True)

AND changing the output to show the stdout instead of the object
return HttpResponse(out.stdout)
Question: Can I pass a variable string into the "Value" attribute for a HTML input tag?

sorry if this is a simplistic question or demonstrates some misunderstanding of HTML, this is my first time using it. If there's any missing info or stuff I should have included in the question, I'm happy to provide it.
I have a python program, app.py, and it calls and renders an HTML page. That HTML page contains a few inputs and I understand how to place a given string as the default value for the input with the Value attribute.
This is the html:
<form action="/new" method="post">
    <label for="qty_wheels">Number of wheels:</label>
    
    <input type="text" name="qty_wheels" value = "Qty_Wheels"     />
    <label for="flag_color">Colour of flag:</label>
    <input type="text" name="flag_color" />
    <input type="submit" class="button">
</form>

Where "Qty_Wheels" is written though, I want to place a string I've passed in from the python end.
This is the python call for this html page:
return render_template("buggy-form.html", Qty_Wheels = given_qty)

I've passed in the Qty_Wheels string here, and I want to print the value of that in the input. Is this possible and how can I do it? I've tried a lot of different syntax variants, but I can't find how I get it to interpret it as a string and print the content.


Answer:
Flask's templates use the {{ val }} syntax:
Put the double curly brackets around Qty_Wheels:
<form action="/new" method="post">
    <label for="qty_wheels">Number of wheels:</label>
    
    <input type="text" name="qty_wheels" value = "{{ Qty_Wheels }}"     />
    <label for="flag_color">Colour of flag:</label>
    <input type="text" name="flag_color" />
    <input type="submit" class="button">
</form>

For more on Flask templates see: https://flask.palletsprojects.com/en/2.3.x/tutorial/templates/
Question: Why does Tailwind declare CSS variables multiple times?

I am working on a react project with the tailwind. I checked the inspection of Chrome and saw the same tailwinds variables multiple times. I thought maybe something is not working properly in our project and checked Shopify and it was the same, I wonder why it is working in this way?
Screenshots are taken from first page of Shopify




Answer:
There are Tailwind CSS default variables defined for ::backdrop in a separate rule from the *, ::before, ::after rule for two reasons:

The *, ::before, ::after selector does not cover ::backdrop. As per tailwindlabs/tailwindcss#8526:

This PR adds the ::backdrop pseudo-element to our universal default rules, which fixes an issue where utilities like backdrop:backdrop-blur would not work because the variables that rule depended on were not defined.


As for why it is in a separate rule, it could be due to browser support. According to MDN, the last major web browser to support ::backdrop was Safari 15.4 which was released on March 14th 2022. The aforementioned pull request tailwindlabs/tailwindcss#8526 was merged on June 6th 2022 and released with v3.1.0 on June 8th 2022.
This means at that time, only very recent Safari browsers would have support for the ::backdrop element. If ::backdrop would be with the *, ::before, ::after rule selector as *, ::before, ::after, ::backdrop, this would break sites on older Safari browsers, since the *, ::before, ::after, ::backdrop rule would not apply since one of the components were not supported. This could be a major regression so they separated out the ::backdrop selector into its own rule in pull request #8567 that was released in v3.1.1.
Question: In Strapi V4, How should I register a middleware to alter the responses returned by /user or /user/:id?

I've started to play with middlewares, it's great !
Here's an example of how I can inject a middleware when calling endpoints /api/playlists or /api/playlists/:id (I edited the file src/api/playlist/routes/playlist.js).
module.exports = createCoreRouter('api::playlist.playlist', {
 config: {
   find: {
     middlewares: ['api::playlist.playlist.find']
   },
   findOne: {
     middlewares: ['api::playlist.playlist.find-one']
   },
 },
})

Of course, I also created my middlewares in src/api/playlist/middlewares/find.js and src/api/playlist/middlewares/find-one.js)
But know, I want to add another middleware to update the responses returned by the API when calling /api/users or /api/users/:id.
Since there is no directory src/api/user in the filetree, how should I register a middleware for this ?
Thanks


Answer:
I finally found out :

create the file src/extensions/user-permissions/strapi-server.js

This is mine. It registers a middleware for each of those endpoints:

GET /users/me : plugin::spiff-api.user-me

GET /users : plugin::spiff-api.user-find

GET /users/:id : plugin::spiff-api.user-find-one
("use strict");

module.exports = (plugin) => {
  //if you see this, the configuration do loads:
  console.log("Custom strapi-server.js for user-permissions");

  //get api routes for 'user-permissions'
  const apiRoutes = plugin.routes['content-api'].routes;

  //add middleware for GET /users/me
  apiRoutes
    .filter(route => route.handler === 'user.me')
    .map(route => {
      route.config.middlewares = [
        ...(route.config.middlewares || []),
        'plugin::spiff-api.user-me'//middleware name
      ];
      return route;
    });

    //add middleware for GET /users/:id
    apiRoutes
      .filter(route => route.handler === 'user.findOne')
      .map(route => {
        route.config.middlewares = [
          ...(route.config.middlewares || []),
          'plugin::spiff-api.user-find-one'//middleware name
        ];
        return route;
      });

  //add middleware for GET /users
  apiRoutes
    .filter(route => route.handler === 'user.find')
    .map(route => {
      console.log(route)
      route.config.middlewares = [
        ...(route.config.middlewares || []),
        'plugin::spiff-api.user-find'//middleware name
      ];
      return route;
    });

return plugin;
};



Then, create your middlewares middleware, eg.
    export default (config, { strapi })=> {
      return async (ctx, next) => {
        console.info("running middleware 'user-find-one.js'");
        console.log();

        //update your query here if needed
        //eg. populate 'favoritePosts'
        ctx.query.populate = {
          ...ctx.query.populate ?? {},
          favoritePosts: {}
        }

        const controller = strapi.plugin('users-permissions').controller('user');
        await controller.findOne(ctx);//this populates ctx.body
        const response = ctx.body;

        //update your response here if needed

        ctx.body = response;
        
        await next();//not sure why this stands for

      }
    }