satoshi-2000/llms-suitable huggingface dataset:
Question: Use python schedule module in an efficient manner

I want to use python scheduler module for an api call.
I have two main tasks.
1-Generate token at interval of 10 mins
2-Call the api using the token generated in previous step
The expected behavior is when a user calls the api it should use the same token for 10 mins and after 10 mins the toekn will be updated in background
Here are the sample piece of code
token=""(This is a global variable)
def generate_token():
 token=//Logic to generate token//


def apicall():
 postreq=//api call using the token//


def schedule_run():
 schedule.every(10).minutes.do(get_token_api)
 while 1:
        schedule.run_pending()
        time.sleep(1)

if __name__ == "__main__":
 schedule_run()
 apicall()
 


When I am running above code the code is getting stuck in the while loop of schedule_run()  and not calling apicall()
Is there any efficient way to handle this?


Answer:
You are getting stuck in the (infinite) loop inside schedule_run. First define the two scheduled jobs, the run the schedule waiting loop:
token=""(This is a global variable)
def generate_token():
 token=//Logic to generate token//


def apicall():
 schedule.run_pending() #will update the token if it has not been done in the last 10 minutes
 postreq=//api call using the token//

if __name__ == "__main__":
 schedule.every(10).minutes.do(get_token_api)
 apicall()
Question: Firebase RT Database Query on field value hash equality

I have a Firebase Realtime Database with users and the email is stored in a child field, for example:
/users/00Vho6lTQke46IqpRv0D5dw6DXs2/email -> user@email.com

I want to query once and get a user based on the MD5 hash of the email address. For example, MD5('user@email.com') -> 'b58c6f14d292556214bd64909bcdb118'.
I don't have the email address of the user, I only have the MD5 hash. Is it possible to query using an MD5 function to transform the email field to MD5 and query on that?


Answer:

Is it possible to query using an MD5 function to transform the email field to MD5 and query on that?

No, unless you save the MD5 as a field inside the database. If you do that, you'll be able to query using the MD5. Your schema should look like this:
db
|
--- users
     |
     --- 00Vho6lTQke46IqpRv0D5dw6DXs2
           |
           --- email: "user@email.com"
           |
           --- md5: "b58c6f14d292556214bd64909bcdb118"

And the query should look like this:
val query = db.child("users").orderByChild("md5").equalTo("b58c6f14d292556214bd64909bcdb118");
Question: Update + join from two databases

Help me create an sql query.
There are two databases DB1 and DB2. In each of them there are tables "users" and "cities" with the names of cities. Users and city names are the same in both databases. But the city ID values are different in the two databases. In the first DB1 database, the "users" table contains the city IDs from the "cities" table.
I need to update the city data in the second DB2 database. For each "users" value from DB1, find the name "cities" and update the values in DB2.users. The problem is that the city ID values are different in the two databases. Only the names are the same.



Answer:
Use an update with join:
update DB2.users set
city = DB2.cities.ID
from DB2.users 
join DB1.users on DB1.users.ID = DB2.users.ID
join DB1.cities on DB1.cities.ID = DB1.users.city
join DB2.cities on DB2.cities.name = DB1.cities.name

The join from DB2.users to DB2.cities goes via DB1's tables, going over using user ID and coming back using city name.
Question: Writing a Nested List of Strings to CSV using CSV helper

I have a nested list of strings that I would like to write to a CSV file using CSV helper. So, it is a list called Entries and each element has a list of strings called Entry.
My current code is exporting a blank CSV file.
Entry example:
{"Date", "param1", "param2", "param3", "param4"}

and the Entries list will have a list of those.
I would like to write to a CSV file with each entry being a new line.
Intended outcome:
"Date", "param1", "param2", "param3", "param4"
"Date", "param1", "param2", "param3", "param4"
"Date", "param1", "param2", "param3", "param4"
"Date", "param1", "param2", "param3", "param4"

My current code:
var rawDataList = await _siteRepository.GetRawSiteData(siteGuid);

var Entries = _siteRepository.GetSiteData(rawDataList);
var writer = new StreamWriter("YodelTesting.csv");
var csv = new CsvWriter(writer, CultureInfo.InvariantCulture);

foreach(var record in Entries)
{
    csv.WriteRecord(record);
}

Help would be appreciated!


Answer:
I used the current code to get it working
using (var stream = new MemoryStream())
using (var writer = new StreamWriter("yodelTesting.csv"))
using (var reader = new StreamReader(stream))
using (var csv = new CsvWriter(writer, CultureInfo.InvariantCulture))
{
    foreach (var record in output)
    {
        foreach (var field in record)
        {
            csv.WriteField(field);

        }
        csv.NextRecord();
    }
    //not sure what this does tbh
    writer.Flush();
    stream.Position = 0;
}
Question: Ansible Variable precedence - what are 'role params' & 'include params'

The Ansible documentation has a list of Variable precedence
Some of them are clear to me but I wonder whether anybody coulde kinkdy shed some light on those 2.

20. role (and include_role) params
21. include params

usage, location, syntax.
I am trying to get Variable declaration a little further to the surface inside a little more complex playbook utilizing 2 roles I am currently working on.
concrete ... some values inside task files of a role should rather be declared as variables in a single location.


Answer:
In a nutshell examples:
role params
(full playbook)
- name: I'm a dummy play
  hosts: localhost

  roles:
    - role: somerole
      vars:
        param1: "I'm a role param"

Include role params
(task only)
- name: Including role somerole
  ansible.builtin.include_role:
    name: somerole
  vars:
    param1: "I'm an include role param"

Include params
(task only)
- name: Including a task file
  ansible.builtin.include_tasks: sometasks.yaml
  vars:
    param1: "I'm an include param"


As a fictive and (most probably bad practice) example: if you include a role passing a parameter and later include a task file in that role passing that same parameter with a different value, the include param will take precedence over the role param.
Question: Why garbage value is not being returned in a recursive call to a function with undefined behavior?

I came across this code somewhere when practicing basic C questions:
int func(num) {
    if (num > 0) {
        return (num + func(num - 2));
    }
}

int main() {
    printf("%d\n", func(5));
    return 0;
}

The code when executed returns 8 as the answer. But I think that the answer should be "cannot be determined".
The recursive calls in my head looks like this:
5 + func(3)  => 5 + (3 + func(1))
3 + func(1)  => 3 + (1 + func(-1))
1 + func(-1) => 1 + (?)

For the ? symbol, I think reading the return value for func(-1) is undefined behavior. I'm basing this on the assumption that for func(-1) no explicit int value is being returned. So reading from func(-1) in the expression (1 + func(-1)) should produce some garbage result + 1 in my opinion.
Why is that the code is returning 8 as the answer and not something garbage?
When I explicitly pass a negative number to func and read the result I do get the garbage value, such as in this code;
int main() {
    printf("%d\n", (1 + func(-1)));   // returns garbage result and not 0
    return 0;
}

Why is that in the recursive call (1 + func(-1)) is being evaluated to 0 value?
I compiled the code on a 64-bit machine with gcc as gcc myfile.c. Compiling the code as gcc -W -Wall -ansi -pedantic myfile.c gives warning about the func function, but that's not the point. I am unable to figure out how 8 is the answer.


Answer:

Why garbage value is not being returned in a recursive call to a function with undefined behavior?

Because "garbage" does not mean what you think it means in this context.  In particular, "garbage" does not mean "random".  (Or if it does, it's more in the sense of xkcd 221.)
Computers are usually deterministic.  You have to work pretty hard to get truly random behavior.  Even a program that contains the worst kind of undefined behavior will quite often return exactly the same strange and indeterminate number every time you run it.
I have used this analogy:

Suppose you go to the store and buy a brand-new garbage can. But it's completely clean! There's no garbage in it at all! It's so clean you could eat out of it! Was this false advertising? Did the store fraudulently sell you a non-garbage can?

See more discussion at these previous questions:
1
2
3
4
5.
(Most of those are talking about the values of uninitialized local variables, not the values of functions that fail to execute a proper return statement, but the arguments are the same.)
Question: DB2 Group by specific columns





ID_1
ID_2
ID_3
LANG




1
11
111
F_lang


1
11
111
null


2
22
222
null




Is it possible to build  a query which returns only these two lines below.
It should be grouped by the first three columns and should take only that line with value of column LANG which is preferably not null, if no value for LANG exists, it should take the line with language null.




ID_1
ID_2
ID_3
LANG




1
11
111
F_lang


2
22
222
null





Answer:
Does this anwser the question ?
with t1(id_1, id_2, id_3, lang) as (
  VALUES
  ('1', '11', '111', 'F_lang'),
  ('1', '11', '111', NULL),
  ('2', '22', '222', NULL),
  ('3', '33', '333', 'F_lang_3_1'),
  ('3', '33', '333', 'F_lang_3_2'),
  ('3', '33', '333', NULL)
)
select id_1, id_2, id_3, lang from t1 where lang is not null
union all
select id_1, id_2, id_3, null as lang from t1 group by id_1, id_2, id_3 having max(lang) is null
order by id_1, id_2, id_3





ID_1
ID_2
ID_3
LANG




1
11
111
F_lang


2
22
222
null


3
33
333
F_lang_3_1


3
33
333
F_lang_3_2




fiddle
Question: How to express h(n) which is the "union" of the two sequences f(n) & g(n)

I am on thin ice both syntax-wise and English-wise here, apologise if my way of expressing maths here is a rambling of a mad-man.
I have two sequences/"functions" f(n) and g(n). They are technically not functions, I have them just defined as a sequence of repeating modulus of 9 and 10.
n = 0,1,2,3,...
f(n): nMOD9={0,3,4,6}
g(n): nMOD10={0,3,4,5,8,9}

This means that f(n) is will go 0,3,4,6,9,12,13,15,18,ect. Just repeating the four numbers in the brackets of modulus of 9. 

g(n) will be 0,3,4,5,8,9,10,13,14,15,18,19,20,ect. Repeating the six numbers in brackets of modulus of 10.

Now I wonder, can I express h(n) which is the list of numbers that is both present in f(n) and g(n)? This will be 0,3,4,9,13,15,18,ect. Either as a function or as some nMODx={a,b,c}? Or some other genius way I have not thought about. 

Currently I do a manual check of both list, and I wonder if can be done more elegant.


Answer:
One possible solution would be to find all values of h(n) up to 90, then h(n) will be those nMOD90, so
h(n): nMOD90={0,3,4,9,13,15,18,24,30,33,39,40,45,48,49,54,58,60,63,69,75,78,84,85}
Question: "BASE64DecoderStream" gives error for javax-mail dependency

I recently changed the dependency
<dependency>
    <groupId>javax.mail</groupId>
    <artifactId>mail</artifactId>
    <version>1.4</version>
</dependency>

to
<dependency>
    <groupId>javax.mail</groupId>
    <artifactId>javax.mail-api</artifactId>
    <version>1.6.0</version>
</dependency>

because Java 8 doesn't support 1.4 version as it uses TLS 1.0.
After changing the dependency this code starts giving error.
Error code:
if (p.getContentType().contains("image/")) {
    File f = new File("image" + new Date().getTime() + ".jpg");
    DataOutputStream output = new DataOutputStream(
            new BufferedOutputStream(new FileOutputStream(f)));
    
    com.sun.mail.util.BASE64DecoderStream test = (com.sun.mail.util.BASE64DecoderStream) p
            .getContent();
    byte[] buffer = new byte[1024];
    int bytesRead;
    while ((bytesRead = test.read(buffer)) != -1) {
        output.write(buffer, 0, bytesRead);
    }

Eclipse suggestion error:
Multiple markers at this line
    - com.sun.mail.util.BASE64DecoderStream cannot be resolved to a type
    - com.sun.mail.util.BASE64DecoderStream cannot be resolved to a type



Answer:
You're using the wrong dependency. You have only added the JakartaMail API. You should use the JakartaMail 1.6.x implementation:
<dependency>
    <groupId>com.sun.mail</groupId>
    <artifactId>jakarta.mail</artifactId>
    <version>1.6.7</version>
</dependency>

This dependency does include com.sun.mail.util.BASE64DecoderStream.
As an aside, instead of what you're doing now (using getContent() and casting to an implementation-specific class), you could also use Part.getInputStream():
try (var in = p.getInputStream()) {
    in.transferTo(output);
}

Also, your use of DataOutputStream is suspect, as you don't seem to be using any DataOutputStream specific methods, so using the BufferedOutputStream directly should be sufficient.
Question: Where should pygame.time.Clock().tick() be called in the script

According to this statement, pygame.time.Clock().tick() should be called at the end of the main loop. However, I couldn't see any differences on my screen display regardless where I executed that method within the loop. Could someone please give some clarification on this? Thanks


Answer:
The documentation say :

A call to the tick() method of a Clock object in the game loop can
make sure the game runs at the same speed no matter how fast of a
computer it runs on

So it is better to call it at the end of the loop because if you do it in the middle of your display fonction, a part of the element will be refresh before the wainting and a part after.You should call pygame.display.update() before that otherwise you refresh the screen after the "frame wait time".
Question: Remote Interpreter DataSpell

I am trying to run scripts on a remote server with DataSpell, so I am trying to configure a remote interpreter.
I follow these instructions and I get the error
Cannot Save Settings:
SSH Python x.x.x user@IP : Python x.x.x
(path/to/interpreter) can't be used as a workspace interpreter

As a path to the interpreter I have used the .../bin/python file that is created in a virtual environment. I have tried virtual environments created both with conda and venv. I have also found this, but I cannot understand the solution clearly.
Any ideas?


Answer:
It appears that Dataspell can have different interpreters for its workspace and projects or notebooks that are opened in the workspace.
I right-clicked the project folder in the Project window, and selected a remote interpreter just for the project. This is what the second source that I posted on my OP says. This works fine.
Question: Gitlab CI Deploy Token cant push to repo

I am trying to copy some files from Firebase into a Gitlab repo.
Using my personal SSH credentials, I am able to do this.
I'd like to use deploy tokens. I generated a fresh token and gave it all the permissions I can. However, when I run the CI pipeline I get an "fatal: Authentication failed"


Answer:
A deploy token is simply not made for that. As per the documentation a deploy token is not capable to write to a GitLab repository. Here is what you can do with a deploy token:

Clone Git repositories.
Pull from and push to a GitLab container registry.
Pull from and push to a GitLab package registry.

What may be a better fit for your use case is a so called Deploy Key. If the deploy key has read-write permissions you should be able to solve your issues.
Question: Replace one button with another when countdown timer ends

I have a simple countdown timer which is launched on the click of the .btn-submit-a button. I tried to make it so that after the timer ends, button A is replaced by button B (.btn-submit-b), but, unfortunately, nothing comes out. How can I achieve this? I will be glad for any help.


jQuery(function($){
  $('.btn-submit-a').on('click', doCount);
});

function doCount() {
var timeleft = 5;
var downloadTimer = setInterval(function(){
  if(timeleft <= 0){
    clearInterval(downloadTimer);
    document.getElementById("countdown").innerHTML = "Time is Up";
  } else {
    document.getElementById("countdown").innerHTML = timeleft + "<span class='remain'>seconds remain</span>";
  }
  timeleft -= 1;
}, 1000);
};
.btn-submit-b {
  display: none;
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<button type="submit" class="btn-submit-a">Button A</button>
<button type="submit" class="btn-submit-b">Button B</button>

<div id="countdown"></div>





Answer:
You had a weird combination between jquery and native code. You can do that ofcourse but I would recommend to stick to either jQuery or native where possible. Therefore I changed some code to jQuery functions.
This said you can hide and show elements with the hide() and show() jQuery functions as you can see in the example below.
If you want to toggle them instead you can use toggle()


jQuery(function($) {
  $('.btn-submit-a').on('click', doCount);
});

function doCount() {
  var timeleft = 5;
  var downloadTimer = setInterval(function() {
    if (timeleft <= 0) {
      clearInterval(downloadTimer);
      $("#countdown").html("Time is Up");
      $('.btn-submit-a').hide();
      $('.btn-submit-b').show();
    } else {
      $("#countdown").html(timeleft + "<span class='remain'>seconds remain</span>");
    }
    timeleft -= 1;
  }, 1000);
};
.btn-submit-b {
  display: none;
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.3.1/jquery.min.js"></script>
<button type="submit" class="btn-submit-a">Button A</button>
<button type="submit" class="btn-submit-b">Button B</button>

<div id="countdown"></div>
Question: How can I get Ansible client IP from target host?

On an isolated network (without internet access to do public IP address lookups), I want to run a playbook from a controller against a number of target hosts where one of the tasks is to download a file via HTTP/HTTPS from the controller without hard-coding the controller IP as part of the task. E.g.
Controller: 192.168.0.5
Target 1: 192.168.0.10
Target 2: 192.168.0.11
Target 3: 192.168.0.12

The controller can have different IPs configured via DHCP, and there could be multiple network interfaces listed in ansible_all_ipv4_addresses (some of which may not be available to the target hosts) so it may not be straight forward to determine which network interface the target hosts should use from ansible_facts on localhost without exploring the idea of looping through them with a timeout until the file has been downloaded. It seems as though the most robust way to determine the public IP of the controller (assuming the web server is listening on 0.0.0.0) would be to determine the originating IP of the established connection (192.168.0.5) from the target host - is there a way to do this?
The motivation for downloading the file from the controller rather than sending it to remote hosts is that some of the target hosts are running Windows and the win_copy module is incredibly slow via WinRM so the Ansible documentation includes the following note:

Because win_copy runs over WinRM, it is not a very efficient transfer
mechanism. If sending large files consider hosting them on a web
service and using ansible.windows.win_get_url instead.



Answer:
Limited test on my machine which has a single ip and with a single target. But I don't see why it would not work in your scenario.
Given the following inventories/default/hosts.yml
all:
  hosts:
    target1:
      ansible_host: 192.168.0.10
    target2:
      ansible_host: 192.168.0.11
    target3:
      ansible_host: 192.168.0.12

The following test playbook should do what you expect. Replace the dummy debug task with get_url/uri to initiate the download.
Notes:

this playbook infers you have access to the ip command line tool on the controller.
I took for granted that the controller IP used to connect to target is the one that the target has access to in the other direction. If this isn't the case then the below will not work in your situation.

---
- hosts: all
  gather_facts: false

  tasks:
    - name: Check route on controller for each target destination
      ansible.builtin.command: ip route get {{ ansible_host }}
      register: route_cmd
      delegate_to: localhost

    - name: Register the controller outgoing ip for each target
      ansible.builtin.set_fact:
        controller_ip: "{{ route_cmd.stdout_lines[0] | regex_replace('^.* src (\\d*(\\.\\d*){3}).*$', '\\1') }}"

    - name: Show result
      ansible.builtin.debug:
        msg: "I would connect from target {{ inventory_hostname }} ({{ ansible_host }}) 
          to controller using ip {{ controller_ip }}"
Question: How to export a PyTorch model for HuggingFace?

I have been training my custom Image classification model on the PyTorch transformers library to deploy to hugging face however, I cannot figure out how to export the model in the correct format for HuggingFace with its respective config.json file.
I'm new to PyTorch and AI so any help would be greatly appreciated
train.py
from tqdm import tqdm

best_accuracy = 0

# Train the model for a number of epochs
for epoch in range(20):
    # Create a progress bar for this epoch
    pbar = tqdm(train_loader, desc=f'Epoch {epoch+1}/{20}')
    
    # Loop over each batch of data
    for X_batch, y_batch in pbar:
        # Move the batch of data to the device
        X_batch = X_batch.to(device)
        y_batch = y_batch.to(device)

        # Zero the gradients...
        # Define an optimizer...
        
        # Update the progress bar
        pbar.set_postfix({'Loss': loss.item()})
    
    # Evaluate the model on the validation set
    model.eval()
    correct = 0
    total = 0
    val_loss = 0
    
    with torch.no_grad():
        for X_batch, y_batch in test_loader:
            # Move the batch of data to the device
            X_batch = X_batch.to(device)
            y_batch = y_batch.to(device)
            
            # Compute the model's predictions for this batch of data
            y_pred = model(X_batch)
            
            # Compute the loss
            loss = criterion(y_pred, y_batch)
            val_loss += loss.item()
            
            # Compute the number of correct predictions
            _, predicted = torch.max(y_pred.data, 1)
            total += y_batch.size(0)
            correct += (predicted == y_batch).sum().item()
    
    val_loss /= len(test_loader)
    accuracy = correct / total
    
    print(f'Validation Loss: {val_loss:.4f}, Accuracy: {accuracy:.4f}')
    
    if accuracy > best_accuracy:
        best_accuracy = accuracy
        torch.save(model.state_dict(), 'best_model.pth')
    
    model.train()



Answer:
You are using HuggingFace Transformers, you can use:
model.save_pretrained("FOLDER_NAME_HERE")

After you saved the model, the folder will contain the pytorch_model.bin along with config JSONs.
Question: AttributeError: module 'numpy' has no attribute 'long'

I am trying to find 9 raise to power 19 using numpy.
I am using numpy 1.24.3
This is the code I am trying:
import numpy as np
np.long(9**19)

This is the error I am getting:
AttributeError: module 'numpy' has no attribute 'long'



Answer:
Sadly, numpy.long was deprecated in numpy 1.20 and it is removed in numpy 1.24
If you wan the result you have to try numpy.longlong
import numpy as np
np.longlong(9**19)

#output
1350851717672992089

https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations