satoshi-2000/llms-suitable huggingface dataset:
Question: 403 for the new pages in the cloudfront distribution

I have a static web site in the s3 bucket behind the cloudfront distribution.
The bucket serves the static site, and the origin is bound to the web site endpoint.
I see a couple of pages if they were added before the distribution
However, when I upload some new html files, I receive 403 for them.
How should I fix this issue?
Bucket policy:
{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "AllowCloudFrontServicePrincipal",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudfront.amazonaws.com"
            },
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::a-test-upload/*",
            "Condition": {
                "StringEquals": {
                    "AWS:SourceArn": "arn:aws:cloudfront::xxx:distribution/yyy"
                }
            }
        }
    ]
}



Answer:
Try to invalidate cloudfront cache. Go to cloud front distribution and click on invalidation enter "/*". Click on create invalidation.
If you are trying to access object publicly, then provide public access to s3 bucket objects.
Question: Get the end_time in a query using the start time of the next sorted column

Imagine I have the following SQL table:
| id | price | start_time |
---------------------------
|  1 |  0.1  | 2023-01-01 |
|  2 |  0.3  | 2023-03-01 |
|  3 |  0.2  | 2023-02-01 |

But then I want to query the prices in that table in a way that I can also get the end time as the start time of the next in time column. So, as an example, if I want to query all the entries in the table I would get something like this:
| id | price | start_time | end_time   |
----------------------------------------
|  1 |  0.1  | 2023-01-01 | 2023-02-01 | // end_time = start_time of the next entry
|  3 |  0.2  | 2023-02-01 | 2023-03-01 |
|  2 |  0.3  | 2023-03-01 |            |

But, I would also like to query that table with others filters, as an example, all entries whose prices are lower than 0.25, then I expect:
| id | price | start_time | end_time   |
----------------------------------------
|  1 |  0.1  | 2023-01-01 | 2023-02-01 |
|  3 |  0.2  | 2023-02-01 | 2023-03-01 | end_time = start_time of entry with id 2

So even that the entry with id 2 is filtered out, its start_time is still used as end_time of one of the entries.
Is this possible to achieve with one single query? I am bit lost on how to solve this approach without doing multiple queries.


Answer:
This can be done using the window function lead() to get the which follows the current row :
select *, lead(start_time) over (order by start_time) as end_time
from mytable

With where clause it can be :
select * from (
  select *, lead(start_time) over (order by start_time) as end_time
  from mytable
) as s
where price < 0.25

Demo here
Question: Next.js 13.4 and NextAuth Type Error: 'AuthOptions' is not assignable to type 'never'

I'm currently working on a Next.js 13.4 project and trying to set up NextAuth using the app/ router. However, I'm encountering a type error that I can't seem to resolve.
Here's my route.ts file:
import NextAuth, { AuthOptions } from "next-auth";
import DiscordProvider from "next-auth/providers/discord";

export const authOptions: AuthOptions = {
  providers: [
    DiscordProvider({
      clientId: process.env.CLIENT_ID as string,
      clientSecret: process.env.CLIENT_SECRET as string,
    }),
  ],
  session: {
    strategy: "jwt",
  },
  secret: process.env.NEXTAUTH_SECRET,
}

const handler = NextAuth(authOptions);

export { handler as GET, handler as POST }

And here's the error message when running 'npm run build':
- info Linting and checking validity of types ...Failed to compile.

.next/types/app/api/auth/[...nextauth]/route.ts:8:13
Type error: Type 'OmitWithTag<typeof import("C:/Users/Luk/Documents/Workspace/zerotwo-dash/src/app/api/auth/[...nextauth]/route"), "GET" | "POST" | "HEAD" | "OPTIONS" | "PUT" | "DELETE" | "PATCH" | "config" | ... 6 more ... | "runtime", "">' does not satisfy the constraint '{ [x: string]: never; }'.
  Property 'authOptions' is incompatible with index signature.
    Type 'AuthOptions' is not assignable to type 'never'.

   6 |
   7 | // Check that the entry is a valid entry
>  8 | checkFields<Diff<{
     |             ^
   9 |   GET?: Function
  10 |   HEAD?: Function
  11 |   OPTIONS?: Function

I really have no idea whats happening here... when looking up the 'AuthOptions' on t he github page from nextauth i see nothing wrong with my code..
I would appreciate any insights or suggestions on how to resolve this issue. Thanks in advance!


Answer:
Okay i solved it myself. For anyone having the same issue, i created a new file @/utils/authOptions.ts:
import { NextAuthOptions } from "next-auth";
import DiscordProvider from "next-auth/providers/discord";

export const authOptions: NextAuthOptions = {
    providers: [
        DiscordProvider({
            clientId: process.env.CLIENT_ID as string,
            clientSecret: process.env.CLIENT_SECRET as string,
        }),
    ],
    session: {
        strategy: "jwt",
    },
    secret: process.env.NEXTAUTH_SECRET,
}

and used it in @/api/auth/[...nextauth]/route.ts:
import { authOptions } from "@/utils/authOptions";
import NextAuth from "next-auth/next";

const handler = NextAuth(authOptions);

export { handler as GET, handler as POST };

I changed the imports a little (import NextAuth from "next-auth/next")
I dont know why this works tbh, but it does... even changing the imports around in my route.ts as it was before wont fix it. Only if its seperated like that...
Question: is defined but never used no-unused-var

I'm New to React, and I'm trying to learn about React Component right now. but when I create nameList.js, add data in there, and export to app.js, it does not show anything in the browser. I read some answers in Stackoverflow and tried it. but not show anything.
App.js
import './App.css';
import nameList from './Components/nameList';

function App() {
  return (
    <div className="App">
      <div className="App" >
            <nameList/>
        </div>
    </div>
  );
}

export default App;


nameList.js in Component Folder

import React from 'react';

function nameList() {
    return (
    <div>
       <h1>Name List</h1>
        <ul>
            <li>Stu1</li>
            <li>Stu2</li>
            <li>Stu3</li>
        </ul>
    </div>
  )
}

export default nameList;

I need to get data in nameList.js into App.js


Answer:
Ensure that nameList starts with a capital letter so that React knows it's a component and not an HTML element.
import React from 'react';

function NameList() {
    return (
    <div>
       <h1>Name List</h1>
        <ul>
            <li>Stu1</li>
            <li>Stu2</li>
            <li>Stu3</li>
        </ul>
    </div>
  )
}

export default NameList;

App.js
import './App.css';
import NameList from './Components/nameList';

function App() {
  return (
    <div className="App">
      <div className="App" >
            <NameList/>
        </div>
    </div>
  );
}

export default App;

https://react.dev/learn/your-first-component#what-the-browser-sees
Question: useState() not updated when state is used in a loop

This code is where I modified the data, wherein I update the "checked" to whether "true" or "false". and this code works because when I console.log() it it updates the state.
    setRolesData((prevState) => {
        const newState = Array.from(prevState);
        newState[targetCb]["checked"] = e.target.checked;
        return newState;
    });
    console.log(rolesData);

Image of the result:

This code is where I loop thru the data that is updated, but the problem is it doesnt return the same data as the log data if tried for the second time.
    // get all checked roles
    const roleArr: any = [];
    rolesData.map((item, index) => {

        console.log(item.checked + " ^ " + item.name);

    });

Image of the Result:



Answer:
In React, components have a lifecycle that is based around the idea of preserving state. When you update a value in a component we effectively trigger a re-render of the component. So if you have a variable at the top of your component with a certain value and try to update this value inside a function, you'll find that this value gets reset. React is replacing the component including the JavaScript variables inside it.
In order to preserve any variables we use useState to persist the state of the component between re-renders. However, in a function, using useState does not save the value to state immediately. useState is an instruction that is sent to React that tells it that when the component re-renders, we need to save that value. If you try to access the value in state before your re-render has begun you will be using the "previous" value instead. In your case, your component will not re-render until the calling function has been completed.
So how do you listen for changes?
UseEffect is a special hook that takes an array of dependencies as "listeners". When we use useEffect and give it a value in state as a dependency, it's telling it to run the code inside the useEffect if there is a mutation to that value.
useEffect(() => {

    rolesData.map((item, index) => {
      console.log(item.checked + " ^ " + item.name);
    });

}, [rolesData]);

This code will run every time myState is changed, including on component initialisation. So we can use useEffect to process logic that requires the newly updated stateful values.
Question: How to capture words with letters separated by a consistent symbol in Python regex?

I am trying to write a Python regex pattern that will allow me to capture words in a given text that have letters separated by the same symbol or space.
For example, in the text "This is s u p e r and s.u.p.e.r and s👌u👌p👌e👌r and s!u.p!e.r", my goal is to extract the words "s u p e r", "s.u.p.e.r", and s👌u👌p👌e👌r. However, I want to exclude "s!u.p!e.r" because it does not have the same consistent separating symbol within the word.
I'm currently using the following:
x="This is s u p e r and s.u.p.e.r and s👌u👌p👌e👌r and s!u.p!e.r"


pattern = r"(?:\b\w[^\w\d]){2,}"

re.findall(pattern, x)


['s u p e r ', 's.u.p.e.r ', 's👌u👌p👌e👌r ', 's!u.p!e.']

I'm just curious if it's possible to exclude the cases that do not have the same symbol.


Answer:
You may consider using
pattern = r"(?<!\S)\w(?=(\W))(?:\1\w)+(?!\S)"
results = [m.group() for m in re.finditer(pattern, x)]

See the Python demo and the regex demo.
import re
x="This is s u p e r and s.u.p.e.r and s👌u👌p👌e👌r and s!u.p!e.r"
pattern = r"(?<!\S)\w(?=(\W))(?:\1\w)+(?!\S)"
print([m.group() for m in re.finditer(pattern, x)])
# => ['s u p e r', 's.u.p.e.r', 's👌u👌p👌e👌r']

Pattern details

(?<!\S) - left-hand whitespace boundary
\w - a word char
(?=(\W)) - a positive lookahead that requires the next char to e a non-word char capturing it into Group 1 (\1)
(?:\1\w)+ - one or more repetitions of the same char as captured in Group 1 and then a single word char
(?!\S) - right-hand whitespace boundary
Question: How to transform Microsoft Graph API query with expand=extensions to SDK code

I am transforming from Graph API to Graph SDK. How can I transform this API call?
https://graph.microsoft.com/v1.0/users/XXX/calendarView?startDateTime=YYY&endDateTime=ZZZ&$expand=extensions($filter=id eq 'NAME')

Expand part is missing. How should I add Expand = ??? or do it somehow with Filter or Select?
var ret = await graphClient.Users[XXX].CalendarView.GetAsync((requestConfiguration) =>
            {
                requestConfiguration.QueryParameters.StartDateTime = YYY;
                requestConfiguration.QueryParameters.EndDateTime = ZZZ;
            });

Thank you for help.


Answer:
Thank you @user2250152 . I made a typo. Solution:
requestConfiguration.QueryParameters.Expand = new string[] { "extensions($filter=id+eq+'NAME')" };
Question: Parsing multiple date string languages and formats

I'm parsing a list of emails in a text file and I need to parse dates in the email headers. The dates are in a multitude of formats and languages:
sexta-feira, 26 de agosto de 2022 16:41
viernes, 26 de agosto de 2022 19:24
2022/08/26 13:30:56
26 de agosto de 2022 13:32:49 BRT

Mostly portuguese, spanish, italian and english.
What would be the best aproach? I have tried Babel but the date parsing is very basic. For now I only have access to the text files exported from Outlook not the smpt sources.


Answer:
The dateparser package provides modules to parse localized dates in most string formats.
The following snippet successfully retrieves all dates in the given example:
import dateparser

text_dates = [
    "sexta-feira, 26 de agosto de 2022 16:41",
    "viernes, 26 de agosto de 2022 19:24",
    "2022/08/26 13:30:56",
    "26 de agosto de 2022 13:32:49 BRT",
]

datetimes = [dateparser.parse(line) for line in text_dates]

print(datetimes)

>>> [datetime.datetime(2022, 8, 26, 16, 41),
 datetime.datetime(2022, 8, 26, 19, 24),
 datetime.datetime(2022, 8, 26, 13, 30, 56),
 datetime.datetime(2022, 8, 26, 13, 32, 49, tzinfo=<StaticTzInfo 'BRT'>)]
Question: Window function, order by clause, between operator

Consider the MWE below
WITH samp AS (
    SELECT '2023-01-01' AS day, 1 AS spent UNION ALL
    SELECT '2023-01-02' AS day, 2 AS spent UNION ALL 
    SELECT '2023-01-03' AS day, 3 AS spent
) 
SELECT day, 
       spent 
     , ARRAY_AGG(spent) OVER(ORDER BY day BETWEEN '2023-01-02' AND '2023-01-03') ss
FROM samp 
ORDER BY day

I cannot figure out what the order by clause is doing here. I'd expect to restrict the entries to those of the selected dates, but dates outside it also have a contribution? E.g., outcome of the above




day
spent
ss




'2023-01-01'
1
[1]


'2023-01-02'
2
[1,2,3]


'2023-01-03'
3
[1,2,3]





Answer:
The clause
day between '2023-01-02' and '2023-01-03'

is a boolean expression, and will only evaluate to two possible values, true or false (1 or 0).  Therefore, your window function array_agg(spent) will compute using an order where dates other than 2023-01-02 and 2023-01-03 will be ordered first, followed by these dates next.
Here is your updated output showing the ordering logic:
     day         spent         ss      order (day between ...)
'2023-01-01'       1           [1]       0
'2023-01-02'       2         [1,2,3]     1
'2023-01-03'       3         [1,2,3]     1
Question: How do I print the user and umask for all running processes?

The username is in column one and PID in column two of ps gaux, so I have:
ps gaux | awk '{print $2;}' | while read line ; do grep -i umask /proc/$line/status ; done

but is there a way to print the username as well?


Answer:
I hope this helps
ps gaux | awk '{printf $1 " " ; system("grep Umask /proc/"$2"/status | tr -dc [:digit:]"); printf "\n"}'

Explanation:

get output from ps
print first column (username) and space
run the grep and remove everything except the actual umask, which is a number (awk does not print the command output, it gets just printed directly from the subshells stdout)
print a new line
Question: How to change color scheme for static methods in IntelliJ IDEA / Android Studio

How to disable italics on static methods called inside AndroidStudio IDE?

I know, it's a personal preference question but is there a way to disable italics? and only keep the colour coding?


Answer:
Settings/Preferences (on macOS) | Editor | Color Scheme | Java | Methods | Static method

Additional trick: to easily find the corresponding color scheme settings do the following:

Put the cursor at the needed element you need to change
Press Shift twice
Type Just to Colors and Fonts and press Enter. Then select the corresponding option
Question: common/unified regex for a set of pattern

I am trying to do some text processing and was interested to know if I can have a common/unified regex for a certain pattern. The pattern of interest is strings that ends with {string}_{i} where i is a number, on the second column of test.csv. Once the regex is matched, I wish to replace it with {string}[i].
For now the python script works as expected for the strings for which I explicitly mention the regex pattern. I want to have a more generic regex pattern that will match all the strings that have {string}_{i} instead of writing a regex for all the patterns (which is not scalable).
input test.csv
bom_a14 , COMP_NUM_0
bom_a17 , COMP_NUM_2
bom_a27 , COMP_NUM_11
bom_a35 , FUNC_1V8_OLED_OUT_7
bom_a38 , FUNC_1V8_OLED_OUT_9
bom_a39 , FUNC_1V8_OLED_OUT_10
bom_a46 , CAP_4
bom_a47 , CAP_3
bom_a48 , CAP_6

test.py
import csv
import re

# Match the values in the first column of the second file with the first file's data
with open('test.csv', 'r') as file2:
    reader = csv.reader(file2)
    for row in reader:
        row_1=row[1]
        # for matching COMP_NUM_{X}
        match_data = re.match(r'([A-Z]+)_([A-Z]+)_(\d+)',row_1.strip())
        # for matching FUNC_1V8_OLED_OUT_{X}
        match_data2 = re.match(r'([A-Z]+)_([A-Z0-9]+)_([A-Z]+)_([A-Z]+)_(\d+)',row_1.strip())
        # if match found, reformat the data
        if match_data:
            new_row_1 = match_data.group(1) +'_'+ match_data.group(2)+ '[' + match_data.group(3) + ']'
        elif match_data2:
            new_row_1 = match_data2.group(1) +'_'+ match_data2.group(2)+ '_'+ match_data2.group(3)+'_'+ match_data2.group(4)+'[' + match_data2.group(5) + ']'
        else:
            new_row_1 = row_1
        print new_row_1


output
COMP_NUM[0]
COMP_NUM[2]
COMP_NUM[11]
FUNC_1V8_OLED_OUT[7]
FUNC_1V8_OLED_OUT[9]
FUNC_1V8_OLED_OUT[10]
 CAP_4
 CAP_3
 CAP_6

expected output
COMP_NUM[0]
COMP_NUM[2]
COMP_NUM[11]
FUNC_1V8_OLED_OUT[7]
FUNC_1V8_OLED_OUT[9]
FUNC_1V8_OLED_OUT[10]
CAP[4]
CAP[3]
CAP[6]



Answer:
I would use sub with a single generic pattern :
with open("test.csv", "r") as file2:
    for row in csv.reader(file2):

        s = re.sub(r"(.+)_(\d+)$", r"\1[\2]", row[-1].strip())

        print(s)


Regex : [demo]

Output :
COMP_NUM[0]
COMP_NUM[2]
COMP_NUM[11]
FUNC_1V8_OLED_OUT[7]
FUNC_1V8_OLED_OUT[9]
FUNC_1V8_OLED_OUT[10]
CAP[4]
CAP[3]
CAP[6]
Question: How to use array variable in my sql statement in plpgsql

total nube in PL/PGSQL. I would like to define an array of strings and then used that in my SELECT... WHERE In statement but can't seem to get it to work, help appreciated.
DO $$
DECLARE 
    testArray varchar[] := array['john','lisa'];    
    ids integer[];
BEGIN
    ids = array(select id from tableA where name in (testArray));
    -- this works
    ids = array(select id from tableA where name in ('john','lisa'));
END $$;



Answer:
You can use any near of testarray.
DO $$
DECLARE 
    testArray varchar[] := array['john','lisa'];    
    ids integer[];
BEGIN
    ids = array(select id from tableA where name = any(testArray));
    -- this works
    ids = array(select id from tableA where name in ('john','lisa'));
END $$;
Question: Adding 3D masked arrays results in TypeError: 'numpy.bool_' object is not iterable

I have two 3D masked arrays (netCDF4 files output from climate model) that I want to add together. I followed this thread and got the following (simplified) code out of it:
import numpy as np
from netCDF4 import Dataset
from operator import and_
from numpy.ma.core import MaskedArray

with Dataset(dir + 'V10.nc') as file_V10:
    with Dataset(dir + 'U10.nc') as file_U10:
        raw_V10 = file_V10.variables['V10'][744 : 9503, :, :] ** 2
        raw_U10 = file_U10.variables['U10'][744 : 9503, :, :] ** 2                                                                                                                                                                                                               
        10m_raw_squared = MaskedArray(raw_V10[:].data + raw_U10[:].data, mask=list(map(and_,raw_V10.mask, raw_U10.mask)))

However, I get the error message:
Traceback (most recent call last):
  File "code.py", line 92, in <module>
    10m_raw_squared = MaskedArray(raw_V10[:].data + raw_U10[:].data, mask=list(map(and_,raw_V10.mask, raw_U10.mask)))    
TypeError: 'numpy.bool_' object is not iterable

If I try changing the mask from boolean to string (in order to make it iterable) by adding mask.astype('str'), I get this error message:
Traceback (most recent call last):
  File "code.py", line 92, in <module>
    10m_raw_squared = MaskedArray(raw_V10[:].data + raw_U10[:].data, mask=list(map(and_,raw_V10.mask.astype('str'),raw_U10.mask.astype('str'))))    
TypeError: unsupported operand type(s) for &: 'str' and 'str'

I have also tried to add the arrays together using a for-loop, but somehow couldn't get that to work without losing a dimension and the majority of the array elements of the data.
How can I add my two datasets together?
Edit: I called for the class of the dataset and got the following output:
<class 'numpy.ma.core.MaskedArray'>



Answer:
You can use np.logical_and to create the mask.
with Dataset(dir + 'V10.nc') as file_V10:
    with Dataset(dir + 'U10.nc') as file_U10:
        raw_V10 = file_V10.variables['V10'][744 : 9503, :, :] ** 2
        raw_U10 = file_U10.variables['U10'][744 : 9503, :, :] ** 2
        mask = np.logical_and(raw_V10.mask, raw_U10.mask)
        10m_raw_squared = MaskedArray(raw_V10[:].data + raw_U10[:].data, mask=mask)
Question: Select the data and generate the SELECT for each day from date to date?

I have this SELECT statement in SQL Server:
select testname as 'Test', 
       tests_morning , 
       tests_evening ,
       Date
from  labtests_hajj 
left join departments_statistics on departments_statistics.test_id = labtests_hajj.testid 
inner join departments on labtests_hajj.dept_id = departments.dept_id

The output now like this:
Test      tests_morning         tests_evening Date 
CBC       null                  null          null 
CALCIUM   null                  null          null   
SODIUMN   null                  null          null   

How can I get the output as script and add the date also depends on date range for example I need the output for 3 days to be like this :
Test      tests_morning      tests_evening   Date 
CBC       null                  null     01/06/2023 
CALCIUM   null                  null     01/06/2023  
SODIUMN   null                  null     01/06/2023

CBC       null                  null     02/06/2023 
CALCIUM   null                  null     02/06/2023  
SODIUMN   null                  null     02/06/2023     
                                           
CBC       null                  null     03/06/2023 
CALCIUM   null                  null     03/06/2023  
SODIUMN   null                  null     03/06/2023  

How can I do the select and put 2 dates and show the output like thie?


Answer:
There are have two ways

use DimDate or calendar table
you create a date table yourself with CTE

With DimDate
declare @StartData date='2023-02-01'
declare @EndData date='2023-02-06'
;with _t as 
(
    select testname as 'Test', 
           tests_morning , 
           tests_evening ,
           Date
    from  labtests_hajj 
    left join departments_statistics 
                     on departments_statistics.test_id = labtests_hajj.testid 
    inner join departments on labtests_hajj.dept_id = departments.dept_id

)
select 
             a.Test
            ,a.tests_morning
            ,a.tests_evening
            ,s.Date_ as  Date
from _t 
cross apply (select * from DimDate where date_ between  @StartData and @EndData )s

If your date range is consecutive, you can create the desired data by setting the start and end dates with CTE.
declare @StartData date='2023-02-01'
declare @EndData date='2023-02-06'
;WITH List
as
(
    SELECT @StartData as Date_
    UNION ALL
    SELECT DATEADD(day, 1, Date_) as 'MonthStart'
    FROM List
    where Date_<= @EndData
),
_t as 
(
    select testname as 'Test', 
           tests_morning , 
           tests_evening ,
           Date
    from  labtests_hajj 
    left join departments_statistics on departments_statistics.test_id = labtests_hajj.testid 
    inner join departments on labtests_hajj.dept_id = departments.dept_id
)
select 
              a.Test
            ,a.tests_morning
            ,a.tests_evening
            ,s.Date_ as  Date
from _t  a
cross apply (select * from List)s