skysight-inc/HackerNews huggingface dataset:
Hey @lewisjoe! YouTube doesn't provide a way to search across all of YouTube based on transcriptions/spoken words. My app performs on-the-fly searches, as @alex_smart noticed in another comment.

My app does as much of the fetching and match finding work that it can client-side. For the few things that can't be done client-side, my app calls serverless function API endpoints to fetch the YouTube channel and video data it needs. Here are the tricks I'm using to make it fast:

- As soon as the "Channel URL" field loses focus, I start fetching the most recent 30 videos on that channel in the background. This way, by the time the user enters the keyword and date range, I've already fetched some (maybe even all!) of the data ahead of time, which means less wait time for them.

- Once a specific video's data (title, description, transcript, etc.) has been fetched once, it is saved in memory. All other searches the user performs from that point on will pull the video data from the in-memory cache, if it's there. Otherwise, it will fall back to fetching the video data over the network. This in-memory caching makes subsequent searches within the same date range (or a shorter date range) take <1 second.

- Network requests to fetch video data are processed concurrently rather than one at a time. So the browser fires off as many as it can in parallel to get them all resolved as quickly as possible.

- As soon as any matches are found, the UI updates to show the user. This way, the user can start scrolling through the matches and reviewing them while the search is still in progress– they don't have to wait until it finishes to start interacting with the matches.

Thanks for checking out my project, and for the kind words! I appreciate it.
Has your team run into any issues with WP checkout?
People are always saying things like "the second amendment protects my right to bear arms".

So can we have another "amendment" that removes this right?
I work in Hong Kong with both Chinese and Indians, and the way it works there is that it's like everywhere, not everyone wants to be a developer and those who want don't always know what's happening around them. The difference is they're the first generation to have some sort of access to wealth, and would rather die than be honest and find a little job in a coffee shop since there are no jobs in coffee shop that would pay near the same.

So you end up with tons of guys who cannot work in software, who cost nothing like us in salary term but produce near nothing. The ones who survive are either unashamed to ask millions of questions (fair enough after all), who have the political acumen to always walk between the raindrops and manage to fail projects without looking like the culprits or are genuinely good and move asap... to Hong Kong or Singapore, sometimes Tokyo if invaluable. These would never stay underpaid for doing the same job as me, they're not stupid, them.
Geographically, but anthropologically aren’t they actually caucasian?
I'm casually browsing but closed it when I saw latinx (not a real word).
There's a section of the internet where you can easily find billions of images or can be generated from moving pictures. Even upscaled. I really didn't expect to have to spell it out.

hint: they are all about one thing and there are a lot of eager volunteers to help on those websites. It would be easy to "normalize/clean/classify" because the pictures would have a consistent theme, thus reducing the amount of parameters.

We are not trying to generate elephants getting railed on a SpaceX rocket flying in oil painting style (although I'm sure theres people into that and its not my place to judge), we are just trying to remove the human cost out of this necessary evil.

I can't believe nobody is investing in "DALL-E-2-4-PORN". This sounds like an X amount of money thrown at a hugely sticky product that can be iterated (with the current trend in hardware) to the point where it literally generates billions of dollars in revenues for ages to come (no pun intended).
I always found corporate credit cards to be similarly dodgy. Firms I worked at offered a "corporate" amex. The employee as card holder was personally liable for the debt, the employer had no liability. At the time these cards did not accrue any points either. And the corporate policy was typically "no personal expenses".

1) you must use corporate card for company expenses
2) you must not use corporate card for personal expenses
3) you cannot accrue points for use of corporate card
4) you are personally liable for corporate expenses on this card

I'm pretty sure amex was giving big perks to CFOs.

If you assume a limit of $10k per card and company (like the one I was at) had over 10k employees (though not all had cards) the amount is pretty astonishing and zero liability.

I once had an issue with Amex because I'd been travelling a lot and the boss was away and then slow to approve.
I hope so. It's a bit cruel to show off and then lock it away.
Round and round we go...

As a foreigner, it's appalling to see every excuse imaginable trotted out every time this happens.

Don't think you're fooling anyone. You're just making things up on the spot for the sake of pretending to have a rational argument, like an intelligent adult would in your place.

But I will eat my shorts if you personally have spent even an hour doing the research before making statements about "what this would do", or "that wouldn't work", or whatever. I don't mean clicking "like" on a Facebook post by a fellow NRA member. Mean literally reading through scientific publications, or looking at statistics published by third parties that are unaffiliated with any United States organisation.

Or just Googling to see what the other 96% of the human race does with guns and laws.

""No Way to Prevent This", Says Only Nation Where This Regularly Happens": https://en.wikipedia.org/wiki/%22No_Way_to_Prevent_This%22,_...

Jim Jefferies (an Australian comedian living in the States) on gun control:

Part #1: https://www.youtube.com/watch?v=0rR9IaXH1M0

Part #2: https://www.youtube.com/watch?v=a9UFyNy-rw4

Can you counter every single one of his points? Really? With facts? That you can link to in a scientific paper that wasn't paid for by a gun lobby in the states?

19 children just died. Please take this a tiny bit seriously and don't just click the reply link to blurt out something in a public forum that can be trivially shown to be false...
Viture One XR Glasses Kickstarter Raises over $3M
You'd have to train it first.
Hey @alex_smart- Yep! You're exactly right. My app performs as much of the fetching and searching work as possible on the client, then calls serverless function API endpoints for the few things that can't be done in-browser.

I replied to @lewisjoe's comment above this one with some more details about tricks I'm employing to make searches fast.

Thanks for checking out my project!
Tested: Where Does the Tone Come from in an Electric Guitar? [video]
I really enjoyed this explanation.
If the functionality doesnt exist, the code doesnt exist, so you have to write it yourself.
Not for causing violence but for addiction. That’s a pretty easy argument and easy to prove causality (more game time = worse school/social life).

Why limit this logic to just social media sites and not other things that kids can be addicted to?
I remember coming back from Iraq wishing the American public should have to see what we were actually doing. America and it's allies export death and destruction, but we dont show or talk about a president bombing a wedding, or sending arms to the Saudis they use to blow up a school bus full of Yemeni kids. Keep quiet about the ~14,000 dead Russian speakers in the Donbass conflict since a CIA backed coup in 2014. Say nothing about all the various North African countries we've couped, or the South American Ones, or the SE Asia ones. Don't mention programs like Fast and Furious sending mexican cartels arms, or sending other cartels guns for drugs that get pumped into the inner cities and make our 3 letters have oodles of off-books black bag money.

I see pundits on Bloomberg cackle about Lockheed and Raytheon outperforming the market on national television and radio. News hosts talk about missles sent to Syria against our own Constitution as beautiful. Warmongers given platforms on supposedly left shows. Unnacountable bankers financing it all.

SV is so full of myopic hypocrites. Show the gore, sure, but be sure to show it all.
What is the reason Google published their research details about Imagen?

Why don't they just keep their findings to themselfes and build products on top of them?

Public companies can't do stuff just for the fun of it, right? So there must be some commercial reasoning behind it?
This is just an example of what a person has accomplished in a limited amount of time by himself. Noting stops you from pushing your hamster wheel at a FAANG or thinking about FIRE. You can even do this and a small app at the same time because it doesn't need a huge amount of time.

You don't even have to do a small app. You can do the next Uber or Airbnb. Or you can do something that yields the same monthly amount you get from your FAANG but have the advantages of having no boss and working for yourself. And if you decide you want to do something else, you can sell it for a nice amount, while quitting your job doesn't yield you any money.
Demonstrating significant energy savings using neuromorphic hardware
> Gun owners consistently misrepresent and come up with inconsistent interpretations of any gun control.

Maybe sometimes, but there's also gun laws like in California that affect large populations of gun owners. I don't think they're making their dissatisfaction and confusion with the gun laws up. Owning a gun in the US isn't very simple; different states have wildly different laws you have to observe, and those are in effect at airports as well.

I don't think it's wise to say, "they just don't understand" to the people who actually have to live with the laws.

As a gun owner, I do think there's reasonable forms of gun control. I don't mind having my background checked or paying a tax stamp. I don't mind psychological evaluations or closing gun show loopholes. Restricting guns, ammo, attachments, magazine sizes etc are not effective forms of gun control imo.
The problem with your solution is:

1. If the verified field is not set, your program blows up at runtime. I don't think it's acceptable for programs to blow up at runtime for errors that can be caught at compile time.

2. Your example still has the same problem when it comes to `MaxMessageBytes`, what if I really meant to set it to 0? The `WithDefaults` function would eat that. You could use a pointer, but then that just complicates the API (you need to do new(int) as a separate variable and assign it
Your welcome :-)

In the end my real point is: I have no special revolutionary solution, just suggesting to do their best to live the life since we work to live not the contrary and living is our real "target".

The tooling part is just a suggestion and a reminder about how much we need and how useful are classic desktop systems where the user is at the center, not the service like modern ones...
It's just, people like me simply don't want to enter their payment information to an additional thing.
It feels obvious now that kids should not be on tv at all, by the same logic that they can't consent to sex. They can't consent to the huge ramifications of public exposure.
Now think how many children suffered the same fate, but we’ll never hear about it because they did not become famous.

Reminds me of “dance moms” or “cheer moms”, “football dads”, etc.

If you’re a parent, remember not to live your unfulfilled dreams through your children.
You don't realize how hard it is to make an inclusive, cleaned up dataset. Take a look at this Notion from BigScience detailing their workgroups. Three of them are related to preparing the dataset.

https://bigscience.notion.site/10743770aae24ff3bdc1b938cf454...

And this is just for a text-only scrape.
nice pun haha
Stripe funded Fast
Its just mind of matter otherwise everyone would be using it for birth control!
There we go. This is the comment I was going to make, but you've done the idea justice.
> dishonestly claims that link was related to the claim about crime rates — when it clearly wasn’t.

I’m sorry for assuming that the final sentence of the comment and only citation was related to the rest of it
the article you quoted says:
“Homicide figures may include justifiable homicides along with criminal homicides, depending upon jurisdiction and reporting standards”

Why don’t we talk separately about “murders” vs “suicide” vs “justifiable homicide”? Each of those terms is fairly clearly defined in most people’s minds (maybe the last one a little less so), and then we could focus on the different mitigation that might be needed for each. I feel conflating them doesn’t actually help move the discussion forward given how different they are.
On a moral basis, I think the situation where a law enforcement officer killed the recent texas school shooter is very different from that of that shooter who murdered innocents. So when people conflate such very different things by using terms like “gun deaths”, it makes many gun owners like myself suspicious that people are not arguing in good faith. Or am I missing something? Is there some benefit to doing so that isn’t just political?
Agreed, that vector won't happen again. The #2 hopefully will never happen again (Tulsa race massacre). But the #3 (Oklahoma City bombing) could happen again with a sufficiently dedicated person.

But people are creative, and some other method could be used in the future. The Boston Marathon bombing was only 3 deaths, but 264 injured, and that was just a pressure cooker. The Trollhättan school attack was 3 killed with a sword.

Guns are a tool, a very deadly tool, but a tool none the less. Removing them from the US would be near impossible at this point (with the 2nd and 4th amendments getting in the way). It's also not all that popular in this country. Stricter gun laws don't seem to help (see the Buffalo shooting, in a state with red-flag laws).

One interesting point I heard raised is that the phrase "going postal" was born from an era where workplace shootings became common, but have since near disappeared (with background checks being a large part of the reason). Figuring out why our younger population is doing down the route they have the last 25'ish years is something that has yet to be corrected.
You seem to be deeply ignorant of basic finance. Have you ever actually read a bond rating from one of the big rating agencies? They explain the metrics they use.

If you think you have found an error in bond rating procedures then put your money where your mouth is. Go long Tesla bonds and short GM bonds, and make a fortune. Be sure to lever up to maximize your returns.
I think it would be easier to create better laws around it, than removing the amendment.

For example, removing the PLCAA, for starters, to create liability from manufacturers:
https://en.wikipedia.org/wiki/Protection_of_Lawful_Commerce_...
I really wouldn't be surprised if 30/40/50 years from now "child stars" will seem as absurd and barbaric as child labor in general does today. It just seems so terrible and exploitative in so many cases.
There are roughly a million violent crimes a year. It is not credible there are 3x as many defensive uses of firearms that are somehow being missed.

The page you reference indexes a few thousand over several years. Seems likely an undercount as the page claims but not by that much by probably a couple orders of magnitude.
I would say salaries are massively under-inflated
That really depends on your definition of mass shooting, and you seem to be quoting a statistic which uses a very broad definition.

The kind of thing that happened in Texas has not happened 44 times this month.
In my experience, scraping the data is the easy part. Once you've scraped it you've got to get rid of all the garbage, which is where the issues arise, especially if you're just blindly scraping everything you can find.

For example, in a generative model I'm working on, I have a dataset consisting of ~5M images just blindly scraped from a website. After filtering, this drops down to ~500k images, yet a model trained on that performs worse than one trained on a set of 30k curated images (picked based on a manually evaluated list of "known good" artists), where filtering brings it down to ~18k images. The larger dataset, while containing more information, also contains more errors, many of them pretty hard to filter out.
I feel like you’re intentionally avoiding the point by saying things like “I don’t see the word totalitarian” and then ignoring the obvious implication that the article describes camps which are forcibly quarantining people who never traveled, including taking measures like searching unrelated cars who happen to be nearby.

I think you’re responding in bad faith because it would challenge your view of yourself and your nation to admit that forcible imprisonment in camps for quarantine is what people mean when they say “totalitarian”.
>taycan

Porsche already had 9 recalls for things like suspension, brakes, lights, full power loss, then there is this:

"“Six out of ten Porsche Taycan” ever delivered have a problem with battery management that affects and damages battery cells, requires replacement of cells and batteries, and is causing vehicle fires, according to a source working at Porsche's headquarters in Zuffenhausen, Germany.Dec 1, 2021"
Euh,

Azure is much bigger than Google cloud.

And Azure functions + OpenFaas exists for serverless in .net.

For Microservices, spring ( Java) and Dapper seems to be the way to go.
I suspect you're probably right, but I'm not sure it's to the point that it discounts their point. I think the "institutional rot" is a smaller factor than other elements impacting mental well-being.

A hormonal, bullied, enraged teenager where I live is less likely to do monstrous amounts of damage to others. They can get easy access to a knife and might own a car. Out of hundreds of people here, I think I know one person who has a gun, and they live on a farm with a rifle that I'd imagine emerges once a year to shoot foxes or kangaroos.
Surely there are confounding variables? For instance, if one does/does not have a criminal record, has/does not have a propensity for violence, or maybe most importantly does/does not have curious children in the house.
That's interesting. I feel like the way to really get this business off the ground would be for someone like 1password or Chrome to replace the credit card number they store with one they own, and then when you use it to fill in a form with your credit card information, the merchant charges them and they charge you a 1% service fee or something. Overnight, you are taking 1% of all Internet shopping revenue.

The ethics are, of course, terrible. I made myself a little sad just writing this!
Taking the examples from https://www.youtube-nocookie.com/embed/hA1ZsxE8VJg I am sharing how I approach the simple problems in the video without using Python or having any knowledge of CSS selectors.

Retrieving the HTML

echo https://www.nytimes.com|yy025|nc -vv proxy 80 > 1.htm

yy025 is a flexible utility I wrote to generate custom HTTP from URLs. It is controlled through environmental variables. nc is a tcpclient, such as netcat. proxy is a HOSTS file entry for a localhost TLS proxy. The sequence "yy025|tcpclient" is normally contained in a shell script that adds a <base href> tag, something like

#! /bin/sh
yy025 5>.1 >.2
read x < .1;
echo "<base href=https://$x />";
nc -vv proxy 80 < .2|yy045;

yy045 is a utility that removes chunked transfer encoding.

The benefit of using separate, small programs that do one thing will be illustrated in the solution for Problem 3.
Already does. It works by magic, we put images on one side and text on the other side and then power it up. Let it simmer for a few months at megawatt power levels.
I've ordered things from not-Amazon and had them arrive weeks late, or they ship the wrong or damaged items, or returns are a hassle, etc. One-click checkout is not the solution to any of that.
It feels more gratifying to watch walkthroughs.
I'm not sure I want the UAE deciding what is serious crime across the planet.

People have been jailed for 3 years simply over criticising Islam.

https://en.wikipedia.org/wiki/Blasphemy_law_in_the_United_Ar...
Nice job on PhraseFinder, @SteveDR! I am indeed doing something similar for my app. The user enters the YouTube Channel URL, the keywords and the date range, then I perform an on-the-fly search to find the matches.

I replied to @lewisjoe's comment above this one with some more details about tricks I'm employing to make searches fast.

Thanks for checking out my project!
I bet that was pre-linked size. But that corroborates my expectation about Chrome code quality.
Game theory says that when you leave the table I can take all your chips. Flippant, but we're talking about humans and power, as another has noted here. This isn't a market we're talking about.
Mass shootings aren’t common.

Also, most “mass shootings” are by pistol by gangs — you’re citing a statistic on the largest ones, which are rare even among rare events.

Generally, we shouldn’t base policy around outliers of outliers.
Problem 1 - Extract the values of <h2> tags from NYT front page

NB. In 1.htm, NYT is using the <h3> tag for headlines, not <h2> as in the 2020 video.

Solution A - Use UNIX utilties

grep -o "<h3[^\>]*>[^\<]*" 1.htm |sed -n '/indicate-hover/s/.*\">//p'

The grep utility is ubiquitous, but the -o option is not.

https://web.archive.org/web/20201202103125/https://pubs.open...

For example, Plan9 grep does not have an -o option.

This solution is fast and flexible, but not portable.

There are myriad other portable solutions using POSIX UNIX utilities such as sh, tr and sed. For small tasks like those in "web scraping" tutorials these can still be faster than Python (due to Python start up time alone).

Solution B - Use flex to make small, fast, custom utilities

Create a file called 1.l that contains

int fileno(FILE *);
#define jmp (yy_start) = 1 + 2 *
#define echo do {if(fwrite(yytext,(size_t)yyleng,1,yyout)){}}while(0)

%s xa xb
%option noyywrap noinput nounput
%%
\<h3 jmp xa;
<xa>\> jmp xb;
<xb>\< jmp 0;
<xb>[^<]* echo;putchar(10);
.|\n
%%
int main(){ yylex();exit(0);}

Then compile with something like

flex -8iCrf 1.l
cc -std=c89 -Wall -pedantic -I$HOME -pipe lex.yy.c -static -o yy1

And finally,

yy1 < 1.htm

This is faster than Python.

Solution C - Extract values from JSON instead of HTML

The file 1.htm contains a large proportion of what appears to be JSON.

I wrote a quick and dirty WIP JSON reformatter that takes web pages as input called yy059. https://news.ycombinator.com/item?id=31174088

yy059 < 1.htm|sed -n '/promotionalHeadline\":\"[^\"]/p'|cut -d\" -f4

Sure enough, the JSON contains the headlines. One could rewrite Solution B to extract from the JSON instead of the HTML.
Related to applying a non-linear function to randomly distributed points is the Unscented Transform[0]. It approximates how the function changes the distribution's mean and standard deviation by selecting several points, applying the function to these points and calculating the transformed mean and standard deviation from the transformed points.

[0]: https://en.wikipedia.org/wiki/Unscented_transform
Absolute power, corrupts absolutely. Perhaps it’s a game theoretic approach to leveling the playing field.