skysight-inc/HackerNews huggingface dataset:
My current project is a Huffman coder built in Zig, it's been a lot of fun to build. The Huffman tree generation is simple, but there's a lot of different nuances and variations to account for.

For example, a "canonical Huffman code" can save you space encoding the table through some clever trickery. In short, you can encode the table by counting the number of bytes that use each bit-count, and the bytes used in the file. You don't need to store the patterns at all, since in the canonical algorithm the patterns are regenerated from the bit-lengths. [1]

Right now I'm trying to implement the package-merge algorithm,[2] which will allow me to create the table without building an inefficient fully-allocated tree, and more importantly will allow me to limit the lengths of my codes (the maximum code length is n-1, where `n` is the length of your alphabet. Working with bytes and using 255 bit codes is obnoxious). Unfortunately all explanations of the algorithm I've found are very academic and mathematical, so I'm having trouble working it out.

Some of you might be interested in the short video Tom Scott made about Huffman coding.[3]

1. https://en.wikipedia.org/wiki/Canonical_Huffman_code#Encodin...

2. https://en.wikipedia.org/wiki/Package-merge_algorithm

3. https://www.youtube.com/watch?v=JsTptu56GM8
What did you like about the movie? The book made the whole “surviving a zombie apocalypse” trope...I hesitate to say realistic, but at least more realistic than other treatments at the time. What I remember of the movie, a standArd blockbuster. Nothing wrong with that I just felt that it was much weaker than the book.
> I think addressing one issue does not 'trivialise' everything else.

Yes, it does. On my Twitter I have "racism" and "race" including many other trendy SJW keywords blocked. The reason is I simply don't have the time and energy to tend to every single accusation of racism or whatever-isms is out there. And most these accusations have nothing to do with racism. It's just some bully has found a new target to harass.
You're committing money in order to gain a financial return. Investments don't mean good decisions inherently. I think speculation and gambling are indeed more accurate descriptors, due to crypto volatility. But it means it's a more volatile investment, not that it's not an investment.
Postgres regex search over 10k GitHub repositories (using only a MacBook)
Good question... You wouldn't land safely. That's part of the reason the design of the engine is such that if there's a failure, the engine cowling is supposed contain all the debris so that there is no damage to the wing, airframe or associated systems.
Honest question, what do you do with it multiple times a day? You use it as your regular bank and a debit card or something?
What's the relation between COVID and this genocide?
The problem is UX: if data privacy laws forced the issue, I think we could come up with a nice way to specify the pod that is accessible to non-techies.
> ...that is because ML tools are able to cheaply do a really hard job of moderating

I mean , YouTube is huge, but we had moderation-at-scale before ML was a popular phrase to bandy around.

'ML' and 'Algorithms' are just convenient non-human bogeyman that any corporation can pin all their woes on without batting an eye right now -- a convenient explanation tool for when things go wrong.

I am going to take a bold stance with this next statement:

An algorithm cannot be at fault -- it is always the fault of those that implement such algorithms.

YouTube and their community is huge -- YouTube has revenue fifteen times greater than Yahoo. 'Too big to X' should not be a tolerable excuse from any industry -- especially so since YouTube/Google has been pushing the idea for years that being a YTer is an actual profession.

Imagine for a moment coming into work and being told that you're fired because an internal corporate policy checker decided that something you did in your past work history was worth of termination; and then 24 hours later being told to continue coming into work because a mistake had been made, but with no further explanation.

This kind of 'workplace' would be intolerable in any real 'work environment'. Just more evidence that YTers are not really treated like any other partner or employee anywhere else in the world -- and that they should probably curtail that language if they want expectations to match the reality that their 'employees' experience.

>People forget that they get these free services for free,

YouTube is not a free service. This fallacy needs to go.
I don't want to go on too much, I know this topic can be rather divisive and people get sort of heated about it but I really think in many cases things like the black/white list changes are counterproductive.

There are actual race issues genuinely affecting people every day. Making a big deal out of what are essentially non-issues i'd be willing to bet 99% of people never think about or genuinely care about is just going to make the average person roll their eyes and not bother caring.

Which again, is counterproductive. If you want the average person to care about the actual systemic issues, violence and actual racial oppression, forcing them to adjust their language and take time to address issues like the gp's comment is not the way to do it.

Policing every word people say and making them feel guilty for using the wrong ones will not make them care, if anything it's going to make people who are otherwise neutral, who would maybe care, turn hostile.

It pushes people to extremes and leads to places like 8chan or Parler, where people take their language to utter extremes and the opposite extreme where people are finding offense in nearly every little thing.

Both are unreasonable.
> Why do we need laws around this? We need to hold the people accountable [...] [T]he people that okay'd its use need to take responsibility.

Genuinely asking: what do you think profitable companies would do to their ML engineers over a product issue like this if there was no law under which they would be sued?

I think accountability without an enforcement mechanism is just a wishful suggestion.
Is Eagle a viable alternative to KiCad these days? I stopped using it around KiCad 4. (2015?) At that time, they seemed to be feature-equivalent, but KiCad has been improving fast since then.
You misspelled misspell. (Sorry.)
Sort of like Soviet dissident authors developed a roundabout way of writing to bypass the “algorithm” of the state’s censorship bureaucracy.
As an ignorant American, I feel like I pay enormous taxes to defend international trade that mostly seems to benefit non-Americans. For 30 years, I was told “trade causes democracy” and “democracies don’t fight each other”. But, that doesn’t seem to be working very well. Why should the US continue to spend on defense so heavily? Why should we care?
Despite new language fatigue, this does look interesting.

A few things that stand out:

1. use of Exceptions. These get so much hate these days, but the articulation here of the reasons for allowing them is better than most of the Hacker News comment threads about them. I might just link to this section of Passerine's docs the next time somebody tries to tell me how Result<> is the only thing any programming language needs.

2. The use of fibers/coroutines as built-in error boundaries. This is such a pleasant natural extension to the Erlang proposal I find myself wondering why so few of the languages I'm familiar with have taken the same approach. Presumably I need to get out more.

3. Not sure what to think about the left-associativity of function calls, nor the lack of explicit call syntax. One of the things that bothers me about Ruby is the extra work you have to go to to pass functions as arguments to other functions. Naively, lacking explicit call syntax seems to me like a premature optimization. I'd appreciate someone's perspective on these two (separate?) points.
How many people have died in this so called genocide?
"Master", as in "Grandmaster" is obviously problematic, with its obvious reference to north-American slavery.

They should change to Grand Wizard or something like that.
You may feel it doesn't matter now if you're already familiar with the culture but what happens in the future if they come after the word "git" which is a slur for low intelligence people or people with what is probably a mental illness? Mental-health-ism and the intelligence-ism may well appear since they're logically consistent with racism and sexism. Yes, you'll be able to just change that too but it'll be more total manhours for everyone to do that just as blacklist is probably more work than nigger.
Colonial Diplomacy plays a lot faster than regular Diplomacy but the end game is a bit more brutal.
> What's hilarious to me is that I think the people who find there to be a problem with these words are the racist ones themselves.

That is what I believe as well. I am not that preoccupied with races to make such associations. In all fairness, I still have no clue what "blacklist" has to do with races or slavery, even after having been "shown" this association, because it makes no sense given the context.

There might be people out there who genuinely do not know what the term means in the context of CS/IT, but then instead of assuming that everyone using the term is racist, they should try to figure out the actual meaning.

Killing a child or a parent in the context of computer science is something completely different than... actually killing a child or a parent. :) I can imagine "just kill the child" being misinterpreted, but that can only be the case if someone takes it out of context. Once the context is known, there should be no confusion or mystery.
So, "... doesn't mean they're not allowed to have an opinion"?
That's pretty much all of them except LZW (which is an LZ78-alike, rather than LZ77). gzip has a number of Huffman tables for literals, distance, and for building Huffman tables at runtime... Yes, the instructions for building the Huffman tables used for decompression are themselves compressed using... more Huffman tables (HCLEN)!

Anything to save a few bits...
> What about child pornography?

It's amusing how frequently this comes up. If someone distributes child pornography, arrest them using the existing laws of the land.

"But what if it spreads from there?" I hear someone saying. Well, they're still posting illegal content, so arrest them too.

It's silly to assume that the freedom of speech implies some kind of inherent free pass from consequences of your speech.
If you’re referring to comments like mine challenging the idea that private websites banning anyone for any reason has anything to do with free speech, you’re wildly mischaracterizing your US audience.
Jump on the #debian-mentors IRC channel (OFTC) (also bridged to Matrix) and folks there can walk you through it.

PS: lots of different ways to help Debian:

https://www.debian.org/intro/help
Picture frames are a solved problem. Metal, wood, whatever. Build your item to known demension and let people order exactly what they want
A Python package to sizably boost your ML performance
>The practical answer is that during extreme weather, there are never enough people willing to voluntarily freeze. The demand for heat when it's below freezing in a state where homes aren't insulated and people don't own winter clothing is relatively inelastic.

Not exactly. While it's true that people are willing to pay infinite dollars to not freeze (aka stay alive), there are a variety of ways to do it that don't involve using electricity. eg. starting a fire (risky I know), or cohabiting with relatives.
> the state government is taking steps to reduce risks of outbreak from hotel quarantine.

Stopping the system entirely is certainly a way to reduce the risk ;)

> I also disagree with your statement about mental health not being taken seriously. I think it’s difficult to talk about, but I believe mental health resources have been made broadly available. I’m not really sure how much more seriously it can be taken without making mental health checks mandatory.

I was mostly thinking of the way the pandemic has been handled in general by the Victorian government. Locking people inside for 22-23 hours a day, curfew, etc. There is very little if no scientific evidence that these drastic measures, which severely impact mental health, have a measurable effect on getting numbers down. Not all lockdowns are the same, NSW's North Beaches lockdown allowed people to picnic outside for example (and made gathering exemptions on Christmas day). This again points to the benefit-risk evaluation mentioned in the article.

Similarly, banning Victorians out in regional NSW from returning home: forcing them to make a dangerous dash across half of the country to get back within the day. Again this could have been handled with a much stronger emphasis on benefit-risk but instead the wellbeing of Victorians was disregarded.
I thought that sounded like a fun project to learn web extensions and decided to make a Chrome extension for that today: https://github.com/bdibs/DeKarmaHN. Hope it works for you.
A few years ago, I was on a transatlantic flight about to depart from the east coast (accelerating down the runway) when there was a loud bang and the plane skidded to a stop.

One of the engines somehow managed to shred itself to pieces and fall off the wing onto the tarmac. The bang was apparently shrapnel and scrap metal hitting the fuselage rather than an actual explosion.

As we hadn't actually taken off yet there wasn't all that much drama, but boy it was a long wait until we finally got off the plane and could use the overnight lodging vouchers the airline provided.
I a good way to find things you use that need help is the how-can-i-help tool:

https://wiki.debian.org/how-can-i-help
Have you tried journal_mode=wal?

https://sqlite.org/pragma.html#pragma_journal_mode

This lets sqlite read w/o a write lock (among other things)

Since it stores the log to an adjacent file you have to make sure the process can write to the whole directory containing the db.
The current state of the art in ad fraud detection has basically become "here is a bunch of weird random stuff, lets see if you get the right answer." That stuff is delivered by dynamic JavaScript.

What you are proposing is AMP Ads and is universally hated by advertisers and publishers.
What happens in China does not stay in China? Public perception in other words
If you omit “so called” this might be a good question.
> But does it not strike to you at all problematic, at the moment, that 'black' is a synonym of 'bad' and 'white' of 'good'?

The inverse solution to this stop calling people with darker skin "Black"?

I kind of agree with what you're saying, but also it isn't true:

- Blacktie

- Black friday

- Blacktop

- Back in black

- Orange is the new black

- Men in black

All of those use black in a positive connotation. In my head black/white list has always been a metaphor for light/no light. As in black list can't see, white list can.

In asian Culture white is often associated with the colour of ghosts / the dead... So it'd get awkward if we keep modifying language to suit cultural references especially as English is a global language and while slavery was(is) a global problem it persists only really in the US as a current popular cultural issue.

In replying to you I've convinced myself at least the solution here can't possibly be to change all of the possible references of black to something else... it verges on new speak.
Give it five years and you can change the title to When Engineers were Nihilists. "The ironic fact here is that humanism, which began with Man’s being central, eventually had no real meaning for people."
You're argument is flawed because someone is going to come along and say "$200 isn't enough to really test most advertising methods...I'd throaw at least $20,000-$100,000 on testing something." Then along comes Procter and Gamble and they say $10m-40m. It's all about scope and market size.
Exclipse got removed from Debian because it became too hard to package.
by wear out means losing power/heat efficiency

you absolutely can wear out a GPU with excessive mining.
In either of those cases, not being calm is of no benefit either.
Is it more complex than a text classifier though? If it truly took into account who is commenting, and context, etc., it would have noticed very few chess videos were being flagged for what seemed like racist language.

It seems more than likely somebody put in a very crude black box to process text without context. (Or should I say, opaque box, so the horde doesn't flag me?)
I've found as long as everyone is equal opportunity backstabbers then it's fine. It is when you get people who won't backstab Fred but will backstab Barney every chance they get that you have problems. Those people would have problems playing Mario Party too.
I'm glad the resources on the basic trees exist, since that's what got me interested. But I would love to see more resources on canonical Huffman and especially the package-merge algorithm!

The former was confusing at first but mind-blowing when it clicked for me, and the latter looks awesome but the implementation is incomprehensible to me —I guess I'm stuck allocating trees until I can figure it out!
That fire through the thrust reversers burned all the way to the ground. It was probably lubrication oil, hydraulic oil, or CFRP burning.
> more of a standing

So you think ignoring facts and making nonsense gives you credibility so long as you hate the right people?

Maybe in your own head.
It's actually pretty good. Kind of chill beats for background music.
If no platforms exist there is no means of distribution and there is less motivation to create. You can talk about feudal lords all you want, but the simple fact is without a marketplace no one is exchanging anything except in very ad-hoc manner. Here is a clip of what it might have been like in pre-agricultural revolution era [0]

[0] https://www.youtube.com/watch?v=YHjYt6Jm5j8
> private websites banning anyone for any reason has anything to do with free speech, you’re wildly mischaracterizing your US audience.

If US citizens already have a hard time discerning this imagine someone from China.

It’s amateur capital driven censorship. It’s inefficient, confusing, and divides a nation. Yes it would be viewed as suboptimal from someone familiar with nationalism and censorship in China.
It would be interesting to know what the "views" of the people doing the study are.
Nearly 75% of Chinese Women Want Their Mate to Earn Double Their Salary (2015)
Invest

1. put (money) into financial schemes, shares, property, or a commercial venture with the expectation of achieving a profit.

I think that's accurate. You're right to warn them about the risks, I always recommend to put in only what can be fully lost.
Oh yeah, might have been remembering gzip. All I remember is that it was one of the old/common ones. Don't know why my mind jumped to lzw instead of the obvious gzip haha.
> But does it not strike to you at all problematic, at the moment, that 'black' is a synonym of 'bad' and 'white' of 'good'?

Black is associated with the night and darkness and death in many cultures. Fear of the darkness is common to all humans.
> 500 * 60 * 24 * 365 = 262800000 hours per year / 1.3x playback speed / 2,000 hours of work per person per year = ~101,077 people to watch every uploaded video.

When you have people watch the video at that fast speed and instantly make a decision, you might as well keep the algorithm - the failure rate won't be too different.

Additionally, these people need holidays, managers, infrastructure and an actual flag will probably not instantly be clear - you'll need to point out the specific offence and make a case. You might even need to look things up. Then you'll have to handle appeals and discussions, because changing the AI blackbox to an appeal-less human black box wouldn't improve the situation much[0]. Next, you need people who can understand the specific language of that video - most will be english, but what happens when a Nigerian video is flagged? So you need people from that country or at least familiar with it. Plus infrastructure.

Overall, you're probably looking at something approaching 400 or 500 people, with this very low flag rate. Assuming conservative 30k$/year [1], this is 15 million USD alone. Doable, yes, but its not the no-brainer you make it out to be.

[0] See this case with FB, which was a human error: https://www.nbcnews.com/tech/social-media/facebook-under-fir...
[1] Which is probably optimistic for reviewers, but we'll have managerial infrastructure to pull that average up.
Are you sure a human, never mind one of below average intelligence, gets this? We have people right now in Silicon Valley calling for renaming CS words because they sound racist to them.
Because you need to defend the exorbitant privilege of having the worlds reserve currency. This means Americans can generally live easier lives by borrowing from the rest of the world. It’s also why wages in the US are so high.
America can’t really be an export manufacturer as long as it’s currency is artificially inflated by massive inflows from global trading imbalances. It could after WW2 for a short while. That was because all others were bombed out and China etc weren’t developed.
Great article!

I have been thinking a lot about the feeling I have that “US manufacturing jobs” has become a political myth, and that real decay is setting in.

I have been ordering CNC machined parts from a machine shop in China. The only US organization I priced from is Plethora which probably isn’t the cheapest but their prices were much, much higher than China.

Of course Chinas prices have been lower for a long time, but in the past my simplified understanding is that there was a large volume of cheap labor that suffered a poor standard of living. But China has grown leaps and bounds in the past three decades. I suspect that now for many machine shops in China the standard of living for their workers is probably on par with that of workers in the US. It could even be better, I wouldn’t be surprised.

So now it’s cheaper in China just because they took manufacturing seriously. In the US we let MBA’s sell off all our manufacturing capability to foreign nations. Which seemed to work great for increasing profits. But the lack of higher level planning has left us with a vulnerability. Selling your business to China works great in peace time. But China has power over us now.

I am hopeful that the article is correct, and our horrendous military spending is coming to a close. But I worry about what will happen in the US to make us finally understand the need to change our ways.

I think the best outcome would be for the US to end its posturing against China and learn to benefit from collaboration.
Also the night is black and dangerous. The daytime is bright and safe. These are ancient concepts and postmodernism isn’t some absolute truth
I know, everything was fine when the YouTube recommendation engine accidentally radicalized half English-speaking world and created a profitable fascist political movement.
They just want more commits under their belt haha
Show HN: DeKarmaHN, a Chrome extension to hide karma and more
The River of Doubt. It is not a biography but about his journey into the Amazon. Roosevelt did so much that you could pick any 3 year period in his life and write an interesting book about it.
I think we are all experiencing the same issue :/ . Let's give it some time I think.

Off topic I am watching your code generation video on nodewood https://nodewood.com/features/code-generation/ . It's seems clever, but I have some difficulty evaluating how much trouble it would save me for example. How do pitch that feature ?
Shameless plug: A couple of years ago I wrote an implementation of the huffman coding algorithm as well while studying it, along with unit tests. I was also interested in practicing OOP. You can find the result in the link below.

- https://github.com/TCGV/HuffmanCoding