skysight-inc/HackerNews huggingface dataset:
Wonderful summary and very well explained! Those graphics could be in a CS textbook :) A bit more (or perhaps another article) on how the tree can be encoded would be useful, since the total size of your compressed example would need to include the size of the tree. Also: JPEGs also have a Huffman tables region in their binary; haven't dug too deeply, but seems like they also use Huffman coding!
It works the *exact* same way. You process the input one byte at a time, build a histogram, construct the Huffman Tree and encode the input.

Why should it work different? Text files are just regular files where the bytes only use a specific subset of the possible values they could have.

If you do have some knowledge about the data you are encoding, you can be a little bit smarter about it: e.g. for the text section of an executable, you might work on individual instructions instead of bytes, maybe use common, prepared Huffman Trees, so you don't have to encode the tree itself.

On a side note: IIRC the Intel Management Engine does that using a proprietary Huffman Tree, backed into the hardware itself, as an obfuscation technique[1].

To circle back to your question: PNG simply feeds the pixel data into the zlib deflate() function as-is.

[1] https://en.wikipedia.org/wiki/Intel_Management_Engine#Design
Even so that's only a month of lag time. Housing crisis was an entirely different scale and set of problems.
With our current knowledge and technology, you are right, but nobody knows what we might discover in the future.

Even if FTL were indeed impossible, I think pursuing it would bring interesting discoveries and technologies.
I’m a relatively new manager and this really stuck out for me

“Try to think through what skills someone needs to succeed without you. Teach those things incrementally.”

I quickly saw my team wasn’t performing at my level (duh otherwise they would be at my level), and have had a hard time trusting them. I have slowly gave them more responsibilities but I like how this makes it very explicit / planned out.
Curious what opportunities you're thinking of? China is hostile to foreigners in a lot of ways, and basically has no naturalization process, so you can't immigrate in the same way you'd immigrate to the US or Canada or Australia.

EDIT: How China may arrest foreigners with no cause or due process, or "exit ban" them to prevent them from leaving: https://harrisbricken.com/chinalawblog/how-to-avoid-china-pr...

China can also just nationalize foreign companies arbitrarily: https://www.foxbusiness.com/markets/china-coronavirus-nation...
Yes, somewhat similar. Thanks for the link. The major difference here is that Pepe runs on a mix network similar to I2P, this helps hide your identity whereas anyone can see what you are doing with plain Bitcoin.

To answer the question in your comment: yes, you can install and configure I2P, you can install iMule on top of it to exchange files and some distributed forum to post on. But the usability of all that is really low even for a pro-user. The performance is also not great because those systems were not designed to be used with each other from scratch.
My car is a 2007 and I've kept it around for similar reasons. The one thing that scares me the most about having a car that old is safety. Newer cars are a lot heavier and stiffer, so even cars that are considerably smaller could annihilate mine in an accident.

It's not quite to this extreme, but IIHS did a test (https://www.youtube.com/watch?v=xtxd27jlZ_g) a while back that showed the difference in safety standards between a 1959 Bel Air and a 2009 Malibu. The Bel Air is about 5% heavier.

Probably less of a worry in a 3/4-ton truck though.
I assume that was with just batteries for storage, not batteries and hydrogen. Adding hydrogen often greatly reduces the overcapacity needed in those model solutions.
You'd be incorrect. The capital of Nunavut in Canada is cleared to land up to an A380, and that's fairly close to the North Pole.

Often that's how OEM's will get their cold weather testing done, by just flying up in the winter.
If your computer is running in dark mode it’s now darkspace.
I recently reworked how zstd builds its Huffman decoding tables (not decoding itself) to avoid unpredictable branches
and speed the table building up by about ~2x [0]. This is insignificant for large decompressions, but if you're decompressing only a few KB, the table building time can dominate the actual decompression.

It sort of goes to show that while Huffman codes have been around for ages, implementations can still improve, especially as the hardware we use changes.

[0] https://github.com/facebook/zstd/pull/2271
If you are on the plane, you don’t care a bit about statistics. “Most” is not your concern. You want no further risk. And by definition, losing half of your engines presents a lot of risk.

There is an unfortunate famous incident in Amsterdam decades ago where a 747 cargo plane lost an engine, and the ejected material caused additional damage. Then the engine sheared off, striking another engine. Then it became a disaster scenario.

Myself an no other passengers on any plane are comfortable with losing an engine. Only people sitting at home, crunching numbers, are unconcerned.
Could you provide some context to what you just wrote? I know it seems like spitting in the wind commenting here, but it sounds pretty random unless you talk about details. By opportunity do you mean academic, business, or something else? Why was you PhD worthless? Is it applicable in China? How?
Coal power plants also carry a lot more radiation into their surrounding environment than nuclear power plants: https://www.scientificamerican.com/article/coal-ash-is-more-...
Yes. I don’t see why they’d have a public profile if they didn’t want people to see it.
In the older definition, having a following that puts some stock into your opinions or recommendations is what makes you an influencer. (i.e. you don't become an influencer by being paid, being an influencer makes you interesting for companies to pay you to advertise)
This is more or less exactly the scenario that played out in Texas this week, Feb 14-19 2021. All the players (gov, regulators, power generators) decided that it was too expensive to weatherize the network for a 0.1% chance event.

Yet... here we are. Entirely predictable.
In the UK, EU and Australia - probably a few other places too - fees to transfer fund between do not exist (unless you are a business).
> Nick Harkaway's "Gnomon" is ridiculously good

You are the first person I've ever seen to mention that book. It is astonishingly and unreasonably excellent, and I recommend it every chance I get. I'm just beginning my first re-read now.
Actually watching every YouTube video uploaded would take ~100,000 people which is possible, but quite expensive. Actually watching every video that’s flagged for banning might take ~100 people and seem like a reasonable effort.

As such arguments about using automated systems are meaningless, they can pre-filter without having final decision making authority.
The still unspeakable one is it may turn out that the public’s widespread belief that cloth or surgical masks are protective may, on net, have led many to use them in situations where they would have otherwise simply have stayed home. Given the increasing evidence that a single, loosely fitting, or non-medical grade mask provided less than perfect protection, it could turn out a message like “stay home unless you had a N95 or kn95” may have been much better. (Along with a federal manhattan project to make and distribute these to as many as possible.) Given the literal lies we were told early on that masks were not effective at all, due to concerns about PPE supply, I worry that those same concerns may have led to messaging on mask that downplayed the difference between good PPE and non n95 masks. The second order consequences of this seem possibly quite bad.
Success in business as a foreigner requires making friends in high places, plus some IP transfer.
>It’s cheaper to have less manufacturing SKUs and just disable features later.

I think you missed the point. If the chips are not stable or have a problem core, then fine, dial them down and sell them as capable of doing less.

But if a chip _can_ do x+y and but you lock y away because you don't want your customers doing y, then there is a problem we should address.
This is the problem. People will choose to ignore context and other grammatical history because of their own personal incorrect views.

It's not like the word "black" is just used with blacklist. We also use it in other places like black market, black hat, and black site. None of these are related to black people, but instead all relate to a shady dark/scary side of things like you mentioned.

When someone pictures a black market they are not thinking something that looks bright and colourful, they are thinking something that is dark and maybe grungy/sketchy looking.

The word policing is something I am really getting sick of.
Clubhouse Chats Are Breached, Raising Concerns over Security
oh no my queen!
I have nothing to add beyond the great recommendations you've already been given, but I'm throwing in an enthusiastic +1 for Diaspora and Gnomon. Quantum Thief, and Harry Potter And The Methods Of Rationality, are also great.
> Blacklist's etymology isn't racist, but if black folks prefer that we use something else, and something as similar as blocklist is there and easy to use, it seems like an easy fix.

Which black “folks” prefer the term “blocklist”? Black people are not a monolith. It seems like you’re just injecting an assumption of what certain people prefer.

Having been at now 3 different orgs implementing the [blacklist->blocklist, slave->replica, etc] routine, it’s always done for the same reason: the unquestionable brand of modern identity politics, diversity & inclusion, whatever you want to call it that is infecting pretty much every tech company (and indeed pretty much every company in general).

It’s an ideology that treats races as monolithic and only recognizes a certain set of opinions as valid. Racial blindness is implicit racism, we need to implement affirmative action policies over the dimensions of the 4-tuple (sexuality, cis/trans, rage, gender) - these are the types of beliefs they enforce.
An “influencer” is supposed to be someone who has the ability to influence people to do things. Whether they sell that ability or not has little to do with it.

Anyone with a big following of people who want trust them in some way is an influencer.
> It is obviously associating racism with something not intentionally racist. But does it not strike to you at all problematic, at the moment, that 'black' is a synonym of 'bad' and 'white' of 'good'?

No. That's how it's always been in European culture and it doesn't have much to do with race. It's probably true of many other cultures as well, although I can't say for sure.
Another option would have been to require to have a sensible cost cutoff point (smart grid). It would have been more expensive to implement, but a great compromise.
Hafnium is not a rare earth.
I see,
lets a person has contributed to open source projects would that make the candidate stand or the person has built a library or any resource that is starred by a lot of people is that going to make a difference
This article does contain very little information.

But this channel is one I watch a lot of. It is dedicated to in depth chess commentary with some bits of chess history and I highly recommend it if you play. This it not the first time this has happened to Antonio. He even discussed it a bit the first time, 7 months ago. In that instance it was a single video that was removed. In that video he says “white will always be better” while analyzing a specific position (interesting to note that in the starting position of chess this is also always true). YouTube said it was removed for “harmful and dangerous content”. Without anything else to go on I’d say it’s a fair assessment the it was banned for racial content. I’m not sure what else could be “dangerous” about chess analysis.

https://extra.ie/2020/07/02/news/world-news/chess-youtube-vi...
In order to operate in Australia, you must report transactions over a certain threshold.

See: https://www.austrac.gov.au/business/how-comply-guidance-and-...

I have it good authority [1], that quite a few local Australian banks report when the threshold is 1000 or more.

For international transfers any amount must be reported:

https://www.austrac.gov.au/business/how-comply-guidance-and-...

[1]: My father works there.
It can’t take off from a standstill with a full load of fuel and passengers with only one engine. Level (or slowly descending) flight doesn’t needs as much power.

If you normally cruise with two engines at 50% going to one engine at 100% isn’t such a stretch.
Author here.

Are you thinking of a specific implementation of materialized views? Most implementations from traditional RDBMSs would indeed be too limiting to use as a general data pipeline building block.

The post doesn't argue that, though. It's more about using materialized views as a conceptual model for understanding data pipelines, and hinting at some recent developments that may finally make them more suitable for more widespread use.

From the conclusion:

> The ideas presented in this post are not new. But materialized views never saw widespread adoption as a primary tool for building data pipelines, likely due to their limitations and ties to relational database technologies. Perhaps with this new wave of tools like dbt and Materialize we’ll see materialized views used more heavily as a primary building block in the typical data pipeline.
There is no utility that isn’t better achieved classically. When that changes I’ll join you on the other side, and won’t hamper my ability to earn money by investing in tilting at the windmills of decentralization - I prefer to approach technology beginning with a problem and looking for a solution as opposed to the opposite - but as always I wish you the best of luck.

I’d suggest you’re in the wrong century, I’m thinking when Isaac Newton lost all his money in the south seas bubble. 1700s?
If the recall is successful, the governor gets replaced by whoever has the highest plurality of votes. In other words, it creates a special scenario where you can be elected governor with much less than 50% of the vote.

I've always thought this is part of why California seems headed for its second recall election in just 16 years.
> we had never used them before

I think that language goes back eons. I think the black/white metaphor is employed in the book of Genesis/B'reshith for instance. People have been scared of the dark for more of human history than not. Nighttime is black, when things putrefy they turn black. Uncorrupted snow is white. The examples from nature are legion. What’s weird is using these terms for people.

Maybe the more sensible way to address the language issue is to stop calling people “black” and “white”. Why don’t we focus the scorching blast of social condemnation in that direction? The colors are silly classifications for people’s heredity or skin tone. Seems like the more elegant fix. Even today using the terms black and white can be “problematic” - some people don’t like being referred to as a black man by the wrong person, many white people consider “white boy” to be a racial epithet. So if we’re doing language surgery, let’s just ditch that part of the language and stop obsessing over the natural metaphors from time immemorial.
Bitcoin is just a way to funnel money from financial markets into energy production.

Since we need huge investments into renewables production, every mechanism for such capital reallocation should be welcomed. Our future is electticity. Currently we are producing a fraction of what we'll need.
"they lacked fundamental knowledge"... you think people should get fundamental knowledge ? on _bitcoin_ ? It is fundamentally worth $0.

The only knowledge I try to spread is correcting anyone who uses the word "invest" and recommend the proper words of buy, speculate, or gamble when it comes to cryptocurrencies. I guess I will add "fundamental" too since I think you meant technical analysis. But even that implies brains when it really is just voodoo unless you are an influencer.
The main key is that you can look at any file as a string of bits! And apply Huffman coding at whatever granularity you like. For text files, you're essentially applying it at the byte level (since each character is a byte (in ASCII, anyways)). For images, you might have one byte per colour channel, RGB. Then you can apply Huffman coding at the byte level, or even at the 3 byte level to operate on entire colours as opposed to channels.
> good luck with scaling, RIP

Why? Plenty of options to scale Postgres. What alternatives do you use?
They link the track in the article.
Venture capitalists are the modern version of medieval patrons---they have money to spend to make themselves look intelligent and forward-thinking through the activities of others.
Yeah but this was a hack and the side-effects should still be addressed. If the intent was to force the price to a point where everyone’s system trips and automatically stops buying but just relied on everyone having that set up then that’s on the commission to make it right when that assumption turned out to be false for some people. They should make everyone whole by charging the real market prices for that stretch of time.
Thanks for the correction! I did think its price had gone up permanently these days though.
Right, try doing any sort of fine manipulation while wearing thick rubber gloves and you'll appreciate how hard it is to design robotic hands.
Yes, a master/slave relationship in technology. Don't drop the context to support policing words. Context very much matters when dealing with words yet for some reason the context is dropped in order to make it seem like a word is being used in an offensive way. Except it's not, you're just forgetting the context to make it that way.

If a process is directly commanding and telling another process what to do I don't see how calling it a master is wrong? It is one of it's definitions.
The Covid-driven centralization of economic power and information control
It's happy that you only use system's Python package. You need manual version management if you install multiple Pythons.
>Nobody should be silenced unless they are directly inciting violence.

>Remember, any exceptions can be twisted around as social norms change.

If no exceptions to free speech can be allowed because they could possibly be twisted, then neither can the exception for directly inciting violence. Absolute means absolute.
In my experience you can find a job in China within days or weeks. They ask you "Can you start tomorrow?" and this is not a rhetorical question.

You can not get naturalized, true. At least not in Mainland China. But they also have a Green Card now.

While I also have a US passport, how do you "immigrate" to the US? It is very difficult, a work visa in China is much easier to get. And believe it or not, I started as an illegal immigrant in China with a tourist visa (While I was not illegal, I was working full time on a tourist visa).

Hostile to foreigners? I don't know. But I speak okay Chinese by now. And doors I tried to kick in in the US that never opened, opened easily in China.
The service is not free.
You can absolutely get complicated data models nailed using views. Some of the views get unwieldly and a bit long but it can be done. The catch is in incrementally loading the aggregate tables or self referencing (even then the underlying views are essentially functions to be included). I scanned the article and have followed materialize.io for a bit and built pipelines that handle what is essentially the awfulness of performantly updating materialized views.

I'm not a master but believe a core piece of truth for data:

Move the data as little as possible

If you can use federated query (cost/performance is acceptable), do so. If you can use materialized views, do so. Data replication has tons of issues, you almost always have the 2 generals problem and reconciliation / recompile procedures. If you don't move the data, the original storage is the source and it is always right.

I think I went way out of responding directly to you, I do think materialize.io and delta lake and declarative pipelines are the solution to 95% of the data problems out there.

I'm speaking conceptually about materialized views as system implementations differ.
Implemented a custom migration using this technique a few years ago, was one of those mind blowing "you can actually do that?" moments.
There is no fee to send money from your TransferWise Borderless UK account to another UK bank account.
The classic simplex method for real linear programming is exponential-time in worst case, but cubic time in the average case. Newer algorithms can beat that.
That has 2500 charge cycles too so...
Spending time offline.

Sitting quietly with our thoughts.

Having faith in others.
> Like was there actually someone sitting there, using an app, seen the term blacklist and immediately started thinking it was an insult to them?

My belief is absolutely not. It comes from an ideology that ascribes simple moralistic explanations to things. “This word has man in it so it must mean it excludes women”. “The word ‘master’ reminds people of slavery”, etc.

Good luck speaking out “publicly” about it in the modern corporate environment. You have to not just accept it as true but actively parrot its talking points. Everyone has to be a believer. That’s how the whole thing works.
A problem with this is most places have drawn the line in a much better place than you are describing.

Did other places have outages? Yeah. They also had more severe weather. With fewer deaths, it seems.
At the end of the day, attention is a resource, and resources have monetary value.

There will always be things like this.
I have used it to pay a large foreign tax bill, so not quite a house. If the amount is GBP£30000 (might have changed now), you can arrange a longer pay-in time (normally it is 24 hours) and they will let you have the cash sit in your account for up to a week and then find the right rate.

Later: Appears they have lowered the threshold to £20000 now (see https://transferwise.com/help/articles/2961856/tips-for-gett...)
This is a good reminder. Many people don't realise if any one of Google's services ban you it takes down your Google account at the same time (GMail, Drive, Docs etc), it doesn't just block that single service where the alleged violation occurred.
My intuition is that consciousness, as an a priori concept, is not useful. Like the four humours in the medical world, it's a metaphor that captures just enough truth to be a blunt instrument. However it's a local maxima. And a honeypot in the neuroscience domain.

So much effort is spent trying to pin down a definition of consciousness. Ask five people and you'll get five definitions. And yet everyone is so sure they know who has it and who doesn't. Slugs don't have it, dogs don't have it, maybe elephants do? Apparently they can talk to each other and mourn the dead.

One possible motive behind humanity's fascination with consciousness, and our speed at assigning or withholding it, is that it gatekeeps the belief in an anthropocentric universe. The human experience is the one we understand, and if you don't have it you are not conscious. Consciousness then, is a Turing test for how passibly human you are. Slugs could get some of the consciousness table scraps if they'd only mourn their dead.

As we learn more about other species; and coming from the other end too as computers (big data really) reveal new sightlines on human determinism; consciousness as a metaphor is strained and loses adherence.
Isn't platinum better for that?
The question was what is the benefit of flatfile over SQLite. SQLite stores the database in a single file, so "it uses a file" is not an answer to that.