skysight-inc/HackerNews huggingface dataset:
Thank you for sharing!

I assume we are talking about analytical, not transactional data:

> Diff'ing data is one of the weakest and most cumbersome ways to verify correctness.

It depends on the use case: if the goal is to assess the impact of a change in source code on the resulting dataset produced (extremely common in ETL dev workflow in my experience), then isn't diff the natural solution? Of course, it depends on how the results are presented. A row-by-row output for a billion-row dataset is useless. That's why we provide diff stats across columns/rows and data distribution comparisons while allowing the user to see value-level diff if needed.

> Diffs are relatively slow

In general – yes, that's why we've implemented configurable sampling. In the majority of cases, developer is looking to assess the magnitude of difference and certain patterns, for which you don't need a large sample size. Our customers typically use ~1/10000 of the target table row count as a sample size.

> when they fail you get a blizzard of errors
We try to fail gracefully :)

> I don't see how this helps with schema migration or performance issues.

For schema migration, you can verify whether anything has changed in your dataset besides the intended schema changes (which certainly happened on my watch).

> or performance issues

We certainly don't claim to solve all DBA issues with diff, but here's an actual real example from our customer: they are optimizing their ETL jobs in BigQuery to lower GCP bill by reducing query runtime. After refactoring the code, they diff production vs. new code output to ensure that the data produced hasn't been affected.

> If you really care about correctness it's better to use approaches like having focused test cases that check specific predicates on data.

Possibly, but

> They're also a pain to code

...which is often a prohibitive pain point if you have 300+ analytical datasets with 50+ columns each (a common layout for companies of 250+).

And another problem: the more test cases, the more failures on every run, and unlike app code unit testing, you can't expect the cases to stay relevant since the data is changing constantly, so those "unit testing" test suites require constant maintenance, and as soon as you stop actualizing them, their value drops to 0.

I think that diffing and "unit testing" are complimentary approaches and neither one is a panacea.
So my recommendation has been to use both:
1) Specific test cases to validate the most important assumptions on data
2) Diff tool for regression testing.
17 years ago...
To bait a story with such a spectacular plot, the author could have us reading tenuous managerial pontification for pages on end, just to get the juicy revelation at the end.

That the narrative payoff never comes is such an infuriating let down, you actually end up confused over what the heck you just read!

I think the fact that the story comes with bespoke cover art just serves to raise expectations even further.
You can read about what USDS has been doing lately in the Spring 2020 impact report [0]

(P.S. Public Sans [1] was developed by the U.S. Wed Design System (USWDS) team at GSA, not USDS)

0: https://www.usds.gov/resources/USDS-Impact-Report-2020.pdf

1: https://public-sans.digital.gov/
https://www.nbcnews.com/news/world/china-forcefully-harvests...

I agree that the US prison industrial complex should be reformed. However it is in no way equivalent to locking people up for political reasons. Political reeducation camps and forced harvesting of organs from prisoners cannot be compared to work release programs or paid prison labor.

Why are so many comments seeking to establish equivalence?
I wonder how cached results are created and is there some kind of copyright violation because they store the content.
https://www.youtube.com/watch?v=5JAHAFvcr2o
Yeah, I've only hear about it as a typical result of pneumonia
c.f. P.G. Wodehouse, defended by George Orwell.

https://www.orwellfoundation.com/the-orwell-foundation/orwel...
Her "mistake" was assuming everyone in the federal government is on the same side, including the FBI and whatever org USDS had oversight on in this story.
That should be possible with EC2 nodes for your ECS cluster rather than Fargate.
Lion is similar in that it’s intended to be customized from the start. For something more “batteries included,” see https://shoelace.style.
But leaded gasoline was still sold in the US at least into the 80s, in my memory.
Microsoft: Cryptocurrency system using body activity data
Whoever the author is, I hope that he doesn't have a Uighur relative in Xinjiang in jail trying to get reformed and a friend in Hong Kong trying to free the country.

If you were in China, you couldn't even write an article criticizing Tiktok like here the author here is writing an article criticizing SV. Have some perspective.
It’s funny. IMO Hacker News which I only recently discovered is what I was looking for in twitter. Now I’m completely off twitter and with HN I get all the neat computer news that I crave. Also I like that for the most part, people keep the discussion civil and constructive when discussing things over which there may be disagreement.
I imagine this is something that can be quite exhausting for people in the federal bureaucracy.

There's nothing to be done when the American public elects a muppet President.
Are politics like that allowed on here? Doesn’t sound like a good idea. ‘Old Biden was a good man anyway, but be has one confirmed brain aneurism and may have a second, plus we are in a depression that everyone calls a recession (to avoid panic) so doesn’t the final choice have to be Trump Good, Starvation Bad? By the way, there is no way to deface Reddit and Russia are gone-be our BFFs in a Trump second term —Russia plus USA will remind everyone what China rhymes with.
The reason (basically) is because Republicans in the states trend older and by extension, less technically savvy. Since content that both incites strong emotion and allows that person to represent themselves a certain way tends to be the most shared, highly clickbait/inflammatory/misinformation right-wing content rules the roost.
ahem

> "OH," BUT YOU MIGHT BE TEMPTED SAY, "THAT WASN'T JUST FOR A GOOGLE SEARCH! AND MANY OF THOSE WHO ARE CITIZENS ELSEWHERE? LIKE THE CHINESE GOVERNMENT, I DON'T BELIEVE THAT CITIZENSHIP IS VALID." YOU WILL OF COURSE UNDERSTAND THAT THE PARENT POSTER DID NOT PHRASE THE MATTER IN THOSE SPECIFIC TERMS, AND THAT IN ANY EVENT THIS IS NOT A COMFORT.

thank you
Unfortunately, I think the solutions are context specific. It depends on what the larger organization's goals are.
I think the point of the story was to demonstrate what non-risk-averseness looks like. It was probably hard to find a way to make that explicit because it's basically a form of bragging.
What about parachutes?
Almost every automatic nowadays gives you a couple of gears to play with. I always downshift on curves or when braking to a stop. I also drive a hybrid and downshifting guarantees regenerative braking whereas with the pedal it’s easy to over do it and engage the pads.
I'm sure that's among the cases he meant when he said "fringe cases". Most Americans don't have families in China
Very cool implementation. I am wondering have you considered using something like https://github.com/cliqz-oss/adblocker which can sit on top of headless browser and do not require bridging to an extension.
OTOH monomorphisation multiplies amount of code to optimize. For large functions that is problematic.

Rust's standard library uses a pattern (written manually) of "hoisting" generic code out of large functions, e.g.

fn generic<P: AsRef<Path>>(path: P) {
non_generic(path.as_ref());
}

fn non_generic(path: &Path) {
// lots of code
}

That's cheaper to compile and reduces binary size compared to having one large generic function monomorphized for every AsRef<Path> type.

It'd be great to have this as a compiler optimization to automatically have "half-monomorphic" functions.
Forgive the dumb question but what is False about ‘∀x eggplant(x) ⇒ fruit(x)’
Would COVID have been a "pandemic" if we did not have the internet? The Swedish perspective is interesting. I wonder if Google trends can be broken down by nation to reflect what the concerns of different countries are. How many COVID searches in Sweden vs the US vs other nations ?
to challenge the premise -- it is worth thinking this through: why is it necessary to drive what you're working on over the finish line? if you reduced your inputs of energy, time and caring to moderate levels that can be sustained without negative physical or mental health impacts -- would you personally receive comparable output?

re: burnout

> [...] level of caring couldn’t be sustained in the absence of results.

> My clients are perfectionists [...] They have very rigid ideals in terms of win-lose [...] Their expectations of success are through the roof, and when their reality doesn’t match up with their expectations, it leads to burnout—they leave no room for error or failure at all in their formula.

> Older workers, as it turns out, have more perspective and more experience; it’s the young idealists who go flying into a profession, plumped full of high hopes, and run full-speed into a wall. Maslach also found that married people burn out less often than single people, as long as their marriages are good, because they don’t depend as much on their jobs for fulfillment. And childless people, though unburdened by the daily strains of parenting, tend to burn out far more than people with kids. (This, too, has been found across cultures; in the Netherlands, a recent survey by the Bureau of Statistics showed that twice as many working women without children showed symptoms of burnout as did working women with underage children.) It’s much easier to disproportionately invest emotional and physical capital in the office if you have nowhere else to put it. And the office seldom loves you back.

-- https://nymag.com/news/features/24757/

re: not burning out

> Poor is the person without Slack. Lack of Slack compounds and traps. Slack means margin for error. You can relax. Slack allows pursuing opportunities. You can explore. You can trade. Slack prevents desperation. You can avoid bad trades and wait for better spots. You can be efficient. Slack permits planning for the long term. You can invest. Slack enables doing things for your own amusement. You can play games. You can have fun. Slack enables doing the right thing. Stand by your friends. Reward the worthy. Punish the wicked. You can have a code. Slack presents things as they are without concern for how things look or what others think. You can be honest. You can do some of these things, and choose not to do others. Because you don’t have to.

> Most times are ordinary. Make an ordinary effort.

> Make sure that under normal conditions you have Slack. Value it. Guard it. Spend it only when Worth It. If you lose it, fight to get it back. [...] Make sure to run a diagnostic test every so often to make sure you’re not running dangerously low, and to engineer your situation to force yourself to have Slack.

-- https://thezvi.wordpress.com/2017/09/30/slack/
I’d put money on this lady coming from a privileged background. Story was a person who works recreationally being disgusted by having to interact with all the filthy proles who need to put food on the table.
If the scientist isn’t concerned with going splat and doesn’t do something about it then he or she may not reproduce.
Found the military industrial complex shill.

Easy find. What a loser.
No. I wasn't trying to say that. He is right that there is a lot to know to make any contribution to physics. I was trying to make a more general comment that people shouldn't be afraid to try something even if they don't know all the background. Make mistakes and learn as you go. That applies more to "easy" things like software.
This.

Have been taking hour walks in the sun since May.

Never have I gotten so much sun before! Feels great
> I may move to Hawaii when the plague passes, and then my day would start at 6am. That'd be rough.

I know someone who did this. If you're a morning person already, the 5am wake-up isn't terrible, and finishing work at 2pm means you get prime surfing time every day. Not a bad way to go.
For the alternative meaning, I think you'd say, "The company couldn't make the car safely enough." So yeah, adjective and adverb, I think you're right. For that matter, "The company couldn't make the car quickly enough" would resolve the ambiguity in one direction, but I guess not the other.
Oh totally. He's insufferable and absolutely hilarious. I don't read him to get a decent judgement of a film, I read him for "old man yells at cloud" vibes.
No, in this case it was a simple way of saying "This group of people can be slaves and that group of people cannot be slaves" which is why we get weird demarcations about how you are still "Black" even if you are only 1/8 African: So slave owners could rape their slaves, father children upon them and treat those children as slaves for several generations.

In the US, our weird definitions surrounding "Black" and "White" are a polite euphemism for "People officially marked as someone you can victimize and people officially marked as someone who can do the victimizing."

Which I didn't really want to baldly state because it isn't something Americans really want to hear.
Its worth noting Atlassian's roots in Australia mean they have had a long history of working across borders. Both the founders just bought the two most expensive houses in Sydney adjacent to each other so presumably they don't want to be bound by geography in running their business either.
I don't know how truly "zero-cost" the compiler implementations are, but the premise of functional programming languages like Haskell seems to be that programmers should be able to freely think in abstractions like map, fold, monads, etc.. without worrying about performance.
I think your assumption/implication that "was traumatic" is inconsistent with "happier now" is questionable.
Louisiana was not aggressive about reopening; we have Democratic governor
> a user-owned host device

So, they can transfer "ownership" to me whenever I use my personal computer that happens to be located in Google's cloud which, through a series of tubes, is in fact connected to my LAN-based network.
Thanks @geofft, you made a lot of great points.

I don't think rootless approach is fully aligned with what we're doing right now. True, we both rely on user-namespaces, and we both emphasize the security angle, but our goal is to expand the number of applications/functionality that can run in containers, which is something rootless approach may struggle with for some time.

Please see here if you haven't done it yet: https://github.com/nestybox/sysbox#sysbox-is-not-rootless-do...

In regards to our dependencies, we can operate with or without shiftfs. In both cases user-namespaces are always utilized. The rest of your approximation is correct: we need most of what you mentioned in your second paragraph, which btw, is already there (thanks to Canonical/LXD folks) starting in Ubuntu 5.0+ and 5.5+ for other distros. As you know, shiftfs is only present in Ubuntu at the moment, but as i said, we can live without it.

Which leads me to your question: why would you wait if the functionality you're after is already there? If having dockerd running as an unprivileged user is not a real must-have for you, then Sysbox provides a fairly secure solution while giving you all the functionality.

Sorry, i'm not familiarized with vpnkit yet, will take a look.

Correct, those are some of the benefits Sysbox-EE offers at the moment. That, plus efficiency & scalability features and hardened testing.

Thanks a lot for your detailed feedback @geofft. Please ping us on slack anytime.
Show HN: A free blog for devs you edit over SSH
I think it's just greed and poor risk management. Up until recently, china couldn't even manufacture ball point pens (which I think only 2 companies in the world produced). There's no money to be made at the bottom of the inverted tech pyramid. They rather put their energy in buying ball point pens and using those pens to write stuff that will in the short term make them more money on to of a foundation of sand. Similarly all of the investment from giants like Alibaba etc went into developing e-commerce on top of a tech foundation of sand
> Textualism doesn't care about intention or purpose and will gladly apply an interpretation that would have seemed absurd to anyone who drafted or voted for the law before the implications (e.g. the insanity of the world) laid bare the ambiguity.

This is probably why originalism is more what one hears about these days. One could argue that it's just textualism with the serial numbers filed off in practice, but in theory at least an originalist would reject an absurd interpretation that was obviously counter to original legislative intent. But my cynical side agrees that textualism or originalism is just another tortuous path to reach conclusions that have already been decided on before hand.

> In Western legal philosophy it's usually considered a violation of separation of powers and of justice more generally for a legislature to directly overturn the result of a particular case, as distinguished from changing the law for future cases.

There are various laws passed that granted veteran's benefits to specific individuals. There's also the Palm Sunday Compromise[1]. The latter was certainly politically controversial, and raised criticism along the lines you mention, but it was within Congress's power.

[1] https://en.wikipedia.org/wiki/Palm_Sunday_Compromise
Authors' 'invisible' words reveal blueprint for storytelling
Hottest Recorded Temperature on Antarctica Measured in 2020
Sweden has social programs where people can do their own lockdown and not be worried that they're whole life will fall apart.

Eg. They don't have to keep working or risk not having healthcare.

The US needs the big government sponsored lockdown because otherwise US companies will force people to go to work when it's not safe
Same reaction exactly, yes!
Average new cases per day has dropped from about 1200/day in late June to less than 300/day now. There's a slight uptick over the past week.

https://91-divoc.com/pages/covid-visualization/?chart=countr...

Downtrend in average deaths per day continues, currently at 6/day.

https://91-divoc.com/pages/covid-visualization/?chart=countr...
The Carlos Ghosn case highlights some of the points you raise about the arbitrariness and closed-box nature of Japanese nativism.

I have worked for a Japanese client (big firm, I will not name them). I think it is a miracle that Japanese innovate at all. Their hierarchies are extra rigid, and promotions are more or less dependent on seniority. It was difficult to find any tech manager who was not around 60 years old. Peer-to-peer email from our firm to Japanese engineers was strictly forbidden. Except for the money involved, it was like a humorous game of Chinese whispers - a perfectly legitimate technical point raised will come back as completely illogical gibberish because it was been distorted by layers of management in between.

Engineers at my (Indian) firm in 2001 were saying that Sony's days are numbered, and Samsung will eat their lunch. I refused to believe it since South Korea had just gone through a major crisis. 20 years later, it is patently clear that they were understating the trend.

The often deserving worship of Japanese culture that many indulge in, leads to vast blind spots about Japan's cultural disadvantages.
I believe she was using the concrete experience to speculate on why she ended up being different enough from the rest of her peers that her manager could know that it “could only be her”. Escalating this issue to the FBI went beyond what was called from her.
So far I've only discovered the aftermath, drained husks hanging in a web. If I can catch a spider in the act though that would be great and I'd definitely film it.
Not necessarily.

Lots of evidence now that aerosol covid is a major risk with prolonged exposure.

Scientist have found the virus in ventilation systems of hospitals supporting this.

This means the ‘aura’ of effect for an individual is quite large, especially in buildings with shared ventilation systems. This still requires much greater exposure time due to virion density decreases over distance from source, but that time would be available in a quarantine. Therefore, a steady trickle of infection is likely because not everyone lives in a house, and because there will never be enough N95 masks (we had to start prepping for this 10 years ago to make enough masks).

That, and the fact that people move between borders means it will not go away on a meaningful month time scale because there is no way to coordinate an entire world response. As you can see, the easiest geography to protect is islands...
N.Y. Schools Can Reopen, Cuomo Says, in Contrast with Much of U.S.
I'm mostly meaning vpnkit in the sense of a it's thing that plugs into rootless Docker to provide networking - it seems like you could also be a plugin to upstream rootless Docker to provide sysbox-fs and your shiftfs management, at least in the long term.

Will try to remember to join the Slack next week, this is definitely a cool project :)
It's a perfectly reasonable set of questions.

Please note I stress that I am commenting on the virus "regime".

Is the virus real? I think we should assume it is real, regardless of the facts. Let's say it is real.

Should we concern ourselves with whether it is (a) natural phenomena, or (b) man-made;

and whether regardless of its provenance, the spread of the virus was due to (a) chance events, or (b) intentional release?

Again, I say, to seriously answer any of these questions is an unreasonable burden. So, let's call it an "act of god".

Now, we are at the point to discuss sensible response, rational response, reasonable response, to a collective health issue: a virus that has reached pandemic stage and posses certain risk probabilies across various demographics.

Does the above scenario mandate that governmental norms be bent if not broken? For example, is it acceptable that from now on society will be governed by executive decrees and not legislative bodies?

How long should we tolerate such encroachments on what we know to be the basis of free societies?

Should not a health care matter of such grave import and impact on our lives be subject to the input of a wide range of experts in related subject matters?

What "authority" has designated certain institutions, such as the Imperial College as an example, as fonts of indisputable claims regarding the gravity of the situation and required remedies?

Why are dissenting experts and workers in the "frontlines" subject to media blackout, de-platforming, and loss of income?

I could go on and practically write a little book here. But that should suffice as to "why" some of us are rather concerned, to say the least.
What is there to master? It's drawing or typing on a board.
And our leaders wine and dine with people who we call allies and good friends yet have their journalists, activists and political rivals killed without flinching.

You don't see anyone in power losing sleep over the Saudis or Israel constantly assassinating people they don't like. Or even bothering to care about a madman like Duterte from going on extrajudicial rampages and killing tens of thousands of his own citizens. Apparently they're the good guys because they fall into line with us.
Slavery is a competitive construct.
>The threat in the US is not from the government, it is from the private sector.

Isn't that covered by the OP? Regardless of the private sector's surveillance capability, they don't have access to violence. This by itself makes it less worse than the CCP.
hehehe. chuckle. it's like giant orgs make an art out of dysfunction, more sophisticated and sophisticated levels of dysfunction. as if refinement of dysfunction to its highest potential was the ultimate aim.
I’m curious about what an analysis of the COVID stats would show.