skysight-inc/HackerNews huggingface dataset:
(Oil author here) Thanks for mentioning it -- Oil is intended for exactly that kind of thing, in theory. That is, a clean shell language for gluing together distros, which are inherently heterogeneous and messy.

In practice, I think there is limited time / resources / appetite for cleaning up a bunch of 15 year old bash, with the potential to break many things. Shell, and especially bash, can be very fragile.

(I'm not speaking for the Nix team, I'd be interested their viewpoint)

OSH runs many shell scripts unmodified, but not Nix's, even though we implemented many bash features that Nix uses. I mentioned that in the latest release: https://www.oilshell.org/blog/2023/03/release-0.14.2.html

I'd still be interested in bug reports -- non-interactive language features tend to be prioritized, and are realistic

---

So to "concede to reality", I think the approach of having new builders makes sense -- it's ADDING rather than REPLACING. It's sort of like what Rich Hickey says about functional programming -- you don't refactor your code, you just keep adding stuff.

Having only used Nix a tiny bit, I didn't even realize new builders are possible. Is this the only other one?

Is it possible to create some sort of "interface" for them? If there was a documented interface, maybe it will help people understand how to build packages, in addition to plugging in new builders.

---

BTW for people wondering about the project, I just wrote a FAQ:

Oils 2023 FAQ - https://www.oilshell.org/blog/2023/03/faq.html

tl;dr We're stabilizing OSH, the "good parts" that runs your existing scripts. In theory Nix could adopt this, with the harder "replacing" strategy.

And we're renaming Oil to YSH to delineate the stable and unstable parts. YSH could be used with the "adding" strategy.

Though the idea is that you can seamlessly upgrade from OSH and YSH, and not have two different things forever, but the costs/benefits are different for each project. And of course as I say, YSH isn't yet stable. (nushell is also pre - 1.0 from what I understand, but I guess version pinning helps with that.)

Much more detail in the post. We can still use help :)
If stress is hurting your ability to focus, I would recommend trying exercise. Like a lot of it. 40 hours a week even. I know it sounds really shitty, but going through hell is usually the fastest way out. I've tried the curl-up-in-a-ball thing a few times and it always makes it worse than if I just endure and push through.

The human mind has an incredible ability to develop master strategy when you force the body to endure unhappy experiences. Right now, it sounds like the suffering of your body is not well-aligned with the suffering in your mind. As the physical suffering equals or exceeds the mental suffering, you may begin to feel relief.
I think it's difficult on Intel to be fully free because of Intel ME. As far as I know, it's not possible to fully be rid of it, so you'll always have some proprietary blob running a possible backdoor.

If you want to be fully free, you may be interested in the Raptor Talos II, which is based on OpenPOWER and has free firmware. It is pretty pricey however.
I hope in the next decades we gonna fight for our rights here in the US. Economy is not what it used to be. We need insurance that is not tied to an employer. We need safety net. Otherwise it all can get derailed really quick.

I hope grandpas in the government understand that nobody works for one company for 40-50 years anymore. They need to address that gap asap.
>The US did not enter a new armed conflict in 2022.

It didn't?
As a Russian student studying in Paris, it was a terrifying realization that the mayor can, apparently, prohibit any and all mass protests by simply issuing a decree, no higher authority needed. (This was in 2018 IIRC.) This is apparently part of the anti-terrorist law or something? I dunno, and nobody seemed to care except for the participants of the protests that were happening at the time.

Compared to the constant, unceasing reminders by Russian activists that the de facto prohibition on protests in Russia is a legal sham and the actual thing would require the president declaring a state of emergency (that only happened in 2022 and then not fully), the silence in an established democractic society was deafening.

The tired “freeze peach” responses on HN evoke similar feelings, as do the “private businesses can deny anything to anyone”. (Never seen a less-than-loyal artist’s concert cancelled the day before because the venue received a suggestion—not a command, mind you, a suggestion—from a city official’s phone? Have I got news for you.)
DS9 is my favorite ST, and I very much enjoyed Babylon 5. Its that same kinda show.

That being said, the cast isn't quite as charasmatic as the best of DS9, but surpassing Garak is a tall order.
> which republics are missing from the public education diet? Both Athens and Rome are studied to death in public school. Sparta wasn't a republic (or anyone's idea of a model for a good, or successful society.)

Athenian democracy is discussed, but nots its constitutional structure and evolution. Sparta wasn't a republic, but it had elected leaders. (If we want to split hairs: proto republics. Also, right here is a point missed by many: elections a democracy do not make.) Similarly, at least my American education about Rome focussed on the fall of the empire--not the the republic.

Adding to the traditional corpus: the Vajjika League [1], Carthage, the maritime republics, Britain, the Haudenosaunee f/k/a Iroquois. Possibly also confederacies: the Holy Roman Empire, the Swiss, et cetera. Finally, other extant constitutional systems. Basically, how do the bones of government work, what have we tried, what are we trying, and what lessons can we draw from the past.

> anyone's idea of a model for a good, or successful society

There is merit to studying failed societies. The Spartan Gerousia bears striking resemblance to certain ossified structures in modern governments.

[1] https://en.wikipedia.org/wiki/Vajjika_League
Doesn’t most malware target Android these days?
Sure, central gov _chose_ to cracked on on the sector, but the denial is that will lead to doomer level of collapse portrayed in western media. Here's a recent primer on PRC land finance and why market remains resilient / impact on gov finance overblow.

https://www.bloomberg.com/news/newsletters/2023-03-25/china-...

The real denial is western analysts with little understanding of why real estate crack down won't lead to PRC collapse. At end of the day, realestate is about as wasteful as US extravagant health care spending in % terms. Inefficient sure, but not end of the world in terms of misallocation. Even less so when ~100m surplus/vacant housing units still not enough for 200m that needs to be rehomed to bring urbanization from 65% to 80% in next 20 years.
Why do you continue to use HN, which contains URLs to nonfree software?
So this is funny because I was trying to figure out what Darklang is and what its used for, so i was looking at its doc. The page for usecases/limitations [0] says

> Dark is really good for building web applications, APIs, and things that speak HTTP. Dark is not intended to be usable for embedded systems, to run existing applications in existing languages, or for code that needs to be extremely high performance.

> Similarly, Dark does not have support for many use cases such as bitcoin/crypto, AI/big data, image or video manipulation.

lol.

Anyways, some bold claims in here.

> Do we expect that in 3-5 years time developers will still be typing out artisanal code at a keyboard? I struggle to see it. With recent announcements like ChatGPT Plugins and Copilot X, even three years feel generous

I wonder how true that is.

[0] https://docs.darklang.com/discussion/when-dark
This is a fascinating subject. Chinese naming taboo focuses on emperor names; there's also a more general practice, common I'm Aboriginal and Bantu languages, where the sounds from in-laws's names cannot be uttered: https://en.wikipedia.org/wiki/Avoidance_speech
You know they wouldn't be doing this unless the point was to expand the government's ability to harm its citizens. Which sweeping and terrifying powers does this one introduce?
Did he look like a paper cutout?
Rolling-Mill Oops
Can GPT4 help process this data?
Same for American news
My EV will do 250Wh/mile at 60mph.

But it'll do a hell of a lot better at 15mph!
Hello whinvik, I totally agree the debugging experience of Spark is suboptimal.

But why?

I don't think the main reason is too much 'magic', I think it is because of the coupling of user's logic and the complicated backend logic. When there is an issue, not only you have to wait for a long time to get the error back, but also you get the error of the whole execution chain, and you have to find the real error in it, which is challenging even for experienced Spark users.

Not only Spark, if you use Dask or Ray, you will find very similar experience. Can we directly make Spark Dask Ray easier to debug?

I doubt. This is because the complexity of these systems are necessary. And since the problem can happen anywhere either on the user side or on the backend side, the whole stack has to be investigated. It is inevitable, otherwise, people will start to complain the real errors are not captured by the system.

What can we do about it?

Fugue's idea is to let you iterate on your own logic locally before using these complex systems. So you can get quicker and less noisy feedbacks. The challenge is how to unify the semantic so you don't need to change code moving between local and distributed. And how to trim down the system errors so you can focus more on your own problems. These are what Fugue solves.

It's all about good practices

It's common sense that if you can keep your logic agnostic to backends, and if you can iterate locally on small data, the development experience can be a lot better. But why big data developers tend not to follow these practices? Because there is too much discrepancy created by the distributed backends intentionally or unintentionally. We created such abstraction for big data developers to get more rewards than punishment when following good practices, so they are more willing to follow good practices.
I do like to run tcpdump regularly. For a while I ran my home laptop thru an allow-listed proxy, and I was quite surprised at the results, and how little I had to allow to get what I needed done. A lot of software is phoning home. I would have thought that open source folks didn't do that, but standards or expectations have shifted. A browser that took over your DNS would have been anathema in the 90s, but now we trust the local networks even less.
US looks rather involved in the 2022 Ukraine-Russia war from my peasant POV.

Russia is not fighting the US military, but they are fighting US military vehicles etc
My read of the word "Islamic" in that case was, uh, for want of a better word, inclusive, in that I took it to refer to "the culture that was flourishing in the Arabic world in the 7th C". Given when the subject of the original article was born (c. 630), it is, I think, both harmless and descriptive to conflate the terms Arabic and Islamic: Islam was born in Arabia and was at that time purely Arabic, and it spread quickly enough that most/all of Arabia was Islamic during the physician's lifetime.

I don't know how fruitful it is to attempt to determine the extent to which the respect for knowledge then prevalent was a result of Islam or of other factors. The simple fact is that for a period that covered at least the physician's life, preservation and extension of knowledge was prized in that area.

Contrast that with the predominantly Christian cultures to the northwest-ish, that were at that time far more concerned with dogma, their own intellectual flourishing still a few centuries. What we in the west know of the ancient Greeks we owe to the Arabs, who were predominantly muslims, of that period.

It's far more complicated than that, of course, but to this day, writers conflate "Islamic" and "Arabic" when referring to that period (cf, e.g., [1]), and I think forgivably so. I think.

(I hesitate because on the one hand it is both difficult and fraught to tease apart the various influences, on the other hand western society was predominantly Christian and many of the western values I cherish arose and flourished in those Christian societies, on the third hand there was definitely a Jewish influence in some of those societies and an Islamic in other, on the fourth hand I share many of those values despite being none of Christian, Jewish, or anything else theistic, and on the fifth I am often irked when Christians claim (wish I could find a source, but alas, I cannot, but I've seen this many times) that those values could NOT have arisen elsewhere... not enough hands to balance these complexities.)

[1] https://greekreporter.com/2022/09/01/how-scholars-islamic-go...

(I am often reminded of how applicable to social discourse is Postel's Law, written in regard to TCP, "be conservative in what you send, be liberal in what you accept" - be liberal, be open in one's interpretation of the wording of others, seek neither offence nor exclusion when one can charitably and gracefully assume none was meant. Nope, he didn't mean it that, but it does seem a useful heuristic.)
Yup. I don't understand why people choose to rag on Microsoft when literally everyone else who has something to sell you besides the OS itself, does this. Maybe it's because Windows is from the era when MS was not a services company, and people, by force of inertia, expect windows to continue behaving as if MS is not a services company.
Do you need to stay in rural Indiana? Sounds like it would help you greatly if you could move to a place that offered ample opportunity.
Can someone explain why a bill is needed to ban TikTok?

Can’t the administration or a governmental agency (such as but not limited to FTC, FCC etc.) ban it?

I think that’s how it works in many other countries.
There are two problems here:
1) Hanlon’s razor
2) The author doesn’t understand health insurance data.

I’m not trying to excuse the other bad behavior, but within the data itself, he’s experiencing a combination of health insurers’ incompetence, the kludged up data models they’ve had to build to represent the output of the multiple generations of claims processing systems and other administrative processes, and the general mess that provider identifiers are. Every payor calculates values differently. Every payor uses different codes (beyond the standard CMS and CPTs). Every payor has different arrangements that are difficult to represent in standard schema, eg capitation in Florida, delegation in California, or the oddness that are Taft-Hartley plan.

There is a link in the article to a discussion with CMS. Another participant in the discussion works for IQVIA, a long-time claims data aggregator (and CRO and a bunch of other things), and clearly understands what’s going on. It would be extremely difficult to do this work at all without significant experience working with multiple payors’ data, which requires time and access, and pays well once you do have that specialized experience.
The UK police seem to be doing a disturbing amount of policing around what people say online.

I'm not sure if this is part of the anti immigration/brexit campaign or not, but it certainly caused me to cancel my previous idea/fantasy of living in the UK, so good job there!
I used to live in Brisbane which had two "one in 100 year" floods in relatively quick succession: 2011 and 2022. When the second one came there was some talk about communicating the risk more effectively. Instead of saying "one in 100 year" it would be better to say there is like, a 0.2% chance of major flooding every year.
Yes it does. It doesn't have a Constitution, but that's not the same thing as not having a constitution.
In response to your second statement: Technical people have been able to explain their ideas well enough to get them funded and implemented in a business setting. So we must ask, what is the difference between getting something done in business vs. in the government.

Politicians, in order to get reelected, need soundbites that can be understood within the average attention span, which is not long enough to capture the more subtle details. Though tech people might provide them with these details, they are often lost in the execution or distorted by special interest lobbyists.

Ideally, leaders would listen to the tech experts, translate that information into plain english, compile it into a
detailed and concise plan, present that plan to voters within the average attention span, and execute the plan faithfully.

Unfortunately, most of our current congress people, to use a metaphor, are not willing to 'rock the boat'. We need our politicians to put logic and reason above obedience, have high standards for legislature, and make this legislature in service of the nation.
> “not apple”

It’s a non-starter for some people for a variety of reasons.
> the Russian propaganda technique "whataboutism"

We're so fucked.
How did you create these onboarding and landing page videos?

What software did you use for voice and to create this videos.

I am trying this. Thanks
The strange case of ICMP Type 69 on Linux (2015)
Some people guard secrets that they think need to stay hidden for more than a lifetime.
You're completely correct.

What I wanted to articulate though was that Facebook isn't delivering an essential service. The consumer still has a choice not to use it. So if it as a company is known to abuse the information you give it then don't do business with that company.

Email, phone, and even your personal address you don't have a lot choice around not using. And that stuff also isn't really protected without encryption. (excluding mail by postal service, mostly).

Basically if you don't want communication to be intercepted and potentially used against you then do it in person inside of faraday cage. I'm mostly joking here.
Apple launches buy now, pay later service in U.S.
The emergent behavior is much more obvious in GPT-4 than in GPT-3.5. It seems to be arising when the data sets get extremely large.

I notice it when the AI conversation is extended for a number of interactions - the AI appears to take the initiative to produce discourse that would not be expected in just LLMs, and which seems more human. It's hard to put a finger on, but, as a human, "I know it when I see it".

Since injecting noise is part of the algorithm, the AI output is different for each cycle. The weights are partially stochastic and not fully programmed. The feedback weights are likely particularly sensitive to this.

In any case, it's early days. Check out the Microsoft paper, Sparks of Artificial General Intelligence: Early experiments with GPT-4
Never forget, you are the carbon they want to reduce.
I switched to GNU/Linux after reading essays from the "Free Software, Free Society" book. It took years, but I did it. Also, I'm using an FSF-endorsed OS on my phone right now (Librem 5 with PureOS).
That's not what the concern of the bill is and it's completely transparent. Just read the first 2 pages of the bill and you can see the concern is, in the words of the bill, a "foreign adversary" and their ability to "sabotage or subversion of the design, integrity, manufacturing, production, distribution, installation, operation, or maintenance of information and communications technology products and services in the United States".

It's quite simple - the US and China don't get along.

It's never been about privacy nor internal (to the US) security. It's very openly about cross pacific adversaries. Only meta-tech commentators have tried to apply some weird narrative of privacy.
> I’ll update this comment when I find the source

So... you gonna update your beliefs in light of the changing landscape? We have the source; you probably heard it when it was true, but things have gotten worse. Or are you just going to seek alternative ways of supporting your pre-existing beliefs?

Everything is politics, indeed.
Can someone explain or share references on how HVM (or GHC or similar computational models) beat optimized compiled imperative languages like C, C++, or Rust?

I took a quick look at the HVM runtime and can vaguely make out the shape of the memory and code that reduces terms. While it's cool to have a computational model that evaluates everything only once and avoids duplicating computation, if it does not compile programs to machine code to exploit hardware, then the single-core performance will be worse than C/C++/Rust.

Something that generates a few machines instructions in C/C++/Rust requires reducing a graph of terms - many CPU instructions for each reduction and many memory accesses that may have poor locality.

Is the idea that programs contain enough parallelism that is out of reach of human programmers so that HVM will make up for the single-core performance drag through parallelism?

Maybe I'm missing what the HVM compiler does?

Maybe hardware acceleration with FPGAs or ASICs could be applied?

In other words, can this beautiful and elegant computational model be implemented with efficiency that matches C/C++/Rust?
Indeed. And not if that layover is overnight and you ask to take your bags off Atlanta airport whilst you sleep at the hotel nearby, scrapping that ATL>Costa Rica leg the next day as no show.
That’s another trick I haven’t seen mentioned around: the extra ‘dummy’ leg at the end sometimes make a huuuge difference.
> iOS version never have these issues.

This is not true. It’s much rarer but there’s nothing special about iOS in this regard when it comes to abusing 0-days.
I'm talking about migration within US borders. It has nothing to do with immigration.
As long as it remains interesting and worthwhile. So far that has yet to go longer than a few days.
The issue is that tiktok told congress that US data was inaccessible in China but there is evidence of Chinese nationals accessing american journalist's data in China. Also I'm sure big tech lobbying is playing a large role here too, but that's the cover story at least.
Tinder/Hinge/Bumble should do the same.
GPT-4 readily hallucinates data[0], so that seems unlikely, though it also appears unlikely that it would make it any worse!

[0] https://news.ycombinator.com/item?id=34775853
Because it endangers all US business interests in foreign jurisdictions.
https://en.wikipedia.org/wiki/Foreign_Corrupt_Practices_Act#...
A startup says it made a jumbo meatball using the genetic sequence of a mammoth
1) I understand that the CCP bans sites and apps. Let's not sink to their level. (Instead, let's build more and more technologies to enable their people to get around said bans).

2) This is embarrassing. You don't like TikTok? Man up and compete. Don't ban it.

3) The complexity in this bills makes it reak of corruption. There will be winners and losers in this bill. The losers we can bet are 99% of the population.
I feel like no less than 10 years if the singularity doesn't kick in before that. Hardware and energy isn't progressing as fast as we'd like, and that is the main bottleneck. As in, imagine a world where we actually had the same computing power required to train (not run) GPT-4 in 1s in a phone? That kind of world is way beyond AGI and the cure of cancer IMO. Which is great, because it gives us a very objective goal to achieve these things. Sadly, I don't think we're nowhere near that. What was even the total energy consumption of GPT-4 training? Very hard to imagine computers will get that much better anytime soon. IIRC we have some kind of data that the a smartphone today has the power of the best supercomputer... of 30 years ago, right? Don't remember the source, sadly.
Was thinking this was something new. But is two year old article.