skysight-inc/HackerNews huggingface dataset:
The author is obviously technically capable, but makes some ridiculous/ignorant claims about XML and Character Encoding.

Anyone who works with texts today as a programmer who doesn't understand the basics of character encoding is likely in trouble, I find it quite ignorant to claim "Only 90s Kids Will Remember".

Also, most modern XML parser will handle/detect the correct encoding for you, so whilst you should understand character encoding, you don't need to do the heavy lifting yourself.

There are also inaccuracies in this article, the author says:
"Instead, we’ll use a streaming approach (the XML folks call this “SAX”, for reasons I didn’t care enough to research)."

Erm... actually there are multiple streaming approaches available. SAX is just one of them (which was pioneered in Java).

The author also says:
"I decided to go with the quick-xml library because it had good documentation, good performance, and roughly the API I was looking for."

Erm... but... quick-xml provides a StAX like API, NOT a SAX API. The former is pull and the latter is push; quite different things, with different performance characteristics depending on your needs.

I guess when everything looks like a nail to the author they reach for the rusty hammer. The amount of time and code they put into solving their problem in a limited way is rather staggering.

I would have suggested using - the correct tool for the correct job! If you have a small amount of XML (38GB isn't big these days), then work with it as XML.

Alternative options would have been to:
1. Load the XML document into a Native XML Database, which takes all of about 10 minutes whilst you make a cup of Tea. There are several Open Source ones if that's a requirement for you. Write probably less than 10 lines of query, and voila you have your CSV.
2. XSLT 3.0 has a streaming mode. You could write a small amount of XSLT 3.0, execute it using an Open Source processor from the command line (e.g. Saxon), and voila you have your CSV.
3. If you wanted to go really fast, and can spare a few dollars, they could have rented a cloud instance with more than enough memory for 1 hour, stuck it all in RAM and processed it quickly using whatever temporary scripting language they liked...

If there was an article down-vote button, I would be using it!
> Far more importantly: you get a declaration of the encoding right at the start, so you don't have to guess.

True, but it was a lot worse than that. XML forced even the worst programmers to at least think half a second about the existence of the whole encoding concept.

Some horrors from the past: receiving so-called text files generated by concatenating multiple exports. Every chunk had a different encoding, and there is no hint when the encoding changes mid-file or what was what. As most of it is ASCII, it seems to work and people don't even understand what the problem is.

Another one: XML-support by pasting a CSV file between the beginning and end tag. <data>...CSV goes here...</data> . Of course, the CSV between the tags is not XML,violates escaping rules, and has no defined encoding.

One of the worst I ever received went like <textfile><byte value='65'><byte value='66'>.... </textfile>. Now have fun converting every element to an individual byte, then finding out the encoding of the resulting text file.
If you're looking for a low-cost alternative to Adobe Photoshop, Illustrator or InDesign, take a look at the Affinity suite of apps. Although they don't match Adobe apps feature-by-feature, they are professional grade apps with a lot of features. They are cross-platform (Mac and Windows) and they even have iPad versions of the Designer and Photo apps. The Affinity apps are not based on an online subscription model.

The Affinity Spotlight website showcases what people are using the apps for and the type of content they are creating:

https://affinityspotlight.com/

I have all the Affinity apps and don't miss Adobe one bit, but the Affinity apps are a bit clunkly in their UI (but far less clunky than Adobe's apps).

After Effects (AE) is the only Adobe app that has no real rival. A new motion and animation app called Cavalry recently launched but it's far too early to see if it can rival AE. It also uses an online subscription model (https://cavalry.scenegroup.co/).
I for one am happy I don't have to manually write database persistence layers any more.
I mentioned this in a HN post in the past regarding standing desks, mentioning that I'd created an over-engineered standing desk controller (using a Raspberry Pi) over lockdown as as little fun project. Somebody asked about a write-up for it, so here it is
Not all writing is thinking, and there is definitively more to thinking than writing.
I think you missed the opportunity to re-enforce a few more times how much you dislike the genre of music in question. Just so the reader knows.
So 0.00000343477% IS below 0.00%.
I am yet to find a service that properly translates to European Portuguese with proper use of pronoums, so no big hopes here.
GPT3 Function in Excel
While "practice makes perfect" is not a very new idea, it is still nice to be reminded about it from time to time.
But your IDE could understand your code for years with:

/* @return MyObject */

Returning an array with unknown items has always been bad practice imho.
This is a bit off topic but should be expect the same regulatory dance with Airbnb after IPO?
> It is not so much the fact that many projects host their projects on GitHub, it is the fact that many projects haven't secured the code outside of GitHub!

If I'm the maintainer of a project of course I would have backups (even if I trust Microsoft because GitHub could be hacked). And for important open source projects there are probably many people who have a copy of the repo.
For me (neck and upper back pain) the trick is not so much the model of the chair but the presence of neck and head support. One of the standard Ikea models (Markus I believe) offers that and gives me essentially pain-free computing where thousand dollar chairs without that feature cause agony at the end of a day.
I don't think it's so much about density as it is about planning. Even in a massive area, the correct way to distrubute people in it is so that they can access what they need with ease. Let people live next to the store, their office, the bus stop and so on. How far apart clusters of people are will determine the overall density. Density doesn't say much about whether people are clustered or more evenly spread out however.
Huh, no I haven't come across dbt. Looks interesting! Jinja'd sql scares me a bit, but to be fair the string interpolation in my lib here scares me as well :)
I'm sure I've read an old Joel on Software post somewhere about how Adobe/Photoshop got big in the first place, but I can't seem to find it anymore.

The story as I remember it went like this: X had a practical monopoly on professional image editing and printing, everything else was for amateurs. (Was that Quark? not sure). Then X did something that pissed off a lot of their users, and within a few months everyone had pivoted to the newcomer, Adobe.

The moral here is that if you've got a monopoly or at least a comfortable majority in an area of software, you shouldn't just be worried about someone else making a better product - switching is effort and people don't like that - you should be very worried about doing something that annoys your existing customers enough that staying but adapting to the new UI / pricing model / etc. is perceived to be as much effort as switching.

(Whoever authorised the new UI for firefox on Android might want to think about this too.)

What I can find is Strategy Letter III, saying that in this situation the best thing for a challenger to do is make sure their product is "backwards" compatible: https://www.joelonsoftware.com/2000/06/03/strategy-letter-ii... Example: How Excel (that Joel was working on at the time) grabbed the market from Lotus 1-2-3.

When people like @prox below say things such as "I switched to Affinity completely, and I also slowly see some people in the industry change.", I'm starting to think that Adobe might be ready for its Lotus 1-2-3 moment. Not letting people install their old Adobe software they've paid for is the best incentive to try out something new that I could think of.
that's why I like the "shape" type in Hack - the ease of returning an assoc array, but with field and type checks enforced.
Power Platform French Summit
Why would you have any reasonable expectation to have privacy in your house?

Americans do.

"The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized."
Spring was first MVC framework, but Laravel in PHP and Django in Python were inspired by Ruby on Rails.
A small but complete JavaScript engine
Can a fellow geek knowledgeable in PHP gives me a few pointers?

I have inherited a 12 year old PHP app written by 2 interns. The code was written with Notepad++ (no IDE), no comments except when they copy pasted something from the internet, most variables are single letter and it's the biggest spaghetti bowl I have ever seen.

To add insult to injury I have no experience with PHP (apart from peeking at this code to fix a minor bug here and there). To be fair the app works OK and has done the job for all those years.

This app has never been updated so it's PHP version 4 or 5 I think, it runs on an old Debian 5 VM, it's not facing internet, internal use only. It was supposed to be deprecated last year, but now given our budget (or lack thereof) and the sheer incompetence of our new provider I don't see it happening soon.

Is there a way for me to have an IDE with a kind of "compiler" that would help me convert this to PHP 8 so I can migrate it on a more recent VM?
Of course there's 0 budget for this.
And I’m using Adobe software from Photoshop 6, and upgraded with each new version to gain features and performance that wasn’t available with previous releases. I wasn’t happy with cloud option, but it’s still cheaper than buying boxed versions of Adobe soft. And no, not even one professional will use CS for anything other, than amusement on old hardware.

> Today, I’d be hard pressed to choose CC over a pirated copy or other alternative, and I’ve known a few designers that stuck with cracked copies of CS.

He’s not a designer, his a pirate. If you do a professional work, you pay for tools you use. Simple as that. If you want to say, that they are pricy then use some free alternative (they existed, and are great), or cheaper pro software like from Serif.
> It would have left us with exactly 1 format for data transfer and configuration that had XMLs advantages without its verbosity and enterprisyness

Hmm.. that works for data centric applications, but what about document centric applications? An advantage of XML is that it can do both, whereas JSON has no support for mixed-content.
I wonder what exactly causes "instability" when overclocking/undervolting in a processor. Transistors don't have enough time to switch, but it matters where it happens.

If there's an error in e.g. floating-point unit, it would just cause random errors in the output, but the game would probably keep running. That might be completely fine, if it's rare and you could undervolt just that unit a lot. Especially in somewhere in the graphics pipeline, where it would just generate visual noise. Also, I think energy consumed / heat is proportional to voltage squared, so it has a big effect.
It's a different culture. They go around the problems differently. Also the way the write programs seems simpler and leaner (composer in mind)... probably not ultra safe but a good enough ratio I guess.
I've found the reverse to be prevalent for me.

Instead of all hours being work hours, once I'm done with the tasks I've outlined for the day and see that there's not enough time to pick up something new - I just wrap the work day up with some time to spare.

If I was in the office, I would've been forced to sit in front of a computer, even if I'm mentally clocked out.
This sounds interesting. I haven't heard of "tree-folds" on XML before - do you have any links to further reading and/or examples?
Because a PhD is useful for a lot more than getting a permanent academic position. My first "proper job" was at a research division of a big corporation. They had ~20 PhDs when I started and expanded to ~100 over 3 years. I don't think any of those people would say their PhD was wasted because they didn't get an academic job at the end of it.
They do kill projects, but these are not really comparable. People paid directly for Google music. They also did not really kill it, but rather replace with an inferior version aiming at the same business. This is not even comparable to the chat/hangouts/meet migration.

I can find a reason, even if I don't agree with it, for most of their killed projects. This one is just... different.
Right, sorry for the confusion. Though "machines" makes me think of the self-replicating machines concept, so "von neumann architecture" would be a better fit.

https://en.wikipedia.org/wiki/Von_Neumann_architecture
I'd need to check if this is still the case in PHP, but Reflection was orders of magnitude slower just a few short years go for cases such as ReflectionClass->methodExists() vs method_exists()
So I'm probably not going to Egypt as Telegram is the main (in fact - the only) way communicate to members of my family remotely.
Yeah my only issue with how this played out is that the proposition on the ballot written by Uber/Lyft gives ride-share companies a specific exemption, rather than just repeal AB5 for everybody. I would be a lot more forgiving towards they're lobbying effort if it was focused on fixing a bad law rather than just helping themselves only.
As someone in a similar position, my first impression is that this is going to be very useful in saving time coming up with big sql monstrosities. I will be following your project with interest and hope to be able to use it in the daily grind.
This might be a valid statement for leaving the EU and US by land in some places of high trust (Switzerland, Norway) but certainly not a general rule by all means and certainly not by any other means (sea/air/train) where your passport or ID will be checked by at least the airlines and other companies (who act as deputies of regulations) and it will be matched against passport validity databases and other various databases (no fly lists, wanted criminals, interpol ...). And I know of several (if not all) EU countries where you have to scan/show your passport to access the departures gates in Airports (into an automated gate that scans it). Some even require such scan just to enter the airport since the last wave of terrorist attacks in 2015-2016.
https://ibis-project.org/ is another great alternative. It provides a fluent/linq style api but doesn't abstract the SQL away too much.
IDE: Visual Studio Code with the PHP extensions, free.

Compiler: you are looking in the wrong direction, PHP is interpreted. There were some compilers, but it's not what you need.

Converting to PPHP 8: do you need that? what are you trying to achieve? You can keep the app another year or so it it is working, don't fix what is not broken. There is no way to convert to 8, you need to rewrite it, PHP become more object oriented with each version and that is a paradigm shift there is no converter for.
What language _isn't_ trying to be Java these days? What the old jargon file described as "bondage and discipline lanugage" applies to all major programming environments now, as Java proved that you can actually commodify programmers quite well and both old-school enterprises and the SV juggernaut need their harvest.

C#, TypeScript, Go, PHP -- they all converge on the same methods, tools and workflows. Tech choice these days is about what you want to have on your resume.
Hmm, interesting and good to know, thanks. (and damn, subqueries are awful to read :( )
If there was appetite, it would be straightforward to make `csql` work by generating subqueries instead, might be something to add to the TODO list!
> than what you would from a course taught by nobel laureates to a room of top-performers

Do you really think you're getting a full course taught by a Nobel Laureate? Do you really think the room is full of top-performers? Sure, the average might be better than a state college, but not much better (with some exceptions, surely)
I went swimming in the sea yesterday evening, it crossed my mind that there was a non-zero chance I might die ..

If I wrote online "Gonna go swimming in the freezing sea, in the dark" I can imagine my death-day estimate could easy move by many many years!
What’s Going on with Julian Assange?
Kentucky Fried Chaos Engineering
I'm reasonably sure that it was a pottery class (or study) in which students/study participants were asked to produce either the best piece they could think of or as many as possible. I think it was actually a study that I read about.

It might need some internet sleuthing to find it. I'll try later.

I do not think that it is surprising that practice improves skills though (well, except for people with an exceptionally fixed mindset ;).

(For today's lucky Ten Thousand, "fixed mindset" refers to https://en.wikipedia.org/wiki/Mindset#Fixed_and_Growth_Minds... )

(For today's lucky Ten Thousand, "today's lucky Ten Thousand" refers to
https://xkcd.com/1053/ )
Ooooh, that's going on my 'to play with' list, thankyou.
You responded to

> He either had to pay $55, drive over 100 miles to appear in court, or not pay the fine and become a criminal.

with a comment listing some unreasonable alternatives. You didn't list the reasonable alternative of making a written response. My disagreement was with the situation presented in your comment and the one you were replying to; I live in a different country and don't know New York's actual legal system.

> I never understand why people can't understand: it's a rules based system, if you don't have to follow the rules, no one does and then it collapses. Americans in particular seem to really struggle with this but its just life...

I don't think anybody here (and only a very few ideologues elsewhere) is arguing against a rules-based system, or for people to disobey laws out of principle. I am arguing that the rules must be fair and reasonable, that people of all levels of intelligence and income can afford to follow. It would be unreasonable to force people to travel long distances or pay for an hour of a lawyer's time to address police mistakes. It would also be unreasonable to expect Pennsylvanians to know how to dispute a New York ticket in writing without clearly informing them (I don't know if that's what happened here, it's just an example).
I wonder how the Airfish is progressing. I haven't seen an update in ages. https://www.wigetworks.com/
The user could enter an email-like identity, where the domain part determines the provider.
> Anyway half of the US has a vendetta against the environment.

Citation please
Duplicating a record with its relationships in Laravel
But a really big archive won't be usable unless it is in the Transcend and who knows what it might do then?
XSLT 3.0 streaming with Saxon requires Saxon-EE (closed-source Enterprise Edition), see https://www.saxonica.com/html/documentation/sourcedocs/strea...

Are there Open source xstl 3.0 processors with streaming support?
Your example of multiple sorts is problematic.

> SELECT * FROM (SELECT users.* FROM users ORDER BY users.id asc) ORDER BY users.id desc

A sort should never (for some value of never) appear in a subquery because it's meaningless; it can't affect the result. In tsql it's explicitly checked for and reported.

Not having a go, just fyi.

Edit: now I'm more confused by that. The bracketed subquery doesn't have a name but it's apparnetly called 'users' because the last clause is 'ORDER BY users.id desc' - but that's illegal. Again tsql correctly rejects that (once I comment out the illegal inner order by).

Edit 2: sorry about this but FYI, I'd expect the inner sort order to be disregarded anyway, and assuming this is mysql it explicitly is.

"If ORDER BY occurs within a parenthesized query expression and also is applied in the outer query, the results are undefined and may change in a future MySQL version"

https://dev.mysql.com/doc/refman/5.7/en/select.html

(Actually, what the heck is it saying? An outer order by in the presence of an inner order by is undefined overall??)
DOJ’s Crypto Framework Is ‘A Complete Disaster’ for Digital Privacy Rights
Isn’t the fundamental problem here that old people are living a lot longer than we thought they would? So you have granted tenure (and pensions and retirement) to people who are gonna seek it for longer than you thought they would so you don’t have place and money for younger people. The same thing applies in the job market, where there is literally no place for someone hired in the last decade or two to grow in management because no one above them in the hierarchy is retiring. Then you have government jobs that are paying out pensions for way longer than anyone expected them to.

Over and above this the the government will not dare to cut any policies that would adversely affect old people because they are the only age cohort that votes reliably. What they will try and do is mess around with social security, which, ironically, is one of the few programs that both predicted the baby boomer issues and accordingly created a trust fund to handle that and is actually paid for.

All these costs are being picked up by younger generations and all they get in return are the olds complaining about why they can’t just do a part time job to pay for their college degrees like they did in the 60s (nvm that the changing pricing and salaries mean students today could work multiple full time jobs and not be able to pay their college tuition without loans while the boomers complaining could pay their college fees with their weekend burger flipping positions with no debt).
How did you get YouTube-dl? That was the second thing I tried but I get SSL issues with both wget and curl.

I also tried installing some npm packages but ran into permission issues.
Ad1. In my opinion it's the variety of problems an engineer has solved that matters. There are plenty of crud developers that are helpless whenever a more complex problem occurs, even though they did hundreds of crud pages in their life.
The news a couple of months ago was that it had been accidentally stranded on that beach: https://news.ycombinator.com/item?id=24305279

> Today, the Ekranoplan–the only Lun ever built—lies stranded off a beach in the Caspian Sea. This summer, the craft was in the process of being towed from a nearby naval base to a museum when something appeared to have gone terribly wrong. Instead of making it to its new home, the Ekranoplan ended up stranded in shallow water, where it's drawing visitors. https://www.popularmechanics.com/military/aviation/a33808381...

However, this article today claims that the beach was the intended destination:

> "Lun" will be the star of Derbent's planned Patriot Park, a military museum and theme park that will display different sorts of Soviet and Russian military equipment.
Construction of the park is expected to start later in 2020. For the time being, Lun will sit alone on the beach.

Checking the famous photo article that was already linked from that article two months ago, it seems that the “something appeared to have gone terribly wrong” was just speculation by Popular Mechanics:

> This summer, Russian urban explorer Lana Sator, 31, read about the ekranoplan being towed by sea to Derbent, a city in Daghestan, where it is due to be moved into a "patriotic park” and put on permanent display.
https://www.rferl.org/a/photographer-sneaks-inside-the-legen...

Derbent is on the west shore of the Caspian Sea, close to Azerbaijan: https://www.openstreetmap.org/search?query=derbent%2C%20dagh...