skysight-inc/HackerNews huggingface dataset:
Ask HN: Why are there no easy-to-use highly-available open source databases?
I'm a single engineer who wants to run a database by myself, no cloud vendors involved. I don't really care about scalability to a point, I don't really care about the API to interact with the database. I just want something that I can set up on multiple machines with minimal effort and have the loss of one of those nodes not cause catastrophic failures. I want the experience of something like etcd without being limited to a few gigabytes of data in the cluster (a Terabyte or maybe a couple hundred Gigabytes as a limit would probably be fine). I've spent a lot of time looking through the various options and wrote up my thoughts on why each of them wasn't a good fit.

Postgres: A great database but HA options either have far too many moving parts, require manual intervention, are proprietary, or are Fake Open Source. Seems like there's lack of interest in addressing this because companies that sponsor development generate a lot of value off complex operations

MySQL: This database has some odd behavior but that would be fine if it weren't Fake Open Source

MariaDB: HA Options are poor or abandoned

SQL Server: Proprietary

Oracle: haha

CockroachDB: Fake Open Source

YugabyteDB: Fake Open Source. Special shout out here for not even linking to instructions for how to build the database in the readme.

MongoDB: Proprietary, and even before the license change it was Fake Open Source

Cassandra: Not fun to run, and even though I said I didn't care about API I don't necessarily love how it works. But it comes probably the closest?

ScyllaDB: Fake Open Source

TiDB: Fake Open Source

Singlestore: Fake Open Source

FoundationDB: This one comes close but its beginning as a proprietary database really hurt its community, which is way smaller now than it should be. Could grow into something great if more folks got behind it.

etcd: not suitable for use above a couple GB of data

What do I mean by Fake Open Source? A project that has a large percentage of its contributors beholden to a single organization/entity to me is not really open source in spirit. I'm looking for a project where I can feel confident my contributions won't effectively end up behind some proprietary license down the line if/when the VC backed organization that primarily sponsors development decides it needs to protect itself from AWS. If there's an "Enterprise" product and the organization calls the source code for the main project the "Community Edition" or something like it, it's not Real Open Source. If a single organization shuts down and contributions fall off a cliff (https://github.com/rethinkdb/rethinkdb/graphs/contributors) it's not Real Open Source. There are lots of Real Open Source projects with great communities of users/contributors, but many of the newer databases don't have legitimate open source development communities behind them, in my opinion.

I've probably missed some examples. Mostly, I wonder why there hasn't been a general purpose open source database that does the operations stuff as well as the proprietary databases do. Am I missing something?
What types of software/tools do you plan to build? See a few potential opportunities in patient care and insurance.
Have you ever been bit by the bug of absolutely needing to get something done and finished just so you can keep riding the high of finishing something? You chase that feeling into the next thing, and the next thing, and so on.

They were doing that. Finishing and shipping anything feels really really good. If you're a small team and you feel a deep personal investment in the product then shipping becomes addictive.

Brandon Sanderson and writing is a similar example. Is the quality always there? No. But, the guy is very very good at finishing and clearly rides to wherever his passion takes him. His output is, in comparison to other authors writing in similar genres, incredible.

Also, games were much simpler and players had much simpler expectations.
Well said and I appreciate you being a good person. Have a nice evening
Not true at all. It is hard, but not impossible. Technologically it's very challenging because we don't have non-TC programming languages that are able to keep up with modern ones. But, for example, we have (safe) Agda which is not TC (all programs marked --safe should prove to halt) and you can write useful programs. Here, "useful" is a subjective qualifier, but I personally implemented many parsers in Agda (including a json parser) so for my definition of useful, Agda is one such language.
The thing is, if you WFH, you don't have any need to live in a high cost of living area.
Yeah looking at the commit graph for the keybase client shows a pretty obvious acquisition date..

https://github.com/keybase/client/graphs/contributors
Agreed, I was talking about Vanced, my comment wasn't clear. Revanced is a bit more gray, but still very dark.
Timeui, minimal profiling tool for the macOS
The real trick was sneaking into China to infect the market. Or, infecting key people whose travels would take them to the market.

Why the hell is this sh*t on HN?
We Need International Agreement on How to Handle These Dangerous Technologies
What’s your point? One whistleblower alleged the NSA of the US was illegally conducting bulk surveillance across the entire world, including its own citizens. A court later ruled this program was illegal and unconstitutional.

Are you saying we should discount all whistleblowers because it doesn’t fit some preset narrative?
Felt the same. Most complete example seems to be the installer for the language which is written in it. This covers some of the things you mentioned

https://github.com/cotowali/konryu/blob/main/konryu.li

and the generated shell script

https://raw.githubusercontent.com/cotowali/konryu/dist/konry...

There is also the website, heavily under construction, but has some more documentation on the language than the readme

https://cotowali.org/
> If it's Turing Complete you can't prove that it's hermetic

Turing Machine model doesn’t apply to i/o so it is a meaningless statement.

Bazel and Nix achieve this by sandboxing i/o no turing incompleteness needed (tho bazel is still purposely non-tc but for totally different reason than hermeticity)
I'd say Oracle has been a better steward of MySQL than Sun was back when Sun was a separate company.
Yes of course. If your goals are retro computing, yes by all means go ahead and use retro technology.

That's my point though: today, BIOS is retro technology.
Ah yes, having the client push messages back to the matrix network for later retrieval on other devices makes sense. It would however mean that the device running the bride should be either running most of the time, or that the hypothetical API of that proprietary service must be ready to be accessed from multiple clients at once, in case my home and work laptop are running at the same time, and both have a client side bridge, because usually only one of them is online.

Realistically I'd run a dedicated bridge on a home server or vps then, but I guess not everyone wants to do that.

But one step at a time I guess :)
Yep, your assessment seems broadly accurate to me. I was going to suggest Cassandra until I saw it in the list. I'm {interested in/optimistic about} FoundationDB too, although haven't had a chance to use it in practice yet.

Out of curiosity: what would your preferred choice(s) be to fit these requirements using existing proprietary products?
The reason I turn to piracy isn't that I don't want to pay, it's that there's no legal option for the manga I want to read.

Do you see a realistic path to a world where scanlations are unnecessary because any manga I would like to read has an official translation?
Clickable link: https://github.com/linkdd/tricorder
You can do that today, you could ship five Chess or Candy Crush games in six months if you wanted to. It's just a matter of where you set the bar for complexity and effort.
Yes the wooden table was the first thing that came to mind.

Oddly enough I have recently been drawing geometric designs, and then got the idea to draw them with a drawing ro bot. The current process is :

1. Draw design with ruler and compass

2. Photograph paper

3. Edge-detect image in Inkscape

4. Draw over the edge detected lines and save to SVG

5. Possibly convert to a graph, and run the Chinese postman algorithm over it

6. Convert graph to turtle commands and send to robot

So not far off. Also step 2 often involves a wooden surface...
I don't understand this take. If a woman doesn't want a kid she can get an abortion. If a man doesn't want a kid he better hope the woman feels the same or he is screwed. Women have almost all the power in this situation.
You ever wonder why people in old rust belt areas complain about "the globalists" all the time?
What do you mean for what?

Why I was at the bank? I had to cancel a credit card I never used, and had to show my latest ID.. Here in Europe banks are supposed to validate ID regularly now which is a real pain in the **. I don't get why either, it's not like I change identity very often :) It's just more distrust from the government I think.

No I didn't go there to withdraw money or to pay bills. Though it can still be done. In fact for some government transactions, like getting a new ID, it has to be done. You have to go to the bank with a government form, pay your 20 euros into the government account, and the bank stamps the form as proof that you did it. Then you take that to the police and they issue a new ID.

This is also in Spain by the way (like the article).

Yeah I hate this process too. But I like that old people can still do things the way they understand.
I loved Keybase and had brought so many people on to the platform through work. It’s really sad to see what the platform has become.
Prevalence of prostate cancer increases with age >70. Of confirmed cases ~2/3 are low grade, require only "active surveillance". Around 20% are intermediate grade tumors and ~10% are high-grade, aggressive cancer. The basic screening tool is the high-specificity PSA lab test, with biopsy considered the definitive diagnostic procedure.

Except for the highly malignant variety prostate cancer is generally an "indolent" tumor, that is slow-growing. So in the majority of cases chances are good that the patient would die of something else before the tumor becomes the cause of death. However prostate cancer is a leading cause of cancer death in men. In the US ~30k/yr men die of prostate cancer, so it's not a trivial condition.
Yeah. Laws can always be ignored or selectively enforced. They're worthless paper until something actually happens, and it will only happen after the fact, after people's rights have been violated.

Technology puts a stop to all that by making it harder if not impossible for them to abuse power in the first place.
> What I was wondering was: is there a better number? e.g., is there a way to calculate the amount of USD put into a cryptocurrency across a timeframe?

Velocity is a very important factor that critics of asset-wealth ignore.

When it comes to criticizing crypto-assets specifically, people just turn their brain off or are completely ignorant to how assets they respect work in order to hold crypto-assets up to a fictional higher standard. but even when articulating their standard its like "do they even know what they're talking about?"

for example, when comparing crypto assets to currencies, due to the "cryptocurrency" misnomer and skeumorph in the name, the illiquidity and relative few transactions in comparison to the marketcap seems like an important area to focus on, to them, while completely missing that currencies are broken down into 4 segments for this exact same reason. M1 being that tiny sliver used for transactions with M2 and beyond being illiquid allocations of the currency, the similarity of behavior ironically bolsters the currency aspect of crypto in what was supposed to be a criticism.

Many of these criticisms focus on the conversion to a fiat currency, and neglect the ability and reality of acquiring goods and services and investments directly with the crypto.

I see something like this over and over again.

These all factor into how one would go about valuing any particular asset. If a replacement for marketcap was sought after. But "dollars in over time" is not good enough, as it misses how liquidity can change at any moment, and misses the velocity of activity within any one crypto economy.
E.g. you can require all programs to halt, which makes the language non-TC since the language has a trivial solution to its halting problem. There are other ways too, but this is approach e.g. Agda takes.

Note that you can still build useful programs this way since such languages can still allow certain forms of recursion such as primitive recursion, structural recursion etc... You can also use sized types [1] to allow even a larger set of (halting) programs.

[1] This is a way of compiler statically analyzing the size of each type (i.e. how many recursive constructors necessary) which puts an upper bound on the number of recursions need to be performed.
Some people live in tiny apartments without even a balcony.
Quota Plan May Destroy Oscars
Surprised to not see an outcry about this site's lack of https support. I personally think the importance of https is overstated for casual sites like this, but the argument gets always ridiculed on HN.
If investing is truly a passion of yours, I would spend some time working for an institutional investor before you get involved in the business.

Most folks are better off using a boglehead style investing strategy and not working their regular job.
The functional stuff in Nickel is really pretty vanilla, and the syntax seems much more inspired by ML than Nix. There's some info in the repo's `RATIONALE.md` that specifically answers "how is this different/better than Dhall/JSonnet/Cue/Starlark".
Read everything on James Hague's "programming in the twenty first century", but start with "Living Inside Your Own Black Box":

https://prog21.dadgum.com/66.html

Libraries can be helpful. But at the baseline, what you have to solve is the problem of adjusting your dependency load such that adding new target platforms is a matter of rewriting small details, which means taking a language-designer's view of your system and saying "OK, if I need to automate the rewrite, then I need a compilation mechanism". Sometimes you have to change asset formats or runtime I/O mechanisms, but you can select what layer of the design you are aiming to standardize and go from there. e.g. records of chess games have a few defined formats. They don't need to track the assets used to draw the board, or the account information of the players.

If what you want is all of it: every pixel and frame of output the same, all input devices treated identically across all environments, then you'll gravitate towards a VM type of design, like Another World, Z-Machine or SCUMM, to name three well-known examples of such a design.
Check out telegram. It surprisingly awesome in so many ways. Ultrafast native client on all platforms, very user friendly, time tested, stable, fully packed with features etc.
It's really cool to hear that this has been develop to be able to store 'any website'. Previously, my understanding was that it was set up only for wikis.
I bet there are plenty of medium-sized cities, with good universities and small tech poles (because let's just say it, you want to be around educated people, and yes, it's nice) with much lower costs of living than where most techies end-up living.
Drug design software is 40+ years old but finally profiting from the staggering increase of computer horsepower and the kinds of statistical analysis that can be done. If any of the readers tire of ads, data collection or transaction software, these are some of the areas where "programming expertise" can work in harness with the experimentalists to (ideally) hasten the appearance of new drugs or at least decrease the failure rate of the current system.
Apple forced to allow sideloading and 3rd-party app stores under new EU law
Help me out here. I'm going to issue a hundred trillion MostExtraordinaryShitCoin (MESC). What's the bid for MESC from these AMM pools?
Did bizden / sales have to go through the same videos?
YouTube blocks WION for reporting Russian statement
RecordFlux
https://github.com/Componolit/RecordFlux
Good question; had to go to the supplementary materials of the paper in Science [1]. It is called a Bragg Mirror [2], which basically seems like a special coating consisting of layers of alternating refractive index. I don't know the details but this isn't _necessarily_ more exotic than, say, the antireflection coatings on eyeglasses or photographic optics. The supplementary material states that the mirror is 99.998% reflective in the wavelength under study.

[1] https://www.science.org/doi/10.1126/science.abm2543
[2] https://en.wikipedia.org/wiki/Distributed_Bragg_reflector
Seems to me the thesis that "mammogram positive" implies "possible cancer" is the faulty one.

In fact, "mammogram indeterminate" is a better diagnosis.
Must have been awesome.

My dad was a physicist and we lived next door to a physicist from caltech so ... I met Feynman twice.
Only if the Fn key is configurable. Nothing worse than having an entire key on the bottom left dedicated just to media controls as they often are.
You could still produce a AAA title with a small team (e.g 10) in the early to mid-90s. By 2000 an average team was about 100 people. There are exceptions to this in both directions but that was the overall trend.

In that period, you moved from mostly 2D to mostly 3D games so more complex code generally, plus the need for artists to do 3D modelling, texturing, rendering, optimisation, etc etc. Sound designers, composers, etc etc. AAA Games just got bigger, plus multiplayer. Hardware got more powerful (sound and graphics cards) so there was a desire/demand to make more use of that hardware. Generally, it all became a lot more "Hollywood".

There are still great games made by small teams of people in a short period of time. But to hit the AAA mark you need to be with a big studio and have the marketing budget behind you. Again, Hollywood.
> it's 1/3 of the jobs, so chances that both can do WFH are not that low

It helps that those are not multiplicative probabilities. They apply to each job, so each partner will have access to ~1/3 of the jobs, totaling an access to ~1/3 of the total jobs.
Using calibrated Bayesian reasoning can certainly be an innate trait. Some people seem good at making good predictions and assessing uncertainty of the data and knowledge they have whether they've been thought or not and it seems like an unusually difficult thing to teach. Will you make that much better predictions even if you read all of the lesswrong.com cannon?
Yes and no.. assets were simpler, games were simpler.

But they had to write a lot their tools to create the assets. They had no middleware, no engine to license. They had to write stuff in assembly. The computers they used were super slow and you could crash them so easily. At some points in iD's early history they were smuggling computers from Softdisk out of their offices and working over the weekend and then taking the computers back Monday morning. The tools were terrible. Documentation was a lot harder to come by. A lot of the people in iD at the beginning were also juggling a day job, they were moonlighting making those earliest games. IIRC Wolfenstein was the first one they worked full time on.
id Software went independent after working for a company named Softdisk making a monthly floppy disk called "Gamer's Edge":

Softdisk is most famous for being the former workplace of several of the founders of id Software, who worked on a short-lived game subscription product, Gamer's Edge. Gamer's Edge was a monthly[3] PC game disk started in 1990 by John Romero.

Having a monthly deadline making MS-DOS games on floppy is why they were so prolific at that point. None of the Cmdr Keen games are much different from each other, and there were 6 of them.
Intel Virtualization and Apple Silicon
This has been my assumption as well. You'd think this would be a project the Gates foundation would jump on though
This guy types. Much more eloquent than I am in expressing the importance of a mechanical keyboard.
Doesn't mean you can't run that Ubuntu instance in a VM on the same machine. Wait does it? Can you run VMs on these guys?
Non-keybase user here. What has it become?
Maintaining test automation is equivalent to maintaining the code base. A dedicated qa team should be doing that as well as manual testing, infra tests, etc.
We had a dedicated qa in a project, and when he was taken away twat automation pretty much died. Nobody maintained it.
OK but I'm responding to a claim about BTC.
Yeah I was thinking the same thing. This would be a cool product.
As much as I appreciate Python, I much rather use Perl as a replacement for complex shell scripts. Perl 5 is ubiquitous, it has been stable for the best part of the last 20 years and it's unbeatable for all those tasks that involve heavy text processing.

Python is good for complex, structured applications, but shell scripts are not that. Shell scripts are glue, and there's arguably no better glue than Perl.
I love Telegram, but I want to state it clearly: the default chat scheme isn't end-to-end encrypted, which Keybase is. Therefore, it's not a drop-in replacement.
Okay, so definitely cryptocurrency shonk then.
> I don't feel compromised.

I get it.

It’s a long road from “dont be evil” to illegal wage fixing, mass ad fraud, omnipresent dragnet surveillance, and cooperation with totalitarian nation states; all of those with compunctions are long gone, the rest are like you.
there isn't much citation needed; it's common practice at many dealerships for certain eras of cars.

the 90s era hondas up to about 2001 use various key-turn-rituals to enroll/program keys into the immobilizer, the later ones use the Honda HDS system which is just a specialty Toshiba/Panasonic ToughBook with an obd dongle and special software.[0]

I've enrolled keys myself for my 04 BMW with bootleg 'BMW MODIC' and 'BMW Rheingold' software packs pirated from The Pirate Bay.

You don't need existing keys for either system.

The trick (used to be) at the time that BMW keys were difficult to cut, and the key cutters were well controlled. This isn't the case any more, and in reality if a key was the deterrent you could always just program an immobilizer chip from another key, tape the key/chip to the column, and then use a pry bar and screwdriver to break the key tumbler and turn the switch without a key. This is neither rare nor hard to do -- and it used to be the defacto way to steal pre-immobilizer Hondas (breaking the column/tumbler, that is).

It was common enough that an in-joke at the Honda dealership I worked at was that a flathead screwdriver could be referred to as a 'lazy CRX key', a majority of those era cars encountered were so worn that a flat head would turn most of their tumblers by the time I got to work on them.

[0] : I was a Honda tech from 07ish to 09ish
I’ve been surprised recently by the availability of plugins for libpurple (the lib on which Adium is based) which supports Telegram, Signal, Facebook …
12 bucks is about two or three days of work where I live. I will do everything that is in my hands that won't get me arrested to save money and improve my quality of life. Google does the same, so why shouldn't I?
If you want to see something really interesting, there's the occasional time when you can see crews working on an open NYC Verizon (former NY telephone, then NYNEX) manhole in a street hauling out defunct 600 and 1200 pair copper and associated splice cases...

Looks like this, but old and decrepit and dirty.

https://www.google.com/search?q=1200+pair+phone+cable&client...
https://wire.com/en/ is what i have used for years. YMMV.
rent control