skysight-inc/HackerNews huggingface dataset:
I think most platform TOS say "forever". No sane lawyer would allow you to put "2 years" in the TOS.

What if part of the user text is stored in a backup, and the backup is misplaced and it is not deleted in time? What if the file is marked as deleted but it's not zeroed and erased from the disk?

What if the deletion time coincides with a big outage an it is delayed? Or the don't want to delete anything until they are sure all the databases are consistent?

What if the text is in one of the cache just at the time of deletion?

What if the user saves the text as snippet and later the user remove it? When do you start to count the two years?

What if the user submit the sample as a suggestion for an improvement, and it's copied in the automatic rules? What if one of the developers do this without the permission of the user? What if multiple user make the same suggestion?

What if the text is copied in a screenshot in a marketing material? (Probably not applicable in this case.)

What about legal request to track some users? What if the user sues you and you need the info for the trial?

Just put "forever" in the legal terms, and if you want to be nice try to delete at approximately 2 years.
I think this is some kind of supernatural "free will", and no mechanism has been found for such a thing in science. It's associated with mind–body duality, souls, homunculi inside the body, that kind of thing.
Airlines Were Supposed to Fix Their Pollution Problem. It’s Just Getting Worse
>If you provide a platform you should not be allowed to compete with those who build on top of that platform, because it wouldn't be a level playing field.

Isn’t this exactly what grocery store chains do?
No, still not the opposite. If you only have public water systems than only public water systems can fail.

Private water companies don't even make sense, as they can only exist as monopolies with the only incentive of squeezing as much profit as possible forever, pushing boundaries on water and service quality to pocket more money, lobbying for weakening of quality standards, etc. It cannot possibly be cheaper and better than fixing regional quality problems through very competitive water filtering market individually (you do know how cheap filtering systems actually are today?).
Just how much ink needs to be spilled about how California and Bay Area are doomed. I have probably read a hundred article over the last 15 years predicting the demise of both, but in reality the exact opposite happens as California and Bay Area economy continue to crush the rest. The gap between SF and rest of the country when it comes to venture investment has actually grown as the cost of living has gone up. For example, in 2018, $45bn was invested in Bay Area startups whole only $1.5bn was invested in Austin and even less in Denver, Utah etc. It is not even close. Weirdly, the other fast growing tech regions are similar to California - expensive, dense, and left leaning (NYC, Boston, Seattle).

Maybe this time the prediction will be true, but I am willing to take a large bet against it. And this is because several things, but the primary reason is that California continues to overindex it's policies towards people, while other competing states overindex on business friendliness. But in the knowledge economy, a thirivng talent market is far more relevant to a business than some tax breaks. If you can't execute and be profitable, what's the point of a tax break. California optimizes for people through various mechanisms - non competes, inclusivity, diversity, environment, labor protection, safety net etc. If a place is great for people, it is great for business.
Honestly what should your salary be if you have domain expertise and are good with data science tools (and tools surrounding them like git, CI/CD, etc.)?
One more condition normally, I think -- it shouldn't be relevant to your employer's business. That's a biggie, and makes it very dangerous to do any kind of programming in your own time if you work for a big company.
Right, but then

wget ftp://cddis.gsfc.nasa.gov/gnss/data/daily/2019/brdc/brdc0690.19n.Z

seems much simpler
I used to be in this business. It’s a good practice to think of everything in terms of probabilities alongside capacity/liquidity. If an all cash merger hits the news, there’s a race lift prevailing offers on the one stock as there is a much better expectation of value due to the news.

Now let’s say there’s negative news on the Chinese economy. Someone could be running an algorithm that sells down companies with Chinese exposure. But, inherently, there likely is a much lower probability that this news is uniform or material across companies with China exposure. A strategy like this likely would get put on small positions and get in and out quickly as it could get run over depending on how humans process the information and make bigger discretionary bets.

There definitely is a risk too all of this. Roughly 10 years ago, somehow an old united Airlines bankruptcy article hit the wire and the stock sold off hard. Eventually, United released a statement saying the article was innacurate. This isn’t typically the type of strategy you would make large, single bets with, or hold the positions over a long period.
regarding #3 -- I kind of despise the Valley in this regard. When you are a tech firm not in the valley, inevitably you get a middle-manager type that comes in and tries to emulate what the latest hip SV firm is doing to boost your street cred. When I look around the offices I have worked at in the South, and up in D.C. in the 00's and early 2010's, we have never had a diversity "problem". There was plenty of diversity, not just male/female diversity, but also multicultural diversity, and diversity of thought and opinion(I've actually worked with some conservatives! They are real!). SV to me pushes their "we have a white male broculture" projections out, so suddenly now it's everyone's problem. Now we have to be explicit with our "diversity" initiatives, despite the fact that we were already much more "diverse" than the SV intellectual and social monoculture you have now.

Anyways, just kind of ranting, but yeah. SV's problems in tech somehow always branch out to be everyone else's problems that are in tech, at least media and high profile twitter accounts and bloggers would have you think that, but it isn't really so.
Running a neural network on paper abstracts can probably get you 80% of the way to removing the need for human submitters — figuring out the remain 20% is an exercise for the reader :)
At the same time they are taking away free charging, seems like a double lie.
> > the rest of the world times their orders taking transmission latency into account.

> Could you elaborate on this please? Say I want to place an order for 100 shares of XYZ at $100.00 per share. I know my trade is going to take 50ms to get fulfilled. What do I do with this information? Predict where the price is going to be?

Simplistically, If you're making an order that's too big for the order book on a single venue to get a sensible price, you might chop it up and buy some in NY, some in London and some in Tokyo. Let's say you're sat in NY: you can work out that it'll take your order 1ms to hit the NYSE, 150ms to hit the LSE and 600ms to hit the TSE. So rather than dumbly hitting 'transmit' on all your orders at once, you send the Tokyo order first, then then the London order 450ms later, and then the NY order 149ms after that. That way, your orders (and the public information that your making those orders) will hit each venue at exactly the same time.

This is common practice for any broker or institutional trader.
I was an engineer for a high frequency trading desk at a firm not mentioned in this article.

Flash Boys is a great book and we required our new hires to read it as it got a lot of things right.

Front running is illegal now, but arbitrage opportunities still exist between exchanges. Being fast is still an easy way to make money.

You're right about dark pools though. Those are the future (honestly they are already up and functioning) and they hide trading activity very well.
Really like the clean design of the site!
Fiber is about 0.6c and the cables are much longer than direct line of sight.
From an outsider's point of view (still a CS student):

If you are a CS/Physics/Math person who loves to optimize systems then this is a good way to get paid for doing exactly that I guess. Many other problems/industries just require good enough solutions, with HFT it seems that most optimizations are worth it, which makes it so much more interesting.
"Substantive" has to do with the signal/noise ratio of the original comment.

As for your last sentence, that's dismaying to hear. If you're saying it in good faith, suggestions about how to do better would be welcome. Keep in mind that it's my job to post moderation comments and try to coax people into following the site guidelines. (In case it helps, they're even more tedious to write than they are to read.)
> > market makers

> How many market makers are there?

However many the market will support.

How many shoe shops are there?
If it causes a problem where solving the problem once doesn't solve it in the future, and where the fixing is a total waste of life that doesn't teach you anything, or where an easily solvable problem is compensated for in higher level abstractions, it's probably "tech debt".

Personally I would consider a stupid name or out of place capitalization to be tech debt of it leaks out into config parameters or something such that it can confuse more than just the people looking at one isolated chunk of code.

(Or call it "tech necrosis", because americans with college debt tend to view "debt" as an inevitability in life and that's not a helpful attitude for getting stuff done)
Market makers need to be able to close arbitrage opportunities or they won't participate in the market. Speed matters just as much to them as it does to the prop traders.
Good question!

Note that any answers should be HFT specific. That is, if they apply equally well to trades on a second or minute scale as on a millisecond scale, they probably don't answer the question.
I didn't see a price anywhere. Did I miss it? How much do they plan to sell these for?
Won't those tissues spread more than one disease?

polyvalent vaccine <grin> - much the same as 'Triple Antigen' which used to combine diphtheria, tetanus and whooping cough vaccines.

I initially laughed at this - it's definitely a money making scam - but it has a hidden basis of truth. It's certainly not a lot different really from fecal/faecal transplants.
The post is not spoiled.
Also advertising....
> Although I would never admit to this publicly I am a sensitive person

you say it as if it's a weakness but why? Rhetorical question, but a valid one.

> Getting chewed out in an aggressive tone, getting yelled at, or chewed out for extended periods of time can significantly impact my productivity

Mine too! That stuff can cut deeply; don't imagine it washes off me just because I can usually handle it. I've been left literally shaking after someone's had a go at me, and it has added to my long-term mental health problems. It affects my work very negatively - it deteriorates. I remember wanting to punch myself in the face on the train on the way home I was so upset, less than a year ago. It should not happen.

> in spite of my efforts to toughen up

I really do not know if that's something a person can do to themselves. Or should? I don't mind if I fuck up and get it in the neck but if I get it in the neck cos someone's rowed with their BF last night then I shouldn't have to deal with it at all. But that's life I suppose.
I don’t have any statistics but I have an anecdote.

I built a trading platform ~6 years ago that had a 100 microsecond tick to trade (a term that means from the time a market update hits your nic to the time an order leaves your nic).

We knew then that we were too slow for large swaths of the HFT space & wouldn’t even attempt those trades.

I haven’t kept up with the industry but I’d guess tick to trade times at the speed edge are in the low hundreds of nanos now. In an environment like that a 1ms time improvement is out of the capabilities of your software/hardware engineers and can only happen at the network.
Porsche Taycan 2020 decides its date to declare war a sophisticated American car
Paying lower taxes is generally another one. Employees generally can't deduct much.
Burning plastic is perhaps not ideal. But I'd guess it's better than pretending to recycle it while actually dumping it somewhere.
Driving is the worst. I want to be able to walk everywhere, or at least being able to take public transit.
The Bill Gates software story makes me think of the first iPhone demo: they were ready in case the device in Steve Jobs' hands crashed. Although I suppose there's a difference between unfinished software and impossible hardware (what Theranos was promising).
Speaking as a Montanian, it’s been done before.

https://en.m.wikipedia.org/wiki/RightNow_Technologies
> How does a flash crash hurt the average investor?

Maybe it can trigger "stop loss" orders?
Apple Maps Flyover Reverse Engineering
The dice is fully loaded in favour of HFT pioneers and incumbents, facilitated by the ever-growing and nouveau riche (e.g. Zayo) Tier-1 ISP's, who control most of the backhaul and dark fibre estate.

However, over the next few years, the advent of maturing ML frameworks, coupled with the intensifying war and ratification of standards over 5G/IoT based infrastructure, might prove to be serendipitous for nimble-footed enterprises.

https://en.wikipedia.org/wiki/Zayo_Group
It was far from "all factual", and clearly in dispute since someone immediately disputed it.

If you post strong political claims using pejorative language like "disease", "dictatorship", and "disregarding the people" (which in fact is almost all of what you said), you're going to take threads like this further into flamewar whether you mean to or not. That's bad for thoughtful discussion and therefore off topic here.

Most of us are living in contexts where our strongly held political/national views feel obvious and unobjectionable. But the minute you open up to a much broader context, such as a highly international forum like HN, it becomes painfully clear that this is not the case.
It's probably less important what might have happened, that what is required by law. If I recall correctly some bottled water is basically just bottling the municipal supply (i.e. no additional treatment or filtration).

I don't really use bottled water, being lucky in where I live (and filtering). Does anyone know how much enforced transparency there is on the industry? In other words, can I in practice find out what processes they have applied or not to particular water, especially if the company has multiple plants ?
The article starts: Let’s Encrypt is a revolutionary new certificate authority

... and in my opinion fails at that precise point.

I think that SSL/TLS itself is the correct starting point and not a particular implementation. The article could have been titled "Using TLS for internal servers". The LE implementation could have been one of a few.

Don't forget that you can manage your own Certificate Authority lists and really ought to do so if you actually give a shit about IT security. Abrogating your responsibility to MS, Apple, Mozilla, Google etc is way too easy, dangerous and probably immoral.
If the jar is 10% empty presumably you want to multiply the estimate by 10/9.
There also are luxury brands that cost as much and don't have any easily visible logos, even if they do have some expensive showrooms. All depends on how exactly one wants to wealth-signal.
Hold my beer ...

https://www.youtube.com/watch?v=L5cZaIZ5bWc

And that was more than 6 years ago, one ticker symbol only. You can imagine now.
It’s been shown by many (see Flashboys) that those “benefits” are completely exaggerated and are just an excuse to do HFT. The only real thing HFT benefits the bank accounts of the traders, and the traders will be the first to admit that.
Same reason why EVs are a net win even if they are ultimately powered by coal.

My impression is that pollution scales roughly linearly with the size of a power plant, while power output scales exponentially.
A lot less than their used to be. The margins have been driven out of that business. Like most industries that’s led to consolidation.

I’d guess Citadel/Virtuu/Jump capture the vast majority of that business.
Well, you would have lost out on a lot of innovation for one. I would be willing to bet that nearly half of the world's high performance computing specialists cut their teeth with a trading firm at one point or another.
By your logic, there would have been no benefit to video games or going to the moon even though each of these things have helped deliver large amounts of technological advancement.

IE, Facebook is a giant waste of resources but they employ many of the world's smartest engineers. Because of that they've spawned things like ReactJS which has completely changed web development for thousands of other companies.
> I'm interested taking a look at this if you want :)

I'd rather not, the thing in question was around three years ago and I don't think I even have my AUR PKGBUILD for comparison. It wasn't a cry for help, and I'm not naming packages or individuals for a reason. Just voicing some frustration at the process.
not sure if citing that as an example is such a good idea, considering everything surrounding it. You do have to reject an awful lot of evidence to claim with certainty there was never a missile.
Two factor authentication......
See https://news.ycombinator.com/item?id=19355603.
I've heard the (facetious) comment, before, that if we ever do discover a faster-than-light theory it'll be because of HFT, not academia.
It appears that some clients just universally query third-party servers, thereby giving out a list of every host name that they're looking for to a third party.
`dpkg -S /path/to/file` only works if the file belongs to a package. `/etc/resolv.conf` gets created dynamically at runtime. `/etc/hosts` gets created at installation time because it includes the hostname (which ought to get replaced with nss-myhostname).
I've read a lot of Java, C++, and Python code at work and some Go, and verbosity I've seen is Java > C++ > Go > Python.
Check these articles out, he had at least one mentor

https://motherboard.vice.com/en_us/article/pgajmv/these-are-...

https://motherboard.vice.com/en_us/article/z43e7y/fbi-says-s...
Of course market makers wouldn't want to compete in a market where they are forced to make much slower trades than anyone else. But market making existed long before HFT, so it can't possibly be a unique benefit of it. You can argue that market making has become better because of HFT, but not that it requires HFT.
Is there any finite initial condition for Conway's game of life that will expand to fill an arbitrarily large area of the plane with density that does not approach zero?
And CVS is clearly the cutting edge of source control.
Good point, the same way if someone phones you out of the blue and asks you to confirm your name and date of birth (or other personally identifiable information), you don't tell them anything.
In the long run, everyone who tolerates email is dead.
Particularly on a new design variant, there are probably a lot of design/manufacturing issues that can only be found when operated in the field.
And not just Airbus. Hard to imagine how you're going to do much regional travel and avoid Embraer, Bombardier, or ATR.
I know a lot of people who take melatonin to fall asleep. Trying to understand if that's good or bad here?
If only an "18650" was always the same! Despite the name, they're never exactly 18.0mm diameter by 65mm long. Some have protection circuits built in, some don't. Some have solder tabs, some have wired connections (2-pin or 3) and some fit into holders the way your garden variety AA battery does. Then there are a variety of capacities and charge curves. 18650 Li-Ion is not a standard the way that a AA alkaline is. The core can may be "standard" but by the time that gets packaged into a cell in a product, there are an infinite variety of options.
It's too bad the author started the article with that line.. I almost stopped reading right there. "Smart" people move here because it's relatively easy to get hired at a job that pays six figures even for the relatively inexperienced. The closing line about "lovely houses" was also a stretch. More like million dollar tract housing.

In between though, the article had a lot of interesting points.
There’s a ton of similar content on Youtube.
Crossing into personal attack will get you banned here. Please review https://news.ycombinator.com/newsguidelines.html and don't do that again.

Accusing someone else of astroturfing or shillage without evidence is particularly out of line.

https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
Interesting - they are expressed in such a way as they aren't tied down too much to the specific weapons available at a point in time. I wonder if they have been rephrased to be more universal.
I have some workout clothes with no visible branding, and I think it's of pretty good quality (at least the manufacturer is wuite famous for their high-tech fabrics and general quality)but interestingly it is much more expensive that your run of the mill Adidas or Reebok (not quite sure why I ended up having it either...).
I just finished reading the book "The Confidence Game", and the stories of Delvey or Holmes would fit right into that book.

It's easy to look from the outside in and question how these people could fall for these types of frauds, but I think most people are more susceptible to them than they realize.

Really, what cons are about is selling people a version of the future that they want. With Delvey, her associates were probably motivated to believe her because she spent money freely, and she seemed to belong to a "higher" social order.

What Holmes was selling wasn't blood testing kits, she was selling the idea that healthcare could be "disrupted" and progress the same way that software could. I'm not exactly sure who Tom Draper is or what he said, but for wealthy older people, who may have significant financial stakes in Theranos, you can see why this would be an attractive reality. In contrast, the alternate reality -- that you got suckered in by a fraudster and the progression of healthcare will be as slow and bureaucratic as ever (which means the quality of your life over the next 30 years will slowly, then quickly, decline probably past the point where you will logically want to live anymore)-- seems far less appealing.

It reminds me of someone who I worked with who was a Libertarian. He used to go around trying to "convert" people, and the only things he would ask was "Wouldn't you like to pay less money in taxes? If you didn't have to pay taxes what would you buy? Can you imagine how much money you would make if you didn't have to pay taxes?". He managed to convince like 90% of people that way.
How Transformers Work – Model Used by Open AI and DeepMind
If you're used to flying a plane for several thousand hours that's not going to be second nature at all (when the procedure has changed).

You really should watch this video just to get an idea of how catastrophic this error is:

https://youtu.be/baK39CrgW-4?t=363