skysight-inc/HackerNews huggingface dataset:
Ask HN: Cheapest way to compute large batches for science
Hi all,

I'm currently working on my PhD, where I will have to fit a few hundred million high-dimensional curves under somewhat complex constraints. My data - around 3.5 TB - is stored on a Postgresql-Server in my basement.

My current approach is as follows:
- Segment the data into chunks and upload to AWS-S3
- Use AWS Lambda to create an AWS Batch Job to run the computation on EC2 spot instances
- Store result on S3 and fetch

While that works like a charm, it also burns through my grad student budget like a wildfire, so I am looking for less cost-prohibitive ways of running that computation.

I have considered either running my tasks on EC2-comparable instances with another (cheaper) provider, since the AWS Batch functionality is mostly convenient, but I don't really need the flexibility. I know exactly which results I need, since my data set is already complete. However, I don't understand enough about cloud computing to exactly quantify how many requests or hours of computing I need on what type of processor.

Alternatively, I might just buy more computers, and heat my flat with those for a few months, before selling them again. Issues here are the high upfront cost, but it would give me cost safety over time. I also have no idea how long my computation would need.

How would you approach this problem?
Implementing custom IValueTaskSource – async without allocations
Well, the impetus for enhanced airport security was first the spate of hijackings in the late '60s to early '80s, and later 9/11.

While you can hijack a train, you can't fly it to a friendly country, and you can't fly it into a building, so it's a very different threat model.
> There are also a few RFCs that demand terminating your connection say when you've negotiated SSLv3. Emacs doesn't terminate and won't terminate before asking the user first (with caveats).

That seems bad. There's plenty of evidence that if you ask the user "hey, do you want to continue but be insecure?" they'll blindly click "yes".

While non-browser focused protocols may still rely on clients not forbidding insecure protocols, we should absolutely be trying to move away from blindly allowing insecure protocols (or effectively allowing them, given users typically click "yes").

I'd strongly suggest failing hard, and allowing per (host, port) exceptions in the config file, rather than asking every time.
> but it is truly annoying the amount of subsidies the flight industry get

Please could you illustrate some of those for me, as compared to rail?
> It tells me they're trying to minimize the number of users (which reduces their costs) by raising the price while still making it profitable.

Is the marginal cost of serving extra users really a big part of Google's costs? Bandwidth is cheap; I would have expected that building and updating the map data was the expensive part.
Webassembly Fast Tutorial (windows)
Actual search engine is needed by other criteria like age range I would say. They also have things priority and maybe plan for more.
Want to give you guys my Instagram Growth ebook ($9.99) for FREE
True. But they don't take hits and vibrations like a military 4x4 would.
Just my opinion of course, but I see the basic problem as employees using twitter. Many companies encourage employees to use twitter, social media, etc. for more direct interaction with customers, the community, etc.

What is often overlooked, is that in communication you are first an employee of a company, and your own opinions and feelings come way later.

As is clear in this example, not everybody can do this (all of the time).

She is certainly entitled to this opinion. But as an employee, she should not have expressed it. Most likely the company didn't reflect on this before allowing/encouraging her to use twitter to communicate.

I'm speculating that the company didn't have a communication strategy to contain a shit storm. And in the end just fired the employee. Firing the employee only makes sense if she has a history of causing the kinds of issues.

Again, just my personal opinion.
Chinese Hackers Targeted Internet-Of-Things During Trump-Putin Summit
It's funny because your comment is itself textbook hysteria. You provide absolutely no evidence at all that any of your claims are true. You attribute behavior to some undefined group of "people", and then insist that this behavior is being driven by another shadowy group ("the media") and this leads you to the very strange conclusion that the "media" is focusing too much on shills.
>“you know we would have preferes if everyone was on windows and coded on VisualCode for .Net”

If you mean VisualStudioCode it's cross platform and you can program .NET in it with .NET Core on Linux.

If you mean Visual Studio then that's only good on Windows.
Ban Muslims

Alright ban me but we all know this is the truth. 2.8T USD and so much productivity has been lost since 9/11.

Anyway I'm going to get banned but I hope this username can be recycled and my sacrifice shall not be in vain.
The only alternative would be to develop everything in house and to purchase and run your own servers.
> We disassembled the phone and asked security researchers to probe it to find out what it is. Verdict: It's wild.

I could have used a better summary. Now I had to read half the article anyway.
That depends on what you mean by "what Apple is doing".

I'm not a fan of Apple either, but at least they are not hypocrites.

If MS stopped pretending that they <3 Linux. (see my other comments)

And if they stopped pretending that they are this new friendly open company. Forcing upgrades to win10 and forced telemetry that is a pain in the ass to turn of permanently, is not at all open or user friendly. What a privacy nightmare.

If they did just that, then at least I wouldn't hate them for being such hypocritical assholes. They'd just be a company whose products I don't like.
Ask HN: Is there a place where I can get Sales pitch reviewed
I am a dev and trying my luck in selling (cant hire a full time person as of now). Is there a place for me to quickly get the sales pitch I use on the call to get reviewed? Will you be interested in a platform like this?
Deploying an OpenStack Cloud on Kubernetes
Earth's average temperature without the atmosphere and the greenhouse effect would be quite cool.
Guess depends very much on the spacecraft design, how you absorb and radiate.
I helped maintain a large public pool a few years back, and I saw first hand how disgusting they could be. I now swim almost exclusively in creeks and rivers.
>Is killing the boom the key to supersonic air travel?

No. Cost is
Figma should aim for their own Linux or BSD distro focused on full stack webdev (designers who code, coders who design), and maybe even tailored hardware in the future. There's so much potential in this space, I wouldn't sell to MS. They could become an Apple alternative that is respected by designers!
> Because they keep churning out garbage. And the only "good" stuff they do make, they don't open source, or even release it on multiple platforms.

.NET Core runs on Mac, Windows and Linux and is BSD licensed, and totally not all garbage (though I'm guessing you will disagree with me here with no backing facts because what defines garbage is mostly opinionated views). VS Code is the same scenario. Wow it astounds me the lack of understanding coming from you.
I suspect that the solution to the problem may look very different to the planes and drawings of planes shown in the article.

I know nothing of aerodynamics but what I do know is that innovation in aerodynamics is more likely to come from the world of F1 than the bowels of Lockheed Martin these days. This has actually been the case since the 'Gurney Flap' that was the first innovation in aero to come from motor racing to make its way to regular aerospace.

In 2017 at the Singapore Grand Prix the Force India F1 team surprised the fans with a new gadget to get extra downforce. In 2017 F1 cars had a 'sharkfin' engine cover and on top of this Force India had what looked like a hedge trimmer, this was 30 or so very small wings that worked together to create two powerful vortexes with the mini-winglets reinforcing the same vortex created by the mini-winglets at the front of this 'hedge trimmer' arrangement. From this they were able to get good downforce from an area of the engine cover/sharkfin that was not supposed to be downforce creating. The innovation obviously had to be banned and this is no longer a feature in 2018 cars.

F1 max speeds are lower than a plane and the aero is not about getting airborne, however, due to the rules there is constant innovation. Regular planes look as aerodynamically sophisticated as F1 cars from the 1980's. I suspect that to do supersonic without the boom then aerospace will have to borrow more than just the Gurney Flap from F1. Furthermore, incentives have to be better than pork barrel politics or else the Lockheed Martins of this world will be on this gravy train forever.
For a fun, artistic experiment, yes, it’s worth showing off the amount of people involved.

I assume there will be hundreds in a week or two and the site will be something completely different.

For example, a PR just came in, containing a time-travel functionality so that you can travel between commits, on the fly!

If that isn’t cool and something worth celebrating, I don’t know what is.
>can I gently request you take that slight back. Or at least qualify it to be "perl regex".

Regex has the same complexity in any language, that can't be a valid reason to denigrate Perl. My biggest issue with Perl is that functions don't even have proper parameters and you need somewhat cryptic syntax to work with arrays, scalars and dictionaries.
>I don’t want to live in a world where the best language humans could think of for working with data was invented in the 70’s

Yeah, because languages go stale... especially those based on mathematical abstractions like SQL.
Software developers get religious about their OS like they do their choice of a text editor. :)
Was it covered or outdoors? The wiki says it can be deactivated using low doses of UV, I wonder if being exposed to the sun is enough.
they have definitely changed, they have re-aligned themselves with what matter today. There earnings has just surpassed $100 billions which is the direct effect of their re prioritizing the product lines and philosophy. https://www.theguardian.com/technology/2018/jul/19/microsoft...
More specifically, a TeleType Model 37 from 1967. None of this modern ECMA-48 stuff from 1976. (-:

* https://news.ycombinator.com/item?id=17557575

* https://news.ycombinator.com/item?id=15779194
Indoors - a public one that was close to end of life.
>And the relational calculus itself isn't super lovely as a base... Turing completeness is a nice thing to have, after all.

And then you have C++ templates all over again...
Thanks for the feedback!

I focused on showcasing use cases for each pod in the video but that might have taken away from conveying the overall goal. Pi is intended to act as one hub for all your productivity tasks throughout the day. Is there anything specific you think could help make things clearer?
Presumably some end customer may pay close to iPhone X prices for the counterfeit version. And regardless, when they logon, their Apple ID and password can be harvested.
Even if you live in a strict Windows world, the console is important. Plenty of sysadmin-type tasks can only be done from the command line, or is at least faster/easier than the alternative (Exchange, VMware, SharePoint, ...). At work, ConEmu is among the first programs I start after logging in.
It seems like the cache link I previously pasted does not allow to click on deeper links on the site. Sorry. At least it helps to have a quick view to the content.
Go 1.11beta includes WebAssembly
> Flying is absolutely catastrophic in terms of greenhouse gas emissions

Uhm, flying is OK in terms of emissions per passenger mile. Not as great as train, but better than driving solo in a gasoline car.

The problem is transcontinental flights. While they're OK in terms of efficiency per mile, the number of miles is staggering.

The problem is, those trips can't be made by train.

So in terms of flying, I think the main focus should be not to fly long distance if it can be avoided. Sure, it's much better to take the train if possible, but if you fly between countries within europe a dozen times a year, that's not a catastrophe.
24% of Tesla Model 3 orders have been canceled, analyst says
How to Protect Your Legal Practice from Payment Redirection Fraud
Lol, try living in California. 6-12 hour plane ride to anywhere. Trains? We have them. But why would you when it takes literally days to cross the country.
UK here.

> Aviation is currently undertaxed, according to economists from the IMF and World Bank, compared to other forms of transport. Airlines pay no fuel duty and VAT is zero rated meaning that no tax is charged on sales and tax charged on inputs is refunded.

> ...

> CE Delft calculated in 2013 that the shortfall as a result of these two anomalies amounted to around €27 billion annually in Europe.

> ...

> At a global level, the World Bank and IMF favour a relatively low tax universally applied per tonne of CO2, which could then raise money for climate change mitigation.

> In the UK, the Liberal Democrat and Conservative coalition considered in 2010 replacing APD with a per plane tax to include freight aircraft and provide an additional incentive to airlines to fill empty seats. However, this was dropped following concerns, the Government claimed, about concerns that it could be challenged legally as representing a proxy fuel tax (contrary to bilateral air service provisions with other countries).

> Another alternative would be a form of taxation that discouraged frequent fliers.

https://www.aef.org.uk/issues/economics/taxation/
To avoid confusion: Perl 6 is not (anymore) on the defunct list, it is still on the "beautiful contenders" list.
There are ways to get much better terminal experience for typical web dev https://blog.ropnop.com/configuring-a-pretty-and-usable-term... (also works for tilix)
Prevent Payment Misdirection Fraud with Two-Factor Authentication
Here's an interesting article about 19th century Germany when no copyright laws existed yet:

The Real Reason for Germany's Industrial Expansion?

Did Germany experience rapid industrial expansion in the 19th century due to an absence of copyright law? A German historian argues that the massive proliferation of books, and thus knowledge, laid the foundation for the country's industrial might.

Found at http://www.spiegel.de/international/zeitgeist/no-copyright-l...
The children accused of witchcraft and murder
I was originally going to say that they should open source those. But then I changed my mind to just releasing it on multiple platforms as a start. And so Windows doesn't make sense there. I should have proof read that better. Thanks for pointing it out.
Are there companies doing wastewater monitoring on pools for public health?

All kinds of disgusting things get flushed down toilets in homes and businesses. I think that monitoring a swimming pool would give more helpful data, because the bacteria/viruses/pollutants were all carried by humans.
Then they would have just another generic Android phone in a sea of generic Android phones
A page to track ECMAScript proposals
(Perhaps a question for the mods: Why are so many of antpls's comments flagged/dead? Their other behaviour doesn't seem malicious, and this seems like a reasonable question to me even if it doesn't directly address my comment.)

> How is that different than having only 1 version of the API and updating it?

Using a service discovery document makes it easier to put /v1 and /v2 on separate servers/domains instead of putting both on all the servers/domains. Serving the service discovery document is cheaper than having your front-end server route all requests to backend systems (like using nginx or an Amazon ELB in application mode).

Putting applications on different domains is desirable if /v2 has performance gains (or cost gains) obtained by using the new protocol. Or if they're separate codebases.

> Clients code will still break if a structural data change is pushed (for example, removing a field from the responses)

It's very difficult to predict how a client will react if you haven't seen/vetted their implementation, but there are things we can do that go beyond service discovery.

For example:

My advertising service allows publishers (people who make web pages and content) to choose how to render my advertisers' ads in those pages. They do this via a server-side call -- I don't do any tracking of this, but I want to be sure that the client is doing the right thing because I'm still representing the brands.

I can look at the web page and sign off on it, but I want to make sure that it'll support a wide range of ads -- including ones I haven't thought of.

So I put a lot of energy into patterns like fuzzing the clients, and doing things that I hope will require they do things securely:

- I require retries so users always get an ad

- I randomly giving them a bad SSL certificate -- one that if they accept, I block their account. I usually only do this during onboarding or for my crawlers (recognised by IP) which check for sneaky stuff.

- I require certain utf8 characters to make sure the query is utf8 encoded

- I randomly insert random fields to make sure I can add new ones later

- I randomly reorder fields in the response to make sure they're not counting

I think that if you do all these things, and perhaps other things, and clients are used to it then you probably can have just one version of the API and update it. But this is hard work, and the enterprise is not (in my experience) very fond of hard work.
What makes you think that nobody would use it? If Libreoffice is good enough, then why doesn't everybody use it instead of Office?
Azure Maps, Apple Maps, and TomTom all use TomTom map data.
Why are sudden temperature changes bad? When I was a kid in the north of Scotland we'd always jump in as gradual exposure to cold water was miserable - the shock of jumping in was always quite bracing ;-)

Mind you - as I doubt if it was much above 20C most of the time I doubt the temperature difference between air and water was that large!
> The whole duty free forced march -> endless tread mills
> -> gate -> bus routine is harrowing busy work.

Is it not a purpose of it, at least in theory the opposite, to distribute flow os passengers in space and in time?

Sorry, I do not really see how self-driving helps here, looks like a solution in search of a problem. Technology already changed a lot of things (self checkins, etc.) but it did not make flying a bit more enjoyable.
I'll point out that although it's indeed atrocious compared to a train ride, flying is still more efficient than driving a car. So if traveling is required and you're debating between driving vs. flying (say SF <-> LA), you should prefer flying unless you can carpool. I don't know the exact numbers since it depends on where you're going and what plane/airline you fly, but I would expect around 50-75 MPG for a reasonably full airplane [1], whereas your car might be more like 25-30 MPG on the highway, so you'd want at least 2-3 (ideally 4) people in the car to make it more efficient.

However the thing about flying is of course that it burns fuel at a much, much faster rate, so if it means the difference between you driving to workplace A vs. flying to workplace B regularly, then you'll be burning one hell of a lot more fuel by flying.

[1] https://en.wikipedia.org/wiki/Fuel_economy_in_aircraft#Airli...
Linux terminal emulators are by no means uniform about this. Scrolling speed, and whether to not re-render for every character, are points of contention amongst the XTerm, Alacritty, and VTE people.
I love stuff like this, but I think he left out a fairly important number: the amount of time he spent making it, and the $ value he'd put on that time.
From the article:

"For performance, it runs with .NET Core on Windows, macOS and Linux, works with Python 2.5 through to Python 3.7 and supports the latest language features"
This is funny:

Paul Graham, September 2001:

> "And if you manage to write something that takes off, you may find that you were merely doing market research for Microsoft.”

Google is today's Microsoft :)
This definitely seems like it's true. I wonder if there have been any reputable studies or surveys on how much time office employees waste on average like this?
That's not out of nowhere, since you have exact control over when it goes out of scope.
If we're doing anecdata when Reader was killed off I tried Feedly, didn't like it, stopped consuming via RSS. I've recently tried Newsblur but honestly I never check the thing despite having it on every device I can. I don't think RSS is a thing for me anymore outside of technically podcasts
Potentially it acts as a relay circumventing any local firewalls.
> Reference counting isn't more efficient than GC

Isn't it more efficient in terms of memory usage?
SingHealth's IT System Target of Cyberattack
> DOM manipulation, of course. But not everybody knows about it. So most people default to Photoshop, which is what these people did.

They're web developers, and you're suggesting they don't know how to manipulate the DOM?

> This email is FORGED, and I'd like to know why.

The fact you're using a throwaway account to make the accusation suggests you're not too sure yourself, and you know your "evidence" is beyond flimsy.
People would take their infant children swimming in "swimmies" (diapers) at an apartment complex pool I used to often get ear and eye infections from using. Once I learned that's what those children were wearing in the pool I ceased all usage. Never had those problems before or after that particular rental.
> could have been avoided if the streamer had hedged just a bit better

If you find a live claymore in the middle of the company meeting room, exploding it can be avoided by circumspect behavior.

If that's all you do about it, you might as well start calling it "boss".
To get to low cost you need a lot of routes to fly on. Without a sonic boom can fly over land which opens up more routes. Therefore, the lack of a sonic boom does contribute to the comeback of supersonic air travel.
I worked at a pool as a lifeguard instructor throughout high school and college. The aquatics director used to try to explain this to people who would blame her for misbalanced chemicals when there was a strong chlorine smell. It would always get so much worse as soon as the pool got busy, like during a swim class or children’s swim lessons. Getting people to shower before entering the pool was a hopeless plight.