skysight-inc/HackerNews huggingface dataset:
Background:

We have a team of 6 DS working on parallel projects. We tried the Java service approach. It's great for a one-time model, but very painful to iterate on.

We develop on top of Sagemaker, and since we're a funded company, can somewhat get away with the 40% price increase of an "ML Instances".

We have a mix of R/Python models. For each, we keep a separate repo with a Dockerfile, build file, and src code.

Training:

If the jobs are small, we train them locally, package assets into the container, and deploy. If it's a bigger job, we leverage Sagemaker training jobs and S3 for model storage.

Serving:

We have boilerplate web service layer with an entrypoint that DS fills in with their own code. Yes, this allows almost arbitrary code to be written, but we do force code reviews and enforce standards. Convention over configuration.

We do the feature engineering using Python/R, which when parallelized, has good enough performance (sub 200ms latency on Sagemaker prod). If we need latencies in the 1-10ms range, we'd consider refactoring the feature engineering into a separate layer written in a more performant language. It's always FE that takes the most time.

One learning from tuning Python services is: for max performance, try to push the feature engineering work onto consumers. Have them fully specify the shape of the data in a format that your models expect so you do as little feature engineering in the serving step.
Last I checked you can use WSD without account just can’t save, but iirc it records to the url so you can still kinda work it. Also; could be wrong but I thought it allowed federated logins.

It’s worth it.

The syntax is stupid easy and you really would have to try to mess it up. Fast to make workflow diagrams that other people can edit.
I don't really understand the argument, most startups explicitly have a goal/off ramp. The government is the exact opposite in terms of goals...
Sounds awesome! Can you please share the bash script?
Just an update - Cardboard doesn't work as the DayDream headset doesn't have the "button press" mechanism which taps the screen, and since Bluetooth doesn't work, it doesn't work at all.
> If we had been smarter, we would have introduced DBT after gen 1 and not gotten into the gen 2 mess.

Is this a roundabout way of saying you wouldn't have exposed looker to the underlying staged data?

We have two layers within our data warehouse: The ingestion/staging layer, and the organized/cleansed/segmented layer. We only give Looker projects access to the latter.

We still have a lot of chaos within our Looker projects (views with select star, dimensions for every column bc of select star, hard-coded schemas, etc...). Slowly working it out though.
SaaS businesses succeed based on the service part, not the software part.
Not to mention the fact that the premise of the whole damn thing is about the misuse of political power to dictate the course of investigations.

Pot, meet kettle.
I don't think anyone can predict the applications because creating new materials could have many, or almost no, novel or useful properties. Look at how many different ways we can manipulate iron and create different properties, now there might be even more. Or maybe it isn't stable in bulk atmospheric amounts, but can be held under pressure and has some unique property to make it suitable for computing, or detecting certain kinds of energy or radiation, or forms piezoelectric properties or who knows. Maybe it can be used in optical data processing or amplifying.

Most likely this isn't going to revolutionize our world, but there is always that small chance, and we will certainly find some use for these novel states of matter somewhere, if not just more physics experiments.
I general I believe that people don't like to mix entirely different tasks.

Chatting with friends is not the same as buying stuff. It can really poison the atmosphere. The very act that you could send money makes you less engaged.
There are also state laws requiring a "certificate of need" that artificially limits the number of MRI machines, keeping the cost high.
There is nothing similar to Apple’s business model to Google’s.

- Apple’s users are its customers. I give Apple money and they give me stuff. Google’s customers are advertisers and others trying to reach users.

- Apple is mostly concerned with its own platform and doesn’t have services across platforms with a few exceptions (iTunes on Windows, Apple Music on Android, Apple TV+ everywhere). Google wants to be ubiquitous.
I think it's fine. I'm not sure that new linux users would honestly prefer something extremely fast like i3, and unity is extremely clunky and unappealing.
Coming to DirectX 12– Mesh Shaders and Amplification Shaders
If you like the text based ness of WSD, you might like https://mscgen.js.org - which is 100% open source & client side only.
That's good advice but it's hardly simple.
Are those drawings supposed to be clickable? When I do so, it just opens and immediately closes a new tab or window. Firefox 70 on Windows if it makes a difference.
The SoftBank Effect: How $100B Left Workers in a Hole
The claim does not have any evidence for it whatsoever. However, instead of stating it as a fact, Snopes starts to editorialize about "deeper truth" in it, which it doesn't do in other cases where claim is presented without ay evidence. The fact check would be "false" or at least "unproven" (if Snopes would be consistent assigning "unproven" to every claim having no evidence behind it) and leave editorializing to non-fact-checkers.
Show HN: Spherical Geometry in the Browser (In ClojureScript)
Do you feel that your queries now are approximately similar to those you used to get good results on?

I can see a world in which the parent comment is correct, and we've "dumbed down" our queries because it gets us good enough answers most of the time. For example, perhaps I used to google "halo game walkthrough OR tutorial" and now I would type "halo tips."
Tuning Nginx for better requests/second
We had a similar problem with SageMaker re: cost. We tried a few different things out, but ultimately wound up sticking with Cortex https://github.com/cortexlabs/cortex/
Yeah weird the article didn't push back. Lotteries don't create wealth (except for the person running the lottery). If NAACP was funded by a tax on the innumerate and superstitious, well fine. That's how many states are run. But don't pretend its somehow virtuous.
It matters a whole lot. There's lots of code written in various languages and it's often prohibitive to use it in your own project because making the languages speak with each other (especially if you have performance concerns) is a lot of work. I don't expect WASM to make this trivial, but to say that it doesn't matter is patently wrong.
No greater mistake can be made than to imagine that what has been written latest is always the more correct; that what is written later on is an improvement on what was written previously; and that every change means progress. Men who think and have correct judgment, and people who treat their subject earnestly, are all exceptions only. Vermin is the rule everywhere in the world: it is always at hand and busily engaged in trying to improve in its own way upon the mature deliberations of the thinkers. So that if a man wishes to improve himself in any subject he must guard against immediately seizing the newest books written upon it, in the assumption that science is always advancing and that the older books have been made use of in the compiling of the new. They have, it is true, been used; but how? The writer often does not thoroughly understand the old books; he will, at the same time, not use their exact words, so that the result is he spoils and bungles what has been said in a much better and clearer way by the old writers; since they wrote from their own lively knowledge of the subject. He often leaves out the best things they have written, their most striking elucidations of the matter, their happiest remarks, because he does not recognise their value or feel how pregnant they are. It is only what is stupid and shallow that appeals to him. An old and excellent book is frequently shelved for new and bad ones; which, written for the sake of money, wear a pretentious air and are much eulogised by the authors’ friends. In science, a man who wishes to distinguish himself brings something new to market; this frequently consists in his denouncing some principle that has been previously held as correct, so that he may establish a wrong one of his own. Sometimes his attempt is successful for a short time, when a return is made to the old and correct doctrine. These innovators are serious about nothing else in the world than their own priceless person, and it is this that they wish to make its mark. They bring this quickly about by beginning a paradox; the sterility of their own heads suggests their taking the path of negation; and truths that have long been recognised are now denied — for instance, the vital power, the sympathetic nervous system, generatio equivoca, Bichat’s distinction between the working of the passions and the working of intelligence, or they return to crass atomism, etc., etc. Hence the course of science is often retrogressive.

-- Arthur Schopenhauer, "On Authorship and Style"

https://ebooks.adelaide.edu.au/s/schopenhauer/arthur/essays/...

Emphasis, and wall of text, in the original.
I'm sorry, but paying money to YouTube/Google to get around their adds and garbage recommendations seems too much like rewarding them for bad behavior.
This is why direct primary care is becoming a thing. Doctors are opting out of the insurance system in order to see fewer patients and reduce overhead. For $50-$75 a month, you get unlimited access to your physician, including via video call, access to wholesale meds and labs, and reduced cost diagnostics.
> the people who will benefit from it don't know how to use it? Or something?

Something like that. Another point is that the complexity adn increased risk (yes) added by complex automated (auto-scaling-)systems might not be worth the price you save instead of running at 5x mean capacity all the time.
Not a complete solution, but if you're already using Looker...

https://looker.com/platform/blocks/embedded/create-a-data-di...
Tesla announces new Giga factory near Berlin
Risk of having a statement misconstrued or taken as a much bigger deal than it is is why people (who want to actually land a job) are so fake in interviews to begin with. It ends up mostly being a test of whether you can put on your "interview face" without letting it slip (any slip must be the tip of some awful iceberg, surely!) for the desired amount of time. Which is why the "slick" folks who aren't good get through while the non-slick but good person who slipped up and let something genuine and not-at-all-a-big-deal-actually get through is lumped in with the "abrasive" folks (any hint of something is taken—not entirely unreasonably, to be clear!—as proof of a problem) and doesn't get an offer.

The actually-good end up having to do precisely the covering-up you mention, because genuine and entirely fine attitudes and behaviors are, in interviewers' imaginations, often magnified to their worst possible extremes. It's very valuable information for a good employee to have that they need to cover up or avoid certain things in interviews that aren't actually a problem, because they do need to do that, and they may not realize how things that they think aren't a big deal and in fact are not are coming across in an interview context.

I write this as someone who is, I gather from feedback, pretty decent at that part of interviewing. Doesn't make it less gross-feeling and stressful.
What if Street Crime Statistics Matched Those of Cybercrime?
You can have a look on LibHunt next time. Apart from project activity, you can compare different libs by their relevant popularity. That should help you make a better decision. E.g. https://ruby.libhunt.com/categories/21-a-b-testing

(disclosure - I've built libhunt)
> For everything else, though, I'd be much happier if I could just have a price list up front and shop for the provider of my choice.

Well the price list should be up front, but that's the case regardless of whether you're paying for all of the care or just some of it.

> Emergency care is what insurance _should_ be for - the stuff you can't plan for, must have, and can't comparison shop for.

Wouldn't that discourage preventative care?
FreePascal, Delphi and Oxygen, already three possibilities. And there are a couple of other variants still around.

Lazarus, Visual Studio Code, Delphi IDE, Oxygen IDE, Emacs, Vim, Eclipse as IDEs.
Looks really interesting. Any plans for Android or Web?
What we do at https://quillbot.com

Training:

Currently we just use a bunch of beefy desktop workstations for training (using Pytorch).

Deployment:

This is the vast majority of our cost, each time a paraphrase comes in we add it to a queue through google cloud Pubsub. We have a cluster of GPU (T4) servers pulling from the queue, generating paraphrases and then sending the responses back through Redis pub/sub. I think ideally we would have a system that makes it easier to batch sentences of similar length together, but this seems to be the most cost effective way for models that are too computationally expensive for the CPU that is relatively simple to put together.
Berkeley just has CS164 [1] which is all in one. They haven't taught a graduate compiler section in years (decades?). Stanford is on the quarter system. They have an even more compressed single quarter compiler class, CS143 [2], but they also have a graduate advanced topics class, CS243 [3] as well. Moreover, CS243 doesn't require 143 as a prerequisite. As an aside, Berkeley is very strong at computer architecture (RISC, RISC-V, ...) while Stanford is very strong at compilers (Aiken, Lam, Ullman, Hennessy, ...)

http://www-inst.eecs.berkeley.edu/~cs164/sp19/
https://web.stanford.edu/class/cs143/index2018.html
https://suif.stanford.edu/~courses/cs243/
One thing missing here is to avoid synchronous communication. Sync comms tie client state to server state; if the server fails, the client will be responsible for handling it.

If you use queue-based services your clients can 'fire and forget', and then your error handling logic can be encapsulated by the queue/ consumers.

This means that if you deploy broken code rather than a cascading failure across all of your systems you just have a queue backup. Queue backups are also really easy to monitor, and make a great smoke-signal alert.

The other way to go, for sync comms, would be circuit breakers.

My current project uses queue-based communications exclusively and it's great. I have retry-queues, which use over-provisioned compute, and a dead-letter for manually investigating messages that caused persistent failures.

Isolation of state is probably the #1 suggestion I have for building scalable, resilient, self-healing services.

100% agree with and would echo the content in the article, otherwise.

edit: Also, idempotency. It's worth taking the time to write idempotent services.
An automated system can still explain why it's making this decision. Sure, for machine learning that's an open research question, but for most systems you actually find in charge of these tasks it's trivial to give useful information. "Your account was terminated because of unpaid invoices"; "Your application was automatically rejected because you failed to meet our education standards"; "You have to reset your password because your company's admin changed the password requirements"; "Your order was canceled because you didn't pass our credit check"
BentoML(https://github.com/bentoml/BentoML) may help you with the process of building endpoints with both Deep learning models and logistic regression/tree models, and it automatically helps you to containerize the API server into docker image that's ready for production deployment.

It also provides OpenAPI spec for your API endpoint, which allows you to generate API client in Java, for your backend/app teams.
The $500 loan statistic is false.

https://mobile.twitter.com/p_millerd/status/1118071142311288...
Backup is much more important than RAID in most situations. RAID doesn't protect from user error, virus, fire, etc.

I think Windows does remind to create backup (but not very loudly and it accepts local backups to another drive which is poor solution).

Unfortunately, almost everything is cheapest possible. For example, I would prefer ECC RAM as standard. It's very cheap for production (one additional chip per RAM stick), but Intel wants to force people wanting reliability to pay for Xeon CPU and most people don't care.
Huh, I'm not in the market so I'm afraid I don't know. Maybe someone else here who needs this will know.
How is that at all considered to be true?

The original claim was those specific people were voted out of Congress. Not that it's representative of the general quantity.

The claim in the title that they edited might be true, but the original claim from Twitter is not. That's the whole point. From the article:

>this one got the general idea and numbers correct (even if the persons actually pictured in the accompanying photograph are difficult or impossible to identify).

Difficult or impossible to identify? Dozens of others have done it, and even the author admitted it was made up.
I fell for this with "First Documents" for a total of almost $93 a year or so ago. Also received a solicitation from "Local Records Office" as stated in this article. When the second of the two arrived I knew I had been had.

I reported it to my Credit Card company as fraud given the official looking nature of everything, and they promptly refunded the purchase.
I suppose its better that a tax on the innumerate and superstitious gets collected by a community member who has some skin-in-the-game when it comes to the community's health, as opposed to outside organized crime or local government, who have an indifferent or outright hostile attitude towards the health of that community.
WASM modules by themselves can do pretty much nothing besides allocating memory. If you want to use some API, you have to explicitly expose it, as a user.
I'm a fan of M4A, but I think something the gov't could do is allow people to earn tax credits for practicing preventative care that help lower whatever tax increase would be created to fund healthcare. That creates a financial incentive to try to be healthy while not necessarily discouraging people from going to the doctor when they really need to (which can happen with out of pocket expenses, HSA or not).

E.g. on one of my old plans I got a $300 visa gift card just for going to a checkup.
I meant it only as an illustrative example, in this case, of how easy it is to flip the answer. I don't know the specifics of this particular claim.

They are not always using the claim sent to them, and they've been busted dozens of times for changing a claim, instead of changing the result. It's selectively choosing which "original" to use.

For actual cases I've already posted:

https://news.ycombinator.com/item?id=21516623

https://news.ycombinator.com/item?id=21516762
4) Sometimes it's better to fail fast and try again than spend time writing extensive error handling and retry logic.
> There's only one thing HSAs are really good at, and that's creating a huge new tranche of money for Wall Street to suck fees out of.

You can move HSA funds to an HSA at Fidelity and put it in FSKAX or FZROX at no fee.
https://lite.cnn.com/en

Also, the main sites for the BBC, New York Times and Washington Post all work just fine without javascript and in terminal-based browsers (I'm partial to emacs' eww.) In my experience javascript is not necessary for a great many websites, including some major ones. Most links I click through from HN work fine without javascript, although there is some selection bias there.
Can confirm, have seen that exact thing happen.
Is Medicare for all going to be a premium-funded, premiums enforced one like we have in Turkey?
You don't need any fancy word play to express the types of ideas that seem so hollowingly void in these movies.
Excuse me if that has been discussed numerous times, but I distinctly
remember CockroachDB 2.0 coming out, and it being a big deal. Where are
releases 3.0 to 19.0? Was the versioning scheme changed? How? Why?

Other than that, I really wish I could use CockroachDB at "$COMPANY".
Our architects, unfortunately, deem it “too new” and “unproven”. Bah.
Doctors in the UK don’t regularly make $300k+ per year.

https://www.medscape.com/slideshow/uk-doctors-salary-report-...

https://www.medscape.com/slideshow/2019-compensation-overvie...
Copying WeChat, which isn't a good thing btw.
Usually I see HSA’s combined with a high deductible plan so that the six figure medical bill is protected. We were on one when my daughter needed surgery (~$50k). We also had another child the same year ($20k), and all we paid was the $6k deductible which also came out of pre-tax money AND my employer contributed $3k to cover the premium. There were no bank fees involved. i am now self employed. Unfortunately I can not buy a high deductible or a HSA plan in my state post Obama care and my premiums in 2018 where $38k and this year have dropped to $32k. To be honest I kind of miss that $6k deductible.
Exactly. If you have insurance, you should never receive surprise bills that are higher than your out of pocket maximum. Bad insurance might mean a higher out of pocket maximum, but that should mean what is says...maximum.
Appreciated. And given your username I hope you saw the toaster threads!

https://news.ycombinator.com/item?id=21432144

https://news.ycombinator.com/item?id=21164014
https://outline.com/6vbYzr
I'm sure the fake subreddits with amazon referral links are a work in progress as we speak.
Google does this by giving you credits that expire in six months. I think that works well.
Another lesser-known diagramming tool that shares elements of Whimsical and Miro is Plectica: https://www.plectica.com/
Discussed in depth! https://news.ycombinator.com/item?id=21164014
Self-Training with Noisy Student Improves ImageNet Classification
I noticed this big time with Twitch.tv. There's now ads I cannot avoid or skip all over the place.

I'm also finally noticing ads when watching YouTube on Chromecast. There used to be zero ads (which was a little confusing) but now there's ads.
Recreational vehicles as in RVs? They don't use compressor fridges? Is it because the Einstein ones are lighter?
Even for job interviews, liability isn't really the reason.

As a general principle, conveying true information does not incur liability. In the specific case of job interviews, as long as the feedback is true and not of a legally discriminatory nature, there are no grounds to sue, and such a case would be thrown out by summary judgement. Any employment lawyer would laugh someone out of their office who came to them with a "case" like this.

I'm inclined to think it's primarily for the reasons mentioned elsewhere: avoiding back and forth arguments. Frivolous lawsuits from _pro se_ plaintiffs might be a minor consideration, but, as I said, those are easy to get thrown out, provided you're telling the truth and the reason isn't related to any protected classes in employment.
Welcome to Motherfucking Silicon Valley
I tried again and now get the Incapsula crap - maybe related to IP address (which changes often on Tor)?

Edit: the page loads for the first time after assigning a new IP in Tor, but subsequent loads throw the captcha. Odd system.
It's not "chemical waste". It's what was already in the ocean.

(Yes, I know, the same thing in concentrated form is different. But dilute it reasonably, and we're good. It's not like it's mercury or something.)
Can we actually backup the database with the community edition yet?

Last I checked this was impossible.
When Cross-Validation Is More Powerful Than Regularization
You seem to be arguing that sandboxing is not a security benefit. On the contrary, sandboxing is maybe the security success story of the past decade.
Check out Programming pearls by Jon Bentley. Reads like a novel, teaches phenomenal foundations.