skysight-inc/HackerNews huggingface dataset:
I'm a biologist by training. Eventually my research hit a data wall (my simulations produced too much data for my storage and processing system). I had read a paper on GFS and Mapreduce and Bigtable from Google, and decided to go work there. I got hired onto an SRE (production ops) team and spent my 20% time learning how production data processing works at scale.

After a few years I understood stuff better and moved my pipelines to MapReduce. And I built a bigger simulator (Exacycle). It was easy to process 100+T datasets in an hour or so. It wasn't a lot of work, really. We converted external data forms to protobufs and stored them in various container files. Then we ported the code that computed various parameters from the simulation to MapReduce.

I took this knowledge, looked at the market, and heard "storing genomic data is hard". After some research, I found that storing genomic data isn't hard at all. People spend all their time complaining about storage and performance, but when you look, they're using tin can telephones and wind up toy cars. This is because most scientists specialize in their science, not in data processing. So, based on this I built a product called 'Google Cloud Genomics' which stores petabytes of data (some public, some private for customers). Our customers love it- they do all their processing in Google Cloud, with fast access to petabytes of data. We've turned something that required them to hire expensive sysadmins and data scientists into something their regular scientists can just use (for example, from BigQuery or Python).

One of the things that really irked me about genomic data is that for several years people were predicting exponential growth of sequencing and similar rates of storage needs. They made ludicrous projections and complained that not enough hard drives were made to store their forthcoming volumes. oh, and the storage cost too much, too. Well, the reality is that genomic data doesn't ahve enough value to archive it for long times (sorry, folks, for those that believe it: your BAM files don't have value enough for you to pay the incredibly low rates storage providers charge! Also, we can just order more harddrives, Seagate just produces drive to meet demand, so if there is a real demand signal and money behind it, the drives will be made. Actual genomic data is tiny compared to cat videos.

The real issue is that most researchers don't have the tools or incentives to properly collect, store, and use big data. Until that is fixed, the field will continue in a crisis.
Woah, guy. I'm not even in SF. Wouldn't work there for any amount of money.

I'm just saying that the obvious immediate market for this to me seems to be towards b&m or medium-touch product sales businesses that are _in_ areas like SF. Many Bay Area startups are selling services to other Bay Area startups, that's just the reality of this sort of thing.

If you're not their target market, you're not their target market. If you are looking to hire people and this ends up being a viable alternative you will save a ton of money.

I worked in/around call centers for most of my first career, I know the score.
This comes up on the first page of a Google search... It's almost like you didn't search for anything yourself...

http://americablog.com/2013/02/tea-party-obama-is-hitler.htm...

or there is this:

https://kristiann1.files.wordpress.com/2014/06/hitler-t-4-ob...
Emacs users don't have to worry about minor errors because we don't make them. Vi users don't, either (they already made a huge error, so all their little ones don't matter).

On a more serious note, a lot of the features of an IDE (e.g. code completion, method lookup, compiling, and version control integration) are trivially turned on with emacs. I use etags (ctags, etc) and have macros bound to various frequently used git commands, auto-compile when I change a translation unit, etc. My Emacs is an IDE. And much more. Vi has similar functionality available to some degree or other and even if it didn't, with evil mode in emacs vi users can have it, too.
> now he has to pay child-support and alimony and 1/2 their house

As a Brit, I'm not all that familiar with the term "alimony", but the dictionary definition I've read says "a husband's or wife's court-ordered provision for a spouse after separation or divorce". Alimony appears to be a generic term for the total payment from one spouse to another, on divorce. Would that mean he's paying "child-support and 1/2 their house"?

From the information given, that seems a fairly equitable split. Half the house, and a contribution towards his childrens' costs. OK, maybe he's the only one currently paying the mortgage, but in most circumstances, that would be a choice left to him: whether to keep his share (and equity) in the property, or sell it.
> $15 * 500 hosts = $7500 per month.

Oh that's ridiculous pricing. Any team running a server installation at any sort of scale would scoff at the pricing, and that's even before you take into account the implications of sending so much of your metrics data to a third party to be held hostage there if you decide to leave the service.
12 Experts Share Their Best Advice on How to Improve Workplace Culture
In the course of using Walmart in a fairly normal way to buy a gallon or two, I make a teeny mistake and take home 15 trailers of milk and they charge me full price for it.

If only Walmart would have a process in place to notice that I was ordering a spectacular and unusual amount of milk and save us all the trouble.
I've been on the front page of HN multiple times, you won't even come close to the $5 limit of 1 TB. This is ridiculous.
I don't think it's easier to learn that せんぱい is pronounced with an [m] sound than it is to learn that "senpai" is pronounced with an [m] sound. Beginners can be just as surprised by the different pronunciations of ん as they are by "n". The real difficulty is in internalizing the alien, regular phonological rules of Japanese. Likewise, people struggle with the Japanese "r" because it isn't a natural sound for them. Having the romanized spelling around just makes it easier to blame.

Long and short vowels must obviously be distinguished. I think either yōko or yooko are fine. Either Tōkyō or Tookyoo will let a beginner pronounce the word correctly. Some prefer youko and it works fine in almost all cases too, but perhaps there's no real value in clinging to the accidental way with which hiragana happens to represent long vowels; it isn't inherent to the language. Besides, there's a small added benefit in disambiguating real [ou] pronunciations like omou.

Your point about visiting Japan is well-put. There's no arguing with the fact that anyone who wishes to understand Japanese should learn kana and kanji. But I think that there's a real value in putting emphasis on the spoken language first, and a well-developed minority tradition of teaching Japanese like this in a serious manner, not as a crutch. E.g. Eleanor Jorden's JSL textbook follows this principle, as do a number of other textbooks and guides.

"To get started in learning to read modern Japanese along the correct lines, it is essential that first of all you have some knowledge of the modern spoken language. This is so important that it is well to emphasize it strongly from the beginning - contrary to what you may think, you undoubtedly will not make the best progress in learning to read Japanese by starting in your study of the language directly with reading and writing in the Japanese script. Some previous work with spoken Japanese is essential, and the better your grasp of the patterns and the forms of the spoken language, the faster and surer will be your progress with the reading materials in this book" (Roy Andrew Miller, _A Japanese Reader_).
there are "encrypted" hdls, but encrypted no more than dvd discs. Find a right irc channel and ask for keys
I don't know. I think it was more like an agreement from most parties that it is better (more efficient, more moral, etc.) to have affordable healthcare for everybody, rather than dealing with a grossly sick underclass.
Look up git reflog or so.
Interesting opinion. However, this isn't current U.S. law. See, for example, the numerous successful suits against Toyota for unintended acceleration.

In addition, it may soon be questionable to say that an occupant of a vehicle is "operating" said vehicle if it drives entirely autonomously.
Obviously it's the minority (<1%) of users who are abusing the drug at anywhere near that level. I'm sure that the positive effects are long gone by that point, but I think people who abuse to that extent have become dependent on MDMA in order to feel normal (or have convinced themselves that this is the case). While I'm sure most of these people aren't consuming pure MDMA, there's likely some cases where it is still what's really being used.

Anyways, I've seen many people claiming to have done this (or having friends who have done this) on Reddit:

https://www.reddit.com/r/Drugs/comments/45sgbe/daily_mdma_us...
https://www.reddit.com/r/Drugs/comments/1ldmka/so_really_how...
https://www.reddit.com/r/MDMA/comments/2765jb/i_used_100mg_o...
https://www.reddit.com/r/MDMA/comments/211jkj/ive_been_rolli...

Lots of these cases may be false, but there still seems to be at least a few real cases of this. Of course, that's no reason to make the substance illegal (and especially not a reason to stop studies attempting to use it). Better regulation as a controlled substance is probably the best solution (i.e. enforcing the 2-3 month rule by only allowing MDMA to be purchased at those intervals).
> I expect they'd run at like, several thousand USD per device...

You're guessing about an order of magnitude too low, actually. The VU9P FPGAs Amazon is using cost between $30,000 and $55,000 each, depending on the speed grade.

Yes, this means a fully equipped F1 instance costs nearly half a million dollars. Don't count on the instances being cheap to run.
The evidence is mounting that we are confusing cause and effect with drug addiction. Here's a good video about it from Kurzgesagt [1].

[1] https://www.youtube.com/watch?v=ao8L-0nSYzg

Other than your initial point or two I agree with your post. I can understand why a society would have wanted prohibition; people are more generally more productive without drugs.

We can barely manage tobacco and alcohol. Drug abuse continues to exist everywhere facilitated by prescription abuse.

I don't believe we should mandate what people can and cannot do with their bodies, but I am concerned about our increasing desire to escape the real world through pharmaceuticals or entertainment.
DigitalOcean must implement a centralized firewall. This is the missing piece for them.
" Each Snowmobile consumes about 350 KW of AC power"

Wow.
The Case for Memorizing Poetry
Doesn't Google already have a web based, but internal only, IDE? I don't know if that'd be easy to make external, but my understanding is that they've got a lot of internal users on it.
The situation in which you want definitely want SSH enabled on a pi is when you are running it headlessly. This is an incredibly common use case, especially for more experienced users. In such a setup there is no opportunity to ask the user anything.
To ask for more is just to be greedy?
I arrived at the figure by using the numbers I also shared in my post. I didn't know that the $999 was a minimum though, i just added it as a fixed cost.

If you aren't ready for my small project, you aren't targeting small business. We have a small team and the revenue to support it, which is well into the average revenue statistically for small business.

If you don't hire within SF, your numbers don't make sense.
This is great for a simple setup. Obviously there’s no reason not to extend this beyond AWS—

* Login credentials: feed in response to detected phishing emails

* SSH keys: have SSH trigger an alert if certain keys log in

* Database entries: filter out the special ones in legitimate queries

The pain, at least for a small organization, is in managing: reacting appropriately to alerts, ensuring honeytokens are properly rotated.
Get Rid of Sinus Infection in 30 Seconds with This Simple Method
TSA is a parody of itself.
Trump signals the end of 'Made in China'
I LOVE this story. So interesting, particularly Crowhurst and Moitessier. A lot to find:

http://www.pbs.org/independentlens/deepwater/

Short tv doc on Crowhurst, how he fabricated his position to try and fake success:

https://www.youtube.com/watch?v=ZNpaP8TmGkU

Moitessier, who just decided to quit the race and stay at sea:

https://en.wikipedia.org/wiki/Bernard_Moitessier
Tried it twice, and it crashed big time.
Please be extremely frugal with your money. I would be very sad to come back in a couple of years to hear that you shut down. I've seen it too many times.

> Simplest answer is we're going to use the money to build a product that people want.

Obviously. But will spend money on extras? A nice sign for the office? A better office? Nice chairs? Phones for the conference room? These are the kind of not-very-expensive extras companies spend on when they get money. But it adds up, and without an ongoing revenue source it's not worth it.

If you can budget no more than $100K per year, you'll keep your company alive for a long time.
> It was "click on anything and it plays".

> Now it's "click on anything and it plays. Unless your offline. Where search also doesn't work. And when you're offline you probably go somewhere else to see what you're looking for. When online we need to show that too. And buttons to let you download or see the download status. And....

But it seems that your 'before' is not quite right: if you can't download videos, then it's "click on anything and it plays, unless you're offline", full stop! That is, by allowing download of videos, a total-failure mode has been replaced by a mode with different functionality. Although it's technically true that "you can do some things" is more complicated than "you can't do anything", I'm not sure that it's really worse.
I would not count on these instances being useful for development.

The parts Amazon is using cost somewhere between $30K and $50K each. Hourly costs will be substantial -- it will likely be cheaper to buy an entry-level development board than to spin up an F1 instance every time you want to run your design.
It's all self hosted.
People please use any other alternatives, than facebook, google or twitter technology.

They delete your accounts, suppress your opinions, hide your site and whole lot more, when your agenda is not alaigned right with theirs.
I found the more interesting part: Even without the proposed tariffs, Chinese-made goods are already becoming less competive in the U.S. "Made in China is disappearing from the U.S. market," one story warned last year. The story has been circulated by Chinese media several times since. The article explained that of seven items of sportswear the reporter bought in the U.S., including brands like Nike and Adidas, only one was made in China. The others were made in Vietnam, Indonesia, Egypt, Bangladesh or Thailand.

Do they have the infrastructure for large scale orders?
WTF? I never did anything with wordpress plugins in my life. And even if I did, so what? You have some elitism / insecurity issues.
> If you are looking to hire people and this ends up being a viable alternative you will save a ton of money.

That's my point, its not a viable alternative ESPECIALLY if you are going to hire people.
The solution I proposed is for all that other data.
I think all the tools were there (though not necessarily the compute power). What was missing for me (20 years ago) was the guidance to find these sorts of possibilities. Parents and teachers were no help and there was no google to find the right geocities pages. I wish I'd known about linux and all the free oss tools much earlier.
One point: Most Bio type people have not had multi-variable calculus, and many have not had calculus at all. So, it's not really that they can't process it fast enough, it's that their techniques for processing it are stereotypical. They figured out how to do that T-Test (or something else) once, and they stick with it, because they really don't know he math behind it.

Also, though there is a TON of 'data' coming in, most of it is not useful. For example, I have a 500Gb file of a stack of .tiff images per fish that I have imaged in a confocal microscope. I have a GFP filter on the scope and therefore only get the green part of the .tiff files exposed, the red and blue are just background noise. Also, most of the image is the dish I have the fishes in. I tickle the fish, they flick their tails, and I see this all in 120fps. Now, I measure how much of an angle the fish made their tails flick, all in 3-D, because that's what the scope records in. I have a half TB per fish to comb through, and I have ~20 fishes, say ~10TB. At the end, I get a single graph comparing the fish with some gene to those without it, and I have 10TB of 'data' left over. Yeah, someone could comb through it all and find something else to look at. But i forgot to record the precise temperatures, the orientation of the fish, the fish that I knew later died, etc. I had that all in my head. And, hey, what do you know?, the p-value is ~.45 and therefore there is no 'real' difference in the fish and we can't include this in a paper. Now all that 'data' is being kept on a drive on some computer somewhere and is counted towards the budget that the lab has on the shared spaces. It's not really 'data' anymore, in that it is useful to advancing knowledge for anyone (it counts as practice I guess), but it still clogs up space.
First of all, if you have a family history of mental disorders, like schizophrenia or bipolar, I would highly recommend not trying any psychedelic. There exists a possibility of uncovering a latent genetic mental disorder that otherwise would not have appeared.

That said, I wholeheartedly recommend starting with Marijuana. It's not a psychedelic in the traditional sense, but it's a good introduction to mind-altering drugs.

From there, either LSD or Mushrooms. These are more typical psychedelic experiences, with long durations. LSD can last 8-12 hours depending on the dose, mushrooms is 6-10, depending on dose and species. I personally prefer LSD, but you should be cautious and ensure you get legitimate stuff. Taking precautions and purchasing off the internet from a reputable vendor is the safest method in my opinion.

When it comes to LSD, I always recommend swallowing over just leaving it on your tongue, especially if purchased from an irreputable source. The common Research Chemicals sold as LSD are only activated sublingually (through the mouth/gums), whereas LSD is sublingual and oral. For a newbie 100ug I believe is considered a light dose, although you could go down to 50ug and still feel Something.

General trip advise would be to ensure you're in a positive mindset, and in a secure, safe, comfortable setting. Taking a walk through nature is nice, as well. Set and setting is important.

I would advise against trying any research chemicals (25X or 2C-X, where X=[A-Z]), just because they are newer and less known, but with the right research and precautions you could be okay.

Avoid mixing until you know what you're doing. A lot of people like to mix marijuana and lsd/shrooms, and I find it definitely intensifies the experience, while clouding it a little.

Overall, you should conduct a lot of research and read people's trip reports so you know what you're getting into. Tripsit.me is a wonderful resource, and includes an IRC channel for support and conversation while tripping. Erowid is a popular site for information as well.
Ask HN: Why does the Tesla randomly stop 3 times in the Autopilot video?
Amazon vs Google for tech. Does Amazon really stand a chance?
Come hack on Jet.com on Bugcrowd
> An off-the-shelf corporate data scientist, while a critical thinker, lacks the domain specific knowledge to be able to ask good questions.

Yeah, I think that's the other side of the coin here (for biomedicine and most other fields). Even if better engineering is what is most greatly needed, it's not just about just being purely good at code and logistics. As a corollary, if you do have the domain knowledge and are in a position to act, you don't have to be an ace programmer to make a huge difference.
> If one partner earns more and pays a disproportionate amount of the downpayment and mortgage on a house that both partners live in, is the house really only 1/2 theirs legally? Ethically?

That's a thoughtful question, and I'm not sure why it's being down-voted. It's literally one of the questions a court considers, when judging matrimonial division of wealth.

On the one hand, in the case of a spouse who brings significant assets to the marriage, or provides a significant monetary contribution to the marriage, their contribution is recognised.

But also, a spouse who has sacrificed a high-value career to care for children is given credit for that in a settlement, so that they do not lose out on their opportunity-cost. There are fairly standard practices for courts to determine the relative merits/costs when judging a settlement.

All that said, does it happen all that often that a court decides that the assets should be divided far from 50:50?
>This sort of code is trivial in a dynamically typed language: there's no requirement to prove anything ... we just go ahead and use whatever code we like, and hope that we didn't mess something up.
>the [Haskell] type system quite rightly forbids us from writing the unsafe version that dynamic languages would accept

As somebody with familiarity with Haskell in the small but not in large applications, is it not similarly trivial to write in Haskell with the same level of safety that dynamic languages give you (i.e. explosion at runtime)?

Consider as an example the following safe lookup function (standard Haskell)

lookup :: Eq a => a -> [(a, b)] -> Maybe b
lookup _ [] = Nothing
lookup x ((a, b):abs)
| a == x = Just b
| otherwise = lookup x abs

and the following unsafe lookup function

lookup :: Eq a => a -> [(a, b)] -> b
lookup x ((a, b):abs)
| a == x = b
| otherwise = lookup x abs

How is the latter any worse than lookup in a dynamically-typed language where you are assuming the lookup will succeed (and hence don't write any error handling logic)? I know this sort of code is frowned upon in Haskell, but it can be done.
> ... and can provision Docker in a few clicks

One cool thing about docker is I can write a build file for generating reproducible-ish builds that I can run or rebuild anywhere. My Dockerfile can be version-controlled, copied, edited as code, etc.

But now to run it the official docker way in aws I have to click around a gui? That means no version-controlling my configuration, copying the same configuration to a new staging environment, it's harder to test a config change in staging and then be sure I roll out the exact same change to production, etc.

It seems like more and more of these infrastructure SAAS products are focusing on this kind of pointy clicky experience, and I don't understand why. Sure, a gui is potentially easier for deploying a quick sample app to try it out. But how is the "hello world" market lucrative enough that all these companies seem to focus on it to the exclusion of the "professional developer wanting to use modern best practices" market?

EDIT: and this obviously isn't exclusive to this product. AWS itself does the exact same thing. You have to go out of your way to find something like Terraform that lets you treat your infrastructure as code, and nowhere that I've seen does AWS itself encourage any kind of reproducibility of infrastructure or other kinds of workflow best practices.
A month before a heart attack, your body gives you a warning
Show HN: Automate Multiple RSS Feeds in Mailchimp Newsletters