skysight-inc/HackerNews huggingface dataset:
Hi dineshp, I'm Pablo Carranza, and I am currently leading the effort on infrastructure at GitLab, this includes performance.

> what are the major areas where you face performance issues?
> Does the choice of RoR play a part in performance problems?

RoR does not play a part in the performance problems as much as any other language choice.
Our performance problems come from at least 3 different fronts: lack of caching in some specific points making us call the same complex/slow operations many times, NFS (filesystem) performance as a whole, and algorithms that worked really well at small scale, but not anymore, both at app level and at DB level.

I think that RoR is a really good option for building a product fact, and eventually it is necessary to start specializing specific parts that do not perform anymore, and just replacing what cannot be specialized. The key element here is that we need to measure first to see where the problem is.

> Can performance be improved by deploying more servers?

Not really, more front end servers means more load on the NFS backend, so there is no easy solution here.
The first step into fixing this issue right now is this one: https://gitlab.com/gitlab-com/infrastructure/issues/139 and then we will start playing with a distributed FS as a longer term solution: https://gitlab.com/gitlab-com/operations/issues/1

As a final note, we are using some specific issues to measure performance as a blackbox, and those issues are seeing some really good progress lately. So stay tuned :)

https://www.youtube.com/watch?v=czes-oa0yik
I count these as the more respected tech new sources:

Recode
Verge
Wall Street Journal (paywall)

and surprisingly enough BuzzFeed's long form content is actually quite strong. They had some excellent investigative journalism on Zenefits this last year.

Techmeme is the best aggregator I've found.
Elon Musk is an Nth mover (where N is high) in almost everything he's done. Electric cars are not a new concept (they actually predate ICE cars in some ways). Rockets are not a new concept. Paypal was kind of innovative conceptually at the time but in a boring way.
The difference between a startup like Theranos/uBeam and a product based "soft"ware company is that in one case you can make outrageous claims and then use sheer determination, will power and lots of money to make it happen retroactively but you cannot change the fundamental laws of nature.
https://news.ycombinator.com/item?id=12055375
Note that if you're doing this in a terminal, you MIGHT need to first do `%cpaste`:

https://gist.github.com/gknoy/97de793db43ff7ea4d4bfd1bab96b4...

It seemed to work fine when I didn't use `%cpaste`, though, so that might be a vestigial thing? (I remember having to do it on my linux workstation, so maybe it's just my mac terminal behaving differently.)

The cool thing about directly pasting is, if you fumble the indentation like I did when I initially pasted, it auto-indents for you. ;)
re: 2, this is covered quite clearly in the abstract: "There is no relationship between county-level racial bias in police shootings and crime rates (even race-specific crime rates), meaning that the racial bias observed in police shootings in this data set is not explainable as a response to local-level crime rates."
There'a crate for that ;) https://github.com/pnkfelix/tango
Not to look a gift horse in the mouth, but it seems like Corrode misses some other chances to use idiomatic Rust:

1. Rust fn:main doesn't need to return something.

2. The arguments to main aren't mutated, so Rust doesn't need to declare them as mutable.

3. Ditto for the argument to printf.

Anyone know how easy it is to recognize and code for such cases in the transpiler?

Edit: It looks like they might have opposite design goals [1]: "Corrode aims to produce Rust source code which behaves exactly the same way that the original C source behaved, if the input is free of undefined and implementation-defined behavior. ... If a programmer went to the trouble to put something in, I want it in the translated output; if it's not necessary, we can let the Rust compiler warn about it." (Edit2: cleaned up and numbered)

[1] https://github.com/jameysharp/corrode#design-principles
This is very interesting. Thanks for this data.

I've been using the StackOverflow Careers RSS feeds for a small script I've been writing, which gives a good view of Now... maybe I can find something interesting in your data for the Past
Anything that interferes with the absorption of nutrients. Most commonly they would be inhibitors of the catalysts that break down foods.
https://en.wikipedia.org/wiki/Antinutrient

But yes, the paleo theories mostly focus on it having some kind of effect on your stomach lining. It's a bit speculative. It seems they all point to this discussion as the original source: http://thepaleodiet.blogspot.com/2010/01/paleo-diet-q-29-jan...
So assume all police are lying and the people fighting them had good intentions?
The Future of Browser History
One-off unscientific anecdote: A weightlifting coach advised me to take MSM for improved recovery. I tried it for a few months and never noticed a difference, but I was eating plenty of eggs during that time.
it does mean that, yes.
It's not just people at FP Complete `et al` who were frustrated with cabal-install. Cabal-hell was the number one problem with using Haskell for commercial projects, as a majority of people responded who took part in a FP Complete Haskell organized survey (this was before stack was officially released), https://www.fpcomplete.com/blog/2015/05/thousand-user-haskel.... I wrote that survey, if you want to discuss methodology I would be happy to. Anecdotally, after contacting by email most of the respondents, very few complained of a fragmentation the community by virtue of stack being released. We could argue all day over about arguments on reddit, but data suggests otherwise.

FP Complete is a commercial consulting firm now targeting life science companies. We use Haskell because we believe it is the best modern language for robust high-performance software. It is not our goal to own the Haskell ecosystem. It is our goal to promote Haskell use and to develop reliable tooling. Do people feel the same way about continuum analytics for releasing Anaconda and special versions of numpy?
"0.7w seems low to me for standby."

Affordable for many, yes, but IMO, for a LED lightbulb, that's horrendous. Let's do the math: 0.7W full time is 16,8 Wh a day.

In use, a 1000 lumen bright LED bulb uses 14W. If we assume you have it switched on 12 hours a day, that 0.7W at 24 hours a day adds 10% to its power usage.

More realistically, your LED bulb will take 10W, and be on for four hours a day, for 40Wh. That means that 16,8Wh adds almost 50% to your power usage.

And I think the typical bulb will use even less power and an even lower duty cycle. 8W, 2 hours a day, and you have a radio broadcaster that occasionally switches on a light :-)

By the way, 0.7W also is also against the spirit of an EU directive on power consumption (http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:320...), which by now (it's over four years after this came into force) states:

The power consumption of equipment in any condition providing only a reactivation function, or providing only a reactivation function and a mere indication of enabled reactivation function, shall not exceed 0,50 W.
In Submissions

[...]

Please submit the original source. If a post reports on something found on another site, submit the latter.

https://news.ycombinator.com/newsguidelines.html
Also, Stratio's lucene index:
https://github.com/Stratio/cassandra-lucene-index
Hey thanks for sharing! Hope people enjoy the post and the customer focused angle.
Many of the items you've listed here are more cultural than technical.

Confidently saying "we're going to use this because I'm senior and I know it's the right choice" is something that I see as rarely being successful. It has its time and place, but it ain't something that you should be doing often. You mention bringing the right coworkers into the decision-making process, which is the more appropriate move.

Now, of the items you mention not having an answer to, I can give you a definitive answer to a few: ;)

- Do you need CI? Yes if it's helping you manage your risk. Do you need it for your 4 line bash script? Probably not. For an application your team works on regularly? It's 2016; Set up a freakin' CI server.

- Absolute security? This doesn't exist. Manage your risk, and the risk to the business. See: http://scholar.harvard.edu/files/mickens/files/thisworldofou...

- Tests? Yes. As far as coverage and what type, again, manage your risk. Does decent unit-level coverage and a couple of end-to-end tests give your team the confidence they need to develop quality software at a rate acceptable to your team and the business?

- Do we need code reviews? Dealer's choice, but manage your risk. If you're getting heavy on tech debt, or there are big holes in team understanding, and you notice work is slowing down as a result, you haven't been managing your risk.

- Most appropriate database? Manage your risk. Have the conversations with the business that give you a framework for doing a meaningful evaluation.

- Shaving millis off of load times? What value does this represent to the business? There's a cost/benefit analysis that needs to be done to determine whether two weeks of work improving the response time by 10ms produces noticeable business value. Perhaps there's a way to shave 8ms off the response time which will only take one day. How much are 2ms worth to the business?

--

It's nice to have personal experience answering questions like the ones you bring up, but I would only say "nice to have". As the previous poster mentioned, senior is IMO about being communicative and thoughtful more than it is about being right all the time or having a lot of battle scars to show off.
>> what I’m looking for in this transaction is not sexual satisfaction. Do you like everyone at your job?

Obvious Jane says: no, but I'm also not expected to screw them.
> at the current price point, this device is actually plausible to experiment with

Yes, that's really the idea at this point. Neither the hardware nor the firmware have been audited, so it would be unwise to use it to protect anything mission-critical at the moment (which is why there's a big disclaimer to that effect on the order page). On the other hand, the approach does seem to be holding up well to initial scrutiny.
No, my point is that you can both stop poaching and preserve antiques, but not with a blind, total ban on trade.
I did some (very quick) comparisons to Britain's police force, which for the most part is not armed. I ignored the confounding effects of the illegality of guns, assuming that most hardened criminals will be able to get one anyway (a convenient assumption, since I wasn't sure how to account for it any other way).

British police are significantly less likely to be killed, or to kill, than U.S. officers. However, they are also much more likely to be assaulted, and injured in those assaults, than U.S. officers.
> I am always amazed by your trust in our injustice system.

Our justice system gets a lot of things wrong. I'm pretty sure I've called the federal sentencing guidelines "quackery." I don't think plea bargaining is one of the things it gets wrong.

At one point, it was part of my job to read plea colloquies, in a city that saw a lot of drug prosecutions. They looked nothing like what the article is describing. I'm skeptical of the article on the basis of observation and experience, not faith.[1]

> I am sure that all of the filed paperwork for the plea colloquy looks proper. There was the formulaic set of things to be asked and said, and all were asked and said.

What goes on file is the transcript of the oral hearing. If, as the article asserts, the defendant was too much in tears to say anything intelligible besides "guilty" the transcript would reflect that and you would not have a proper colloquy.

[1] Relevant: https://news.ycombinator.com/item?id=5203871 ("Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the “wet streets cause rain” stories. Paper’s full of them. In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.")
If state law in Texas is like Georgia, then it's much harder to stifle the 2nd amendment in a public space at a public event. If it were a registered protest, maybe, or if the organizers of a private event asked people not to carry.

A lot of the argument for the 2nd amendment is that if you put people open carrying, the gov/police will show more restraint, allowing for 1st amendment practice. https://www.youtube.com/watch?v=mSwys0dm_EA
Someone discovered the default functions called on every chat message can be swapped out to process scripts remotely. I'm interested to see if Blizzard does anything to fix this and what the timeline will be for the resolution.

http://us.battle.net/wow/en/forum/topic/20745644941?page=1

https://gist.github.com/Sharparam/11a3cddeaa51aa11dde69b4690...
Can't stop the Signal.

Cute.
T-Mobile gives away your sim card to impersonators
Do they break that down by type of crime? "Crime rate" is a pretty broad term.
I think it is odd that the title has the string "free and open source" prepended to it. It seems like that isn't the primary feature.
What kind of negative experiences have you had with recruiters? I don't know much about them...
Simirarly for the fifth, did it get truncated? I think it is supposed to be:
http://www.sheldrickwildlifetrust.org/asp/orphan_profile.asp...
Like the "hide Library folder always" I hope there's some CLI command to run to override it. Not that I run with hidden files visible, but I can see why it would be useful.
Its been many years since I saw the presentation but I believe they made a point of mentioning that ring buffers are not a new technology. They mentioned how such data structures are used all the time in network devices, such as switches.

The disruptor is an actual library based on ring buffers (other have already pointed out some other features).

In practical terms, this library has been immensely popular. Within a couple years of disruptor's release, practically every (java based) trading firm was using them. I've even seen c++ implementations, only vaguely resembling this library, being referred to as 'disruptors.'

Beyond the library, the original (and subsequent) presentations by the authors popularized the concepts of "mechanical sympathy" (understand the abstraction layer beneath what you are currently working in) and introduced a whole new set of developers to modern CPUs, thinking about cache lines, etc.
The source for that detail is here: https://bugs.chromium.org/p/project-zero/issues/detail?id=82...

> I think Symantec's mail server guessed the password "infected" and crashed (this password is commonly used among antivirus vendors to exchange samples), because they asked if they had missed a report I sent.

> They had missed the report, so I sent it again with a randomly generated password.

I'm not 100% sure I buy it. The follow up comment is about how he had mistakenly sent them a wrong testcase, and he had sent them similar exploits in a zip with the password infected before (see https://bugs.chromium.org/p/project-zero/issues/detail?id=81... from April 28th).

It would be incredible for Symantec to guess the password "infected" for ZIP files. It's possible though!
How? Where would this ivory come from? Why is it important for the pianos to have ivory inlays?
US 2015 Fundamentals:

Number of gun deaths: ~13,000

Number of police officers killed: ~50

Number of civilians killed by police: ~1200

Number of white civilians killed: ~500

Number of black civilians killed: ~330

Percentage of blacks in the population: ~13

We have a large death-by-gun problem in the US as compared to the rest of the OECD.
Guess ycombinator is automatically truncating it.
Raw eggs contain the anti-nutrient avidin, a protein that binds to biotin, making it unavailable as vitamin. If you eat large quantities of raw egg you could theoretically become biotin deficient. Cooking destroys the avidin.

https://en.wikipedia.org/wiki/Avidin

Additionally, the protein in raw eggs is poorly absorbed, which could be because of other unidentified anti-nutrients. Again, cooking solves the problem:

http://jn.nutrition.org/content/128/10/1716.full
A few points:

1. "In contrast to previous work that relied on the FBI’s Supplemental Homicide Reports that were constructed from self-reported cases of police-involved homicide, this data set is less likely to be biased by police reporting practices."

I'm very interested to see if the full article (which is timing out so I can't check) goes into detail on what the reporting practices are, how they are biased, and how this data set solves those biases.

2. I'm curious how this data compares to the Guardian's study: http://www.theguardian.com/us-news/ng-interactive/2015/jun/0...

In particular, this database shows that in 2015, while the "per million" count of blacks killed was 7.27 (and for whites was 2.93), the "in total" count was 306 for blacks and 581 for whites. The statistics in 2016 so far are not much better: 3.23/1.41 respectively per million, and 136/279 in total. Ideally this number would be 0 for all counts, but we don't live in that world.

3. This report was published at the end of 2015, and unfortunately we have seen a massive spike in killings since then. Further, their dataset (according to the title) is only from 2011 to 2014. Is anyone working on a follow-up study using more recent data?
Nice article, thanks for sharing.

In addition to the great information you've provided, we identify features/decisions/choices that require spikes based on assessment that they are high risk and/or we have low confidence in our estimate. In addition, we sometimes use spikes to break xxx-large estimates into smaller pieces.

That way, until a project/plan/item has low (enough) risk, and high (enough) confidence, it needs more spikes.

We've found this model is a great way of identifying what needs spiking, and when a spike is 'done'. And, it also helps to ensure spikes don't end up consuming too much time.

Hope this is useful.
Shoe company uses blockchain to allow customers to authenticate sneakers
Wow. That's pretty cool! I had wrapped ipython in a helper that would always have the Right PYTHON_PATH, but it's neat to see just how much of ipython I have not been using. Thank you for sharing that.
Google and Facebook use Oracle, not for their main product but for their back-office.

eBay also used Oracle in the past (if you dig back in the past of their architecture/design of eBay).
The reason Rhinos are being poached is the fact that its illegal to take their horns. They cannot farm Rhinos for horns so wild Rhinos end up being killed for it. It is in fact possible to remove Rhino horns safely eithout killing the animal.

This issue is another perfect example of people thousands of miles away detached being in an uproar with half assed solutions like "ban ivory" which exacerbate the problem.
I don't believe the claim unless the author lives somewhere with near-zero humidity all the time. People use these frequently for homebrew ("keezer" a la "kegerator"). The condensation is a frequent complaint. If you leave the walls of your fridge wet for months at a time, they will mold. It's cold, but not so cold that no molds can grow. And sponging out a chest freezer full of food is a giant pain in the ass because of, you know, all the food.
Ship Entrance Crash Compilation
Surely with Apple's new file system, the need for these files will go away, right? I'm sure they can bake whatever metadata these directories need right into the fs.
Introducing Verve: minimalistic, static, functional language
Did you read the page? In Estonia plea bargains are limited to cases that carry a maximum of four years of imprisonment and a max of 25% can be shaved off of the sentence. In France, plea bargains are limited to crimes that carry a maximum of one year. Poland has a maximum of ten years. Note that this does not preclude anyone from pleading guilty without a plea bargain.

The whole point is to reduce court load and create leverage when trying to convince someone to become state's witness (which is controversial as well). How does placing concrete limits on plea bargains doing away with them "entirely or almost entirely?" Estonia's and France's systems, for example, allow largely unrestricted plea bargains in the vast majority of cases, thus freeing the court for more important cases that carry large penalties while preventing prosecutors from bullying innocent people for the sake of their career.
Bystander reports are often worth about as much as a deputy's report.

People can't remember how things actually happened.
> Poor defendants don't often get convictions vacated.

Not for want of trying.

There are tons of prisoners who file pro se collateral attacks on their convictions and pleas from prison. So many so that at the court where I worked, there was an entire group of staff attorneys dedicated to reviewing such applications. A plea colloquy like the one described in the article would've stuck out like a sore thumb.
Does the EOD tech running the robot get paid leave like a shooting investigation? Is it an officer-involved bombing?

I'm guessing this situation needs a bunch of new procedures and definitions now.
Assume that people are rational actors that will have no problem lying when their career and/or freedom are at risk. Especially if they aren't aware of any contradictory evidence.
I would not bet my life savings against the NSA's ability to compromise the SC4-HSM if they had physical possession. On the other hand, there is a simple defense against physical compromise that would probably hold up even against the NSA: encrypt the keys with a strong pass phrase. That wouldn't hold up against a monkey wrench attack (https://xkcd.com/538/), of course, but there are limits to what you can do for <$100.
It's actually my sloppiness. I copied the link from a comment I made previously and didn't check it. Fixed now.