skysight-inc/HackerNews huggingface dataset:
The real tricky part is making an algorithm that gives a uniformly random Secret Santa. In fact even generating a random permutation is quite tricky.

Usually the easiest is to just use Fisher-Yates to get a random permutation and reject it if it doesn't fit the criteria (people shouldn't pick themselves etc.). For nearly all practical problem sizes this should be fast enough.

Trying to reject permutation half way and fixing it has a tendency to cause subtle biases.
I think the answer is most likely "no". And I'd be surprised if it even gets kickbacks from leasing companies since it is already getting nice cars on the road with motivated drivers who have to pay their leases off with earnings.
Hype then cool-down is a sine wave... See, not linear at all!
My bet would be that it was a placeholder title while they were finalizing editing the piece.
Another alternative is Transit. It supports bike share data in many cities, including New York. However, it's likely it suffers from the same data quality issues as the Citi Bike app.
Well, with that logic you can bring guns right back into the equation. Soldering guns, glue guns, nail guns, vaccination guns...
Danke schoen :)
it's a shame that algorithm improvements would necessitate a shift away from the name "gzip". It would be better if the intent to compress/decompress was orthogonal to the features of the implementation (compression ratio, speed, split-ability, etc...)
This thing doesn't really have anything to do with zero knowledge proofs.
The headline is (I suspect deliberately) misleading. The actual guardian article is here [1] and is more nuanced, claiming that "One giant container ship can emit almost the same amount of cancer and asthma-causing chemicals as 50m cars, study finds". That's hardly "all the cars in the world", given that latest estimates put this figure at over 1 billion [2].

Ships produce more sulfur oxide pollutants than cars because they use different fuels, and thus produce different pollutants.

A large container ship does produce more more carbon dioxide emission per mile and per gallon of fuel than a car, but ships in general have the lowest emission levels of any other method of cargo transport , producing fewer emissions per ton of freight per mile than barges, trains or trucks.

[1] http://www.theguardian.com/environment/2009/apr/09/shipping-...

[2] https://en.wikipedia.org/wiki/Motor_vehicle
Show HN: Tj – a command line utility for the hardcore WordPress developer
You are being silently down voted because nobody wants to engage the sea-lion.

There are not "legitimate concerns being raised", it's just you, complaining that money is being spent on "political correctness", as if it's somehow bad for a non-profit to spend money to try and improve the world.

Your other points are no more legitimate than "gay used to be a perfectly good word meaning happy how can we express happiness without it" or "people don't object to me when I call them [racial slur] so it must be fine" have ever been. Addressing them directly is exactly the sort of fruitless conversation derail nobody wants.
This was exactly the situation at a previous employer. There were twice as many testers as devs. I tried to advocate for comprehensive unit and integration tests and was met with a "if it doesn't work, the testers will let us know" attitude. Baffling. At my current job we have no QA testers and write much better software.
I found lz4 to be the best for general purpose analysis, it increased the throughput of my processing 10x compared to bz2. Then if you're working with very large files you can use the splittable version of lz4, 4mc, which also works as a Hadoop InputFormat. I just wish they would switch the Common Crawl archives to lz4.

I should probably mention the compression ratio was slightly worse than bz2 (maybe 15% larger archive) but for the 10x increase in throughput I didn't really mind that much. I could actually analyze my data from my laptop!
tar ... tape archive
Qurbani  Eid-ul-Adha ''Travelling''
No. Neither is gz. In fact, how could any compression algorithm possibly be greppable?
I love articles like this! They took the extra step o find the reason for the behavior they saw rather than just saying "yup ...that's how it is".

I've also had multi-millisecond pauses on ext4. In my case, the buffer fills up and we wait for sync_dirty_buffers to do it's job. We tried adjusting the various settings to tune it, but at the end, it's always there. We ended up buffering our own writes in the application so we could get the behavior we wanted.
....or just use gzip and get 90% of the value minus the high probability that that setting will be fubar at an inconvenient time.
That is strange, since far-right usually means strong adherence to property rights. 'Course, "right" isn't all one thing.
No, I didn't know that. Thanks.
That reminds me of some research I read a few years ago about savings habits for people speaking a language that has a time concept (passed, present, future) and languages that don't (chinese for example).

There was a pretty explicit separation on people having no time concepts saving more and putting more importance in the future, or something like that. It was pretty interesting to read how the way your language works shapes how your brain thinks and influences your habits.

edit: https://www.ted.com/talks/keith_chen_could_your_language_aff... :)
We've been hearing about "smart weapons" since the early 90s and they don't seem to be getting any more intelligent as time has gone on.
Ask HN: How should a small high school manage data?
We're a small public high school. Our main student records are maintained by the department of Education in a database that's difficult to access and out of our control. At the moment we run a violent disarray of google sheets and docs trying to manage all the different types of assessment data, compliance records, notes from meetings and professional development and teachers calling home and safety issues and attendance...

And it's overwhelming; I think what we need is a database so I started teaching myself about how to build them from scratch- and my vision, what I think we need, is a staff-only subdomain where teachers can access a dashboard of information about students and input new types of information-- and I'm caught; building our own solution seems like the only viable option at the same time as seeming like a huge mistake.

How do you approach planning for a data management problem like this?

Do you know of any (inexpensive?) services that we can look at for relief / support / guidance?
Everyone should read this comment above^^

EVERYONE.
I think the best defense against the misuse of nuclear weapons is to empower as many people as possible to have nuclear weapons. If everyone has nuclear weapons powers, then there’s not any one person or a small set of individuals who can have nuclear weapons superpower.

Yeah, right.
You're talking about California Labor Code 2870:

http://www.leginfo.ca.gov/cgi-bin/displaycode?section=lab&gr...

The gist: Has to be your own time, own resources, and not related to your company's current or anticipated business.

It's that last clause that's the kicker. For some companies, it's easy to prove -- e.g. wood working is probably unrelated to Twitter's businesses. For other companies that literally work on everything (eg. Google/Alphabet)... it's a pain in the ass. Also: Many companies make reporting the IP or side work a stipulation of your employment contract -- ostensibly, so they can claim that it is related to their business in some way. It's right there in Code 2871. :)

PS: IANAL.
Or used a signed assertion that is verified with a secret.
I'm not sure how you read my post and got that message out of it without purposefully being dishonest.

>Civil liberties should not hinge on looking just like everyone else, following the herd, never daring to be creative, etc.

You even explicitly went out of your way to ignore what I said.

>Now before anyone tries to wage some sort of moral war against me for stating how things are. I don't pass any judgement on if this is "good" or "bad" behavior.

Take your moral war elsewhere, because I'm not interested in this discussion. I'm stating how things are. Not how they "should" be. Not how you want them to be. Not how I want them to be. How they currently are. "What they are" and "what they should be" are not the same thing. Am I being patronizing enough to make my point crystal clear?

>never daring to be creative

This is a strawman. Dare to be creative at hackathons and art conventions - not an airport. Ever heard the phrase "time and place"?
The oldest dream in software business, with a long trail of failed attempts at standard components, visualization, and one true language. The dream is like something out of The Matrix, the reality is something like PL/I.
Can someone put in layman's terms the passage below? please. Assuming the reader has some machine learning experience but no financial knowledge. Thanks

#### numerai_training_data.csv
Use this dataset to train your machine learning algorithm. The first fourteen
columns (`f1` - `f14`) are integer features. Column `c1` is a categorical
feature, column validation indicates a dataset that you can use to validate
your model, and target is the binary class you’re trying to predict.
I think one big problem is that job postings these days are either for 1) super-specific skills, which necessitate years of industry experience (or academic research, in a few cases), or 2) experienced all-purpose programmers (to save money over hiring several niche experts).

To use a personal illustration, at my old job I had a group composed of about 85 programmers, 15 QA, 5 devops and 5 DBAs, with a few miscellaneous non-tech people (PMs, analysts, etc). In that situation I was perfectly satisfied hiring junior folks with only 0-3 years of experience but a good head on their shoulders and a desire to learn, because there was enough slack in the larger org to make up for temporary slowdowns resulting from training & general learnings. In my current role, I have a team of a half dozen, 4 of whom are skilled analyst/PM types. If I want to build something, I either have to outsource or hire one of two of the very best "full stack" developers that I can trust to throw anything from front end design to web + mobile dev + data structures & reporting/analysis on the backend. As a result, I'm almost always searching for people with 10 years of industry experience who have spent all that time learning & practicing a diverse set of skills.

I think one thing that'll cause disappearance of low end jobs is the upcoming/increasing automation of business processes using click & drag workflow platforms (and better leaner ERPs & CRMs). My previous employer has about 200 IT people just keeping Oracle running, and another 150 or so working on manufacturing software. I think the majority of those kinds of jobs in big enterprise are going to disappear in the next 10 years... and that's A LOT of jobs. Fortunately, one of the triggers will be retirement of boomers, so it'll be a more natural progression than massive layoffs would be.
From Wikipedia:

> Nurdles that escape from the plastic production process into waterways or oceans have become a significant source of ocean and beach plastic pollution. Marine life is severely threatened by these small pieces of plastic: the creatures that make up the base of the marine food chain, such as krill, are prematurely dying by choking on nurdles.[7] Nurdles have frequently been found in the digestive tracts of various marine creatures, causing physiological damage by leaching plasticizers such as phthalates. Nurdles can carry two types of micropollutants in the marine environment: native plastic additives and hydrophobic pollutants absorbed from seawater. For example, concentrations of PCBs and DDE on nurdles collected from Japanese coastal waters were found to be up to 1 million times higher than the levels detected in surrounding seawater.[8]

https://en.m.wikipedia.org/wiki/Plastic_particle_water_pollu...
Scare people...not arrestable. I think you misread what was written.
New Rust Plugin for JetBrain's IDEs
Apparently it is possible, as zgrep can search gzipped files.
I highly recommend using atool[1], and never worrying about extracting archives again. It's a wrapper around basically every compression/archive tool in remotely common use.

Bonus: it decompresses to a safely-named subdirectory, but moves the contents of that subdirectory back to the current directory if the archive contained exactly one file. Highly convenient without any risk of accidentally expanding 1000 files into the current directory.

After creating this macro, I've basically never had to care about how to decompress/unarchive anything.

# 'x' for 'eXpand'
alias xx='command atool -x'

# use
% cd $UNPACK_DIR # (optional) (can be the PARENT dir)
% xx foo.zip # or .tar.{gz,xz} or whatever
foo.zip: extracted to `foo' (multiple files in root)
% cd foo/
% ls | wc -l
3

atool actually has many other useful features, but it's worth it just for the extractor.

[1] http://www.nongnu.org/atool/
You're right, guns are pure evil. Clearly, we should take them out of the hands of cops, bodyguards, hunters, and civilians defending themselves.
> All the issues you outline are very much tied to education

Yes, they are. What you seem to be ignoring is that those issues cause poor education. Kids in a bad environment are going to have a hard time learning.

Which of course leads to less opportunities and the cycle continues for their children.
Our "capitalist rulers" really just aggregate the preference of... us. We are on no such journey to any sort of perfect world, because we're far from perfect.

But perhaps we're getting better.
It was the HTML doc title, always a legit candidate for original title on HN. In this case, baity though, so we've changed it to one of the alternates.
Another "Well DUH!" moment in scientific research. I wonder who's grant money got pissed away to report the obvious.
I don't see any evidence of that.

You can't take your wealth with you upon death -- Property owners of valuable resources nearly always maximize exploitation in the short term and could give a shit about the long term.

If they took a long term view, we'd still have old growth forest in the US.
Karen Sandler Speaks about IRS Charity Issues
I suppose that's possibly true in Romania (I doubt it, except for the smoking). It's certainly not true in the US (I drive with Uber and Lyft).
My experience is the opposite. Having independent testers means you're going to be stuck addressing 100 defects if you don't watch what you're doing the first time around.
I agree with you that that is a necessary part of the process and a great first step for small teams moving away from "just release it and see if it works". The unfortunate reality though is that people in the same role on the same team often have the same misconceptions and blind spots.

Now getting devs to see the product through the users' eyes goes a long way toward solving that, but if you have a process and team of devs that are doing that you're way ahead of the game in a lot of ways.
I have no idea what you're going on about, your post is all over the place. I'll try to answer as best I can.

> There are not "legitimate concerns being raised", it's just you, complaining that money is being spent on "political correctness", as if it's somehow bad for a non-profit to spend money to try and improve the world.

OK so first of all, a significant amount of my donation money goes to Mozilla - so this is not me complaining where "money is being spent", it's me being concerned my money is not being spent efficiently.

Second, "to try and improve the world", sorry, what? Grepping out technical terms from documentation is "improving the world" now? You have one low bar for improvements. Mozilla's mission is to keep the web open and free (or so they claim), and political correctness has nothing to do with it. Even if it were a worthy goal to reword such docs (it's really not, and I explained why), it is not Mozilla's job. They're not good at that task nor are they a company you donate to to get such tasks done.

As for what you're going on about with the word "gay" or racial slurs... I'm honestly speechless if you're equating them together, let alone equating them to the word "slave" which is used non-offensively in several communities. If you up and decided that slave is a racial slur now, maybe I should just up and decide that your posts offend me and that they should be replaced with rainbows.
That argument is like saying at the introduction of automobiles that cities with the most cars also have the most horses.
Valve's Steam Controller factory is straight out of Portal
The US Dept. Of Ed's 2016 National Education Technology Plan
Are you referring to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=708423 ? If so, it was fixed in December 2013 and doesn't affect the current Debian release (Jessie), although it does affect the previous release (Wheezy) and there is an open request to backport the fix https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=781496
No they aren't (in the US).
http://www.kiehls.com/facial-fuel-energizing-scrub/615.html?...

it's not cheap but a little bit goes a long way. i use pea-sized amount.
It's very common to see xz files in Gentoo as well.

ls /usr/portage/distfiles/ \
| sed 's/.*[.]//g' \
| sort | uniq -c | sort -n -r \
| head -n 6

3377 gz
3051 xz
1656 bz2
295 zip
194 tgz
107 jar
Lovely essay.
Airport and schools have been targets of shootings and bombings. This isn't some wild eyed paranoia.

While it nice to know some people would rather literally die than be politically incorrect, I prefer to take caution around people I perceive to be a threat.

Again, look at the pics of her "device" and her for that matter. A crude triggering mechanism wouldn't look much different from a distance.
Some companies only list "senior" positions with the expectation that more "junior" developers will apply anyway and they can hire people into the appropriate level.
It is a scientific device. See http://www.ipp.mpg.de/16931/einfuehrung which, although it's ambiguous as to whether any fusion at all is intended to happen, makes it clear that actually producing energy is not a goal. http://www.ipp.mpg.de/17064/strahlenschutz does call it a "fusion device" and suggests that after a few years of operation they'll be putting deuterium in and doing something that produces neutrons, so I guess there must be a plan to do some actual fusion. But, still, intended energy output: zero.
You're absolutely right, assuming we're talking about business/consumer apps and not pacemakers or rocket ships.

An incentive/penalty system for bugs without an incentive/penalty system for features completed leads to paralysis. And a complicated incentive system leads to game playing over productivity.

It's an old saying: be careful what you measure because you'll get a lot more of it.
I think that was his point as well.
And I think that you and the great-grandparent missed the point: It's not people, it's sentience that is dangerous.

Since the dawn of time that meant man or gods. Soon that list will need to include computers.
The previously highly-aggregated content of television/radio now seems to have a sort of burnished nostalgic glow. I mean... "Dallas"? Really?

Of course it burned out and faded away, but at least it was something. We now have meta-broadcast in the mold of "Deadwood"; some of it is striking. "Manhattan", "Vikings", "Fargo".

Only Adult Swim seems to be moving content off the Internet onto cable, and I'm not sure about that frankly. It's not like "The Heart She Holler" has an audience other than people who studied Southern Gothic in college/read Faulkner in earnest. It's brilliant but...

We are just talking to ourselves here.

All the power struggles matter now, but they won't later. It'll all be absorbed as dead matter for whatever tree grows in this compost.
Zgrep
Wow, this thread reminded me of the "best HN comeback of all time":

https://news.ycombinator.com/item?id=35079
The trouble with talking to doctors about cancer is that they are in the business of the possible; the bounds of their world are, rightly, what can you do better with what you have today. But for cancer what you have today is a set of pretty terrible tools, and there really isn't much that can be done to make them very much less terrible other than research and development of new tools, the very thing that isn't in the remit of most doctors.

This is one of those situations where you have to look forward.

I think I'm willing to wager that first things worthy of the name cure for cancer will involve temporary suspension of all methods of lengthening telomeres. Not great for you, but worse for a cancer, and you can wait out its withering away. The technology exists now to suppress telomerase and the known ALT gene products via RNAi or similar, with the only challenge being to solve the problem of reliable tissue coverage every time. All cancers depend on telomere lengthening, no exceptions. The real path to the defeat of cancer is to find commonalities, to escape this business of one therapy, one team, one large budget, one ten year hit-and-miss development process for every single one of hundreds of narrow subtypes of cancer. The commonalities don't get more common than telomere lengthening.
Determinants of Meme Popularity [pdf]
When I owned an Amiga they kept on changing the archive format to find a better one that saved space.

The had arc, pak, zip, zoo, warp, lharc, and every Amiga BBS I got on used a different archive format. Everyone had a different opinion on which archive format compressed things in the best way.

I think eventually they decided on lharc when they started to put PD and shareware files on the Internet.

Tar.gz is used because there are instructions for it everywhere and it seems like a majority of free and open source projects archive in it. It is a more popular format than the others right now. Might be because it is an older format and had more ports of it done.

But I really like 7zip, it seems to compress smaller archives, before 7Zip I used to use RAR but WinRAR wasn't open source and 7Zip is so I switched.

With high speed Internet it doesn't seem to matter much anymore unless the file is in over a gigabyte in size. Even then Bit Torrent can be used to download the large files. I think BitTorrent has some sort of compression included with it if I am not mistaken. To compress packets to smaller sizes over the torrent network and then resize them when the client downloads them. That is if compression is turned on and both clients support it.
We have been concerned about that as well - we have our infra built around Flask already.

Im wondering what's a suitable alternative.... with SQLAlchemy and Celery/RQ integration?
Can you provide more detail? What was the option that allowed you to stay another year?

> i ended up writing all docs myself

What documents did you write? I'm curious and don't know much about immigration to the US.
"Security through secrecy on technology has just not worked very often."

Nuclear weapons come to mind. Would we prefer that the knowledge of how to make them be more widespread?
This post lost me at last * 1000 - number of connection should not be counted. I.e. number of units x frequency gives you FLOPS. With this correction it's just 20 gigaflops for the human brain.

Modern deep neutral network demonstrate that they do comparable decisions with relatively little power. (i.e. modern speech and image recognition running on laptops, for example).

So I say for modern computers it's all about correct programming.
The 'xzgrep' script (and related xzdiff, xzless, xzmore scripts) are part of the standard xz package, though they are an optional feature so YMMV between distros.

~ $ xx /usr/portage/distfiles/xz-5.0.8.tar.gz
~ $ cd xz-5.0.8/
~/xz-5.0.8 $ ./configure --help | grep -A1 scripts
--disable-scripts do not install the scripts xzdiff, xzgrep, xzless,
xzmore, and their symlinks