skysight-inc/HackerNews huggingface dataset:
> I also feel that people conflate duplicating code with duplicating behaviour, which is what DRY is really about.

Indeed. The definition of the DRY principle [1] is:

Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

and not:

Don't type the same characters into the keyboard multiple times.

People often forget that.

[1]: http://c2.com/cgi/wiki?DontRepeatYourself
Thanks everyone for these suggestions. The Allonge book was incredibly helpful
You are describing the problem exactly: it all too tempting to pass text around from user input to command line arguments without any way to validate the text data and assume it's ok because it's easy. It's exactly the same arguments that goes in between static and dynamic typing in programming languages: static typing ensures some sort of semantics is respected. If you pass text around, because it's easy and fast, most of the time you will never validate the data and you have no way to ensure that you are not actually handling a bomb. If the protocol was binary there is no way in hell you would be tempted to pass it's data without validation to an external program because you'd have to respect the API and because there would be not way to just send a bunch of commands. The same goes for sql injections, url buffer overflows, etc. Free form text should only be used for actual human textual data and should NEVER be the interface in between programs. It's way too fuzzily defined to serve as a protocol.
The Allonge book was incredibly helpful. Thanks for the suggestion
How so? It seems like a sufficiently smart comp^H^H^H^Hservice could figure out how to distribute machines across them, even if the user doesn't know they exist.

Turns out it's a moot point anyway, because cells are client-visible, and have been contributed upstream to OpenStack as a Nova extension.
I guess he's saying that for food to be this cheap seen with our eyes, there needs to be a substantial oversupply. And since there are physical limitations to how much food we are able to actually eat, if there's an oversupply a bunch of it will be thrown away, whether by farmers, distributors or consumers:

Either they fail to sell the food, or the low prices gets consumers to buy more than they need.
How much money does an article or a youtube video generate? It seems that this one article alone must have cost thousands of dollars in equipment alone.

And while the iphone 6 bend video on youtube has almost 40 million hits now, the unbox therapy user who uploaded it hits less than 500.000 views on average for their other videos. I'm willing to believe that 40 million views on youtube can be monitized to pay for more than an iphone 6, but I'm not so sure about the 2-300k, so it seems like quite the investment to go around breaking phones.
The saner way to implement this bash feature would be to have a single environment variable containing inherited shell functions. A special prefix is an alternative, but a bit harder to filter.
From what I understand, Apache doesn't send them to bash. It sends the to whatever binary is configured to handle the request (using CGI), which were then calling bash unbeknown to Apache (but implicitly passing the same environment variables).
> if Apache/dhclient wants to put things into environment variables... it absolutely should ... take into account all possible contexts

Sorry if I'm picking on you too much, but I noticed this and thought of an even better counterexample. By this logic, it's Apache's job to prevent SQL injections. It should know that a single quote, in one possible context, can terminate an SQL string, popping the SQL parser into a state that allows for arbitrary statement injection.

It also needs to know about any other database language that could ever possibly be invented, now and for eternity, because we can't know how long any given version of Apache will remain in use.
> Assumedly programs like apache filter out environment variables properly. But unfortunately, in the validation of input data, they fails to validate correctly input data because they don't expect that data starting with "() {" will be interpreted by their bash child processes. If there's a bug, it's not in bash, but in apache and the other internet facing programs that call bash without properly validating and controlling the data they pass to bash.

Bullshit, this may well be perfectly valid data and web servers are not in the business of shielding shells against their own misfeatures.

And if they were where would they stop? It would require that webservers do arbitrary context-sensitive data analysis of everything doing through in case this is a malformed JSON string triggering a bug in GSON while that is data injected unescaped into an SQL variable and the other one's too big for an underlying C buffer.

You can only end up with a webserver refusing to do anything, because some idiotic application somewhere may misuse or misunderstand anything it lets through.
The idea is that postgres would be a storage engine for mongodb, like innodb is a storage engine for mysql
It is for storing huge number of data with dynamic keys.

For example when site admin says "I want new archive that I can fill with items, items will have Id (automatically), Name (string), IsMale (bool)". He also want to do complex queries on this data as well. That's where NoSQL comes to help.

And to answer why exactly MongoDb is so popular - it's because it has awesome driver support for every popular language.

I don't understand what's so hard to understand here. It's a simple solution to EAV/nulltable nightmare.
> because it probably doesn't invoke a shell

Some people write cgi in bash.

Yeah.
Thoughts on Pebble Smartwatch
Needs a Dance Dance Revolution mode and a karaoke then :).
Thanks for the info guys
Extracting keys from maps into a definition, with common-lisp (using my macro defk)

(defk login (user pass)
(and (string= user "Bob") (string= pass "secret")))

(login (list-to-hash '(user "Bob" pass "secret"))
(list-to-hash '(user "Bob" pass "secret"))

Given the following definitions:

(defmacro defk(name h-list &rest body)
`(defun ,name (h)
(let ,h-list
,@(loop for k in h-list collect `(setq ,k (gethash (quote ,k) h)))
,@body)))

(defun list-to-hash(list)
(let ((h (make-hash-table)))
(loop for (k v) on list by #'cddr do (setf (gethash k h) v))
h))
I don't think MongoDB would have much left to do, if PostgreSQL was doing the storage duties. The converse - implementing the MongoDB protocol within PostgreSQL, as mentioned above (mongolike), sounds like a more sensible proposal.
Did I miss something? MongoDB was never ever faster than Postgres. That's nothing new. Most of these things are clear when one reads the MongoDB docs:

MongoDB stores Metadata, (nearly) uncompressed on a per document basis, so of course it uses way more diskspace. It doesn't store the data in any efficient way either.

Also it's pretty much unoptimized, compared to Postgres which has been around for a really long time so it's kinda slow.

Many functions in MongoDB are actually implemented in JavaScript, not C. So that's also a factor, even when I guess it's not the big one here.

MongoDB has a lot of limitations that can really bite yo (document size even though that's the smallest (gridfs), how you can do indices, even limitations in how your query can look like, etc

The only thing that's good about MongoDB is that it's nice for getting something up and running quickly and that it's a charm to scale (in many different kinds), compared to PostgreSQL. If PostgreSQL had something built in(!) coming at least close to that (and development has a strong focus there) it would be perfect.

For all these reasons many companies actually have hybrid systems, because sometimes one thing makes sense and sometimes the other.

The benchmark seems strange, cause there are many SQL and NoSQL databases that are faster and that's a kinda well-known fact. I think everyone who ever had to decide on a database system has known that, even without a benchmark.

This makes it kinda look like an advertisement (look at the company behind the blog).

Using PostgreSQL 9.3 with JSON for a while now and it's great. Also I know it is possible to scale PostgreSQL and it's really nice. Still a lot more complexity involved (again, depending on the use case).

Just use the right tool and please let's stop with such shallow comparisons, because I think it kinda harms the reputation of database engineers and system architects - and the authors of such comparison. When you look for real comparisons and example use cases, typical patterns or just some help one always stumbles across these things and they tend to quickly be out of date too, cause all well-known databases have a lot of active development going on.
Yeah, I meant Apache itself doesn't invoke a shell.
I've walked around in Taipei (where people seem particularly short) every so often you see some westerner who's 6'4 or so (do people still measure themselves with 'feet' any more) walking around - a head above everyone else - they do look weird. (I'm a westerner who's considered 'short' in the west and 'tall' there).

I'm afraid that in the right context you are a freak :-) but yes height shouldn't be an issue
> There are many other ways Bash could do that, though

KSH and ZSH will load functions from files they find via $FPATH.
My understanding is that Apache starts CGI apps in environments with variables for things like HTTP_USER_AGENT that are set by random people on the internet through Apache.
And who's behind it?
Completely anecdotical: I have been in this country 5 days and I have already gained 3 kilos.
Apache doesn't have to know how apps treat data in order to use a secure mechanism of IPC rather than passing variables set by people on the internet.

Random person on the internet shouldn't be able to set shell variables. Not 'x', not anything else.
Motivating examples include:

macros: generating customized implementations given
generalized packages, such as sort.Ints from ints

If this is seriously their proposed solution to the lack of generics then my estimation of Go just dropped even further.
Lots of functions to start another process start a shell instead and is a command line to be executed, e.g. system or popen. The convenience in that case is that you don't need your own handling of $PATH or wildcards or argument parsing. It's pretty standard on UNIXoid systems.
> Somethings are better than others, not discriminating based on gender, where the gender doesn't matter for the job, is strictly better than discriminating based on gender.

Prove it.

> Seriously if this happened in the US you would be up in arms, the Chinese should not be allowed to be different.

Should not be allowed by whom? Last I checked, China wasn't a US state. Are you going to pick up a rifle and go conquer it?
HRM Dissertation Training and Development
I'm the developer of Cursive - thanks to you and others in this thread, I'm really glad you're all liking it!
I'm sorry but throwing food is nothing like shutting down a server. In the first case (food), significant amounts of water, energy and effort have been expended to create what is being thrown away, but in the latter case you merely stop using electricity or bandwidth (which doesn't get wasted).

A better comparison would be buying 3-4 different types of servers because you weren't sure what the final load would be, and then deciding to throw away a few because you bought too many.

California, America's "salad bowl", is facing a drought that threatens to turn it into a dust bowl. So even as a foreigner I'd think America throwing away 35 million tonnes of food every year is a serious cause for concern.
Oh, wow. Think about this. The internet experience may have just changed, and changed forever.
The analogy is very weak, SQL injection should be prevented in your application. Does Apache need to set global environment variables, it seems pretty extreme that it just takes all the user input and makes it available to the underlying shell, there will be a whole class of bad stuff that comes from this!
it really does need a preprocessor though - with a underscore type lib or generics the code would be so much faster to write and easier to maintain.
The great porn experiment: Gary Wilson at TEDxGlasgow
the author commits the generated files to the source repository, so that they are available to clients that use go get

Treating generated files as primary source is a disaster waiting to happen. Someone will end up just doing a quick edit of the derived file because yacc isn't set up right, or running a sed command to do a refactoring, or just not understand the difference. One of the things that I like about go is that it does a good job of keeping the codebase clean; recommending a solution that sets up this sort of trap is a big departure.

I wonder what other approaches are available for packages that intend to distribute in source form, however. I could imagine adding a parallel directory structure for the generated files, to make it clear that they are out of bounds, akin to the mvn source / target layout.
Any articles coming from a company which describes itself as "The Postgres Database Company" and sells "a relational database management system based on PostgreSQL..." [1][2] stating than "PostgreSQL outperforms X" is de facto suspicious.

[1]: https://www.google.fr/?q=enterprisedb#q=enterprisedb

[2]: http://en.wikipedia.org/wiki/EnterpriseDB
I think the same could be done much easier using KSM [1] (Kernel SamePage Merging), which Linux now has.

1. https://en.wikipedia.org/wiki/Kernel_SamePage_Merging_%28KSM...
Yes that was why I asked. I did not know about Bundles or private tracker before downloading this album.
ofc its not a bug its a security vulnerability. Where hes wrong is having an undocumented unnamed unknown function that has eval abilities is plain idiotic; who cares if its a bug?
Activist Hedge Fund Pushes Yahoo to Weigh a Deal with AOL
If the protocol was binary there is no way in hell you would be tempted to pass it's data without validation to an external program because you'd have to respect the API and because there would be not way to just send a bunch of commands

I assume you're talking about Apache - but Apache had no way of validating the data. The protocol just said "this is a blob from the client", which any binary protocol for the task must be able to handle. Apache had no business validating it, anymore than it should validate any other content - how should it know what makes it valid?

Bash, on the other hand, just received that blob and treated it as an executable. It wouldn't matter if the protocol between the server and bash was binary, since it was a valid value as far as the protocol was concerned.

The problem here is the hidden channel between Apache and bash, which never actually talk directly to each other (it's through the CGI binary) but still pass data. It has nothing to do with text protocols.
Poor professors, getting arbitrary ratings from people they meet once or twice a week for a few weeks, which determine their professional future. Sounds like being a student or something. Go suck on a lemon, professors.
Strangely enough, I downloaded Thom York's album yesterday from Bittorrent Bundle, and this morning my torrent is displaying an error: "Tracker gave HTTP response code 402 (Payment required)".

So I guess it's not very torrent friendly, as I cannot seed any more.
I have seen youtubers who live off youtube with average video views of 20-30k per upload.

If an ad plays on 1/10 videos, and each ad makes the person 0.1 cent its still (30 million * 0.1 * 0.1) / 100 = 3 thousand dollars. Now take into account the added subscribers this adds which will make all future videos more profitable, and that I'm probably underestimating ad revenue, it is worth it.
I think you're spot on. I have read many articles suggesting that the problem is not food production, it's the logistics around getting it from the people who are producing it to the people who need it. This is not free, and quite expensive. I recall the first time I became aware of this, they mentioned a BOAT LOAD of rice that had been donating but it was waiting on transportation funds to move to those in need.

It's my understanding that we can provide for everyone on earth so that their is no starvation. Unfortunately capitalism isn't quite up to the task.
The way things work at my university, at least the science faculty, is that word of good or bad teachers just travels trough the grapevine. And it is usually quite accurate.

Then we have (a active, and paid) representation in the faculties decision making body, which leads to, in my experience, the faculty actually dealing with bad professors.

Not everything has to be boxed in by numerics, sometimes simply speaking up and listening is the easiest and best solution.
Notice that if you buy into the signaling theory of education this behavior is instantly explained. The primary purpose of college is achieved by the time you get an offer letter. "We certify that you have a high enough IQ and conscientiousness to get into our school." Spiffy, job guaranteed! Now you get four years to find yourself and drink a lot. But for some reason you are constantly beset by atheist monks who think you should be praying more.
It is a bit astonishing what lengths the go community is willing to, er, go, in order to avoid compile-time code execution (the basis of lisp macros) or generics. I understand that neither of these ideas were invented at Bell Labs during the 1970s, but at some point, can't we just put aside the not invented here syndrome?

Could we all just agree to pretend that Bell Labs folks invented those ideas? Would that work?
> I'm not sure. If you then go head and inline the code after, your unit tests will be worthless.

Local function bindings declared inline perhaps? It seems to me you could test at that border.
They employ several of the main developers of community PostgreSQL and also sell commercial versions of the database.

Why does that make this article suspicious? Obviously they stand to benefit from more people using PostgreSQL, but they've been exceedingly clear with their methodology - their benchmark is available on Github: https://github.com/EnterpriseDB/pg_nosql_benchmark.
It depends. DB2 comes in several editions. The DB2 that IBM mostly sells nowadays is ok but there's also DB2/400, which -as the name suggests - comes with all sorts of quirks and funny mainframey limitations.
okay, let me make this little table. "VULN" denotes the test above (with sh rather than bash) detecting a vulnerable situation, "OK" means the test detects no vulnerability.

sh → bash | bash vulnerable | VULN
sh → bash | bash fixed | OK
sh → dash | bash vulnerable | OK
sh → dash | bash fixed | OK

Why do you want to only test the first but not the third case? You can perfectly well have /bin/sh linked to dash and still be vulnerable, because e.g. dhclient calls a Bash script.
Your comparison doesn't make any sense. It's an obvious requirement for a JSON parser that it be able to parse input from arbitrary sources, including malicious ones. It's not so obvious that a shell should have to deal with malicious environment variables, due to the reasons outlined in the original post.
I don't think you understand the gravity of the situation. It's a choice between rebooting Xen hosts or having them (and you) get owned when the exploit is made public.
I just mentioned the ban as an indication that there may be some drawbacks to the stuff. I see in the wikipedia mention that the UK has 'determined that containers of it should be labeled with "May cause sensitisation by inhalation."' Sounds like healthy and tasty to me.
broken political systems are the primary cause of starvation because they inherently disrupt supply and production; simply look towards Venezuela for an example if Zimbabwe where productive countries took a drastic turn for the worse.

In the US throw in the issue that many companies do not understand the protection afforded by the Bill Emerson Good Samaritan Law or that many facilities needing donations have no way to pick them up. Some cities have processes in place to organize and pick up donations and there are even some charity groups that do it, but its not widespread. A lot of waste in grocery is expired fruits, vegetables, and bakery, that may not be covered under the Emerson bill
I've been trying to think for a while of a comment that explains or even defends this from a cultural perspective but there's nothing. while this probably doesn't reflect public policy or the aspirations of the Chinese people, it's just another of those signs that for all the trains/sports stadiums/airports being built, Chinese society is still sometimes the kid of idiocracy we would associate with the UK a hundred years ago. it's a shame.
> Stripe (who store your money related stuff in MongoDB)

That's a bit scary. There has been several successful attacks against virtual currency exchanges that use MongoDB, utilizing the eventual consistency to your advantage.

If you handle money, you don't want any inconsistencies in your database, no matter how temporary. You can work around these of course but you really need to know what you're doing.
Understanding Jean Baudrillard with pumpkin spice lattes
Rather than doing that, manage your server(s) with code. While there is a little overhead in learning something like ansible, being able to provision another machine to do a dry-run more than makes up for that cost.
The nice thing about using an RDBMS with JSON support, rather than a NoSQL solution, is that you can store all the fixed-schema stuff in column as usual, and benefit from the performance, consistency, ease of joins and so on with that, but you can also store your JSON documents alongside that data in the same table, efficiently indexed.
> Sorry if I'm picking on you too much

Don’t worry.

> By this logic, it's Apache's job to prevent SQL injections.

Curiously, I’d call this an example for my position, not yours: It is not the job of the SQL server to prevent SQL injections, quite the contrary, it is the job of the calling application to ensure that the things it tells the SQL server to do are actually safe. Similarly, it is the job of Apache to ensure that the things it tells its child processes to do are safe.

…

Actually, you’re probably right – if I’m guessing correctly that all the "CGI" standard does is to say "These user-supplied values go into these environment variables", then it is not the job of Apache to sanitise them. It merely means that Bash is an unsuitable choice for an environment with untrusted environment variables, i.e. as a CGI script or even for use as /bin/sh if you cannot trust environment variables.

It gets a bit messier with dhclient, because that explicitly calls a Bash script and hence definitely should know about the issues Bash might have with environment variables.
It's not about text vs binary but about well-specified, documented and understood, versus ad-hoc, convention-based, "infinitely extensible", and taught-through-blogs.
There are plenty of database that don't fit into RAM completely and are perfectly usable. It depends on what you can fit in buffer cache / OS page cache.

The whole point of btree indexes is that they're efficient to query from disk.
Signaling Post-Snowden Era, New iPhone Locks Out N.S.A
I just disagree. You seem to be saying we should use binary protocols and load them directly into memory Cap'n Proto style. But what if I'm little endian and you're big endian? Parsing happens. Text-based protocols fit well into human's heads, and its the humans that have to do the debugging. I think it would only make the process of finding bugs slower and more complex, and give advantage to the attackers.
It's not just bash. Shell scripting in general is "stringly typed" and brittle because of that. Furthermore, when an error occurs and no special options have been set, the script will just continue as if nothing happened. This behavior does not encourage writing correct programs.