skysight-inc/HackerNews huggingface dataset:
One question on avoiding giant IN clauses with Django?

Say I have a class called Fridge, and a classes called Vegetables and Condiments.

Both of these have ManyToMany relationships between themselves and Fridge.

So something like:

class Fridge(models.Model):
condiments = models.ManyToManyField(Condiments)
vegetables = models.ManyToManyField(Vegetables)

And here we have a QuerySet that represents our white fridges:

qs = Fridges.objects.filter(color='white')

First query:

"Given a list of condiment IDs, get me all the fridges that have ANY of those condiments in them (modifying the original QuerySet).""

Second query:

"Given a list of vegetable IDs, get me all the fridges that have ALL of those vegetables in them (modifying the original QuerySet)."

How on earth would I do that without building a list of fridge IDs and adding an IN clause to my queryset?

Here are solutions that do it with IN clauses:

First query:

condiment_ids = [...] # list of condiment IDs
condiments = Condiment.objects.filter(
id__in=condiment_ids).all()
condiment_fridges = None
for condiment in condiments:
qs = condiment.fridge_set.all()
if not condiment_fridges:
condiment_fridges = qs
else:
condiment_fridges = condiment_fridges | qs
qs = qs.filter(id__in=[l.id for l in condiment_fridges])

Second query:

vegetable_ids = [...] # list of vegetable IDs
vegetables = vegetable.objects.filter(id__in=vegetable_ids).all()
vegetable_fridges = None
for vegetable in vegetables:
qs = vegetable.location_set.all()
if not vegetable_fridges:
vegetable_fridges = qs
else:
vegetable_fridges = vegetable_fridges & qs
qs = qs.filter(id__in=[l.id for l in vegetable_fridges])

These solutions seem horrible and hackish and I was wondering if there was a better way to do them with Django. Something like object HAS these objects or object HAS ALL of these objects.

Should I just post this on StackOverflow instead?
Temporary Anonymous Phone Numbers for Craigslist/eBay
http://getroller.com/

A couple of friends hacked together this project. We'd really like to hear feedback from the HN junta. Is it something you'd be interested in using? Try it for free during BETA.
My only concern is that once we realize we're not going to run out of oil for at least a couple of centuries we don't undo the great work of being more efficient with our energy.

From an investment perspective though I can't see $100bbl oil prices being sustainable in the long haul. Its just too tempting to under cut by a 'few dollars' and capture more of the market.
That one always bites me at other people's desks; I'd figure out how to reconfigure it if it was mine. But there's always something jarring like that when you switch between similar-but-not-quite-the-same environments.
[Infographic] How Airlines Spend Your Airfare
> "Speaker Bio
Python Overlord for Heroku. Creator
of Python-Requests. Photographer.
Curator of The GitHub Reflog™.
Knows where his towel is."

Thanks for sharing. The site has a few other interesting presentations by the same speaker.
Yes, that is stating it more clearly than I did.
Switch to chicklet keys and I might as well get a MBP. The whole point of Thinkpads is that IBM/Lenovo & CO leave the design alone.
Should these sorts of widgets be the default implementation of those types of controls in desktop browsers? As long as they can be done with the same input information, this sort of input seems like a "user preference" issue.
Who is anyone on here to decide what grads should do with their potential, or whether they're wasting it? That's the height of arrogance, and I see no rationality whatsoever in such an attitude.
I believe what tptacek is trying to say that an attack on the == operator is not a timing attack but a brute force attack. Sure the function may return quicker but it doesn't reveal any more information and will be a brute force attack of data returned from a hash function.

For other readers, the == operator is comparing a password hash, as opposed to a password itself. One of the properties of a secure hash function is that, changes to the input drastically changes the output. Thus this attack is one of brute force and not a timing attack on the == operator.
The article says to give each password its own unique salt and then store the salt (as well as the salted hash) in the database.

This seems like a bad idea to me. If a hacker gets access to the salted passwords, in this case he'll probably figure out how to get access to the salts too.

I figure if the salt is stored in the code (or a config file...) rather than the database itself, at least it's two different hacks to get (1) the salted hashes, and (2) the salt.

Am I misunderstanding?
"Additionally, complete data on ApoE status, education level, ethnicity, dietary habits, and lifestyle choices were not available for all study participants, so none of these can presently be eliminated as contributory to the results observed."

Background on ApoE: http://en.wikipedia.org/wiki/Apolipoprotein_E#Alzheimer.27s_.... I'd be interested to see how participants' ApoE status relates to these results.
I love Django for making websites, but I'd go with Flask for an API. The only issue I found with Flask is I often found myself recreating stuff that Django already does, and is already done way better.
Salts are not secret. They make you run unique calculations per attacked password and do absolutely nothing else.

You might feel like adding a hidden component but it won't noticeably help security and it's not a salt.
Advice from Ray Bradbury: Be an "Optimalist"
So what is a good alternative?
Yeah, but what are the "costs" when AirBnB "moves in?" Expectations of residential building/neighborhood such as certain amount of traffic and noise, as well as neighbors who are relatively long term (in other words, you can get to know them--example may not apply in some cities).

Face it, AirBnB is just the next hiding place for illegal rentals after tax and zoning officials caught on to vrbo. I don't understand how so many people think that they can turn the property under their control (owned or rented) into a business without liability or regulation. Just write the word taxi on your car, pick up some fares, and see what happens.
If you believe your userbase will object to Google/Twitter/FB, use BrowserID.

Personally, I never feel particularly secure when typing passwords into text boxes on random PHP forms. On the other hand, I feel fairly confident that the folks at FB, Google, Twitter, and Mozilla know how to store a password and secure their infrastructure.
Thanks for posting this. I use Django extensively and I never really felt any pain from the tightly coupled monolithic approach, but I always like improving my tools. I will check out Flask.

Any suggestions for making it easy to build REST style APIs? Is piston the #1 choice still?
Changing operator== to secure_compare or some analog doesn't change this construction's resilience against attacks. This is very much unlike the situation with, for example, an HMAC verification.
one reason we use JQM is for page transitions. what other frameworks provide page transitions? we could obviously roll our own, but are just curious what the other options are. zepto and XUI are two alternatives to JQM mentioned in the article. what other ones have people tried?
Raising Angel Investment in NZ: A Grassroots Perspective
Ask HN: What are the popular forums/news aggregators by language?
For startups that have interesting content written in a language other than English, what are the best sites for getting that content distributed out to a wider audience? In other words, what are the HN, Reddit, Stumbleupon, etc. of other languages (i.e. Spanish, German, French, Chinese, Japanese, Korean, etc.)?
MD5 is not fine. MD5 is very very fast[1] and as such it's possible to simply brute force password given relatively modest computing resources. The space of passwords just isn't that big.

Use bcrypt or scrypt. Don't make up your own crypto.

[1] Crypto benchmarks: http://www.cryptopp.com/benchmarks.html
US labor board: Some limits on employee social media use are illegal
IE10's 'Do-Not-Track' Default Dies Quick Death
Personally I'd say:

- Error handling: Django error page is great, unless of course it's a AJAX request. Pylons does this in a much smarter way

- Django ORM is awful. Easy to use yes, but awful

- Django Auth is a "good intention" but awful implementation

- Django templates are sloooooow. Jinja 2 is an order of magnitude faster

Amongst other warts

Also, the author apparently found the hard way: don't use Django as an api server

I may sound critical of Django, but I think it's a "complex framework for simple usages", I mean, if you need low flexibility go for it, you'll do what you want really quickly.
No, I'm saying I do that face-to-face, and people still can't tell I'm enjoying it. So the tip to say nothing that you wouldn't say IRL is useless to me; I just can't help it.
You don't worry about VACUUM with InnoDB, but that doesn't mean you never feel the pain of the cleanup of old row versions. See 'purge thread spiral of death' for more details.
I mean this in the most polite and professional manner possible, please take five minutes and read http://codahale.com/how-to-safely-store-a-password/ .

It will explain why "salts are useless for preventing dictionary attacks or brute force attacks."

The entire article is excellent - and every colleague who I've ever pointed at it, has come away nodding their head and seeing the light.

The key-takeway (but please, read the entire article) is:
"It doesn’t affect how fast an attacker can try a candidate password, given the hash and the salt from your database."

Salts only help you from precomputed dictionary attacks ("Rainbow Tables") - but, if someone is brute forcing you, the value of a salt just disappeared.
Furthermore, it's important that the salt be unique per password. If you had a single common salt in your code, then two users with the same password would produce the same hash.
Check out Tastypie. You can get a full restful API based around your models up in like 5 lines of code and it's really easy to add extra functionality. Sorry for the lack of a link I'm posting this from a phone
The Internet Was Invented in 1934 (Sorta)
Facebook simplifies how developers monetise the mobile web
10 Employee Engagement Ideas for Skills Based Volunteering
You are. The salt is just to stop the use of rainbow tables, which are pre-generated maps between plaintext passwords and their hashes.

Anyway, you want to store a new salt (not just a system-wide 'this is my salt' salt) for each stored password anyway, so you will need to store that data somewhere. You could obfuscate a little by storing the salts elsewhere, but it seems a little extreme.
http://pastebin.com/JmtNxcnB - 20k++ sample cracked passwords from LinkedIn hash dump released on June 6, 2012. They do appear legit and strong too. It's unfortunate that LinkedIn hashed them using unsalted SHA-1.
Heads up: LinkedIn hackers attempting to log into other sites
No it's not a bad idea and essentially it's what BCrypt does.

Think about it. If you use the same salt for all passwords then I can easily create a rainbow table consisting of "keyword" + salt hashes and doing so I can crack multiple passwords.

However if there is a different salt for each password then this type of attack becomes more difficult as I have to essentially do the same amount of work for only a single password.

Finally, we need to store the salt in the database because it's needed in order to recreate the hash using the user's password for authentication.
"One conversation centered on the ever accelerating progress of technology and changes in the mode of human life, which gives the appearance of approaching some essential singularity in the history of the race beyond which human affairs, as we know them, could not continue."

- Stanisław Ulam, 1958 (referring to a conversation with John von Neumann)

Good's "intelligence explosion" concept is the basis of the modern singularity religion but not the origin of the term 'Singularity' used to mean a technological singularity. Good never uses the term.

So your quote makes my point doesn't it? That a utopian version of the intelligence explosion has replaced the original meaning of the technological singularity. One that favours the views of the AI researchers who went into futurism when the research money dried up.

Edit: I didn't know this but when looking up the date of that Ulam quote I learned that Vernor Vinge seems to be the first person to use the term singularity in the context of Good's intelligence explosion. In 1983. In the pages of that wonderful bastion of great scifi and pseudoscience that I loved as a kid: Omni Magazine.
"Anything above 8GB exceeds what it is designed to handle and can cause major performance problems, such as the database becoming unresponsive for 1-2 minutes."

Can you please expand on that? What version did you test, on what hardware, and what workload?
they also change a minimal amount - $5 i think - that discourages idle registration. i think it works well for them. maybe hn should consider it (it's not for profit - it's for the psychological "bump").
We are in the market for an HTML5 game guru for an optimization job and would happily pay $300 per hour to the right person - I've made offers of more and been turned down.

I imagine others in our position would be willing to pay the same - so I can certainly attest to the demand being there and it making sound business sense in certain situations.
So now Carmack is working on headsets, and Valve is working on wearable computing[1].

That's some pretty interesting support.

[1] http://blogs.valvesoftware.com/abrash/valve-how-i-got-here-w...
I found a similar problem using Phonegap and JQM. I was also loading another script in and the app became very sluggish.

Any recommendations for JQM alternative in regards to UI/design components?
Well of course CMU students have less exposure to the Silicon Valley startup culture than Stanford students. One is right down the road, the other is on the opposite side of the country.
Because loading 50+ kB of code when you probably only need 10 kB of it is really stupid. This is compounded by underpowered mobile devices that can't digest a single-page ExtJS "app" with 2 MB of script.

Simple is best.
Yes, pg_trgm was built for that but someone figured out how LIKE could be hacked to use the trigram indexes (gist_trgm_ops, gin_trgm_ops). So if you have a * _trgm_ops index on the column normal LIKE and ILIKE queries may use that index.

I assume your example would be a false index hit which then is necessary to verify against the real value. The same would apply to make sure 'This' is not a hit when doing a case sensitive search for '%this%'. So index-only LIKE scans are not possible with * _trgm_ops indexes.

EDIT: I just realized how awesome the extensibility of PostgreSQL is. An extension can make a core operator such as LIKE indexable in an entirely new way without touching the core code.
http://news.ycombinator.com/item?id=4075873

TLDR: PBKDF2, bcrypt, and scrypt already do salting, hashing, and stretching, and better than any roll-your-own scheme.
The minimum for deployment of a Netflix node is 5 Gbps within a metro area. You must also peer with Netflix at a common interconnect so that "loading" is free for them.

By your math they are saving a minimum of $5k per month per deployed device.

Any network not meeting these minimum requirements will be served under existing Level 3/Limelight/Akamai contracts.
Problem: Django has features. These features, horrors, reuse each other's capabilities.

Solution: Use something that has no features.

I'm not sure myself how sarcastic I'm being, to be honest. There's some truth in there too (or I wouldn't have posted this). But of course your microframework doesn't have any dependency issues. Rip all those things out of Django and it wouldn't have any dependencies either. Put them all into Flask and it will inevitably have to choose between either having the same problem, or leaving vast swathes of functionality on the table because of the refusal to let the components work together so they stay separate.
How do you find CMU to be a 'surprisingly high ranked school'?

http://grad-schools.usnews.rankingsandreviews.com/best-gradu...
Why C is on the top of programming language?
C is on tiobe(Base on http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html) the top of programming language of choice?
What are the well known application today that is written in C that makes it on the top?
I'm programing in PHP and Javascript a years now. I decide to go back to C and to study it because I felt
my skills are not enough and sometimes it is in the requirements of the employer.
and I want to be comfortable in different language to be flexible in my future job.
Currently I'm doing some basic console programs.
What are the benefits of learning this language for the future?
What are the applications that are now on the market that is written in C?
Thanks.
Yup. The main purpose of salt (aside from increasing entropy by increasing the length and complexity of the hashed value) is that it prevents rainbow table attacks where an attacker pre-computes (or downloads precomputed) hashes for common passwords, dictionary words, and brute-force style variations of same.

A hash unique to the site would require the attacker to create a site-specific rainbow table, but once created it can be used for all passwords. Having a unique hash per password means that the attacker would have to generate unique password tables per user, which for a suitable salt & algo is impractical, even if (as they normally are) the salts are stored with the passwords.
It is important from a developers point of view because those devices don't get tested by Google for API/feature conformance and so potentially are more likely to have applications not run on them correctly.
This is incorrect. There are sponsored links at times inserted via Google Maps API.
Get an open source, portable, future-proof, simple list: http://todotxt.com

If you need a bit more complexity, keeping with open source tools, http://www.redmine.org takes care of the job, easily. Plugins are a way to easily tweak it your way.

But falling for SaaS is easy, with http://trello.com, made by the guys at Fog Creek Software - which you might know already through the resounding name Joel Spolsky - is an awesome simple yet powerful tool, both for business and personal use.
Twitter hits 400 million tweets per day, mostly mobile
Almost every rental agreement I've ever signed disallows sub-letting. They also have rules about having people come to stay (as roommates, rather than as sublessee.) no sympathy whatsoever.

Now why did the landlord decide to list rooms him/herself? That's interesting. Could Airbnb become a way for small-business hotels to book rooms? Generally landlords don't want short-term tenants for various reasons. I wonder if the Airbnb idea, which plays on the social aspect of borrowing an individual's house vs. dealing with an dispassionate corporate entity, enables leasing of smaller properties to work. If that's the case, then a landlord seeking to run Airbnb listings to turn a few rooms in an apartment complex into a hotel might eventually run into trouble if this kind of arrangement becomes common on Airbnb, since then you lose the social aspect that was making the tenants well-behaved. With tenants who might abuse the property a small apartment landlord probably would not want to deal with them.

Maybe an enterprising landlord could put in a clause claiming 50% (or more) of the revenues from Airbnb if any tenant chooses to use it. In fact, maybe the OP should offer such an arrangement to the landlord as a settlement and thus be able to remove the restraining order and continue making some profit.
> Some of the suggestions make PostgreSQL seem less mature than InnoDB, still: "[don't put] sessions in the DB", "[don't put] constantly-updated counters in the database", and "[don't put] task queues in the database."

It's not that you can't do them in postgresql, or even that it will perform slowly. But when you have the chance to offload work from the database server that doesn't need to be there, it's often a good idea to do so.

I wouldn't have worded it the way the author did, either --it's certainly situation-dependent whether those are a good idea. But I wouldn't take it as a "postgres isn't up to the task".
awesome! A stripe for australia. Instantly signed up
There is no guarantee that this will work
Good-Bye Google Website Optimizer
I hope that this thread doesn't get derailed into nit-picking. The Tale of Genji was written in the eleventh century. The Golden Ass is a prose work which Apuleius wrote in the second century. Different authorities will have different definitions of the word "novel," so let's just agree to disagree.
huh, I have a linked in account that I don't check often and my password was on that list. Luckily it was specific to linkedin. I don't believe this is just a small percentage of users. Oh and I never received an email like the blog states (http://blog.linkedin.com/2012/06/06/linkedin-member-password...)... odd...
Correlation does not imply Causation

http://xkcd.com/552/
MD5 is 'fine' for very long passwords. How many of your users have very long passwords? MD5 does nothing to make it hard to blaze through billions of possibilities.
I've had great experience using Zepto, Backbone & custom styles. Was able to field a single-page app with about a dozen views - very interactive and very fast. Total app size about 25K including support libraries. Data loads asynchronously, cached using localstorage. I ran into endless glitches with JQM, but Zepto/Backbone/CustomCSS has been bulletproof every step of the way.
I tried adding my flight for tomorrow, Frontier 662 "F9 662", and I can't find any combination that lets it find that flight. Is the "9" in the airline abbreviation messing it up?
I use Django and Flask quite a bit. They are the first two things I go to for building a site. For a small site, usually without a database, I mostly use Flask. For a large site with all the common features like auth and such, I use Django. Yeah, it has its bad points, but that's ok. It's worth suffering through whatever those bad points are because it has one gigantic good point.

You don't have to reinvent the wheel.

That is the fundamental problem of all micro-frameworks. They are so small they are incomplete. You may not like Django auth, but Flask has NO AUTH. You have to go get a third party auth system, or write your own. That means with Django I can get a site, with auth, up in just a few minutes. With Flask, you have to write a login page for the 1000th time in your life. It may be very simple to do that, but it's even simpler to do nothing at all.
This is only a problem in the sense that once one hash has been cracked the so is the other, however most authentication is done using some user/email attribute also.

Usually we fetch user/email, hash password, compare to hash in database - authenticate if match or deny if not.
Well those people are simply naive.

Many developers have hated the platforms they have developed on but do so because the platform allows them to make money.

It's worked this way since the beginning of industry i.e. decades.
This is not super fast in MySQL either, compared to the alternative of creating a temporary table with your 5000 values in it and doing a join on that.
Except the "native" look and feel is falling out of style fast anyway. The iOS app store is a fashion show and nothing screams amateur now like stock widgets.
How the Buffet Rule Affects Entrepreneurs and Startups
> - 3d maps has been done like this already (by bing and others), though this looks a better resolution.

Bing appears to be using a non-scalable approach for Bird's Eye view. E.g., Seattle downtown contains a few 3d buildings in a very confined area. It might even be manually modeled.

What Google has announced is impressive because it is automated and scalable. Small demos are great and all (e.g. C3) but qualitatively different from getting something like this deployed into the real world.

Disclaimer: I work at Google.