skysight-inc/HackerNews huggingface dataset:
As Eliezer said - they implemented the two most important algorithms from the contest, which advanced the state of the art in recommendation systems & gave the majority of the benefits, and didn't implement the long tail of algorithms that each only gave a very slight marginal benefit & would have been costly to re-train and maintain.

This is one of the advantages of running shorter competitions: normally it takes 1-3 months to approximately hit the asymptotic level of performance on a dataset given the inherent noise in it and the state of the art in machine learning. The shorter competitions are focused on finding the low-hanging fruit that generate large improvements (such as SVD & RBM's in Netflix's case) and exploring the space of possible model structures, as opposed to optimally ensembling across a large number of models to eek out the last 0.01% of performance.

Exploring the space of useful features & possible models enables you to trade off computational efficiency & maintainability vs. model performance in production as well. The $1 million dollars Netflix put to the prize leveraged >> $1 million in human effort to explore the possible models, from which they found and applied the two best suited for their production implementation.

(disclaimer - I work with Kaggle)
Yeah we're actually looking into what categories to expand into. Thanks for the tips!

Yeah that limit is correct, but almost all the things you listed are common discussions on the social web. The great thing is that we can query the entire social web, analyze the data and spit out a rank. So our only limit is the entire social graph.
it's also a risk that might not work out.
Great, added there as well, thanks!
Great article!

I think it gives some insight into the challenges that the TSA faces. They don't have complete autonomy, and like any massive organization, politics plays a huge role.

I have no doubt there are some really smart and innovative people at the TSA, but as anyone who worked in a huge bureaucracy knows, you often spent 90% of your time trying to please everyone and 10% actually doing real work.
PHP is a bad language; it is ubiquitous due to historical accidents and path dependencies. If you have the freedom to pick any language you want, and you don't have some specific urge to write Wordpress plugins or something, then DO NOT PICK THIS. (Also, PHP is likely as popular as it will ever be. There's not really any room for further growth.)

Perl is similar to PHP. It's not nearly as bad - in fact it's a great language in many ways - but it's also old. It's about as widely adopted as it's ever going to be, and it doesn't pick up new features as quickly as it once did. The syntax is also a bit quirky. :) Unless you want to work on Perl codebases, I'd recommend staying away, not because it's bad but because it's not really good enough.

Haskell isn't really suitable for scripts or simple webapps (your listed use cases). It's a wonderful language, and knowing it will make you a better programmer and a better person...but it doesn't do what you want. Stay away. :)

Better choices for you would be:

Python/Ruby: Flip a coin and pick one. Both are awesome, but they also have a lot of overlap; you won't regret either choice. They're both excellent for writing short scripts, webapps, etc. They are objectively MUCH better languages for these purposes than PHP, and at least marginally better than Perl. They also have very serious market share (which makes it easy to find tutorials, help, example code, open source projects to contribute to, etc.)

Javascript: With the advent of Node, it's a decent choice for what you want. And it's still the omnipresent language for client-side programming. Not as mainstream as Python/Ruby, but still pretty popular and widely adopted - but it's not completely mature, and new and interesting ideas are still popping up.

Something else: If those options don't spark any interest - or if you already know them - you might look farther afield. Lua and Clojure are both interesting options, in very different ways. As someone who already knows Python and JS pretty well, Clojure would probably be top of my list of "languages to learn". (I just wish it wasn't JVM based...oh well.)
It's written that they will get back to us on the Monday, the 16th.
My experience matches the previous poster. I used to abhor video games of any kind. Then I was given an XBox for good work or something, and started checking them out.

Project Gotham Racing looked awesome, and played well. Same for Gears. Chromehounds looked crap, but played insanely well. I've spent more time playing Chromehounds than I have any other game, and I play CoD a lot these days.
Planescape Torment was a great game but it also lost money.
The last line of the article is the most important: "If Americans are ready to embrace risk, it is time to strike a new balance."

If we could stop being cowards and accept that we might die, or our families might die, then we might fight these invasive policies. I'd love it if efficacy, privacy, and freedom were once again as given the same weight as security.

I don't hear many calls for courage, so I applaud even this veiled statement.

*edit - spelling.
I see that now, thanks :) I don't mind being downvoted, I'd just like to hear out any rebuttals to my comment.
See: every other comment of mine on this thread.
YC app received confirmation?
Were we supposed to get one? I submitted a long time ago (way before deadline).
No, I'm referring to "external-accessory". This works with both dock connector and Bluetooth iAP accessories. You're right that the other one is only for BT LE (4.0). It's understandable that many people are confused about the CoreBluetooth vs MFi Bluetooth.
Private parties are already allowed to share information amongst themselves under ECPA.
Why Can't Zach Phillips Use His Phone Number on Messages.app?
I liked how the business model of LORD was to get you to pay so you could do it with Violet.
Cool!

It's interesting that you want them to make a procedural language, although if they started with C, it makes some sense.

I wonder if you could broaden the experiment slightly by relaxing your constraints and saying they either need to allow reassignment of variables, or else support lambdas, and in exchange they can assume immutability.
Agreed. I'll usually enable first-party js on a new site, provided it shows something compelling on load and the js import list isn't too sleazy.

I don't use adblockers per se and welcome them if they are polite, respect privacy, and something I'm interested in.
Good point. I hadn't thought of it that way actually but that has got to be part of it.

For me, I sort of skipped over Ruby. I was too busy with other things to really get into it. So since I had more experience (not a lot, but more) with Python than with Ruby, that is what CoffeeScript seems like to me, instead of Ruby.

I always liked Python syntax, its just that I was busy with projects that involved a lot of HTML/CSS/JavaScript/PHP etc.

I have spent years and years writing C#/C/C++/JavaScript/PHP code with curly braces etc. To me, the curly braces and some other stuff are noise. Especially when you get into callbacks and have to remember to keep all of your curly braces and parenthesis straight. Its basically stuff that, as demonstrated by CoffeeScript, the compiler can take care of for you, but in JavaScript it just wasn't taken care of.

In a similar vein, JavaScript (and any other dynamic language or language with static type inference) demonstrates that type declarations aren't necessary, and that a language/compiler can usually take care of that stuff for you.

In a similar vein, C demonstrated (among other things) that managing the stack manually isn't necessary.

Ruby syntax to me is significantly more complicated than CoffeeScript and not really as clean (begin end blocks etc). I think that Rails has some important advantages in terms of software engineering, but the performance and syntax of Ruby turns me off a bit. Still, there is probably more good than bad so I might still end up doing some Rails stuff.

I don't think its advantageous to write an if statement backwards and I think people who wear black wire-rim glasses when they can easily afford contacts or don't even need lenses are ridiculous. I also have never owned a Mac in my life. I dual boot to Ubuntu/Windows 7.
Longchamp Bags, Longchamp Outlet Onlione, 2012 New Style Longchamp Handbags
I loved Planescape! I remember how excited I was the first time I got to an absolute dead end and died. I was getting ready to restore from a save when I woke up on a table... "wait.. this is how you advance?!?!?" So cool, I've still not played an RPG that threw as many curveballs as that one.
Iyc
I thought this too, at first, but only because I didn't know Ruby very well. Now I realize it's heavily inspired by both, but actually more closely resembles Ruby, IMO.
I love to code
I wish I had a professor like you. When I asked why there was no compiler class, the answer I was given was: "compilers are a solved problem; there's no point in teaching them anymore."

Maybe it's just my school though.
How LinkedIn Could Be Hugely Undervalued
The very first thing I looked for was Trade Wars.
I can't agree more! Although I support the idea of paying someone for their hard work, even if it was something of creative artistry. (Time is still money, even if it was spent dreaming of characters in a play, or colors on a canvas) I do NOT agree that we have now set some sort of "bar" for artists/actors/musicians HAVING to become filthy rich for choosing those careers and we can NEVER progress beyond that point.

The makers of horse shoes, sued Henry Ford for putting them out of business. We may laugh at this, but isn't this the same thing we see going on today?
No, you got it right: COM files were raw binary dumps of code and data, a legacy from CP/M, whereas EXE files had the MZ magic number (for Mark Zbikowski) and enough data in the header to allow relocation.

Also, COM files in VMS are the equivalent of BAT files in MS-DOS and shell scripts in Unix-like OSes.
"Think about the implications here. Why should Apple get into payments? A business they have no expertise in, that is nothing like any business they have expertise in, and that is only barely tangentially related to products they make?"

Just like they never had expertise in telephones, yet managed to completely shut that game down.

"More so, payments is all wrong for apple. Payments is about boring stuff like consistency, robustness, and low overhead. That's not what Apple is about…"

iTools, .Mac, MobileMe, iCloud. Email, Calendaring, Cloud storage, etc. Apple has been providing unsexy "consistent, robust, low overhead" services for years.
My mom would not shell out for a second phone line. :(
Incremental can make all the difference between zero- and hours of downtime. Do not underestimate the importance of this extra agility in the modern world of almost daily updates to web apps and backends.
No 3% is for handling in-app payments and 27% for validating apps, maintaining the dev ecosystem, advertising apps, and building iOS devices in the first place.
If you think a website is a software package...
Not trying to be mean here or anything, but I think this is ridiculous.

Reporters have to earn the right to have "open, real and fair" sources. Many reporters (especially unpaid bloggers) are shitty at their jobs. I'm sorry random people you interview don't trust you with their entire life story.

I'd say try harder -- whether you like it or not the person being interviewed has all the power, and that will never change.
They could do something for Android, but iOS and WP7 would never allow a 3rd party "store" application.

They do have a Steam iOS application, but it's an adjunct to the PC Steam application.

I wouldn't mind for them to create cool stuff for iOS (since I own one), but Valve's games don't lend themselves well to portable versions. I haven't seen a portable first-person game done well on any platform (even NDS and PSP). They'd be better off creating new IP or a different genre for mobile (like ID's mobile games - the on-rails shooters aren't bad, and the RPG games are great fun).
Glad you like it! We're doing our best to prevent cheating, but it's certainly not impossible to cheat. If it becomes a problem we'll definitely address it. As for the current scores, I don't believe they are cheating. I've been able to get 310 after playing quite a bit, and I certainly know there are people out there better than me.
The web was built in a manner to separate content, presentation, and later logic. It is those who insist on adding requirements that break it. Which then have to be fixed (see css).
2% after a little fraud and customer support becomes a terrible business.
New Draft Of CISPA Announced: Some Progress, Still Big Problems
Don't listen to any of the PHP haters here. You said you want to build simple scripts and blogs - something PHP is far more than adequate for, and it will be something you can pick up quicker than Python or Ruby. Also popular blog software like WordPress is all PHP.

While the PHP haters may be right if you're trying to get the best software engineering gig in Silicon Valley, for your purposes, I think PHP would probably be the best combination of tons of tutorials and documentation online, power and capability, and low learning curve.

Just my $0.02 as a PHP, Ruby, and JavaScript developer.
Interesting indeed! We'll poke around and ping you guys if there's any "synergy" as the biz folks say...
It's pretty simple... Hotspot collectors are optimized for throughput, not latency. That's where the money is for Sun/Oracle. It's a trade off between one and the other in any garbage collector.

You can tune it to some extent.
Any manager who uses "catch up" for a firing meeting is just a bad person.

You don't put a firing meeting on the calendar. Why would you?

The bad managers make messes for people like OP's manager, who really was working at face value.
Thanks! My email is eocean@fohr.com.
It may well be the case that "allowing knives and box cutters back on board is considered too emotionally charged for the American public"... according to whoever decides these things.

There are people in the political arena who stand to gain from citizens living in fear. And we as citizens would do well to remember that.
Former FDIC Chair: Fix income inequality with $10 million loans for everyone
Style World
"No security agency on earth has the experience and pattern-recognition skills of TSA officers"

really? having watched three TSA officers debate for five minutes over whether or not peanut butter was a liquid and concluding that "well, peanut butter goes on peanut butter and jelly sandwiches, and we don't allow petroleum jelly through, so we don't allow peanut butter through either", I really am skeptical of this claim...
Social Design Strategy from Facebook's Social Design Evangelist
> The title is completely accurate.

Accurate, but a half truth. The prize was for a 10% improvement, but before that solution was produced they had already improved by 8.4%. The headline makes it sound like the improvement from zero to 10% was not worth the engineering cost, but really it was the improvement from 8.4% to 10% which cost too much.
Being Optimistic in All Aspects of Life
Which school do you go to?

Compilers seems like a pretty standard CS course at most schools.
Pretty sure that's just your school.

In the article, he mentions that the students don't get to compilers till 4th year. I find that a little strange. Some elementary understanding of how a high-level programming language translates to executable code is quite useful in other courses in a standard CS curriculum. At my school, in 2nd year we write a compiler for a tiny subset of C++ to MIPS assembly, and in 4th year you write a compiler for Java with a great deal of optimization.
I've sort of had the same idea. I've wondered wether the power required to have a robot process all it would need to in order to move around and interact with the world by responding to all the different stimuli (optical, audio, kinetic) wouldn't leave many CPU cycles left for doing the super human things we're used to computers doing.
Interesting that you say no one has managed to crack first person on a portable platform. Maybe Michael Abrash can bring some of that game changing "Quake" style magic to mobile (although I guess Carmack is a pretty big missing ingredient).
For founders looking at relatively quick exits, they are the product.
~s/anonymity/pseudonymity

You can establish multiple personas and associate various speech and behavior with each, but those are pseudonyms rather than actual anonymous behavior.

Still: anonymous (or pseudonymous) speech is a core component of free speech, and it's a liberty we need to preserve both legally and technically.
Software that predates the iPad, it seems.
OF COURSE that's what it's about, maybe Blow doesn't want to bother to say it because it's so freaking obvious that he doesn't feel he needs to bother. If we're going to get games like Braid, he can be as pretentious as he wants. John Romero was totally full of crap and all that we ended up with was Daikatana.

I personally love the fact that Blow is borderline antisocial and that he hates the industry. Someone needs be that. Most companies are just out there making Call of Duty 12 or awful freemium "casual" games. It's not like there aren't large numbers of artists and so-called creatives throughout history that have shared this sort of personality. I don't need to be his friend and I don't care about being his fan. I just admire and enjoy his work.
What makes an airplane with 100 people on it more "big a deal" than a bus with the same?

The effort spent on security is wildly disproportionate to the risk-- based, it seems, on the supposition that terrorists care fabulously more about damaging faith in American air travel than actually killing Americans.

I'm not saying I know anything one way or another, but that seems crazy.
Valve reveals mystery hardware project: wearable computing
The words I live by.
It still isn't.

http://store.steampowered.com/hwsurvey

Mac HW numbers are in the single digits. Apple needs Valve, not the other way around.
Progressive Enhancement is the name of the game. Still wondering why so many people ignore this
Uncompressed field names - If you store 1,000 documents with the key “foo”, then “foo” is stored 1,000 times in your data set

Oh my god. I didn't know about this. And I hate short, meaningless and anti-intuitive field names. Please fix it mongodb devs!
Are those really the figures he cited? Working only weekdays, and only eight hours per day, that comes out to five lines of production code per hour. Even with the overhead of testing and occasional widespread refactoring, that seems incredibly low.

In any case, Braid was just okay. The level where the flow of time depended on the direction that your character was traveling was great, but the rest of the game was forgettable. There was about as much gameplay/story integration as a Final Fantasy game, and the story itself was mostly overwrought. But considering that the game cost me one dollar to purchase, I'd say it was worth it. :)
How to Learn Scheme
Introspect Python logging with logging_tree
i love ES, but i don't really feel comfortable with it as a primary datastore. We tend to use couchdb to write to, and ES to query against. It all happens automagically with a single shell command.

I won't use ES on it's own, because I have experienced situations in the past where the dynamic type mapping functionality gets confused, ie: the first time it sees a field, it indexes it as an integer, but then one of the later records has 'n/a' instead of a number. The entire record became unquery-able after that, even if it might have stored the original data.

You could fix this by creating the mapping by hand, BEFORE any data has been imported, as it can't be modified later. But what you have then is a situation where you have to maintain a schema to not get it to 'randomly' ignore data.

You also can't just tell ES to rebuild an index when you need to mess with the mappings, you have to actually create a new index, change the mappings and then reimport the data into the new index (possibly from the existing index).

It actually also feels right to me to split storing the data versus querying the data between separate applications, because they have different enough concerns, that being able to scale them out differently is a boon sometimes.
When I flew Southwest a couple weeks ago, a flight attendant specifically stood watch while the captain used the restroom, and had any passengers who walked up return to their seats.
I've unfortunately had to spend a great deal of time working with Labview (a graphical engineering software development environment that works based on data flow. Functions are represented by blocks with terminals for input and output data. You wire functions together to pass data between them). Some aspects are horrible (primarily the maintainability of anything more than a trivial application), but they also do some interesting things that I thought were quite parallel to this demonstration.

Documentation is quite well embedded into the software, and comes up with a similar box detailing the docs for every function you come across. The system also shows you the flow of data through a programme, allowing you to debug and see the data types moving between functions. You can dive into sub-functions and see what data's moving through them. Debugging can be fantastically quick if you constructed your programme carefully. If you didn't though, it's hell (see the point about maintainability).
Thanks!

I think a procedural language makes more sense for them at this time. They are just learning OOP, so it would be hard to implement an interpreter for that.

About your suggestion, that's cool. I think most of them are already assuming immutability and using floats/doubles as the type of all variables.
I think the trick to surviving either big-company jobs (with the tendency toward bureaucracy and risk aversion) or startup (with the high risk and uncertainty) is realizing that it's just a job, not what you have to be doing for the next 15 years. If it's boring for a couple months, who cares? If the company goes out of existence (or fires you) there will be others. I think it's becoming increasingly important to learn this, although it's easier to give this advice than follow it.
This is it. Never has anyone more perfectly described the way in which I want to work. I love to work in teams on projects where I feel I can make an impact to the extent of my capabilities and starting projects myself is deeply entrenched in my nature.

I've been thinking about wearable computing for the past 5 years and even went on to study Computer Vision because of it, but after 3 semesters i switched the field to general computer science, mostly because i didn't exactly wanted to reimplement AR-Engines and learn about all the algorithms behind object recognition but just use them for the things I had imagined. I've been working in the past year with Metaio's AR Framework called Junaio and Qualcomms AR SDK for Unity 3D and will write about some of my ideas and findings in the next weeks. (You can create impromptu Project Glass glasses by taping a HD Screen Phone like the Galaxy Nexus in 10-15cm distance from your eyes onto some fake glasses and add counter weights on the opposite side, it's enough for testing all the ideas you have in that field until the real Glass comes along)

Right now i feel very bad about my decision to switch fields and focus on working more while studying to get more experience, simply because I think that if i would've finished my computer vision studies i would feel more adequate to send them a job application now...