skysight-inc/HackerNews huggingface dataset:
I spent a little while trying to implement a twitter back end for fun during their fail whale period and it's really just a data structure problem. (Edit: Figuring out which of the 240 million accounts they should be sent to.) You can literally get the basic freed process running 10k tweets per second with 100 million accounts on a single PC and a 1Gbit Ethernet connection where the Ethernet connection is the bottleneck. Fanning that out to various places to listen in is a fairly basic scaling issue.

Not that I took it vary far, but my first stab to see what the scaling issues would be was actually plenty fast to run there feed process at the time.

PS: The 'trick' is to keep two lists one everything a user follows, and another is everything that follows each account. For showing the messages to someone when they log in you keep the last 10 message Id's with timestamp or sequence ID so if someone follows 5k accounts you can avoid looking at the vast majority of those messages then sign them up the device to listen to new messages. (Sure, sometimes you will need to look past that top 10 but it's rather effective.)

(And for the downvoters I can upload some code if you want to see it.)
Funny thing...just yesterday I was just considering opening an account with them and Tarsnap as a dual-strategy backup-of-backup.
Do you have to google it every time? The fact that repeated googling is required is a sign that something is off--either you're not growing from your experience or the API has too much friction. I've often felt this way when it comes to linux command switches. Just accepting that having to google everything is OK is the wrong answer here.
What We Learned About Scaling with Apache Storm
Yes, exactly, the old saying goes: RTFM (Read The Fine* Manual). In the 21st century google is the manual. If you're not looking stuff up in google, you're doing it wrong. It's great if you can do some stuff without using google, but memorizing reference material doesn't mean you're a better developer.
Yes, I think the article got that wrong. You are right that Gizzard is eventually consistent. The storage engine for Gizzard is mySQL, so in the event you want to do some node-local indexing, you at least have mySQL's strong consistency, but you won't achieve it in the overall system across keys.
You mean like this?: https://www.kickstarter.com/projects/mohu/mohu-channels-pers...
Oh nice that didn't show up in the schedule below the stream.
A lot of the problem with looking for an answer, especially if coupled to a powerful search system, is that you'll usually find one. Even when you're asking the wrong question. A large part of becoming a better developer, it seems to me, is learning to ask the right sorts of questions.

Which is difficult to do if you don't have a reasonably wide general knowledge and an understanding of why things are done a particular way, as compared to some other way, and what the trade-offs in that choice were.
Delphi pools were invented by RAND, not Brunner, IIRC, have been studied over the years (Armstrong's _Principles of Forecasting_ has some papers on them), and work differently from prediction markets: Delphi pools collect a small number of experts, asks for written analyses and breakdowns and qualitative assessments (not necessarily the precise numerical probabilities of prediction markets), feeds each expert's analysis to the others, asks them for a new analysis incorporating the others, and repeats this for 2 or 3 rounds until a rough consensus has been reached. This is not very similar to prediction markets.
An alternative to the distributed file system suggested in the article is a distributed in-memory data/compute grid.
also there were magazines with full working programs (sometimes event 4 to 5 pages of "beautiful" BASIC) that you manually copied - letter by letter, number by number - into a trusted C64 (or other pre-historic computer thingie).
You can go here [1] and select "Register as a developer with App Studio" - Free, and create apps to run on your own Windows Phones

[1] http://developer.windowsphone.com/en-us/featured/devpreview
If you're still suffering that pain you need to switch to better build tools. Source map support is pretty mainstream now.

If I open Chrome Dev Tools on my production site, it automatically grabs source map files from my server (accessible only to me), and I see the whole source tree of original coffeescript files, with complete access to breakpoints and stack traces.
Watch programmers who don't Google how they code, it usually ends up complicated, slow and worst of all insecure. If the only tool you have is a hammer everything starts to look like a nail. Using string.join instead of paremetherized SQL queries is a typically example of this! Any tutorial on top of Google will show you the correct solution to this but if you are too smart for this and reinvent your own way of creating queries then this will bite you in the ass one day.

Even if you think you know how to do something, spend 5 minutes on google to see if this is the defacto optimal solution for your problem and if there are any gotchas you have to consider. I'm not advocating blindly copy pasting code from random stackoverflow answers but at least look at them and understand what they do, then you can copy them.
You have a right to speak to those who choose to hear you (to wit: not be punished for doing so). That is the point of the ruling in question: Citizens United made a film (acknowledged by the court as none other than a 90 minute campaign commercial directed at a particular candidate) and accepted money to "speak" (via pay-per-view, prepaid by corporate supporters) to those who made a deliberate and positive effort to listen; current law forbade them from doing so, and there was no other way to allow it while preserving the bulk of the abridging law without instilling a "chilling effect" on other lawful political speech (which would be deemed so by voluminous case-by-case analysis, there being no bright line).
in theory Microsoft COULD support linux directly,would be a bit of work, but they would not even need mono.

Now from a business persperctive would it make sense? i dont know,Microsoft business relies heavily on Windows. But hey,anything's possible.

Problem with mono is ,mono or not,.net dev without VS kind of sucks,and VS is Windows bound.
Who's talking about a witch hunt? My article is about my personal decision to choose another browser, and the conditions for reconsidering Firefox. None of those conditions involve him losing his job. I don't think you actually read it.
I have grown a lot in the last few months by consulting library code & documentation first when I hit a road block & only turning to Google/Stackoverflow if I hit a weird error message I don't understand or shit really hits the fan. One major benefit of opening up the source code when I'm using a library is it often sparks my interest to return to that library later just to learn more about how it works.
GJP is still running and just added some new IARPA questions today. Registration for the next season may not be open yet, though.
Thanks for making your point clear.

I don't see any evidence that they are going to abandon other devices. The Kindle Fire didn't stop them from releasing an iPad app for Amazon Video, for example.

In reality the number of platforms they need to target is fairly small. There is the iOS world, there is Android, Roku, and HTML5. The vast majority of TV and blu-ray player apps are the same HTML5 app.

I would be concerned if Fire TV and Kindle Fire became too popular that they might abandon/neglect the other apps, but that's not likely to happen.
I've been using Fotolia. It's one of the better stock photography sites, and their prices are pretty good if you:
a.) Pre-pay for credit
b.) Don't need images larger then 1200px X 1200px
How is supporting two desktop operating systems any less realistic than supporting a desktop operating system and at least two separate mobile form factors on three different platforms? Both have the potential to suck if one platform doesn't get enough attention.
For the vast majority of Romanians, or anyone else living in the Soviet empire, the surveillance state made no detectable difference in the outcome of their lives. It was conducted under the rule of law. Why be bothered about it?
Equal treatment: Everyone is allowed to marry one person of the opposite sex.

Now some disagree that it should be "of the opposite sex" and instead want it to be: Everyone is allowed to marry one person.

Soon the next special interest will come along and want it to be: Everyone is allowed to marry one or more persons.
So is this Silverlight but for WinRT apps instead of WPF apps?
I'd like to see them incorporate the recycle bin into the taskbar, similar to the dock in OS X.
Developing is pretty hard. And, although you can get great information, Googling doesn't get you very far.
I can't stand the desktop with all the window management that is involved, so I generally prefer to just swipe between full screen apps. Sometimes you need to have multiple windows on screen, however, and so the desktop has a purpose, but as Windows moves forward it should be relegated to a secondary role.
The GJP prediction market is pretty similar to how Intrade operated, but a head to head comparison is going to be very difficult inasmuch as a) Intrade has been shut down for like a year now; and b) Intrade's contracts overlapped only a little bit with the IARPA questions.
Show HN: Twitch Plays 2048
If I were you, I'd be launching my startup.
But then you'd be me, and you wouldn't know what to do about the monkeys,
and what would be the point of anything, then?
The market exists for Nintendo games, not for shitty consoles with the same games you can play in your phone.
I wanted to read this but got hacked off with the letterbox created by the tall footer banner, which was also a big distraction. It would be nice if there was at least a way to hide that without having to fire up Chrome developer tools :)
Apps making networking easy
Axios Open Source Modular Ergonomic Keyboard
After using Windows 8 since release, I kind of prefer the current start menu to the old school one, but I guess I'm in the minority.
Being a little loose with my estimates and a bit of Fermi Math, thats only about 300 years of computing time on a small home built GPU cluster.

Basically tells me that 4 random words are definitely crackable and 5 are theoretically possible (and definitely doable with 5-10 years of Moore's law)
I doubt you'll be able to disable it, but presumably you could just remove all the tiles if you really don't want them.

Personally I think it might be a good idea. It seems like a reboot of what Microsoft was going for with the Windows Sidebar (or the Start screen, obviously) in a format that actually works with desktop Windows's overall UI paradigm.
Insurers Are Really Mad at Sovaldi, the $1,000-a-Day Miracle Drug
> writing HTML/CSS

That's never a rote process for me. Every bit of HTML or CSS I write requires human judgment. Any part that doesn't is handled for me by preprocessors.

I'm guessing that instead of "writing HTML/CSS," you'd have designers meticulously dragging objects and setting properties in a GUI. Isn't that just the same thing with another skin? If I have to click to set a property instead of typing the property in a text file, have I saved myself any work? Have I eliminated the need for some bit of technical knowledge?

> linking together various libraries

Concretely, what sorts of tasks does that entail? Generally there's a few moments to drop the files into the right folder, and a few more to add the appropriate tag(s) to the <head> or the bottom of the bottom of the <body>. How do you eliminate those steps? Or does the user still have to perform them, except in a GUI environment instead of a text editor? If the latter, then I'd argue the same as I did above about HTML and CSS.

Or do you include certain standard libraries by default? If so, that's fine, but the same is achieved in more traditional, text-based frameworks.

> pushing code through processors

Which processors do you have in mind? SCSS and Uglify are quite trivial to run. I wouldn't call that a pain point at all.

> wiring up a basic UI to a database

I'm still not convinced that can be automated beyond the basic interfaces that something like MS Access provides. Which don't cover most use cases. Can you give an example?
So I just bought a new TV that has Amazon Prime, Netflix, Voice search integrated into it. I don't play games. Is there any point in me getting this vs getting an Apple TV or a Chromecast? The only thing I would use an Apple TV (all computers in my house are macs) for is to screenshare wirelessly. I believe Chromecast lets me do that for specific apps.
Yeah, I'm all in favor of compiling better languages into Javascript, but it requires a degree of maturity and completeness from the surrounding toolchain.
Microsoft says Windows for the Internet of Things will be free
I think Qt does it decently, as far as the desktop platforms are concerned.

I mean look at something like Origin or the Battle.net desktop app.
I think this is similar to the question "does using turn-by-turn GPS make me a better navigator?" The answer could go either way. If you depend on the GPS to the exclusion of paying any attention to your surroundings, you'll probably be a worse navigator, because it will take you much longer to learn even the common routes in your day-to-day life. If, on the other hand, you use GPS-aided navigation as an opportunity to learn the back roads that take you between two places more efficiently, but that you never would have discovered on your own, and internalize that knowledge, GPS can help you become much more intimately familiar with an area than you otherwise might be.

I see Google/Stack Overflow through the same lens. Lean too heavily on snippets of other peoples' code, and you'll never learn to solve even simple problems on your own. But if you take the time to actually learn why a particularly elegant answer on Stack Overflow works, and generalize that technique, you can grow your skillset much more quickly than by trying to figure out everything yourself from first principles.
Michael Lewis: “The Stock Market Is Rigged”
I believe there is something more nuanced going on with Chromecast "casting" performance.

Netflix is fast obviously because you are just telling the chromecast to stream netflix itself, not "casting" it to chromecast. Same goes with youtube.

However "casting" performance for me is even worse than the described .3fps on my Chromebook Pixel that isn't running ChromeOS. And my LAN is fast enough to use mplayer over ssh/x-forwarding... Trying chrome tab casting in Debian on a Chromebook Pixel burns the Pixel up, with it being very obvious that the Pixel is the bottleneck.

Trying to video encode the "cast" tab without using hardware support maybe? I'm not sure, I haven't really investigated it further.
https://www.google.com/search?q=gays+who+oppose+gay+marriage

Easy to tell the good guys from the bad guys when you think in black and white, isn't it?
A single tweet is closer to 2500 bytes on average in most of the data feeds. However, your point still remains that this a "human-scale" data model that is relatively small and with a low data rate. It is pretty easy to engineer systems that can keep up with this even if you are not an expert in designing real-time database systems.

By comparison, many complex machine-generated data sources (e.g. real-time entity tracking) that are sometimes fused with the Twitter firehose operate at millions of complex records every second (often tens of gigabytes per second) that need to be processed, indexed, and analyzed in real-time. You can't deal with this kind of data model using something like Twitter's current architecture because the several order of magnitude difference in velocity and volume exposes the limitations of most database platform designs people typically use.
For insertions and lookup it is probably hard to find a situation (for large n hashes are asymptotically superior).

Trees provide some other nice properties such as maintaining an ordering (so it's fast to get the smallest argument or interate of the elements in numerical order) and always perform at the asymptotic time cost (a dynamic hash table occasionally takes O(n) when the table is resized so it might not be a good fit in a realtime situation)
Slightly more weirdly, Google TV does have an Amazon Instant Video app. But non-TV Android does not. Well, Kindle. But non-TV Google-standard Android does not.
WinJS playground
You're right, if the winning idea is "build the control software for a self-driving boat that can search for a missing airplane," then the tech certainly will not be easy (and would be pretty awesome). What we were getting at with that line was that there are lots of great business ideas that can be prototyped without solving monumental technical challenges, and having a solid prototype isn't enough for a startup to succeed - it's only the beginning.
I've loved Ubuntu One for years. This is too sad, I would've wanted to have Ubuntu use it automatically for all kinds of things such as home directory backups etc.

Are there any good open-source self-hosted options that I could run on my little box at home? Preferably something that doesn't require a special setup or deployment on a server.

I could imagine there are file-sync solutions that just need an operable SSH account somewhere and merely automate the use of rsync to do the transfers, watching files and taking care of conflicts.
Days like this make me sad that I can't have the best of Windows Phone, iOS and Android at the same time. A "Cortana for Android" would be great. Some day perhaps phones will be standardized enough that, like the web, we can develop once and run anywhere without major sacrifices.
The Verge link doesn't mention Windows 9 anymore. And the screenshot actually contains the "Windows 8.1 Enterprise Build 9600" string (check the right corner).

https://pbs.twimg.com/media/BkPBkEXCcAEIbN4.png:large
right click >> Properties >> Disable metro

I hope there is an obscure setting somewhere that would disable metro completely. As a programmer I see no benefit to it.
How U.S. stock markets are rigged
How to Run an Entire Data Center as Easily as a Cellphone
Whenever he finally wakes up.
lg(65k^4) is very nearly 64. If you worry about 4 random words being brute forced, you should worry about 64 bit symmetric keys being brute forced. I don't know where the current recommendations come down on that.
I bet you can count the number of people who did that on one hand. They're not shuttering the service because it was popular. They are open sourcing the code though, so you can run your own version and not change any code.
Julian Assange to Publish ‘When Google Met WikiLeaks’
That's where the grandparent's idea of being able to run Don't-Call-it-Metro apps in a windowed mode comes into play.

Doesn't seem like it should be hard for Microsoft to accomplish. Metro's already supposed to be resolution-independent, so all that's really accomplished by forcing them to be fullscreen on desktop monitors is keeping ease of task switching and information density to an absolute minimum.
I'm fascinated that they still keep the Building Windows 8 blog up these days. You can go back and read their justifications on why they made the major changes that are now being backed out of win 8, for example start menu: http://blogs.msdn.com/b/b8/archive/2011/10/11/reflecting-on-... and http://blogs.msdn.com/b/b8/archive/2011/10/03/evolving-the-s... I would love to see an analysis of how with so much data and insight they built something people didn't want. Was it all just confirmation bias causing them to see what they wanted in the data?
Why isn't there more smart traffic management research?
Then you may want to consider staying out of the software business. It's layers all the way down.

The relevant question is only how leaky each layer of abstraction is.
Quoting Albert Einstein: "Never memorize something you can look up in a book."
Yeah – it's an easier way to do donations, but the tax implications and fees are the same as always. It even comes out of your paycheck pre-tax... making your tax end simpler as well.
The lady in the photo looks like she's confusing the remote for a candy bar.
damn, intro to python at 12? I wish my grade school taught me cs :(
Windows will be free on phones, small tablets and the Internet of Things
Like all tools you have to aware of its limitations, but I think shading can be a fantastic way to visualize uncertainty.

For example, the Bank of England occasionally uses it in plots of economic forecasts, where time is on the x-axis and things like GDP might be on the y-axis, eg. here http://www.bankofengland.co.uk/publications/Documents/inflat.... The fading out of intensity over time is a great visual reminder that predicting the future is hard.

It is much better when your chart is supposed to be targeted at the general public, because the "smearing out" of the data is very hard to misunderstand, unlike confidence intervals.
The amazing thing is that you can potentially write all your code in C and have backend/frontend logic for the web, Windows, Windows Phone, OSX, FreeBSD, OpenBSD, Tru64, VAX, Arduino, Linux Kernel, Windows Kernel, Mach, Hurd, and practically every processor or operating system ever built, and even iOS/Android (via Xamarin). Wow.

You can even link to and reuse that code from languages such as: C++, C#, Java, Ruby, Perl, Python, assembler, lua, scheme, lisp.

The future is gonna be so 1970.
On my own projects I'll sometimes "reinvent the wheel" on purpose. It's a good way to make learning mistakes. For better or worse, I tend to learn from making mistakes.

At work though, my employer isn't really paying me so I can experience personal growth. I'm try to deliver value as efficiently as I can.

I've done this long enough to know that the "easiest" problems can have subtle bugs. Think of the singleton pattern: if you write one "without looking at the answer" you invariably make a small mistake that introduces a race condition. That's why I Google even "easy" problems.

That said, I can't do math in my head very well any more - calculators have robbed me of day to day practice and my 8th grade math skills are flabby now. (My degree is in math so it's especially embarrassing.)
I'd suggest you replace the music with a voice-over that explains what's going on.
Why debug mode is not good for production (clear-text password)
Cortana (yes) and Many, Many Other Great Features coming in Windows Phone 8.1
Jetstrap Free for a Year
If you're picking with structure (including "phrases that spring to mind"), agreed. If you genuinely include enough entropy, then it doesn't much matter what mnemonics you layer on top.
I have often thought that we could do much more to improve fuel economy, safety, and travel times by getting cars to communicate with each other and a traffic control center. Tightly bunched cars should be able to travel through intersections at full speed by simply adjusting their timing to avoid other traffic. Why hasn't there been more work on something like this?
Can anyone recommend software that works for the German market (but also available in English), ideally that can integrate with Elster?
Isn't a PR blog, by definition, more than a little disingenuous?

I imagine certain data was cherry-picked to tow the party line (Ballmer?) but now with the new talent coming into power, things are changing.
You can have Metro apps side-by-side, including next to your desktop apps, right now. Just grab from the top of the screen and drag left or right.

http://i.imgur.com/3IwDXVA.png

Or, don't use Reader. The inclusion of a free PDF reader app with your computer purchase doesn't get in the way of using a desktop program instead.
I am wondering whether they can make Windows Runtime's GUI avaliable for existing desktop appliacations. There are many professional applications needs a brand new API for user interface.
Because it's just a higher level wrapper around a unittest framework? I have no idea what you mean when talking about various roles. What roles?