skysight-inc/HackerNews huggingface dataset:
Ask HN: Best framework for parsing thousands of feeds?
I am currently working on what I hope will be a startup (lean, bootstrapped etc) and I am dealing with thousands of feeds.

Presently I am batching 5-10 feeds to download in batches of threads from Ruby using FeedZirra (https://github.com/pauldix/feedzirra) and then parse.

Has anyone been in a similar situation and done something particularly innovative they care to share? I plan on ranking feeds by frequency of updates after some analysis, but in the mean time I am resigned to pulling everything down in as quick a time as possible.

I would love to use Superfeedr for this, but cost is prohibitive for me and I do not want to stump up the cash to pay for the credits whilst in development (although I could move to this in the future).

Not so bothered about the technology/language - this is a hodgepodge of Ruby, Ramaze, MySQL, Solr and good old file system storage.

Advanced thanks and appreciation of any and all comments!
No I don't think so. He is talking about a single request here, you seem to be talking about a session.
Twitter is almost the same. Except that with Twitter you can receive useful information/links from those who tweet about a specific niche. I can't stand when some tweet about being drunk, getting a new haircut, etc. Fortunately there is an "Unfollow" button.
Where are all the London hackers working on startups? Places like TechHub and White Bear Yard, mostly. Working.

(Speaking of which, nearly everyone I know – including us – is hiring.)
love to hear that :)

Do you remember the name of the website back then?
Yes I've written XMPP libs at the base level. It's mental. There's 15 hoops to jump through just to say "hello!", forms must be filled out in triplicate, several nested stanzas must be entered into. It's absolutely ridiculous at the protocol level.

I agree it's probably pointless, but I find it odd that XMPP is now apparently 'cool'. I thought it was commonly thought to be an abomination of the highest order.
I've had a lot of dreams in my life to the effect of coming to a brick wall and reaching forward and turning the door handle that didn't exist until I reached for it and opening a door and going through the brick wall, shutting it and watching it turn back into a brick wall for my pursuers. Sorry I can't explain better what needs to go on in one's head to walk through brick walls. But it can be done.

Peace.
One portion of my application to grad school (in English Composition) explained how my participation in /., Digg, Reddit, and Hacker News (among others) got me into the habit of daily writing. It's probably the largest factor in my ability to write.
How to Navigate the New World of Database Technologies
After Oracle's purchase of Sun and the Dalvik lawsuit, I think it may be time to consider Mono and the Common Language Infrastructure more seriously. There seem to be far fewer legal issues there, surprisingly (mostly centered around using the Microsoft APIs, not the core language).
> I didn't have rebellious teens because there was nothing to rebel against.

I'm curious about this statement. Does that mean that when your child become a teen you placed no restrictions on them, and let them have a free reign?
Thank you for the idea :) We thought about adding a "store" for when there was a match but I have to admit that we never thought about doing that at the sending !

We definitely have a lot of options to explore !
Yeah, but they do care when the app they want to use won't work on their phone because their software is too old.
I disagree.

The (quite old; it'd be interesting to see how this panned out) question came across to me as someone who, at the end of their many years in education up to this point, has got very used to directed learning. They are used to learning by being set a challenge with some accompanying theory and completing the challenge. They may well be good at this and enjoy doing it, but they haven't become used to searching out the material for themselves.

In fairness, for a corporate setting this may not be a great limitation! When you're working on someone else's project doing maintenance work as most developers end up doing, the need to seek out new problems and work out novel methods can come later; for now, your work domain is fairly closely controlled.

We may well have a CS graduate without the passion to stick at development, but this isn't necessarily the case. I did very little personal work then, because I had other interests as well and was doing plenty of programming for my course, thank you. I think we've got someone with the self-awareness to realise their limitations but insufficient experience of learning outside of education to know how to address this.
I am not sure why you see socialising with fellow geeks in the real world as a popularity contest.
could you give some examples? bjonathan could benefit from learning more about the other services (what worked for them, what didn't).
I remember back in the day there was a Facebook app that did something like that. Though I'm not sure how well it did because I never used it. Or did I...? (No, I actually didn't :-) )
There is not really much there yet, but the idea seems to be aggregate all your information related to interaction/social-networking/communication in one place. One place under _your_ control. That's feature number one.

Then, it could present this information in a way that is actually comprehensible by mere humans. Keeping track of contacts/addresses/events/news/photos/videos/conversations spread up between Facebook/Hacker News/lesswrong/Flickr/YouTube/LinkedIn/blablablablabla is not easy.

It may be of much less interest to you if you like to just forget about what you've done in the past (as it is practically done outside the digital world). Or if you do have an actually working sync-solution cross all your communications means.

Besides having a better overview of my digital self through this, I think I'd use it to come back and convince myself regularly that yes, I also have said so immensely stupid things in the past.

I think it is a great idea -- but to make it useful, it requires a tremendous amount of work.
Use the same trick to get root access on a linux server at school. As the Webmaster of our site on the server I had some sudo rights, thus the trick was actually not so clever ;) But I had sudo rights using Vim and when I went to the command shell in vim I had root rights.
Nelnet Business Solutions - Lincolnshire, IL is hiring entry-level Java web developers.

See http://www.nelnetbusinesssolutions.com/.
I personally don't know of any surgeons who performed surgery before going to medical school or outside of medical school for enjoyment.
I'm really interested in the difference between your perspective (which I share) and the perspective of Mr Pavlina (which my friends share).

I see people's attitudes towards FB fall into these two distinct camps:

1) People who feel compelled by FB somehow, and end up deleting their profiles or "Quitting" entirely, and

2) People who have a profile, but don't feel compelled to check it or interact- except at their leisure.

I think it has something to do with implied disrespect when you don't get back to people quickly; Ignoring a text or phone call sends a message. Does 'ignoring' a facebook ping send that same message? What about failing to respond to an indirect message on Facebook? There must be some sort of social phenomenon putting uncomfortable pressure on people somewhere in the setup of facebook.
Thank you, yeah I would definitely have a lot to learn from that !

We thought about doing a Facebook app but ToS seem not very friendly with apps doing anonymous action :)
I guess the main takeaway is that programmers like to ask questions and answers on StackOverflow because it's very helpful and popular. Even through newsgroups and other forums existed previously, StackOverflow is taking it to new heights by making it easier to ask questions and get answers.

Also, the site admins have cultivated a peer pressure of having answers/questions in good English on the site forcing people to sit down and formulate their typing instead of bashing the keyboard like it happens on other programmer forums. This is similar in effect to the editorial guidelines on HN, the same link on Reddit gets a very flamebaity headline whereas it's pretty tame on HN because of peer pressure.
I have worked at startups in the past.
The Myth of “Write Once Run Anywhere”
The Techie Dissidents Who Showed Egyptians How to Organize Online
Suggestion for Amazon: Open source the Kindle apps
Google's "number of results" is a wild guess (bottom of post)
I think you're missing the importance of feedback, in particular the points of speed and quality. Writing more doesn't help if you're just repeating the same mistakes over and over.

In the traditional writer's workshop, you write something, it's passed around, and people make comments. Depending on the workshop, a good portion of the feedback can be useless. And it's quite possible that few or even none of the others are trying to write the same sort of thing as you, so you don't have a good source of contrastive examples.

Read the SO response he got again. Near instant feedback, from a peer group, with counter examples to show what worked with the readers and what didn't. Faster, better quality feedback that leads to accelerated improvement.
I agree. But I don't see Facebook as a direct communication link. It is just a feed that I consume at my leisure.

If you want to contact me and are in my circle of people that have my cell number that is the best way to directly contact me. But even on my cell I sometimes choose not to pick up immediately and listen to the voicemail. This way I can decide if the message needs my direct interaction.

Anyway people put to high of hopes on tools. Social interaction is about people and not the tools.
to get pgsql to force a particular index, and forgo destroying the site for another few days.

It's just a belly feeling but I smell a deeper problem in there. If your live queries are so complex as to require advanced massaging then perhaps you missed to collect some low-hanging denormalization or caching fruit earlier in the game.

The PostgreSQL planner has always done a flawless job for me in terms of choosing between indexes and tablescans - on reasonable queries. I can count the occasions where it made obviously bad decisions on seemingly simple queries (due to thrown off stats after some slony confusion) on two fingers.

However, if your site depends on cascades of sub-queries to build interactive views then I'd first take a step back and re-evaluate your persistence strategy before putting blame on the database.

if the original author of our schema hadn't made some critical mistakes

Okay, perhaps my belly feeling isn't too far off?
>For example, what happens if you discover you've been hacked and all of your encrypted passwords have been stolen? No big deal.

Encrypted passwords are just as bad, because anybody that can steal your password store can probably get at your source code, or wherever it is that you store the key.

Store hashes instead.
Well, the informative thing for me was that one of his original goals in starting StackOverflow was to improve the writing skills of programmers and not just(like everyone would assume) to create a programmer question/answer forum.
To be clear. If someone submitted this code to you for code review you'd say, "nice and concise -- approved"? Really?

I get this person may have been working with constraints we don't know about, but its not straightforward. Give this to 10 working C devs and ask them to tell you what it does without running it.

The standard cues of straightforward code aren't there -- well named variables, well named functions, comments, indentation, etc...
That depends very much on the question. If I just want THE ANSWER as my tld, and care about nothing else, then it IS the answer.
Microsoft adds H.264 support back into Chrome on Windows 7
I think there's a communication barrier here, but I'll just let it go for now. Gawker's passwords were unsalted according to the first few hits on Google and that's all the research I did on it, not that it really makes a difference since a salt isn't the point here anyway; the point is that you don't have to be using bcrypt to be a responsible datakeeper and that bcrypt isn't an end-all as it's made out to be.

You're right that I don't have a very deep cryptographic background, but from everything I've seen and read, dictionary attacks still work on bcrypt'd passwords, just more slowly than otherwise. This is the part I want to know if I'm wrong about; is there some a reason a dictionary attack doesn't work on bcrypt? I understand that it will take longer to get 100% password recovery out of a bcrypt'd database than a database hashed with SHA-256 or SHA-512. Let's say that's not relevant because our theoretical attacker has A) a computer from the future which is fast enough to blow through bcrypt just as quickly as SHA-* or MD5; or B) is only interested in one user's password, so the expense is bearable. In these situations, does bcrypt provide extra protection that isn't routinely implemented in a standard hash?

My understanding is that bcrypt does not provide much besides the slowdown. I'm not trivializing the slowdown, I understand completely why it is beneficial to keep passwords away from bad guys as long as possible after a data leak. I am merely saying that if I'm correct and the only thing standing between bcrypt and SHA-* for a captured password database is X iterations of silicon, I don't really see using SHA-* hashes as "completely irresponsible".
I'm 24, and my wife is pregnant with our first.

Reading this article blew my mind. I grew up with AIM, ICQ, MSN Chat, whatever. I had a cameraphone when I was 17. I didn't see any of this, though.

I'm not sure whether I'm more scared to have a girl that has to live in a world like this, or to have a boy who may be like the guys in that story. I hate both outcomes.

I know many will argue that pornography isn't a big deal, or that the internet hasn't changed much. But as a parent who doesn't want to raise a daughter in that kind of environment, or a son who encourages it, what can I do? What would you do?

Is homeschooling the answer? I don't think so. I am very interested in seeing my kids develop socially in ways that they just can't do at home. Living in a small town? I think that's worse, because there's nothing else to do besides sex, get drunk, and do drugs. Shutting off the internet in the home? Seems unrealistic, but it is necessary?

I know, what I'm saying probably makes me look insane to many of you. But for those who are dealing with this with their kids now, what are you seeing? What should a future-parent strive to do?
Given that I agree more with Malcolm Gladwell's thoughts on the impact of social media's importance (or lack thereof) for revolutions, I found this article quite interesting. I think the main lesson is that while social media may not do much for getting outsiders to be heavily involved and emotionally tied to whatever's at stake, perhaps it is an amazing tool that makes organization much easier for people who are on the ground (at least until the ISPs get shut down by the government).
Attacks on reporters is worthy of cover because it is often done to drive reporters away so that no one will know when things go really ugly.
Or you can change. Now. It's not only possible, it's not really that hard.
Cliques of children harassing other children! What a new and amazing discovery! This is unprecedented! </sarcasm>
I'm working part-time on my own startup
I'm excited about project. Hope I'll get rid of custom scripts that download contact data and place into repository.
Both the future independent Scotland and the United Kingdom of England and Wales would presumably be constituent nations of the European Union.
We're actually using hardware-accelerated CSS-based transitions based on jqtouch. The difference is that jqtouch doesn't scroll you back to the position you were on if you navigate back - you always scroll back to the top, then transition so if you're on a long page, click a link, then go back you lose you place. We think that remembering you position is a very important usability feature that's expected on the web so this is included in jQuery Mobile. Unfortunately, it introduces a blink or jump which makes things feel much less smooth.

Page transitions, especially on iOS are an incredibly important issues for the team and we're actively working on improving this. In fact, we have 3-4 different branches exploring various approaches:
https://github.com/jquery/jquery-mobile/issues#issue/455

We weren't able to find a solution in time for a3, but this will continue to improve.
In case you didn't know, orkut had this feature for quite sometime. I'm not sure if it's still active though.
"Every so often, POF sends you an email with your password so you don't forget it."

I threw up i my mouth a little when i read that. Stupid crap like that is what made me switch to 1password, so if some random guy in a coffeeshop looking over my shoulder does see a plaintext password, it will be too difficult to remember, and will only open up one website.

Hello, haven't they ever heard of auth tokens?
"Where do you work" asked 9 days ago:

http://news.ycombinator.com/item?id=2145250
I actually think he accented on "2003" and not on "big money".

Anyway, I would suggest that you create a facebook application to go with the website, you could get quite a following.
Verizon breaks first day sales record with iPhone 4 pre-orders, in only 2 hrs
The author wrote early on in the article "It happened not because I was a reporter", so while I can see your point in general, it's not particularly relevant to this article.

The fact that, in some cases, reporters are being directly targeted is however of real importance. Not because something is worse when it happens to a reporter than when it happens to one of the protestors, but because of what it signifies.

It signifies a mixture of two things, which are Mubarak supporters being mislead by propaganda into blaming reporters for what is going on, and also the fact that Mubarak is doing everything he can to prevent free press.

I don't know how much of each of them is in play, could be just one or the other, though I suspect it's around 50/50. Mubarak has shown he has no problem using violence to try and improve his situation, and he's shown in numerous ways that he doesn't like the reporters who aren't state-controlled, so it's not a huge leap of assumption to think that he is encouraging his supporters to attack reporters.

"It happens every year ... in the USA"

Not promoted by the White House in order to try and prevent the country from understanding what is going on. THAT is why this is relevant, not because we have to care more about reporters.
So the solution is to lock away the computers in the gun cabinet?
Oh ok I didn't know, I gonna take a look. Thank you for the tips :)
May I use this to flirt with my boyfriend?
I switched to DDG about 3 months ago. If nothing else, the bang syntax is a huge win for me. Among my favs are !cpan !jquery !js !perldoc !php. Makes looking stuff up nice and quick.
imho, cs degrees are not supposed to be about programming. their focus should be purely theoretical side of computation e.g. heavy emphasis on algorithms, data structures etc. etc. which pushes them mostly towards applied mathematics imho. most programmers are autodidacts anyways, and picking stuff up is generally not that hard.
I'm working on a project on the side.
Recommended Tech Startup Lawyer and CPA in Bay Area?
We are thinking about incorporating a subsidiary for our startup (exvo.com) in the USA.

Can any one recommend a good startup lawyer and CPA?

Thanks!
Congress Questions Facebook CEO Mark Zuckerberg About Users’ Data Privacy
Wouldn't that still be a finite state automata?

The key is the bit about the "recursive call."
Yeah sure! Lot of our users are sending 1 crush, and we are guessing that it's at their loved one .
Kopp-Etchells Effect on Military Helicopters
What about an 'I am not involved or particularly interested in startups', and an 'Other' option?

(This is in the interests of fairness and not out of any personal bias.)
>Which, regrettably, has not been my company's experience with Oracle 11g --- we haven't resorted to explicit hints, but we've restructured queries in other ways for order-of-magnitude improvements in performance.

He's not claiming that query planners are so good that you'll never have to restructure your queries. I've written many queries that seem perfectly reasonable, but once I start optimizing I can see I'm doing things completely backwards. It's just like almost everything else programmers do. If I dash off quick program, I don't expect the VM figure out how to make it perform perfectly. Why would I expect database software to be different?
It's interesting to compare Greece and Ireland.

Greece was profligate, openly cooking the books since at least 2002 (http://en.wikipedia.org/wiki/Greek_Financial_Audit,_2004) whereas Ireland was doing the noble thing by imposing austere measures on itself.

Yet both needed bailouts by the EU recently, and both will probably wind up defaulting.
Self-organizing systems in Egyptian protests
Determining the SEM Budget
Hands on: Android Market website improved, still lags AppBrain
A) I anticipated problems and taught them "wisdom" before the issue came up so they wouldn't be stumbling around doing stupid, destructive stuff.

B) I'm not a rejecting, judgmental, controlling, authoritarian kind of parent. So there isn't anything to rebel against. For example, when my oldest would say he wanted to play video games for a living when he grew up, I would say "I don't think that's a realistic career goal. But don't let my lack of vision stop you. Feel free to prove me wrong." So he never felt compelled to go make scads of money as a competitive gamer just so he could throw it in my face. He was free to have his idea of what a career looked like evolve without making it a power struggle with his mom.

My kids are very well behaved, ironically because I never required it. On the one hand, I imagine it would take many, many words to adequately explain (and maybe some day I will get back to developing the parenting site I own). On the other hand, I also feel like there really isn't anything to say. Just love and accept your kids. No big. (And work on your own crap. Ninety nine percent of the time, if kids are a-holes, they are just reflecting demons in the parental subconscious. I spent a lifetime hunting down personal demons instead of kicking the crap out of my kids for being reflections of my personal demons.)
Mailinator - averaging 4.5million emails/day on a single 2Ghz machine (2007)
Boston Mobile Company to Buy MySpace?
Worth pointing out it's just been covered in TechCrunch EU as well:

http://eu.techcrunch.com/2011/02/04/not-sure-if-she-likes-yo...

Sounds similar to FitFinder and the like... Web 3.0: The flirting network...
How to convert a bookmarklet to Google chrome extension
Yeah, I agree that the overall utility and technical meat of his posts has dropped off in the last couple of years. (Critics will likely jump in with "What technical meat?") I don't think that's an issue with this particular post, however.

I'm not sure why you doubt his sincerity. Stack Overflow is where he's spending most of his time these days, isn't it? Why wouldn't he be mentioning it in most of his posts? It seemed appropriate for this post. What would constitute a sincere blog post?
I think 1405 days is long enough to bring this back: http://news.ycombinator.com/item?id=7859

Inspired by this thread: http://news.ycombinator.com/item?id=2179545
It sounds like you need to read an entire article. The article argues that they're not incompetent but evil.

Essentially, plaintext allows them to email users their passwords periodically, which in turn allows them to increase user retention, security be damned.
deal with it, it's established at the RFC level so at this point I'd prefer XMPP to <insert proprietary protocol here>