skysight-inc/HackerNews huggingface dataset:
How can I improve my estimates?

I've found that the more detailed my estimation process, the less optimistic my estimates are. Generally speaking, if I break down what I'm planning on doing into units which I think should take a month each, I end up with an estimate which is 2x less than the actual time I will take; if I break things down into units of a week each, I end up with an estimate which is optimistic by a factor of 1.5x; but if I break things down into units which I think can be done in a day each, I tend to come very close to being correct.

I don't know if this trend is something other people experience; but if it is, the obvious solution is to think more carefully about all of the small steps which make up the big steps.
Exponential backoff also works exceptionally well in the case of dead batteries.
I tend to cycle through different patterns, often depending on workload, season changes, etc. I'm currently up pretty much all night and sleeping in until noonish, and since I set my own hours I often just get done communication stuff during the day then work on programming at night, with some socializing in between :)

It can be challenging though, since staying up late can create a tendency to overwork, and often in a fairly unproductive way. I try to keep semi-balanced with a less-is-more approach. You don't need or want to code 12 hours a day. If I can do 6 hours I've had a really good day. If I get at least one good bit of work done, even if it took an hour to do, I'm happy with that. And that seems to be a realistic pace compared to other programmers anyway, in my experience at least. Not to say others are slow, I just mean that throwing more hours at a problem is often a lot like throwing more programmers on a project: there's a decrease in efficiency with each one added.
No one should call back. What you were jabbering about wasn't that important anyway.
TuneCore Tells Us Where We Can Shove It
I like this overall. I read Joel Spolsky's book on hiring techies and I've tried his interview process for the past ten interviews I've done. Some things work and some things don't.

I would add to this article and to Joel's approach that you need to ask some basic coding questions during the telephone interviews to at least pick up on things to review during the in-person interview. Example: for a programming job, I want to know if a person knows the size of an "int" or if they can tell me what the difference between a struct and a class is. If they can't, then I need to make the decision right now to drop this candidate or consider them for a more junior position.

Here's what happened to me recently: I have a job opening for a junior programmer (still do actually) and I had a phone interview scheduled. I prepped my notes and asked what I thought were good questions ("Tell me about that job. Tell me about a challenge you had. Where do you see yourself in 5 yrs?"). The candidate breezed through these questions and I was excited.

For my in-person interviews, I had decided to use a 10-question Basic Skills test. It is handwritten and very, very easy: "Write out the numbers 1-10 each on a separate line in a console app", "describe how a web app works in detail as though you are describing it to a brand new tech writer". It is timed - it's really, really easy - and I allot up to 20 mins.

I handed my candidate the test as the first thing and he failed miserable. Next!

My fault for not having a better phone interview process in place.
No thanks. I looked and, instead of telling me what it is, it asks me to register.
I struggle to see how anyone can be productive at all working all your waking hours. The only result of that is going to be burnout or stress related illness.

I don't have or work for a startup granted, but when I am working on something hard, I reckon I have 5 - 7 hours a day in me and then I am just wasting time - it was the same studying for exams and my degree - I have never pulled an all nighter for an exam, but still managed to get the top grade - proper rest and relaxation are just as important as the working time - properly rested you will get far more done in less time too!

Oh and leave some time for exercise - that helps you productivity, stress and well being no-end!
This is probably a good start to get to know more about some of these subjects:
http://en.wikipedia.org/wiki/Category:Compiler_optimizations
I would say you're right about the differences between MyISAM and InnoDB.

The "feature configuration" system at Facebook is pretty good. That said, it's not perfect: the fact that there are two different caches whether you're running in CLI or Apache SAPI mode has been a problem for me in the past; I still haven't figured out a way to run scripts from a crontab while getting access to the data cached by web pages. Running curl might be an option...

For a CDN, it really depends on the number of files you have, and how much data you're sending out. If you're paying for a hosting plan with more bandwidth in order to serve all your static images, then it might be interesting. You can calculate the costs yourself, actually: http://calculator.s3.amazonaws.com/calc5.html
There are alternatives to S3 (I compared Akamai and S3 in a study in the web company I work for, S3 came out cheaper for our traffic patterns.) This is not to say that cheaper is better, you obviously have to look at the whole picture.

I general, the main problem we have to examine before moving our files to a CDN is the cache policy. S3 has datacenters both in Europe and the US, and there certainly is a propagation delay when you write a file.
There are techniques to avoid this, such as naming your images with a version number, but you have to make sure your clients are going to find the files when your pages start linking to them.

Another important issue is the downtime. S3 recently had a several-hour downtime... this has never happened with our local hosting provider. What will your plan be when this happens? If you can detect that your CDN is down, can you redirect the traffic to your host? Will this risk bringing you down to, if you downsized the equipment after the CDN migration?

It's certainly not an easy decision to take, but if done right and properly planned, a CDN can save you a lot of money.

Anyway, have a look at Memcached. We added a "Cacheable" attribute to class definitions in our in-house ORM so that cacheable data is taken from Memcache instead of querying the DB (that is, if it is available in the cache; otherwise there is a miss + Memcache SET).
For our relatively large website (~1000 people connected now, 1.7M subscribers), Memcache is saving 1 billion queries a month. I would suspect that APC is saving at least that many also.

Having this feature in the ORM makes it transparent for the developers. This is important, because having to wrap hundreds of DB GETs in

if(val = get_from_cache()) {
return val;
} else {
val = get_from_db();
cache_set(val);
return val;
}

is pretty tedious and leads to unmaintainable code.
It is somewhat important to hackers what the future of tech policy in the United States will be. Net neutrality? FCC regulations on different parts of the spectrum? FCC regulations on ISP's? Funding on new internet backbones or litigation against the previous telecoms on their failure to implement high speed access with previous funding? Higher education funding and research grants?

All of these are directly related to the production of new/better hackers or the ability of existing hackers/internet startups to reach people.

While, yes, any other day this would certainly be off-topic, in our two party system when one party decides on its nominee for president of the United States, it is some-what important for hackers to know what the two possible outcomes of the future of Technology policy will be.
Technological and Development Support for the Global Poor

Socially responsible technology development projects aimed at stabilizing and then improving the conditions of the global poor can easily be created from America's scientific and technical base. Many projects already exist, such as the One Laptop Per Child initiative, but these projects are seen as stand-alone initiatives, rather than parts of a coherent policy approach to the developing world.

Target areas could include global access to clean water through simple technologies like solar water pasteurization, development of new appropriate technology power systems, low cost basic medical technologies and so on.

The close analogy to the Air-support-only defensive pacts is that the US would take responsibility for developing and testing these technologies, but would only make the designs available free of charge - essentially providing design and laboratory services, and leaving nations, NGOs and individuals to capitalize the designs and handle resource distribution. This keeps costs down, and makes clear where US responsibility ends, which is an important consideration when considering how to reach outside of our borders on non-military levels.

Another critical area is farming. Roughly 50% of the world's population makes a living by simple farming, mainly using manual and animal labor. One recent study shows that modern organic agriculture techniques could roughly double the food output of these farmers (Perfecto et. al. "Organic agriculture and the global food supply.") Organic agriculture practices could be disseminated using modern ICT. For example, a farmer could send an SMS message to the local American embassy while standing in their field. The GPS data about their location could be correlated with satellite imagery showing the condition of their fields over the previous few years using techniques from precision agriculture, a common first world farming practice. Individualized recommendations could then be delivered. The cost per individual helped is relatively small: the heavy work is done by computers. But the effect on global food security, and on how the US is viewed by perhaps 50% of the world's population is beyond price.

Similar approaches are possible in medicine. Carefully prepared information packs, tailored to the health risks of each climate, geography and lifestyle could be coupled with existing medical expert systems to provide very basic, but still very useful, global health support at relatively low marginal costs. Education is similarly an open door: primary and even secondary education materials, vetted for accuracy and cultural appropriateness, and then translated into many world languages could help countless people at very low cost, and the more educated a population is, the less vulnerable they will be to rumors and propaganda.

http://guptaoption.com/2.long_peace.php

==========================

There is a lot to be done with computers in the developing world.
Ask YC: How to Hack & Have a Social Life
I am a startup founder and i have come to a recent conclusion that my social life is now next to zero. The only people i "socialize" with, are fellow hackers and programmers like myself. This is beginning to bother me, because I have more of an online life than a real life and i am becoming somewhat awkward in real life social interactions. I will like to ask fellow hackers here, how can you hack and still have a "Real Social Life"?
This happens in Europe as well: Something Rotten in Hungary

In the end, Steve Ballmer is doing what any good CEO should be doing, by recognizing where to best put his efforts to best sell his company to the world. Clearly, Mr. Ballmer sees the young minds as being valuable enough to invest 10 billion HUF (~40,000,000 Euro) in them. He's even has done something even better for his share holders: he has figured how to make this investment using someone else's money - namely, the EU taxpayers' money.

http://www.theserverside.com/news/thread.tss?thread_id=49451
EBay Should Get Out of E-Commerce
all good sense.. we'll be looking at memcached soon, and our use of Propel should make it easy to integrate caching into the ORM like you suggest.

Question, though: I admit i've not given this much thought, but isn't the hard part working out when to clear the cache for individual rows/objects when the db is updated? I'd be grateful to hear any tips from how you've delt with this, considering it sounds like we'll be implementing something very similar..
Some Changes Needed in Research and Innovation
Interesting article on how conservative American Research is becoming and pointing out the ties to Business
The Machine That Changed the World: Inventing the Future
Web OS? Not an OS
It's not as good as something simpler from a pure SEO perspective, but one thing about Google in its current state is that traditional SEO like meta tags and easily indexable content matters less and less as Google gets better and better at indexing the web -- if you have the best content for certain keywords, even if it is tied up in flash, they will find it eventually.
.Mac and Me rumors: TUAW inside look at Apple's next-gen service
I'm not sure why you are defining a 'real social life' as socializing with non-hackers. A 'Real Social Life' consists of hanging out with like-minded individuals. I would imagine that most people spend their down time with co-workers and school friends - people they see on a regular basis and build relationships with. It would follow then that if you are a programmer, you will be spending most of your time with other programmers.

Unless you mean that you are only talking with them online and stuff. In that case, you should probably just go out and get drunk or something.
Disclaimer - I am not working on a startup right now, rather have a regular 9-5 job with a few projects on the side. I guess what I am saying is take my advice with a grain of salt.

Having said that, my simple advice is - Get Outside. Like a lot of other hackers, trivial social talk bored me. I love intellectual conversations and topics, and reached a point where the only "real" conversations I was having were online. But lately, I have been pushing myself to go outside and meet other people. Try taking a genuine interest in them, and you will come to realize that not only are other people as interesting and passionate as you, but a lot of them have the same issue as you, in that they find it hard to meet interesting people.

Over the weekends, I have been getting out of my apartment and heading out to a local coffee shop. Familiarity helps, going to the same place a few times means you get to know the staff and some of the regulars. Strike up a conversation with the person on the next table.

Throw a party. Ask your friends to call their friends. Its easier that way because you have overcome the first hurdle, and that is - "What do I say?". Start by asking someone knew how they came to know of the party!

Join some social groups - book clubs, dancing classes, martial arts or a meditation group (whatever interests you). These are great places to meet new people with similar interests (and no, the Local Linux groups does not count :D)

At the end of the day, social interactions take practice, much like anything else. You need to pursue it and treat it like any other task ("Deliberate Practice"). Over time you will come to realize that its not that hard. Yes, sometimes you fall flat on your face, but would you rather not try?

There are several books that can help, if you like to read up on it, two come to mind -

1. How to Win Friends & Influence People
2. Never Eat Alone
3. How to Talk to Anyone: 92 Little Tricks for Big Success in Relationships

Good luck!
It's a very good introductory text. If you get interested in going deeper then try: http://www-csli.stanford.edu/~hinrich/information-retrieval-...
I am running a marathon, not a sprint, so I frame my long working hours within a "healthy schedule". My typical day:

7:00 am - immediately start coding last night's plan, then email, headlines, HN
8 to 9 - exercise
9 - breakfast & internet
9:30 to 12 - code
12 - lunch & internet
12:30 to 6 - code
6 - dinner with family (home or restaurant)
7 to 9 - code
9 to 11 - computer off, pencil/paper, analysis, design, detail plan for tomorrow

Exceptions: one to two days per week consulting (still keep my night schedule), one day per week with family, occasional sports on TV. I could keep this up forever.
See http://news.ycombinator.com/item?id=206538

Also see http://news.ycombinator.com/item?id=116121
Ask YC: Beta Registration System?
Hey guys.

We were wondering whats the best way to handle limited beta codes and user registrations for our app?
We were just looking to see if there is something out there we could buy/use to handle an invite only beta registration database/system without writing our own...

Thanks
Um... Children AREN'T supposed to use computers. It's not just the Nigerian Minister of Education who believes this. Children should be using their hands, developing coordination, developing social skills, and solving problems through imagination. Ideally, they should be outside in the natural world. Unequivocally, they should be away from all screen media.
My non-web app startup: http://www.insidedesi.com/

:)
it feels to me that more assholery has resulted from "HN-vs-not HN" than what would have from a political discussion
It is a problem, that's for sure.

Let's take a simple example, the display of the list of available forums on every page.

In our class declarations diagrams, we have something like this:

+----------------------+
| Forum |
+----------------------+
| name: string(1,100) |
| topic: text |
| ... |
| CACHE(shared, 86400) |
+----------------------+

This creates a table for a forum definition, with its entries in the shared cache (Memcache) for a day - CACHE(local...) is in APC's user cache.

This means that when a developer writes ForumModule::getAll(), it builds the SQL query:

SELECT * FROM Forum;

and uses it as the key to query the cache. In case of a cache miss, the query is sent to MySQL and the data added to the cache with a lifetime of 86400 seconds.

When we add a new forum in the DB, it has no impact on the website, since the data is taken out of the cache 99.9%+ of the time. We then have to manually erase the cache using a maintenance script, running ForumModule::removeAllCache() that will generate the same key and remove the item.

Note that this is not always possible... For instance, if you're caching the list of forums but have dozens of SQL results stored in cache, such as:

SELECT topic FROM Forum WHERE id = 42;
SELECT id FROM Forum WHERE category = 1729;
...

Then these will be used as keys to get results from the cache, and we won't be able to remove them. Last I checked, you couldn't enumerate the keys from Memcache - you can with APC.

We usually solve this by setting low TTL values. A query per day or a query every hour is not going to make a difference, but guarantees that all your changes will be visible within a short time. In some rare cases (large upgrade of the code base), we have flushed the caches completely. If some people have to wait an hour to have the new forum displayed, it's not an issue. Certainly we tune these values depending on the user experience that will follow.

There is something that I didn't mention but is quite important: Do not always rely on the user to invalidate your caches. If the cached value is a list of forums, fine. If it is a large tag cloud with hundreds of queries, update it using a cron. Otherwise if 50 clients go to the tag cloud page just after the cache expired and all get a cache miss and run the rebuild process at the same time, you'll get a big hit on the DB.
On a related note, don't rely on Memcache, the data might not be there. Always have a fallback data source in case the cache is not giving you what you expected.
This can mean using other caches for datasets that are complex to compute - for a tag cloud, I would store the compiled data in MySQL, not in Memcache; I don't want the client to regenerate them and I don't want the client to see a message saying that the data is unavailable at the moment.
Learning Rails Free Online Course, Learn to Build Web Applications with Ruby on Rails
I'm working my full-time job and in my startup (along with a couple of personal projects) - and I was close to burn-out several times during the past couple of months.

I always have a flexible work schedule, but I usually start around 8-10 am and stop at around 8-10 pm :).

LOTS OF EXERCISE (jogging in the nearby woods)

And I'm definitely working on getting closer to 8-10 hrs per day ...
What's wrong with jottit? It's good for me. Are you kidding here?
Canonical Announces the Ubuntu Netbook Remix
I completely agree, I'm only really productive for 4-5hr during the day (this is when I come up with my best ideas, work or solutions) the rest of the day is spent doing general admin stuff. If you push yourself too hard you are likely just to burn yourself out. Working too long and hard is generally counter productive (at least thats how I feel)
I subject myself to constant distractions but I did get through the first couple chapters and enjoyed them. I forced myself to focus on the material by writing the examples in Ruby (http://romej.com/archives/590/programming-collective-intelli...). I've only skimmed the other chapters.
"this community is slowly being evaporated"

No it's not. Have faith in pg and the rest of us. This stuff happens all the time and we survive just fine. How could you know this after only 28 days.

"my Karma has dropped roughly 20 points within the past hour or so, I just really would love to know"

You never will, so don't waste any mental energy worrying about it. That's just the way it goes.

"confused as to what I said that has offended the community"

You haven't "offended" the community. Some people just downvote without commenting. Once it gets going, sometimes it snowballs. Forget about it and move on.

"All of my submissions (all ask YC/HN) have been to consult the community on decisions that affect my life to an unbelievable extent."

Great! Keep it up and do your part. Forget about the other crap.

I have noticed that discussions about what is "appropriate" get lots of upvotes ot lots of downvotes or both, without much logic. Hacker content threads, OTOH, are our bread and butter and are generally well received and appreciated. Stick with the latter.
Articles about net neutrality and other political issues of relevance to hacker entrepreneurs are certainly relevant to HN. General political stuff of the sort that's already saturating the media everywhere else is not.
This story reinforces what Tech Crunch is to me, as I commented a few days ago:

"Yes, TC is a click generator fueled by drama."

(http://news.ycombinator.com/item?id=205986)

... I'm even happier now to have unsubscribed from their useless, noisy little RSS feed.
Grouply Claims to Blow Past Ning; Now What?
Why is it that every time a political link is posted on here, it gets up-modded like crazy, while at the same time collecting dozens of comments full of meta-discussion about what types of links should and should not be on HN? Seems to me that if you don't like a link, you should just not up-mod it, and certainly not waste your own time by commenting.
I agree. I am also amazed that the word "assholery" made it throught the HN spell checker. Wonder what that means.
Breaking news in 2015: FreeBSD switches to Git!
I think that a 'Real Social Life' should consist of more than hanging out with like minded individuals.

It's important to get different perspectives on life non?
We had a repository that grew to about 1.5 gig on github recently (due to hundreds of copies of SWF binaries). Neither git nor github even hinted at having any issues with large repos.

I'm down-modding the above because it sounds like FUD. Git has no issues with large repos as far as i'm aware, and based on my own experience.
I'm just the opposite. I hate discussing politics. When I'm with family or friends and stuff like this comes up, I go in the other room.

Oddly, political discussions here don't bother me so much, just as long as they are "occasional", not regular. I like to hear what people here have to say.

And it's easier to click the back button than to go in the other room.
Did Price actually tell them to "f*ck themselves" or was that just made up for sensationalism?