skysight-inc/HackerNews huggingface dataset:
> Someone said to me, "I couldn't find coverage.py on Github." Right, because it's hosted on Bitbucket.

I was curious, so I went to bitbucket.org to try to find coverage.py. Experience was awkward to say the least:

1) Bitbucket homepage does not have a search bar, nor is there a link from the homepage to the search [1]

2) Guessing that the URL is /search puts you in the team profile for account `search`. finally you see what looks like a search bar at top [2]

3) Searching for coverage.py doesn't show `ned / coverage.py` in the first page! There are lesser-known forks of coverage.py that show up higher than the original project. [3]

I'm sure these will improve over time, but discoverability is still incredibly difficult on bitbucket.

[1] http://i.imgur.com/soRXB98.png
[2] http://i.imgur.com/z5YmcL2.png
[3] http://i.imgur.com/uhg8rbc.png
I'd like to point out that the Python standard library offers an abstraction over threads and processes that simplifies the kind of concurrent work described in the article: https://docs.python.org/dev/library/concurrent.futures.html

You can write the threaded example as:

import concurrent.futures
import itertools
import random

def generate_random(count):
return [random.random() for _ in range(count)]

if __name__ == "__main__":
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
executor.submit(generate_random, 10000000)
executor.submit(generate_random, 10000000)
# I guess we don't care about the results...

Changing this to use multiple processes instead of multiple threads is just a matter of s/ThreadPoolExecutor/ProcessPoolExecutor.

You can also write this more idiomatically (and collect the combined results) as:

if __name__ == "__main__":
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
out_list = list(
executor.map(lambda _: random.random(), range(20000000)))

In this example case, this will be quite a bit slower because the work item (in this case generating a single random number) is trivial compared to the overhead of maintaining a work queue of 200000000 items - but in a more typical case where the work takes more than a millisecond then it is better to let the executor manage the division of labour.
I think there is too much emphasis on the last mile speeds. I see complaints all the time that other countries have 100mbit links to the home whereas the USA is stuck with 5-10mbit. Personally, I have 25mbit down and Fios is always trying to upsell me to 50. However, I never get those speeds to real internet sites. Sure, my speedtest comes in right at 25mbit.
No it's a social problem. I do not search for code using function names or snippets of amusing for loops.

I want to find code based on "it's simple, you know my kind of simple, like made it into the BSD manual simple, but I also need it to be in python or bash because I am going to need to back on it but not spend so long on it that it becomes full time (leaves out C++ and haskell)

Ok now it must work with rabbit MQ and preferably install via pipy and run with decent test coverage using py.test and

you get the idea.

Anyway no search system does that - what does that is newsletters and emails and recommendation and ... social interactions
GitHub got content creators in the door by being easy to use like that. It became the de facto search engine for users looking to find projects because once you have a critical mass of the creators, that's what happens.

If whoever comes along with a competing product runs it in a centralized manner like GitHub, then they'll create the same issue the OP has with GitHub. If they offer a decentralized, easy to use solution, they need to also offer a way for the person looking for a project to search the decentralized net. Without a unified search, nobody is going to oust GitHub from their spot.
Where do you store your pictures and home movies? A movie of an event is near 1GB in HD.
To my great sadness, TTYtter is currently unmaintained (I sent in a bug report and that's what the dude told me). So I expect it to stop working at some point. Until then, best desktop Twitter client for me by a mile.

I wish Twitter, Inc weren't the absolutely stinking worst maker of Twitter clients. :(
Well here is one, but this is really for the device level: http://nepp.nasa.gov/workshops/etw2012/talks/Tuesday/T04_Hei...

If I find a good system-level paper, I'll post it.

Anyway, SSD drive wear tends to be rated like this: 5 years for x number of full drive writes per day. For good drives, x is 1. For really good SLC NAND based drives x could be as much as 10, but they are very expensive. For cheap consumer drives x could be 1/10. Based on vendor sales meetings, the aggressively large drives also have low write endurance (but then they say they are not optimized for heavy writes).

Better drives should have a built-in write counter so you can track the remaining life.

Another interesting parameter is their power-off "data retention" time. At end of life, this tends to be specified to be just 90 days.
Thank you, I used code W4JXTPWL4HTR.
Going for a week of traveling tomorrow, so will test it out with my Pocket account.
> I wonder if [Netflix is] space limited or iops limited.

I don't think you're asking the right question. The more iops, the less space and the less cost. I'd say Netflix is always optimizing for both.

The right question to ask is what is the break-even point between space and cost where these drives make sense, and that would depend mostly on how popular the most popular content is. If everyone were watching the same few things, but enough that it can't just all fit in RAM, it would make a lot of sense to have a lot of these drives. But if the watching is spread out across a lot of content, then not so much.
There are no enums in Erlang and Elixir. If you have a case expression and there is no match with the given value, you'll get a run time error. This is deliberate and is part of the general approach to error handling in those languages.

Statically typed languages like Haskell and Rust require the programmer to list all possible cases when matching on a value.
One of the interesting changes has been folks have been thinning out silicon wafers. Typically 700 microns there have been examples of working wafers as thin as 5 microns. If you stack 5 micron silicon you can put 100 chips in the same place you put 1 before, assuming you can figure out how to talk between them. Thru-silicon-vias (TSV) is the current poster child but there are better alternatives coming. Should be interesting when that comes to market.
But when they do, hundreds of thousands of developers sit there waiting for it to come up.
I actually much prefer Unfuddle to GitHub, but everyone is on GitHub, so I use GitHub.
Very cool. But this has me worried:

> through a weighted vote based on present and past consensus

How can Truthcoin protect against votestuffing?
It's so completely trivial to generate examples of things that Lisp macros can do that Haskell's laziness cannot achieve, it muddies the subject. Try implementing lambda-case or anything written in template-haskell in laziness: you cannot. That's where Lisp macros shine. :-)
The assumption that these things should be cheap is not new.

I bought my first washing machine - a high speed front load type - about 12 years ago. It was ~$1k, but for several reasons I thought it was a good buy. The cashier made a big deal about it when I went through the checkout line, laughing loudly at my stupidity for purchasing such an expensive model and actually exhorting the cashier in the next lane to do the same. Didn't I know that I could have got a washing machine for $300 or less? I was really embarrassed; I've never bought another appliance in-store.

I'm still using the washing machine. I have had no problems with it and it would not surprise me if it's the last washing machine I ever buy.
The entire point is that pretty much any other (read: non-Apple) laptop/desktop you buy nowadays is going to come with at least 500 GB, and generally 1 TB nowadays.

Is that the case? Most machines I've seen around the cost of the Macbook Air still come with hard disks or 128G SSDs - maybe a couple might push that to 256.

Indeed, I don't think I've even seen a laptop with 1TB of SSD, aside from the very-top-end-I-upgraded-it-especially Retina Macbook.
GitHub has already won due to networking effects. It's like trying to unseat Facebook as the dominant social network. It will require a sea change in technology used for version control or sharing code.
Realistically, I don't think that knowing Haskell really would benefit most programmers, especially those working on typical, everyday software systems.

Just curious. How much Haskell/FP experience do you have?
I also own my company, there is a limit to your salary in one direction, Zero.
I'm afraid I don't actually know any numbers on Linux users' willingness to buy software, but I am aware of the reputation that Linux users generally prefer their ecosystem of free apps.

However, I bet that many Linux users are also Windows or OSX users, and if a password manager is to be effective, it has to be omnipresent. That's why I have begrudgingly paid for LastPass, while my dual Windows / OSX license for 1Password 4 has sat unused. I tried for awhile, but I couldn't stand the gap in coverage.

So I narrowed it down to RoboForm and LastPass, and I chose the latter because it seemed more popular and I hoped that to predict company longevity. Because as nice as lifetime licenses are, once software is no longer supported, esp. for things like intelligent form-filling, chances are you won't want to use your software anymore.
This example is not too realistic and just narrows it down to the case where a job can be divided into isolated tasks with no shared data/state.

Often times threads need to update shared dict/list etc... With multiprocessing this cannot be done. You can use a Queue for this but it's horribly inefficient.

Generally speaking if you need performance and Python is not meeting the requirements then you are better off using another language.
Yeah, you are unusual in my view. :) My mom, who barely uses the computer - and has a laptop, was able to get up to 200 GB within 6 months. How? Simple -- videos and pictures she takes of the kids from her smartphone, that she insists be saved and archived. That's my mom.

I, on the other hand, am running out of free space on my 4TB capacity desktop, and will be purchasing larger storage devices soon.
Fedora here, Debian's not an option (which would work as it's got the libgmp every "modern" Linux distro has).

CentOS binary is the only choice and it won't work. FWIW CentOS 6 is equivalent to Fedora 12 or so (latest is Fedora 20).

Anyway, it's done, but a serious PITA, hopefully in future there will be a more sane way to get started exploring the latest & greatest.
That's John Wiegley. Worked with him at FP Complete for over a year. He loves Haskell more and more every day.
My definition of AI was based on human-like intelligence. I see your point that AI won't necessarily have human-like intelligence but could have something completely different, something that my puny human brain can't even comprehend.

Then again, if an AI system can't understand how itself works, is it really intelligent?

I don't know the answer to any of these questions. I'm just asking them in effort to help myself try to understand technology and AI.

Maybe simple curiosity and the ability to learn and retain data is the key to AI.

Will the first generation of true AI be like a simple human child or a god-like "being"?

Why even bother making pure AI systems when we are so close to BCIs? Why not just take recently deceased people, keep the brain alive, and attach a computer system to it?
You can use bitbucket just fine to host git projects.

I use a mixture of bitbucket and github for different projects, not wanting all my eggs in one basket, really. I find them pretty much equal in many respects - except being able to have private repos on bitbucket is very convenient, and github pages is also very useful. I wish bitbucket had something similar.

https://bitbucket.org/dfairhead/streetsign-server in case you're interested.
Wouldn't the OP get the same result by staying in C# and changing the outer if-else to a switch? I'm not super familiar with C#, but many C++ and Java compilers do that. (Pattern matching is still a cool feature, as some of his later examples show, I just don't think his first one motivates it.)
Ok, let's see... this is a draft and I don't have a compiler here, so I may write something wrong, but it would be something like this. Assuming that you don't want to use simply a translation table, and you really need different code for each element from the enum, you would first create an interface:

public interface ILanguage {

String convert(int num);

}

Then you would subclass for each language:

public class SpanishLanguage implements ILanguage {

public String convert(int num) {

String ret;

switch (num) {

case 0: ret="cero";break;

case 1: ret="uno";break;

default: ret="~~~";

}

return ret;

}

}

And the same for English, German or whatever you want. Yes, it gets too verbose, that's why I'd prefer just a translation table... in any case, you can abstract common code for languages that only implement translation for 1, 2 and others, and create more complex subclasses for different behaviours. But I digress...

Now that you have the interface, you would declare the enum as follows:

public enum Language {

SPANISH(new SpanishLanguage()), ENGLISH(new EnglishLanguage()), GERMAN(new GermanLanguage());

// This is a member instance.

private ILanguage translationImpl;

// And this is the one and only constructor.

public Language(ILanguage translationImpl) {

this.translationImpl=translationImpl;

}

public String convert(int number) {

return translationImpl.convert(number);

}

}

If you do this, you need to supply the implementation of the language to the constructor, otherwise your code fails to compile. So you can't add FRENCH or JAPANESE after GERMAN(new GermanLanguage()); you need FRENCH(new FrenchLanguage()), JAPANESE(new JapaneseLanguage()) and so on. Of course you can do JAPANESE(null), but it's immediately clear that you're doing it wrong.

I'm completely aware that there is a lot of Java-ness (in the bad sense) in this code, so maybe in C# you can do it more elegantly. I haven't touched C# for a good 10 years...

EDIT: formatting.
I don't see what you mean with d). For ECDHE_RSA you still need your high value certificate key online to sign your public Diffie-Hellman parameter. Or do want to presign a few thousand of them and use them one after another in normal handshakes?
Gitlab is a non-starter since it requires developers who can barely manage to use Git to also run their own servers. If that weren't the case, then we'd just have another GitHub-lookalike. I'm old enough to remember when self-hosted Subversion was a popular geek "badge of honor", and the frustration of trying to pull code from some guy's VPS that had been offline all weekend.

Something like Fossil's model seems better, where issue tracker and wiki are instead built into the repository. Sure, essentially you're still running a little mini web server on the local machine, but the presentation is completely different (a single binary rather than a huge Ruby webapp with many dependencies).

The problem isn't really that GitHub goes down, but that we can't mutate or make use of certain pieces of state while it is down (mostly issues/wiki), and nobody can see our changes until it comes back up.

Actually Git comes with all the tools necessary to send pure data in the form of changesets by mail, its just that its usability sucks unless you have a working sendmail/mutt config, which 99% of devs don't bother with any more.

That seems the more interesting thing to fix, rather than just having another self-hosted SourceForge clone or whatever
https://en.wikipedia.org/wiki/Assassination_market
SRS = serious (not reddit).
That a person is applying for a programming job is the first, and most important, positive sign of interest in programming. These days, in fact, thanks to the ever increasing emphasis on having a github profile as part of the recruitment process, I view having a github profile as being little different from plugging in the Language Alphabet in the right places on a resume. It's empty filler that may or may not have any use.
Wow - that is significantly more elegant than what I discussed in the article!

I wasn't aware of the concurrent.futures library, thanks for pointing it out.
The key point for me is that most of the recent criticisms I've read about TDD have almost always been about TDD as a testing tool. If you look at the reasons folk list on why they do TDD testing is rarely, if ever, mentioned.

For me TDD is a design/development practice, not a testing practice. That's where it delivers value for me.
Hi I'm the author. I'm sorry my article offended you. Can you please tell me why you are so angry?
Well, I'm no fan of Apple at all and haven't owned an Apple product since the II GS. And I'd argue against storing any significant data on a laptop or desktop drive, as this is one of the most likely components to fail. Data should be on a storage server with RAID. All you should have on your primary drive is your OS, applications, and active working set of data. Any archival data or larger files should go on the storage server.
Not just google, though. I routinely use the word 'bing' instead of 'google' when I tell people to search for stuff, mostly just to see the reactions. It's quite odd - some people laugh, some people actually get hostile. Someone asked me if Bing would work their Yahoo. I told them to just keep doing what they were doing and not worry about it.
Gillmor Gang: Pattern Baldness
What Killed Google+ And What Can Save It
I also own my company, zero is not a limit.
Unless Gitlab or a similar product starts offering a way that I can search all the Gitlab projects, it's not going to amount to much. Github has taken over the space not only because it's easy and powerful for creators: it's also a powerful search engine for users. As the article notes, the first thing many of us think when looking for code is "Lets search GitHub".

Google is not efficient at searching for code. If each creator spins up their own project sites using Gitlab or other tool, there needs to be some way of searching the network of Gitlab sites if we want users to actually switch.
I thought Logo was, too, and Logo is based on Forth, but I checked Wikipedia article and Forth is not mentioned. Also, don't know about the popularity of Logo back in the day, but I thought it enjoyed some.
I was considering adding this but I wasn't fully sure that it would be good content for a "first intro to parallel programming" article. Perhaps a good candidate for the next one?

Thanks for mentioning it though.
I totally agree with the author.

Worse is, when answering to the question what do you use for version control software, I hear quite often people saying "github" and not git, and not understanding the difference between the two.
Do you even understand git? Literally push the whole repo wherever you want. git remote add blah. It doesn't matter where it's hosted. Many popular projects push their code to multiple services -- it's not harder than pushing to one.

If github goes down and your devs are twiddling their thumbs because they can't push, you have larger problems.
As a side note, GitHub wikis are actually git repos that use Gollum:

https://github.com/gollum/gollum / git@github.com:gollum/gollum.wiki.git

As such, you can clone wikis, work on them locally, and push changes later just like any other git repo.
Nope.

Claim 1: white-ish lines on light grey paper.

Claim 2: As above, but more closely matching white and grey.

Claim 3: Either of the above but with a slightly colored paper instead of gray.
Gonna need a citation on that claim against Audi. The vast majority of Audi/VW cars are equipped with a 2L turbo that is probably the most bullet-proof motor made today. I personally put 120,000 very hard miles on a VW GTI equipped with the signature 2L turbo.

I've seen multiple folks on the MKVGTI forums approaching 200k miles with no issues as well.
It's important to note that most companies these days that have 'big data' typically follow the WORM (write once, read many) pattern. They utilize technologies that write their data to the disk once, but never delete.
I agree, the scope of the article is somewhat specific to the "toy" example presented.

Generally I would use C++ or (gasp!) Fortran with either MPI or CUDA for these sorts of tasks if performance was the most critical factor.

I'm excited by the Julia language though!
What’s the best way of harvesting the energy of the sun?
> But at some point you'll need to update issues, and publish your changes.

The second part is solved. You just need a second git server. Maybe Bitbucket, or maybe your own. We just never do it though, for reasons I don't know either. I don't do it, even if I know how. There are multiple tools to push to different servers automatically, but we still go the comfortable route. Maybe it's the psychological thing that "if Github goes down it won't be for long, is it really going to affect me?".

Because of that I don't agree the tools are the problem though. Or even that they are part of some irony. Git delivers on its promise. People just don't use it like that.

The issues part or the dependency with Github to test/deploy software is more of a vendor lock in situation. A lot of people depend on it being up to work, but that's the convenience charge since Github isn't open source. Not much different from people depending on Heroku to be up for their application to function.
It's not immediately apparent from the front accessible parts of Bitbucket's website, but if you refer users with a provided link you also get up to 3 extra free users.
Yep, but unless you train your team up on using Mou or just reading raw markdown, you can't actually browse or publish to the wiki ("use or mutate state"), and again the usability is horrible so nobody will do it.

Usability is the huge selling point of Github, so any replacement would need to be as slick
"Whatever you do, you have to find a balance between doing what you love and making enough money to live the life you want, or what your family needs. It's not always an easy choice."

If you love to program the choice is easy. It's not like programmer wages are so low you have to live under a bridge.

Personally I'm psyched that people will actually pay me to program, because if they wouldn't, I would still be programming.
What is the anticipated price/TB curve of SSD over the next five years? at what point would most consumer storage switch from HDD to SSD?

If we start with a HDD/SSD price ratio of 1:7 (based on a quick check with Amazon) and hope for SSDs to get cheaper/TB by 50% each coming 18 months (?) we'd have a strong incentive to switch by year 2019.
I was going to discuss Parallel Python (http://www.parallelpython.com/) in the next article - have you used that? How does it compare to joblib?
code not valid. :(
GNU C has the Case Range C extension (http://gcc.gnu.org/onlinedocs/gcc/Case-Ranges.html#Case-Rang...) which allows you to write "case KEY_KP0 ... KEY_KP10:".
"We are in no way affiliated to the discontinued Google Knol."

Google trademarked Knol back in the day (2007). A USPTO search shows it is still live:

http://tmsearch.uspto.gov/bin/showfield?f=doc&state=4809:9wz...

Electronic publishing services, namely, publication of text and graphic works of others in the field of general encyclopedic knowledge; providing education, entertainment and information services, namely, providing information via the Internet in the field of general encyclopedic knowledge.