Lily is a hybrid OO+function language I've been building for nearly five years - jesserayadkins2
http://github.com/jesserayadkins/lily
======
dozzie
...and which is better/different than OCaml, because...?

~~~
jesserayadkins2
Lily's made entirely in plain C, and thus it's easier to include as part of
your program. Maybe you'd like to run some script from your program and
provide some program variables/functions to Lily. You can do that. Now, Lily
isn't currently totally ready for embedding, but the important parts are
there: It doesn't read global variables, and it has a single entry/exit point
when you call it.

You can also use Lily as a templating language. The support for tagging is
built into the language, instead of being added to it.

One of Lily's strengths is that, unlike other scripting languages, it takes a
very, very low amount of memory to boot (10K). That makes it idea for,
eventually, truly embedding and being actually useful as a scripting language
for whatever you might be making.

I don't know too much about OCaml, so I can't give a good, complete
comparison. But Lily's syntax and feel is more inspired by OO languages, with
the functional part being more about collections and chainability.

~~~
dozzie
I'm not buying the part about Lily being functional only because it has map()
and filter() equivalents (which is what I suspect at this point Lily
presents), but the rest of the points sound solid.


How To Identify Talent - dorkitude
http://dorkitude.com/post/22729437287/how-to-identify-talent

======
cwollak
Awesome!!!

~~~
dorkitude
Thanks :)


Is Clojure the "New Lisp"? - raju
http://www.lunaticprogrammer.com/2009/02/is-clojure-lisp.html
======
jwilliams
Shrug - call me a critic, but this is another "Why I think Clojure will be
good" article, rather than one with some (new) substance.

I'd like to see some more articles on Clojure in actual practice.

(p.s. It's because it's built on the JVM).

~~~
maryrosecook
Clojure in practice: <http://news.ycombinator.com/item?id=494147>

------
krschultz
I sure hope so More acceptance of functional programming is always a good
thing. I love coding for in Lisp but I haven't really made the leap to
production code since I have to support 4 platforms with all code I write for
work (Windows, Mac OSX, Linux on x86, Linux on ARM), and deploying Lisp code
across all those platforms auto-magically reliably SUCKS. We write all our
stuff on the JVM so this would be easy to bolt on and expose to new people.


Responsive Web Design: A Free Email Course - hv23
http://trydesignlab.com/responsive-web-design/

======
mbrody
How long does each lesson take?

~~~
hv23
Hey! Each lesson is intended to be a bite-sized 10-15 minutes, with related
links and a hands-on exercise you can do after. The goal is to equip you with
the basics and give you a strong foundation that you can use to dive in to
more details later, if you'd like.


New cable cut compounds net woes - justinwhitefoot
http://news.bbc.co.uk/1/hi/technology/7222536.stm

======
sspencer
Three in three days?
[http://www.cnn.com/2008/WORLD/meast/02/01/internet.outage/in...](http://www.cnn.com/2008/WORLD/meast/02/01/internet.outage/index.html)

What are the odds on THAT? Looks less like an accident all the time.


Show HN: jhackers.net - Conversations with Japan's Hackers - hkmurakami
http://jhackers.net/

======
hkmurakami
OP here :)

Would really love to hear the kind of things HN'ers would like to hear in
these interviews, as I'll be reaching out to the hackers named as "next up" by
the first three guys I initially interviewed.

I never would have created this site without HN and my small contributions to
FOSS, so any feedback to make it better for you guys would be awesome!

~~~
Kenzo99
All three are described as "hackers" on the site, yet none of them seem to be
hackers. Engineers, developers, computer scientists - I could see those terms
being used to describe them, but not "hackers."

~~~
hkmurakami
Thanks for the feedback :)

Perhaps I didn't do them justice through my efforts, but all three of them are
open source hackers to the bone. Could you expand on what you see as "hackers"
and what sorts of people you'd be interested in reading about and expected to
find?


Our Approach to Privacy - fjk
https://www.apple.com/privacy/approach-to-privacy/
======
wyc
Apple sells hardware differentiated by integrated software for premium
pricing. They want to build better and more expensive products through
superior design and quality control. Collecting and analyzing personal data
isn't the most important thing for their business. They might see more benefit
by eschewing personal data collection and marketing a privacy-focused message,
which they seem to be doing.

In contrast, Google and Facebook are companies that sell advertising. The
value they can offer to publishers and advertisers completely relies on how
well they know their users. Their competitive moat involves collection of
proprietary data to continuously improve their products. These are clear
incentives to be sticky and greedy with your information, but also to keep it
private and proprietary for their own sake.

With these incentives in mind, I more readily trust Apple when it says that it
will not collect my data compared to data conglomerates, and Apple hasn't done
anything to aggressively betray that idea in its history (to my knowledge).
Combined with the technical security advantages of iOS, I'm inclined to
believe Apple products to be the least-bad option for the security-minded
today.

~~~
mirimir
I do like Apple's approach to privacy. A lot. And I love their phones. While
their PC hardware is still pretty, it's getting more-and-more outdated and
overpriced. But that's no longer their focus, so hey.

And indeed, Apple seems to handle privacy well, in practice. I've managed to
create pseudonymous accounts at Apple's online store, and fund them with
anonymized Bitcoin. And I've managed to make digital purchases.

However, I haven't tested buying physical devices, for pickup in meatspace. I
wonder if they'd require official ID, or just proof of purchase. Anyone know?

~~~
lxmcneill
The last time I went to pick up a product (AirPods) it required government
issued photo ID (Australian driver's license), even though I had the Apple
Store app which made the purchase installed on my phone.

~~~
movedx
That's an issue with Australian law, not Apple. You can also buy such devices
overseas.

~~~
mirimir
Ah, thanks. What sorts of purchases in Australia require ID?

~~~
adambrenecki
Anything over 10,000 AUD (about 8,000 USD) purchased with cash. AirPods are
expensive but not _that_ expensive.

My guess is that they paid online for in-store pickup, and Apple asked for ID
to confirm they're the one that made the purchase. In which case that's an
Apple thing, not a government thing.

~~~
movedx
Bought my AirPods from Apple in the UK: no ID required.

Bought my iPhone 6S from Apple in Ausralia: ID required under Australian law.

Very likely the person also bought a phone with a sim/service, hence being
asked for ID (under Australian law.)

------
greggman
I'm happy Apple is at least trying hard to deal with privacy but honestly I
don't think they are doing enough, at least for me.

For example, I don't really want to give most apps constant access to my
photos, my camera, or my mic but I really don't have a whole lot of choice if
I still want to use popular apps and services like Facebook, Instagram,
Messenger, Hangouts, Line, WhatsApp etc..

I really wish that every time they wanted a photo they had to get it through
some OS level UX and they only got to see the photos I selected. As it is they
get to see all my photos the moment I want to give them a single photo.
Similarly if I take a photo in one of them they require permission to read all
my photos when all I want them to be able to do is save the photo.

As for the Camera I don't give any of those apps access to the camera because
I don't trust them not to spy on me in some way (I assume camera = mic access
so they could be doing the subsonic listening for ads things etc...). Instead
I take the picture with the built in app then access that picture from the app
I want to use the picture in. Of course that leads to the problem above.

Mic access is more problematic. I don't want any of them to have mic access
when I'm not using the mic function directly but I can't talk to my friends
who use all those services to call me if I don't give the apps mic access.

I feel like if Apple was more serious about privacy they'd handle these issues
in some way. The photo one seems mostly straight forward for most use cases.
Don't let the app access them at all, only the OS. The camera one is less
straight forward. I get that there are innovative things apps can do with the
cameras by using them directly. On the other hand most of the current use
cases could be handled by letting the OS access the camera only, not the app,
and then just giving the result to the app.

~~~
guyfawkes303
I think you mis-understand how iOS privacy controls work. An app doesn't get
to 'see all your photos' just because you grant photo access, you still have
to select which photos to put in the app. Same goes for camera, that just
let's the app pull up the camera interface, not be able to access it 24/7 for
whatever purpose they want. Same with the mic.

~~~
LeoPanthera
I don't think this is correct. The Facebook app regularly shows me all the
photos I have taken today and asks if I want to post any of them.

~~~
odammit
I can't tell you how many times I've definitely had a photo show up in the "do
you want to post this" preview that I most certainly would never want to post.

I feel like I'm having a heart attack every time thinking I posted it already.
That feature sucks.

~~~
mixmastamyk
_uninstalls app and uses website_

------
jacksmith21006
Apple chose to let the China government into their China data center and
Google chose to leave the country instead. So not so simple on who you trust.
Personally I trust Google more as they are just a lot better at keeping things
secure, imo. Plus governments getting into data is a bigger deal to me than a
targeted ad. But it is a personal decision.

"A Local Chinese Government Will Oversee Apple’s New iCloud Data Center"

[http://fortune.com/2017/08/14/apple-china-icloud-data-
center...](http://fortune.com/2017/08/14/apple-china-icloud-data-center/)

~~~
et-al
Why was this downvoted? It's a relevant point to the discussion.

------
maxpert
At least one company is "trying" keep my photos private. The other day Google
Photo told my wife it had prepared an album for our trip to SFO. We were
surprised because she already disabled Geo-tagging but whataya know... Google
still figured it out!

~~~
petepete
If you have location enabled on your phone, Google appears to cross reference
your location with your photo timestamps to work out where you were. I know
this because when I import my DSLR images to Google Photos it estimates the
location, usually _very_ accurately.

~~~
ben1040
I've also seen it geotag things based upon the content of the image alone.

I went to the photo store not long ago and had dozens of rolls of negatives
scanned, from a vacation to Europe 20 years ago. I uploaded them into Google
Photos and it geotagged many of them automatically.

They have a public API that does the same, it detects landmarks in images and
can give you a lat/long position for it as well as a confidence score.

[https://cloud.google.com/vision/docs/detecting-
landmarks](https://cloud.google.com/vision/docs/detecting-landmarks)

~~~
mit65
apple cloud did release a lot of celeb pics over the internet...let's not
forget that

~~~
gbear605
That was a phishing attack, nothing to do with Apple other than that the
people attacked by it were iPhone users.

------
gallerdude
Privacy is one of those things I can't tangibly describe why I like it, but it
just feels good to know that nothing is being saved, even in contrast to just
targeting you for ads and nothing else.

~~~
JumpCrisscross
We have a track record of things going south, fast, for societies that
sacrifice privacy. It makes sense to tread warily with it.

~~~
zachlatta
Can you give a few examples?

~~~
JumpCrisscross
Pretty much every authoritarian overthrow that became genocidal relied on a
previous bureaucracy's meticulous record keeping.

~~~
zachlatta
Can you please give a few examples?

~~~
JumpCrisscross
The classic ones are Pol Pot and Hitler. With the latter, the Danish example
is useful.

~~~
zachlatta
In your original comment, you were implying that societies that lose privacy
go south very quickly. As in that loss of privacy is causal.

I can't speak for Pol Pot, since I'm not very familiar with his regime, but
with Hitler and the Danish, my understanding is that record keeping aided in
the German occupation. Not that the Denmark went downhill after it started
doing meticulous data gathering.

Can you give causal examples?

------
newscracker
A few things have struck me really strong on Apple's stand and implementation
on protecting users' privacy:

1\. Though it's understandable that Apple earns money primarily by selling
hardware, it's sort of amusing and alarming at the same time that a
proprietary almost-closed-source software company is focusing on protecting
and preserving privacy whereas partially open source platforms competing with
Apple seem to be nowhere close on this aspect. Do any of the Android forks try
to do as much as Apple does for privacy right out of the box (something a non-
technical lay person could get)?

2\. It's abundantly clear that a lot of thought has gone into the foundations
of a design focused on protecting privacy and in creating silos of information
in/with different SDKs and features.

3\. It's a bit unclear to me as to why Health data is stored encrypted in
iCloud whereas messages aren't. Is there a distinction here between iCloud
sync and iCloud backups? The documentation on messages suggests turning off
iCloud backup as a protective measure.

4\. To me, the weakest link in the ecosystem seems to be third party apps,
where Apple relies more on them adhering to the developer guidelines and on
publishing a privacy policy.

~~~
wyc
Regarding your first point, it's difficult to implement some security schemes
at the operating system level alone. With full vertical control of the
product, you can have nice things like secure enclaves and de-facto hardware
cryptography acceleration.

See here for details:
[https://www.apple.com/business/docs/iOS_Security_Guide.pdf](https://www.apple.com/business/docs/iOS_Security_Guide.pdf)

A lot of the features would be very difficult to implement in Android without
cooperating hardware, and hardware is notoriously expensive to get right and
scale up. Projects like neo900 and Purism regularly encounter delays,
unexpected costs, and pricing issues. It's really tough.

On a broader note, people are spending more and more time in data-hungry apps
anyway, which can send almost anything they want to the network. This is sure
to chip at any device-level security, pushing it towards irrelevance. I wish I
had a log entry every time an app used the location service on my phone along
with a database containing a history of Internet-transacted data.

~~~
newscracker
Thanks for the explanation on the Android side. It still seems weird that
nobody wants to take this up as a USP for their devices (referring to non-
Google entities).

> On a broader note, people are spending more and more time in data-hungry
> apps anyway, which can send almost anything they want to the network. This
> is sure to chip at any device-level security, pushing it towards
> irrelevance. I wish I had a log entry every time an app used the location
> service on my phone along with a database containing a history of Internet-
> transacted data.

I've long wished for network access permission on iOS, allowing the user to
decide which apps can never connect to any networks. To reduce the total
attack surface, I'd want to keep many apps (especially games) running only
within their sandboxes and having access to only the data they create/generate
on-device and no other external resource/server.

AFAIK, Android has had this even in the days of permission requests at app
install time. I don't know if granular control is available on this from
Android 6 onwards.

~~~
wyc
I know that Samsung is trying to push its Knox product to enterprises. I'm
unsure about their technical or financial success. I don't know of anyone
trying this in the consumer space either, but I'd definitely love to see more
activity in this long-neglected sector.

Knox: [https://www.samsungknox.com/en](https://www.samsungknox.com/en)

------
whathaschanged
Apple's approach to privacy also includes being a partner in PRISM, a fact
which they chose to vigorously deny as false allegations until it was proven
to be true.

Every story about Apple and privacy chooses to omit thus huge piece of info.

Why should anybody trust them now? What has changed to make anybody believe
they aren't still lying about privacy?

~~~
dclowd9901
If their technology is built such that even they themselves cannot peer into
the inner workings of your content, what good is their association with PRISM?

~~~
userbinator
How do you know Apple is telling the truth?

One thing we're certain of, however, is that Apple has the signing keys. They
also encrypt their firmware and even other apps to hide how they work.

~~~
dclowd9901
I believe if they weren't telling the truth, _somebody_ would've caught on by
now. We have lots of people in sec watching court cases involving police
trying to break into iPhones. If somehow a department was able to access a
phone without a user complying and no known backdoor was exposed, I'm sure we
would've heard of it by now.

I hope you're not expecting them to open source the secure enclave.

------
spsful
I wish the EFF still made those "Who has your back" infographics, they were
really helpful if you wanted to find out which companies respected your data
both online and in the courts. But IMO Apple is the only large multinational
corporation that is actually taking steps to protect my privacy so I'm way
more inclined to trust them with my personal information. Can't say the same
for Google or others.

------
titanomachy
How much of this can be independently verified? I suppose we just need to take
their word for it?

~~~
Shank
It all boils down to what you see in the public. The FBI was trying to get
Apple to create an iOS variant to extract data from a device, to the point
where Tim Cook wrote a letter refusing to do so and was willing to fight it in
court. In a similar fashion, Apple has given talks and white papers on iOS
security. In tandem with these well documented white papers and talks
discussing the internal functions of iOS, we haven't seen any well known or
trivial exploits appear to surface that demonstrate flaws in them.

In the case of the FBI fiasco, for example, we know that the FBI later paid
Cellebrite for an exploit that allowed them to unlock the device. We don't
know the details of how that happened, but the FBI had to reach out to an
independent company and exploit an older device to do it.

To be clear: if they were lying about the lengths they go to with encryption,
on device security, and user trust, this story would have been different. The
FBI would have already had a backdoor or been able to trivially break the
device, or they would have complied and created an iOS variant to break in.
All we know is what they stood their ground on, and the effort that was
required for the FBI to get in.

iOS 10 security white paper:
[https://www.apple.com/business/docs/iOS_Security_Guide.pdf](https://www.apple.com/business/docs/iOS_Security_Guide.pdf)

Talk about iOS security at Black Hat:
[https://www.youtube.com/watch?v=BLGFriOKz6U](https://www.youtube.com/watch?v=BLGFriOKz6U)

~~~
lern_too_spel
We know that they lied to customers claiming they couldn't help law
enforcement get data off customers' devices.

"Unlike our competitors, Apple cannot bypass your passcode and therefore
cannot access this data. So it's not technically feasible for us to respond to
government warrants for the extraction of this data from devices in their
possession running iOS 8."[1]

After it became clear that it _is_ technically feasible for Apple to assist
with those data requests, they quietly removed that claim from their
website.[2]

[1] [https://arstechnica.com/gadgets/2014/09/apple-expands-
data-e...](https://arstechnica.com/gadgets/2014/09/apple-expands-data-
encryption-under-ios-8-making-handover-to-cops-moot/)

[2] [https://www.apple.com/privacy/government-information-
request...](https://www.apple.com/privacy/government-information-requests/)

~~~
millstone
By "technically feasible" are you referring to Apple's ability to build a
software update that allows for quicker brute-forcing of the passcode?

~~~
lern_too_spel
Yes. Brute forcing pin codes can be done very quickly. Apple realized it was
technically feasible too, which is why they no longer make that claim. That
they didn't apologize to their users and offer refunds demonstrates the
disdain they have for their customers.

------
jvican
After reading this, I have two questions:

1\. Are there any statistics showing the ratio of security issues between
Android and iOS?

2\. What's the most common phone among security researchers?

To be honest, I'm wary of believing this, but I own a Nexus and if I had to
give the benefit of the doubt to either Google or Apple, it would most
certainly be Apple because they don't make business out of my data (or, not as
much as Google, at least).

~~~
pcurve
I'm an Android user, but I feel the same way.

I sometimes wonder what would happen if I were to create a new Google identity
and start fresh. Would they try to re-establish my old email's profile and
data to the new one without my consent?

~~~
arunmib
I tried this as an exercise when they were pushing Google+. Created a new
account, was very cognizant to read through all settings and ensured that
nothing from my old account can be traced back to this new one. I'm not joking
when I say nothing from my old account and tracking: 1\. I timed my move (to
different city inside bayarea) 2\. Changing ISP to wave from comcast 3\.
Bought new computer 4\. Closed my old phone account and only started using
work phone. Never used hotspot in my work phone 5\. Even changed my coffee
drinking habit + free wifi access locations from Starbucks to Peets. Only
thing I couldn't change was my name + SSN. Serious paranoid level of things
for my online life, but they still populated my digital life with same info
and recommendations. After about 10-15 days of doing all this, I enabled
Google+ (because at that time to use youtube account, I had to enable it). You
know who the first contact recommendation that was there for me under my
family - my brother who is living in a different country. I just gave up at
that point. Probably I did something wrong during the setup (or) slipped and
used the accounts for some period of time on same computer or browser (or) the
data companies which sell my offline data have really rich data about me.
Whatever the goof-up was, it was just disheartening to learn that after all
that work my online privacy just lasted at the max 15days. I really wish there
were better ways. Considering the number of security/data breaches (e.g.
recent equifax one) I feel like, I have little to no control on things.

~~~
pcurve
That is pretty frightening.... thanks for sharing your story.

------
mtgx
> While we do back up iMessage and SMS messages for your convenience using
> iCloud Backup, you can turn it off whenever you want.

Wouldn't they be able to hold that privacy promise much better if they
actually allowed people to keep iCloud backup on let's say for pictures, but
still be able to disable iMessage messages? I think many people would like to
use iCloud but without it backing up personal conversations, too.

Also, iMessage's end-to-end encryption was rather flawed last Matthew Green
checked, compared to other end-to-end messaging apps.

[https://blog.cryptographyengineering.com/2016/03/21/attack-o...](https://blog.cryptographyengineering.com/2016/03/21/attack-
of-week-apple-imessage/)

As for their use of differential privacy, when they introduced that it was
essentially a hidden way of gather _more_ of your data than before, not less,
but while still being able to say "hey, we may gather more data than ever on
you starting with the new iOS, but it's _pretty_ private, so it's cool, don't
worry about it".

All of that said, I know Apple is still miles ahead of Google on privacy. If
anything, over the last 1-2 years, Google has become increasingly bolder and
more shameless about tracking users without them realizing (except in the EU,
where they are _forced_ to make it a little easier for users to understand how
they are being tracked, and even that happened because of the anti-trust
lawsuit).

Here's just one example of Google's increasingly privacy-hostile behavior:

[https://www.extremetech.com/internet/238093-google-
quietly-c...](https://www.extremetech.com/internet/238093-google-quietly-
changed-its-privacy-policy-no-longer-promises-to-anonymize-your-personal-
information-when-selling-ads)

~~~
Accacin
I agree it's not perfect. I kind of get around it by disabling iCloud backup
(which includes iMessage, etc.). I just use iCloud to back up photos, and I
back up my phone locally.

------
imron
> Apple has no way to decrypt iMessage and FaceTime data _when it’s in transit
> between devices_

What about when it's _not_ in transit between devices?

~~~
alex_g
Well they decrypt it on your behalf when it's on your device. The point is
that couldn't do that without your device/account.

~~~
jw1224
They then go on to say:

> While we do back up iMessage and SMS messages for your convenience using
> iCloud Backup, you can turn it off whenever you want.

Sure, I can stop backing my data up - but presuming I don't (like the vast
majority of people), does that mean if they got a wiretap order they could
just read the data straight from their backup servers?

I'm not sure if this was just phrased poorly, or if backing up your iMessages
effectively undoes any protection your messages once had.

~~~
DavideNL
"The iCloud Loophole" (article from March-2016):
[https://www.macrumors.com/2016/03/02/icloud-backups-less-
sec...](https://www.macrumors.com/2016/03/02/icloud-backups-less-secure-for-
restoring-data/)

Although in a link in this HN article Apple says backups _are_ encrypted:
[https://support.apple.com/en-us/HT202303](https://support.apple.com/en-
us/HT202303)

So, a little confusing...

EDIT

bah, found this:

"Right now, although iCloud backups are encrypted, the keys for the encryption
are also stored with Apple. "

[https://9to5mac.com/2016/02/25/apple-working-on-stronger-
icl...](https://9to5mac.com/2016/02/25/apple-working-on-stronger-icloud-
backup-encryption-and-iphone-security-to-counter-fbi-unlock-requests/)

------
bgrohman
On the subject of privacy, I'm interested in migrating away from Google
services, but I have years' worth of data tied up there (email, photos, music,
movies, etc). I know Google offers ways to download your data, so it's
technically possible to migrate, but it would be tedious and time consuming,
to say the least.

Does anyone have suggestions for making that process easier?

------
quadrangle
Apple does so many things well, actually. I would return to them for some
things and even recommend them (versus exclusively GNU/Linux and LineageOS) if
they'd only change the stupid iOS terms that prohibit GPL software.

~~~
nnutter
Isn't the issue that the App Store doesn't have a mechanism to redistribute
the source in a way that is compliant with the GPL? Is it fair to say Apple
prohibits the GPL? Anyway that tried to sell GPL licensed software on the App
Store is "breaking the law" so Apple prohibits it? Or is there more to it?

~~~
sheetjs
[http://www.fsf.org/blogs/licensing/more-about-the-app-
store-...](http://www.fsf.org/blogs/licensing/more-about-the-app-store-gpl-
enforcement) goes into more detail, but the summary is:

> Apple's Terms of Service impose restrictive limits on use and distribution
> for any software distributed through the App Store, and the GPL doesn't
> allow that.

------
cyphunk
Wake me up when they let one sync contacts without giving icloud cleartext
access in the process

~~~
mercutio2
Can you say more about why this is important to you?

I mean, more stuff encrypted without provider escrow keys is a generally good
thing, but are your contacts specifically more important than, say, you
calendar events?

What threat model are you worried about with your personal, curated contacts?

~~~
cyphunk
all data is important, indeed. contacts are just an easy to access example.
I'm worried about countries creating laws, secret and public, that force Apple
to hand over my data. Specifically those governments that I spend time in and
that I also spend time criticising or protesting within as an activist. I'm
worried about regime changes. I'm worried about 3rd party data breaches. I'm
not worried about how my data exposure would effect me now but how it would
effect me in the future when mining and correlating data to use against
activists will be much easier.

------
aub3bhat
On Android you can Sideload VPN apps, iOS on the otherhand banned them in
China. Apple mounted the most successful attack on General Computing with it's
walled garden. The whole Apple is good for privacy is marketing.

~~~
disconnected
Android and its community are sorta schizophrenic towards security.

On one hand, users are told that they should never ever, ever install
applications from untrusted sources. They should always use the play store
because applications are scanned for vulnerabilities and whatnot.

On the other hand, we have people telling us that one of the great advantages
of Android is that you can sideload apps - bypassing the store security model
completely.

On one hand, a VPN needs root access for transparent proxying (or at least TOR
does). Changing the hosts file needs root access. Changing gps.conf requires
root. Lots of useful operations need root access, so if you are concerned
about privacy and security, or just want more control, you probably want root.

On the other hand, root is stupidly hard to obtain, and users are strongly
discouraged from doing it anyway because it opens all sorts of attack vectors.
Unless the manufacturer provides a legitimate method to do it, the operation
of obtaining root itself, like on iOS, often relies on an unpatched local
privilege escalation vulnerability. Note that any application can exploit
this, not just the rooting app.

I don't support Apple's walled garden approach, but we can't argue that they
have a much clearer picture with regards to security, and the results speak
for themselves. Malware in the Apple devices is rare, whereas in Android it is
rapidly becoming routine. Unfortunately, you sacrifice flexibility for
enhanced security.

~~~
nnutter
My perception is also that malware ends up in the Google Play Store with much
higher frequency than the App Store. Just do a news search for "Google Play
Store malware" and "App Store malware" and compare.

Also, one can sideload apps, if you have a Mac, onto iOS. Obviously, that's
not anywhere as integrated but maybe that's a good thing. Heck, maybe Apple
even added that so people in China could sideload VPNs. Maybe iOS VPNs are
good enough (no root, no TOR?)?

~~~
lern_too_spel
Add up all the malicious app installs on Google Play Store, and it doesn't
even come close to the 500 million[1] (conservative estimate) users affected
by XCodeGhost. It looks worse when you consider that the 500 million is on an
order of magnitude smaller total iOS userbase vs. Play Store userbase and when
you consider that Google allows third party security researchers to
investigate and publish research on the Play Store while Apple does not, so
XCodeGhost is likely to be the tip of the iceberg.[2]

[1]
[https://www.google.com/amp/s/www.macrumors.com/2015/09/20/xc...](https://www.google.com/amp/s/www.macrumors.com/2015/09/20/xcodeghost-
chinese-malware-faq/amp/)

[2]
[https://www.google.com/amp/s/www.cultofmac.com/128577/apple-...](https://www.google.com/amp/s/www.cultofmac.com/128577/apple-
kicks-security-researcher-out-of-app-store-and-developer-program-after-ios-
vulnerability-demonstration/amp/)

~~~
evgen
LOL, let's start with StageFright (1 billion+ pwned with just a text message),
move on to StageFright2 (because patching is hard...), and then just keep
running down the list of malware in the Play store that is still there months
after being discovered. XCodeGhost OTOH, seemed to have hit around 40 apps so
that would probably not even get it into the top-100 list of Google Play
malware families. Malware families. The Play store is such a shitshow that you
can actually have different strains of malware running around in fake apps,
like some sort of digital syphillus spreading through the brothel that Google
forces everyone to visit if they want the shiny apps...

~~~
lern_too_spel
The number of people actually affected by StageFright malware from Google Play
Store (which is what we were discussing) is very likely to be zero. It simply
blocks the payload. We know for a fact that 500 million+ were actually
affected by XCodeGhost, an order of magnitude more than the sum total from all
malware seen in the Play Store.

Don't conflate unpatched Linux systems with the Play Store. Anybody who uses
Android and cares about the security of their device (like anybody who uses a
Linux-based router and cares about the security of their network) uses vendors
who deploy timely security updates.

------
icomefromreddit
I don't trust Apple. It's a matter of trust, so I have nothing else to say,
just I can't trust Apple.

------
shmerl
I wouldn't trust on issues of privacy to the likes of Apple. Who can audit
their software? It's not FOSS.

~~~
achamayou
You can still contemplate their incentives, their business model, and their
reputation/track record.

It's not as good as auditing the code (and making sure the code you see is
what actually runs), but it's a lot less effort, and depending on the extent
to which you are ready to dedicate resources to improving your privacy, may be
an acceptable tradeoff.

------
phoe-krk
So a cop can put your phone in front of your face while you're tied up and
unable to do a thing. Boom, phone unlocked.

Marvelous.

~~~
R4nger
AFAIK if you keep your eyes closed, it doesn't unlock.

Also, its still an improvement from when cops borrow your thumb. At least this
way, you have less chance of getting hurt.

~~~
megous
I'd rather lose my finger than my head, thank you.

------
williamle8300
All the anti-constitutionalist people at the NSA are getting heartburn as we
speak.


Android as we know it will die in the next two years and what it means for you - venturefizz
http://venturefizz.com/blog/android-we-know-it-will-die-next-two-years-and-what-it-means-you

======
sidcool
Doesn't make much sense. Another doomsday monger.


Be Careful with Python's New-Style String Format - ingve
http://lucumr.pocoo.org/2016/12/29/careful-with-str-format/
======
raymondh
For user defined templates, using string.Template() is a good alternative
because it does not do multi-level attribute lookups. The single mapping
lookup is safer. Also, the syntax is simpler, making it more suitable for
exposure to end users and making it less hackable.


No Smoking Outside Starbucks Shops Starting Saturday - codegeek
http://www.npr.org/blogs/thetwo-way/2013/05/31/187532646/no-smoking-outside-starbucks-shops-starting-saturday

======
DanBlake
Might be the case for locations in more suburban areas, but at least in SF
starbucks does not own/control anything outside its doors.

Its usually just the sidewalk out front, with some chairs/tables. I still
think the city 'owns' the sidewalk though, even though the tables are there.

------
codegeek
Will be interesting to know what percentage of smokers love starbucks ?


TiSpark sits Spark SQL on top of a storage engine to answer complex OLAP queries - jinqueeny
https://github.com/pingcap/tispark
======
elvinyung
Yes! I've been thinking of something like this for a while.

For the sake of simple data integration, I think this sort of architecture is
optimal. As it stands, Spark is basically already a distributed database
without its own storage engine; tighter integration with a transactional
storage engine means that you could get the full power of OLTP and OLAP (HTAP)
under the same interface.

Imagine that you could process transactions in Spark (pushing them down to the
distributed storage engine), and then Spark could automatically use the
changes to update a materialized view, and you could serve the updated
materialized view directly from Spark for real-time decision support, using
SQL plus richer analytics like machine learning, graph processing, etc. It's
not _quite_ a one-size-fits-all [1] database, but it's close.

Put a PostgreSQL or MySQL wire protocol server in front of it, and application
developers won't even have to know that they're using Spark.

(I'm glossing over the fact that Spark currently isn't very good at
transaction processing in the sense that it literally doesn't have much of a
write API right now -- i.e. support for equivalents of SQL `begin`, `commit`,
and rollback`, and updates/upserts in general -- but I think that's reasonably
easy to add by pushing down this functionality to capable storage engines.)

[1]
[https://cs.brown.edu/~ugur/fits_all.pdf](https://cs.brown.edu/~ugur/fits_all.pdf)

~~~
ilovesoup
I'm the main dev of TiSpark. I totally agree. For now we allow trx only in
TiDB. I believe one day those big-data stuff will be unified onto one
platform. With a full-featured distributed db storage layer underneath, there
might be tons of tricks to play comparing to data on hdfs. Ultimately, we plan
to put a mysql layer on top of Spark SQL (maybe or something else as mpp
engine), as you said, to make user not aware of existence of Spark SQL
underneath.

~~~
elvinyung
Just curious -- what other MPP engines are you currently considering, and what
are the criteria?


Online Courses Aren’t Actually Democratizing Education - footpath
http://blogs.smithsonianmag.com/smartnews/2013/11/online-courses-arent-actually-democratizing-education

======
otoburb
>>[...] 80 percent of MOOC students come from the wealthiest and most well
educated 6 percent of the population,” the authors of the paper write.

What about the 20% that didn't have undergraduate degrees? No doubt there's a
lot of skew here, but the focus should be on this 20% that (now) have the
opportunity to take a variety of free online courses. Perhaps this 20%
wouldn't have had a chance otherwise.

Otherwise, the article is right that MOOCs could expand their reach further,
but I think this is a great start and optimistic news. If people thought that
early-adopter MOOC audiences would be primarily comprised of non-degree
holders then now would be a great time to re-examine assumptions.

~~~
ctdonath
Quite. Big-ticket schools have a lot of incentive & budget & momentum to
promote big-ticket degrees. MOOCs just don't have the visibility, so those who
tend to participate are already plugged in enough for awareness of such
obscure options. 20% being "new students" is pretty good.

MOOCs also don't yet provide confidence for all-in degree-scale participation,
so they appeal more to those looking to fill educational gaps instead of
building their academic career thereon. High-school graduates just aren't
ready to pursue MOOC degrees en masse; they will as momentum & reputation
builds, but not yet.


Tim Gowers replies to Elsevier's open letter - ColinWright
http://gowers.wordpress.com/2012/02/26/elseviers-open-letter-point-by-point-and-some-further-arguments
======
Vivtek
I like the way the open letter says that Elsevier supports open access to
public research when Elsevier is actually lobbying against open access to
public research. (e.g. <http://www.michaeleisen.org/blog/?p=807>)

------
revelation
All that needs to happen is that public research grants have to require you to
make your paper available free of charge through a suitable system. Such a
system is obviously trivial to provide on the internet.

Then we can save the time having to deal with retarded "open letters" or
creating petitions like servants.

~~~
tylerneylon
There's a bill that could do this - FRPAA = Federal Research Public Access
Act:

<http://www.taxpayeraccess.org/issues/frpaa/index.shtml>

------
ChuckMcM
The solution to this is sadly to revolt, en masse. Which is to say the
journals cannot exist without papers, and so institutions which are providing
free access to their academics papers are effectively revolting. Elsevier will
not survive the transition but I do not believe it will be missed either.

At this moment in time, this instant, there is an opportunity to create an
alternative to Elsevier and its journals. New journals with a reasonable
policy and non-extortionate pricing. If the idea of an organizing identity
makes sense then its an opportunity that won't come around again for a while.

~~~
pmb
The senior faculty need to revolt first. The junior faculty need to jump
through hoops and be judged by that cohort in order to keep their jobs (i.e.
get tenure). If the senior faculty really want a change to occur, all they
have to do is change how they evaluate which publications "count".

------
jmmcd
_I haven’t tried recently, but maybe if I tried to open an Elsevier article
from the computer in my departmental office I would have no trouble. But all
that would say is that Elsevier didn’t have a completely stupid system for
opening articles. [Edit: Since writing that paragraph I have discovered that I
can open Elsevier papers with no trouble at all from my office, and with only
a small amount of effort from home. But others may not be so fortunate.]_

This leader of the boycott seems to know less about the details than I would
expect. Maybe it doesn't change the principle of the thing.


I Hate Pull Requests - henrik_w
https://medium.com/@pia.fak.sunnanbo/i-hate-pull-requests-17836dd3cc38
======
jordhy
We all do.

~~~
yjftsjthsd-h
I like PRs; they let me easily solicit feedback without risking impacting
master (or other "shared" branches). I could tay, "how do you feel about this
change?", but it's so much easier to say, "I propose this exact diff, what do
you think?". Now, caveat: I'm in a distributed team that's already async and
operates nearly like a FOSS project, so even OP seems to accept that PRs might
suit us.


BSD Unix: Power to the people, from the code (2000) - vezzy-fnord
http://www.salon.com/2000/05/16/chapter_2_part_one/
======
nickpsecurity
Great read. I think the biggest takeaway is how they organize and run their
operation. There's a tiny amount of high talent and dedication people that
really control the process. They're open for contributions by whoever wants to
try. They internally filter the wheat from the chaff to find those who
contributions are truly useful. They give them extra power in both what ends
up in the distro and helping the filtering/discovery process. The rest are
still allowed to attempt whatever contributions they can. The model seems to
hit an optimal point to bring in more good than bad.

Might be worth copying in other projects. Supporting this is the other,
highly-successful organizations that use a similar model. Far as Linux,
there's always exceptions to the rule and lets not forget it depends on GNU
work done differently. That hybrid, immensely popular development doesn't
compare apples to apples with about anything. I treat it like its own
category.

------
justincormack
Kirk McKusick is still an important person in BSD.

This story omits the role of NetBSD in opening up development in the BSDs
[https://en.wikipedia.org/wiki/NetBSD#History](https://en.wikipedia.org/wiki/NetBSD#History)

~~~
brynet
In that sense, it also omits the part where OpenBSD truly opened development
by enabling read access to the CVS repo (log/diff/annotate) to everyone.

[http://www.openbsd.org/papers/anoncvs-
paper.pdf](http://www.openbsd.org/papers/anoncvs-paper.pdf)

[http://www.openbsd.org/papers/anoncvs-
slides.pdf](http://www.openbsd.org/papers/anoncvs-slides.pdf)

Before that, they could arbitrary cut off your access:
[http://www.theos.com/deraadt/coremail.html](http://www.theos.com/deraadt/coremail.html)

[https://en.wikipedia.org/w/index.php?title=NetBSD&type=revis...](https://en.wikipedia.org/w/index.php?title=NetBSD&type=revision&diff=665541892&oldid=656673576&diffonly=1)

------
TimSchumann
Interesting read, thanks for posting. Brings a bit of relevant historical
perspective given that I recently switched to OpenBSD for a project I'm
working on.

------
yuhong
It is funny how USL vs BSDi hurt BSD at first, but then SCO targeted Linux not
BSD for a reason. Terry Lambert has a lot of posts on the backstory behind the
settlement BTW (clue: you can thank Ray Noorda).

~~~
cbd1984
In case you like obscure references, USL vs BSDi was tragedy, SCO vs Linux and
Rationality was farce.

It really is interesting how little the SCO cases actually amounted to.

~~~
stox
If it wasn't for USL vs BSDi, we probably wouldn't have Linux today, and BSD
would be ruling the world.

~~~
pjmlp
Who knows.

Maybe BSD would just be yet another UNIX, and everyone would still be
deploying Aix, HP-UX, Solaris, Tru64, DG/UX, among many other UNIX variants.

~~~
bahamat
We know. The culture in Unix was always sharing source and user collaboration.
When AT&T finally tried to exert control there was a large movement within the
community to go free of AT&T code. GNU and BSD were both a reaction to that.

The rise of x86 and the availability of free Unix (in the form of BSD or GNU)
was destined to destroy the proprietary Unix market. In 1991 neither BSD nor
GNU had kernels booting on x86, leading Linus to eventually release Linux in
August.

Linux had the luxury of being the only readily available Unix-like kernel on
x86 for several years.

The USL v BSDi lawsuit slowed BSD efforts for nearly two years while Linux
gained mindshare and features (specifically x86 features & drivers). Once the
suit was settled out of court development took a long time to regain momentum.

Net/2 was released in June of 1991 (before Linux!), so if the USL lawsuit
hadn't happened, we might all be running that instead.

~~~
pjmlp
That is just playing futurology. We will never know.

Just like some of us would have liked that Apple would be just as successful
with BeOS instead of NeXT, but we will also never know.

In 1993, I learned UNIX on Xenix system. Coherent being the other option.

Linux only became production quality when the likes of Intel, IBM and others
started supporting its development. As far as I am aware they never supported
BSD like that.

------
sudioStudio64
I'm a huge fan of BSD. This was a good read.

------
PhantomGremlin
Very perceptive observation by Bill Joy:

       The really nasty bugs are found by a couple
       of really smart people who just kill themselves.
       Most people looking at the code won’t see anything

How many people looked at heartbleed and didn't see anything? How many people
looked at goto fail; goto fail; and didn't see anything? And those were
relatively simple. I wonder how many "really nasty" security bugs are still
out there?

~~~
sebcat
> Most people looking at the code won’t see anything

Sadly, most people don't look at the code at all. They buy products marketed
as making them safer, or trust central authority figures.

------
swills
Great story, thanks for posting it. Really well written. The section on Gage
at the end reminds me of All Watched Over By Machines of Loving Grace:

[https://vimeo.com/groups/96331/videos/80799353](https://vimeo.com/groups/96331/videos/80799353)


Luis von Ahn: Outsourcing My Research Group - Rod
http://vonahn.blogspot.com/2010/06/outsourcing-my-research-group.html

======
thisisnotmyname
There probably ought to be fewer phd students around, and more research tech
jobs. But, I was under the impression that the current glut of grad students
was because they were so much cheaper than techs?

~~~
Rod
It depends on the schools. A graduate student at a UC school is way cheaper
than a graduate student at Stanford or Caltech. It also depends on the PhD
program, or more to the point, on the course requirements. If a PhD student
already has a MSc when he enters the program, he can start doing research
right away, and his advisor get a return on his grant money way faster.

------
pvdm
I suppose I am too naive to think Ph.D advisors would not treat advisees like
a piece of meat.

~~~
Rod
There's a reason I used to love reading _PhD Comics_... until I started my own
PhD. It used to be fiction, now it's non-fiction (sigh)

------
kunjaan
The next advisor meeting with him is going to be pretty awkward.


Web developers, what SaaS tools would you like to see created? - krogers

There are a lot of SaaS tools popping up nowadays. In the midst of all of these tools. What are some that don't yet exist that you would like to see created to make your life easier?
======
MrDHat
There's one tool I've always wanted - An analysis tool for my database
queries. Something which directly injects into my framework (Django/RoR etc)
and gives me detailed analysis of the time and memory used by the queries.

AFAIK, New Relic provides one such tool but I think there's still a long way
to go.

------
OceanPowers
serious, modern, WYSIWYG HTML / CSS / JS editor.

~~~
mc_hammer
macaw looks good

------
Concours
Baremetrics for PayPal would be great.

~~~
iqonik
[https://news.ycombinator.com/item?id=8783103](https://news.ycombinator.com/item?id=8783103)

Any good?

~~~
Concours
Yep, looks interesting. Will give it a try (added myself for the beta test),
are you affiliate with them ? Thanks for the head up.

~~~
iqonik
Not affiliated, just remembered the "Shown HN" post when I saw your comment :)

------
mc_hammer
draw with my voice - to make vector graphics.


A study on human behavior has identified four basic personality types - T-A
http://www.uc3m.es/ss/Satellite/UC3MInstitucional/en/Detalle/Comunicacion_C/1371223155576/1371215537949/A_study_on_human_behavior_has_identified_four_basic_personality_types
======
brhsiao
A friend of mine who's quite into philosophy once told me that a good number
of philosophers did little but try to categorize things. Rather than draw
meaningful insights, say, or squeeze every drop out of some experience, they'd
simply say, here's nature, here's a grid I've drawn, let's see what falls into
what square. Ok, good job; but what have you actually accomplished? (So my
friend said.)

I don't know if this is true of the history of philosophy (or if it's a
meaningful indictment of these philosophers), but I can't help but be reminded
of it when I see strenuous efforts being made to bucket various phenomena, and
human ones at that. What are we going to do with these categories? Not to put
too fine a point on it, these things are complex. Any differentiating scheme
simple enough will be insufficient to be the basis of any important decision,
and any sophisticated enough won't be neat enough for us to be talking about
it like this.

I say this as someone who spent more years than I'd care to admit obsessed
with MBTI. Wow, look, there are all these different people with their
strengths and weaknesses. I even get a cool framework—a _stack_ —to aid me in
psychoanalysis. And then of course you have Socionics, which shamelessly
tries, using just sixteen categories, to produce a catalog of human
interaction. I am not saying it isn't _fun._

Actually, MBTI did help me in one way: by showing me how diverse humans are.
In my most advanced, intelligent state, I'm still not going to get along with
or be compelling to all of them. That was an important idea for me growing up,
and still is.

But beyond that, and running the risk of sounding unsophisticated: who cares?

~~~
Naomarik
Eleven years ago Kaiser Permanente hired me to help roll out new software to
all their hospitals in Northern California. We had like 3 months of training
before going in and one of the things they had us do was some personality type
thing with some psychologist.

It felt like this lady was trying to shoe horn me into some category, telling
me I am some 4 letter acronym and here are my strengths and weaknesses. I just
kept arguing with her pointing out events and situations in my life that
refuted everything she was trying to say I was.

It seems like an overly complicated horoscope thing where people love to read
descriptions of themselves and others to make sense of the world, even though
by looking at someone's face for a few seconds you can make way more
judgements about the character of someone.

~~~
9erdelta
The "horoscopieness" of almost anything related to personality labeling causes
me to feel deeply rebellious. You can't pigeon hole me! Irrational or not, I
reject that my personality is fixed.

~~~
MrFantastic
Sounds like you fit into the "Rebellious" personality type.

------
jordanlev
I look at these things more like "check out our classification scheme which is
broken down into 4 categories", as opposed to "there are 4 different kinds of
people" (which the headline kinda sorta implies).

~~~
saulrh
Depends on what your methods are for finding your classifications. For
example, if all you're doing is recording a bunch of data over some feature
set and then throwing it at an unsupervised clustering algorithm like DBSCAN
or single-linkage clustering, It's not unlikely that you can say that your
data can be partitioned into N distinct clusters.

Granted, that depends heavily on your features and your data set. It's just as
easy to end up with one big undifferentiated blob, even after running PCA.

~~~
StillHocus
I mean, that's still "who cares"? It just sounds fancier.

"It's not unlikely that you can say that your data can be partitioned into N
distinct clusters."

Of course not...that's tautological. You can say it because that's what it
means. DBSCAN isn't some wizardry that peers into the nature of the
universe...its just a clump finder. Whether those clumps have any connection
to anything interesting is a coincidence. You could roll a six sided die six
times and run dbscan on the results and be like "holy shit! I've discovered
how to partition the natural numbers 1 2 3 4 5 and 6!!"

~~~
nerdponx
This is an unfair and anti-scientific characterization of descriptive
research.

Correlation does not imply causation, but you (generally) can't have causation
without correlation. Finding natural patterns of associations in data is our
only reasonable starting point for finding patterns of causation.

Also, developing reliable and well-researched ontologies can help other
researchers when building models, making sense of other data, etc.

~~~
StillHocus
It's not anti-scientific at all. Looking for correlations to uncover
causations in data is important, and it can certainly be the beginning in a
scientific effort to build a model of the world (especially, I would argue but
I guess this gets rather philosophical, when it's motivated by a belief in an
underlying mechanic that drives the correlated observations, perhaps developed
so fully as to be considered a 'hypothesis'). Finding natural patterns of
associations in data is a totally reasonable starting point for finding a
causation _when a causation exists_.

Feeding arbitrary sequences of samples into dbscan and deciding that because
dbscan produces output, there _is_ causation (or, that there is an underlying
phenomena that can be captured in some type of model), is ridiculous. And
there are tons and tons and tons of natural phenomena that will be happy to
produce clumpable inputs all the time, with no underlying behavior (including
noise).

I'm sure there are also tons of interesting models of human behavior that you
can make out of some set of observations of human behavior via clustering. But
just because you feed some data to an unsupervised learning algorithm and it
discovers features doesn't imply that those features have any useful
descriptive power to help us make sense of the natural world. THAT's anti-
scientific thinking.

~~~
Singletoned
> Finding natural patterns of associations in data is a totally reasonable
> starting point for finding a causation _when a causation exists_.

So if a causation turns out not to exist then finding natural patterns was an
unreasonable starting point? What would have been a reasonable starting point
in that circumstance?

------
watermoose
As a correction to the title, they said four, but then said, "There is a
fifth, undefined group, representing 10%, which the algorithm is unable to
classify in relation to a clear type of behavior. The researchers argue that
this allows them to infer the existence of a wide range of subgroups made up
of individuals who do not respond in a determined way to any of the outlined
models."

So based on the study:
[http://advances.sciencemag.org/content/2/8/e1600451.full](http://advances.sciencemag.org/content/2/8/e1600451.full)

and looking at the results in:
[http://d3a5ak6v9sb99l.cloudfront.net/content/advances/2/8/e1...](http://d3a5ak6v9sb99l.cloudfront.net/content/advances/2/8/e1600451/F2.large.jpg)

the summarization of "Undefined" as "Decides randomly" seems a little wrong-
there's an obvious tendency in PD (Prisoner's Dilemma) for some results:
[http://d3a5ak6v9sb99l.cloudfront.net/content/advances/2/8/e1...](http://d3a5ak6v9sb99l.cloudfront.net/content/advances/2/8/e1600451/F3.large.jpg)

Also, I can't help but wonder whether there is bias effect, as humans seem to
tend to use a small number of groups or factors for personalities. For
example, in literature, J.K. Rowling's sorting hat chose between Gryffindor,
Hufflepuff, Ravenclaw, and Slytherin, and Veronica Roth had citizens of
Chicago choose between Abnegation, Amity, Candor, Dauntless, and Erudite. Even
though there is a mix between races in humans, we tend to categorize- like in
the U.S., typically you must choose from Caucausian, Asian-american, African-
american, Pacific-islander, or Hispanic, even though color and genetic makeup
vary. Then MBTI has four factors (E/I, S/N, T/F, J/P). Friedman and Rosenman
came up with A and B types then later Denollet added the D type. We tend to
categorize like this.

Also, they chose just four games for the study. That could have affected the
outcome.

~~~
gohrt
> Abnegation, Amity, Candor, Dauntless, and Erudite.

She tried so hard for ABCDE, but couldn't find the word "Benevolence" ?

------
dash2
The article's description of the research is unhelpful, and almost certainly
wrong. This doesn't sound like an attempt to identify four basic personality
types. It's an attempt to categorise behaviour in a single, very precisely
defined economic experiment. We already know the five basic dimensions of
personality: openness, conscientiousness, extroversion, agreeableness and
neuroticism (OCEAN).

It's sad that there is so much good social/psychological science out there,
but people know so little about it and fall for this kind of ignorant
reporting.

~~~
fsloth
" We already know the five basic dimensions of personality"

That model present five dimensions, to which the personality traits are
mapped. It does not mean they are the basic dimensions of personality.
Similarly I can map the world into 2 dimensions as in a photograph.

But I cannot say I've mapped the world to it's basic dimensions. I've lost
quite a lot of data already - I'm missing 4 dimensions of data. The 3.rd
coordinate, and 3 for momentum.

~~~
dash2
It's a fair point. People can't be reduced to a 5-D vector of traits.
(Otherwise novelists would be out of a job.) But OCEAN does capture a lot of
human variability.

~~~
disgruntledphd2
Can you provide citations please? All the correlations I've seen have been in
the 0.3 range, which doesn't equate to a lot of human variability.

I actually think this study is pretty interesting. Full text here:
[http://advances.sciencemag.org/content/2/8/e1600451.full](http://advances.sciencemag.org/content/2/8/e1600451.full)

I don't believe in their clustering, but then I rarely believe the output of
any unsupervised learning methods.

Its an interesting approach, the real question is whether or not it captures
incremental variance relative to an OCEAN (or the six factor one they like in
medicine) baseline.

~~~
dash2
Heh, so now I had to read the actual study.

I agree it's interesting, but my original impression was also true: this isn't
about discovering basic personality types. It's about correlating play across
four 2 x 2 experimental games. This is a very, very narrow domain of social
behaviour. Standard personality psychology, by contrast, aims to predict
behaviour across a much wider range of situations. (I still think that is
true, even if individual correlations are low in any one situation. But, if
you know better and personality measures actually a crock of crap, feel free
to correct me!)

Of course, it's possible that there really are four deep types of human
personality, that you can capture them in these four games, and that they
predict behaviour in lots and lots of domains. But I doubt it, and this
article provides no evidence for it.

What's more interesting is the comparison between this unsupervised clustering
method, and more theory-driven ways of categorising play in games, like
inequality aversion models. I think there is an interesting race between
psychologists, who pay greater attention to internal validity, and
experimental economists who focus on theory consistency. I'd like to know
which kind of models does better at predicting out of sample.

------
YeGoblynQueenne
Oh, come off it. Of course the study "identified four basic personality
types". They set out to identify four personality types to begin with:

 _After carrying out this kind of social experiment, the researchers developed
a computer algorithm which set out to classify people according to their
behavior. The computer algorith organized 90% of people into four groups:_

It's not magic, innit. If you choose a bunch of features and run some
classifier on them you'll get some grouping accoring to the distribution of
values of those features in your sample. If you choose a different bunch of
features, you'll get a different grouping. What did you learn? That your
features have a distribution- well done.

Trying to clasisfy peoples' personalities is futile: what constitutes
"behaviour" or "personality attributes" is very, very arbitrary and there's
nothing to say it's not all in the eye of the beholder to begin with.

------
ChuckMcM
_" There is a fifth, undefined group, representing 10%, which the algorithm is
unable to classify in relation to a clear type of behavior."_

They are _divergent_ :-). I found the characterization of "Envious" as people
who always want to win a bit more charged than "Competitive" would have been.

------
d--b
Based on 541 interviews of Spanish people living in Barcelona, a study is able
to classify 7bn human beings into 4 very distinct categories, and guess what,
most of them fall in the 'bastards' category.

------
BIair
"Optimistic, Pessimistic, Trusting and Envious" in case you missed the obvious
like myself.

------
desireco42
I am checking to see if this is a joke :) This is not the first time for
psychologists to come up with divisions like this. Not really sure how
relevant this is, thoughts?

~~~
sridca
It is not the categorization itself that particularly caught my attention so
much as the observation that many people would rather be envious than, say,
optimistic:

> _the latter of the four types, Envious, is the most common, with 30%
> compared to 20% for each of the other groups._

See also: Why are Adults so busy?
[https://news.ycombinator.com/item?id=12494999](https://news.ycombinator.com/item?id=12494999)

------
triplesec
This looks about as useful and sophsticated as the MBTI (ie, not very): 'A
study on human behavior has revealed that 90% of the population can be
classified into four basic personality types: Optimistic, Pessimistic,
Trusting and Envious.' I can't see how this contingent typing, apparently done
for the purposes of being able to group people to do game theory experiments,
can have much external validity, especially as there will be so many edge
cases and double- or even triple-class members. This just doesn't look
worthwhile, from this article.

~~~
psyc
When I read about my MBTI type, I can reasonably say "Yes, I am exactly like
that, and no, I'm nothing like the others." When I read about these
categories, my reaction was "I'm all of these."

~~~
wtbob
That's because MBTI is about as scientific as astrology. Frankly, when I
studied psychology in college ( _real_ psychology, not e.g. some Scientology
'psychologists are evil space aliens' claptrap), I came to the conclusion that
most of it is equally unscientific. It has the form but not the substance.

------
reduxive

      1. The Envious (anything to get ahead of everyone)

      2. The Optimists (everyone else is basically good, have faith)

      3. The Pessimists (most others are evil, favor the lesser of evils)

      4. Trusting Collaborators (will go along with anything to just be included)

      5. Unclassifiable (no common archtype correlating decisions, strategic decision making demonstrates conflicting tendencies)

~~~
gohrt
4 simplistic patterns, and one group of creative thoughtful people. I wonder
how well "Unclassifiables" are represented among the most accomplished and
powerful people in society.

------
reacweb
I have found another one with 6 which seems far less crude and has also
statistic "proofs":
[http://www.comcolors.com/en/](http://www.comcolors.com/en/)

------
tokenadult
Wrong. My hand is in a splint after arthritis surgery, so I can't type much
here, but see any reliable online source about the "Big Five" theory of human
personality to find out better information.

------
Qantourisc
Sounds like 5 personality types IN game theory. Not in personality ? It mean
it doesn't include how nervous / relaxed the person is. Which impacts a lot.

------
kazinator
Sounds like bunk. Optimistic<\--->pessimistic is obviously a complete axis,
which can have envious and trusting people on it all over the place.

~~~
Hoasi
Not to mention these may change upon circumstances.

------
sunstone
Meyers-Briggs are gonna be suing these guys for sure.

------
macawfish
I remember looking at the distributions for Meyers-Briggs quizzes on
OKCupid...

They were not Gaussian distributions :)

------
SnacksOnAPlane
Oooh, I can break it down to just 2 personality types: men and women!

------
max_
How is the image related to the article?


Show HN: Generate a mindmap from an org-mode file, with annotations - porphyry3
https://github.com/vzaccaria/pandoc-mm
======
poirier
That's supremely cool.

Nice work!


Show HN: Ion Browser – Lightweight and Fast Android Browser - bauripalash
https://github.com/xedtech/ionbrowser
======
darkmuck
Is this just another WebView based browser?

~~~
bauripalash
Yes. Actually I was Learning How These Things actually work.


Ask HN: Best documentation for a command-line program? - tomcam
I'm documenting an app that runs on the command line. I'd like to make it as easy to understand as possible for intermediate-level users. What's the best page/site you've seen for such a program?
======
arkanciscan
The typical documentation for a CLI app is a man page or --help arg, but you
seem to be asking about websites? Is that correct?

~~~
tomcam
I’m thinking of something like the example below, where it takes the user
through an example:

[https://www.stanleyulili.com/django/how-to-install-django-
on...](https://www.stanleyulili.com/django/how-to-install-django-on-windows/)


Image Background Removal - Peroni
http://developers.lyst.com/data/images/2014/02/13/background-removal/
======
susi22
Frankly, I'm not very impressed. This problem is a very very well known
problem in Image Processing. Usually people call it background subtraction or
the more general form is image segmentation.

There is many really well working algorithms in this field. A google search
with the major research conferences (ICCV, ICIP, SIGGRAPH etc) will give you
the latest and the greates of these algorithms. You'll also find good (image
segmentation) work if you limit the search for csail.mit.edu.

If you want your uploader to help you, you can also go for one of the
supervised image segmentation algorithms. Otherwise you'll need an
unsupervised algorithm.

Given that the authors also want to detect what is in the image, this might be
helpful for them:

[http://people.csail.mit.edu/mrub/papers/ObjectDiscovery-
cvpr...](http://people.csail.mit.edu/mrub/papers/ObjectDiscovery-cvpr13.pdf)

This guy is also doing some great work in that field:

[http://www.engr.uconn.edu/~cmli/](http://www.engr.uconn.edu/~cmli/)

~~~
mistercow
My thinking exactly.

The really crazy thing is that they seem to be reinventing the wheel so that
they can lean on the optimized code a graphics library provides. I think
that's flawed thinking to begin with. In my experience with graphics
programming, a reasonably optimized direct implementation tends to beat the
hell out of a nine-step filter chain, no matter how good your graphics
library. Even if they used the same algorithm, a direct implementation could
condense steps 2, 3, and 4 into a single convolution [edit: whoops, no you
cant; Sobel has a non-convolution step].

But more importantly, they've tied their hands by limiting themselves to a
tiny set of operations. Combing the computer vision literature would have been
a better use of time than trying to chain filters together.

In fact, some background subtraction algorithms effectively do as an
_intermediate step_ what Lyst wanted as an end result.

~~~
zackmorris
> The really crazy thing is that they seem to be reinventing the wheel so that
> they can lean on the optimized code a graphics library provides. I think
> that's flawed thinking to begin with.

Actually I disagree with you, even though what you say is correct in
principle. On any given day, the vast majority of my workload revolves around
working with other people’s code. So I may spend 7 hours trying to get a new
library to compile or figure out why I’m getting an exception or how to get
out of dependency hell and only 1 hour getting “real work done”.

For me, it’s exhausting to find a new library or API that loosely does what I
need, only to find that I have to install a new language, new framework, new
compiler, or even new package manager to use it. Developers have a tendency to
copy other developers (even when the “normal” way of doing things is not
ideal), so many libraries have no binary that I can test, and in fact no
example of usage or up to date documentation.

Then there are subtleties with new libraries such as speed or memory usage. So
perhaps a library that does exactly what you want runs at 1 frame every N
seconds while the highly optimized function in a mainstream graphics library
runs at many hundreds or even thousands of frames per second by utilizing
concurrency or the graphics card.

So in fact when it’s all said and done, I tend to think more in
transformations. I ask myself if I can express a solution slightly differently
if it allows me to use an existing tool, then encapsulate it in a black box
that has the same inputs and outputs as my ideal solution. Then my frustration
is that other developers don’t seem to think this way. “Make one tool that
does one thing well” has become the mantra and driven us into this fragmented
ecosystem.

This is just an aside, but: The only truly general purpose language that has a
syntax that doesn’t make me want to club myself over the head is probably
MATLAB, but unfortunately their licenses are too expensive for me. So I have
high hopes for Octave, and after that, maybe NumPy. So maybe one point we
could agree on is that we shouldn’t need a graphics “library” in the first
place. If we had a good mainstream concurrent language, then many of these
algorithms become one paragraph code snippets and run with speed comparable to
C or Java.

Edit: I wanted to give a concrete example. Low level languages like C are
overly verbose in their implementations, by 10 or 100 times usually, because
they try to leverage the wrong metaphors. Notice how compactly concepts like
image compression can be expressed with the right ones:

[http://www.mathworks.com/help/images/discrete-cosine-
transfo...](http://www.mathworks.com/help/images/discrete-cosine-
transform.html)

~~~
mistercow
I agree that using optimized libraries is the better choice when those
libraries do something _close_ to what you are trying to do, but this
algorithm is Rube Goldberg-esque.

Also, OpenCV includes functions for image segmentation. If they couldn't use
that, it would have been nice for the article to at least touch on _why_.

~~~
jbondeson
_but this algorithm is Rube Goldberg-esque_

I think that's being a bit unkind, it's simply clear that the people are not
experienced with computer vision and/or image processing in general. They
approached the problem from their domain and found the solution that worked
for them. If I had gotten to the point that I wanted to do a domain-specific
image segmentation heuristic I would certainly build it up from a series of
filters and image morphology steps. If the performance (both accuracy and
speed) was satisfactory, I don't know why you would invest in optimization at
that point. Also, from experience I know that implementing algorithms from
papers is a slow and many times painful process as the original authors
generally don't publish reference code, and if they do it's probably in
Matlab. And if they don't have someone adept at image processing their
potential for success would likely be low.

 _Also, OpenCV includes functions for image segmentation. If they couldn 't
use that, it would have been nice for the article to at least touch on why._

I agree that a quick "Here's what we tried that's readily available" would
have been good for others to learn from, because for many the general
solutions would be sufficient.

------
Synergyse
This was posted on HN a while back, really impressive:
[http://clippingmagic.com/](http://clippingmagic.com/)

~~~
babo
I've tried it with the sneakers sample but the results are disappointing. It
had a hard time differentiate the brown of the background and the welt.

~~~
jacobn
I thought it worked ok?
[http://clippingmagic.com/images/4953782/public/86ede827aa2fa...](http://clippingmagic.com/images/4953782/public/86ede827aa2fa9d9104776f68737dc43226554c5)

You need a couple of extra markings, but that's life on images with background
colors close to the foreground colors.

(I'm one of the creators of clippingmagic.com)

------
zk00006
Image segmentation is one of the most studied problems in computer vision. A
really good lectures about this topic is by Prof. Daniel Cremers:
[http://www.youtube.com/watch?v=fpw26tpHGr8](http://www.youtube.com/watch?v=fpw26tpHGr8).
In one of the talks, he describes methods that were popular in 80' and that
are pretty much what the author developed.

------
jallmann
Sounds like a saliency detection algorithm would be appropriate as a pre-
processing step prior to edge detection or thresholding. The choice of
algorithm may depend on the image characteristics; eg, [1] utilizes a global
center-surround, works well for large and distinct foregrounds, while [2] uses
a FFT, works better for noisier backgrounds. There is a lot of literature on
saliency detection, segmentation, etc.

[1]
[http://infoscience.epfl.ch/record/135217/files/1708.pdf](http://infoscience.epfl.ch/record/135217/files/1708.pdf)

[2]
[http://www.klab.caltech.edu/~xhou/papers/cvpr07.pdf](http://www.klab.caltech.edu/~xhou/papers/cvpr07.pdf)

------
jcampbell1
This might be of interest, to people looking for a more human guided approach,
where rough edges are defined by a human:

[http://www.alphamatting.com/](http://www.alphamatting.com/)

~~~
sjtrny
Came here to post the same thing (current researcher in this field). Although
you could actually automate the labelling by building a classifier for the
type of object you wish to extract.

------
tan
Another automated background removal method that worked best for me on complex
backgrounds was Graphcut Algorithm, by Microsoft research. Gives the best
results and has a implementation in OpenCV.

~~~
zenbowman
Yep, that works remarkably well.

I think it is called GrabCut - [http://research.microsoft.com/en-
us/um/cambridge/projects/vi...](http://research.microsoft.com/en-
us/um/cambridge/projects/visionimagevideoediting/segmentation/grabcut.htm)

~~~
gpcz
The advantage of GrabCut compared to the parent article is that it takes both
color difference and similarity into account rather than just difference. This
is done by combining all that information into a very cleverly-laid-out Markov
Random Field where making an energy-minimizing graph cut roughly corresponds
to finding a better pixel classification.

The disadvantage compared to the parent is that it's a semi-supervised
algorithm: it requires a bounding box to be drawn around the desired image. If
you had a bunch of very similar images you could pre-generate the Gaussian
Mixture Model that GrabCut expects and turn it into a supervised learning
algorithm, but then you'd lose the major innovation of GrabCut compared to the
graph cut methods discovered before it: you would not be able to re-run with
the newest "best-guess" and be guaranteed that the energy would monotonically
decrease. It would also fail spectacularly if it encountered something it
hadn't seen before.

~~~
gpcz
Another thing to consider is that GrabCut is patent-encumbered for commercial
use -- it relies on the mechanism described in Zabih, Boykov, and Veksler's
"Fast Approximate Energy Minimization via Graph Cuts," which is patented in
the US since 2004 (Patent No. 6,744,923).

------
mxfh
I'm still looking for a white to transparent filter. This program fails to do
that properly, if i look at shadow in the transparent examples with the boot.

Any recommendations other than keeping Photoshop CS5 around just to use
_KillWhite_ [1]? (Why _Adobe_ axed _Pixel Bender_ [2] is still just beyond
me.)

 _MathMap_ for _gimp_ looks promising to create pixel-filters, yet I haven't
tried it yet. Being able to to this on commandline would be nice, but isn't a
requirement.

[1][http://mikes3d.com/extra/scripting-
plugins/killwhite/](http://mikes3d.com/extra/scripting-plugins/killwhite/)

[2][http://www.adobe.com/devnet/pixelbender.html](http://www.adobe.com/devnet/pixelbender.html)

[3][http://www.complang.tuwien.ac.at/schani/mathmap/](http://www.complang.tuwien.ac.at/schani/mathmap/)

~~~
morsch
In standard gimp this functionality is available in Colors - Color to Alpha.
Maybe I'm misunderstanding what you're looking for.

~~~
mxfh
Thanks for the hint, just tried it and it not behaving as I need to.

Expected behavior in HSL color space would be anything with the same Hue and
Saturation value (with optional adjustable thresholds) of a selected color
gets assigned an alpha value of 1 - lightness. Where lightness of 1 being
white and alpha of 1 being opaque.

KillWhite seems to do exactly this: [http://mikes3d.com/extra/wp-
content/uploads/2010/07/apple.pn...](http://mikes3d.com/extra/wp-
content/uploads/2010/07/apple.png)

I guess by now, it's easiest for me to re-implement this as a filter for HTML5
2DCanvas.

------
pj_mukh
I am working on a similar problem and came up with similar methodologies. I
also used the Canny filter which worked quite well
([http://en.wikipedia.org/wiki/Canny_edge_detector](http://en.wikipedia.org/wiki/Canny_edge_detector)).
However, it all broke down with complex patterns in the foreground/background.
I'm currently trying to incorporate more structural information about the
foreground objects. Learning algorithms is probably another option here? Just
not sure how to frame the problem for a learning algorithm.

------
swframe
I've worked on this problem quite a bit. The hard thing to accept is that
there can be large areas in an image where there isn't a significant
difference between the foreground and background. In other words, the raw data
doesn't contain enough information to detect an edge there and no segmentation
algorithm will find what is not there in the data. Humans see edges
"semantically". We know there should be one there and so we "imagine" one. I
work on image segmentation and recognition and my code cycles between those
two goals. I segment and look for noisy and smooth edges, then I postulate
where there should be a smooth edge (for example, if you recognize a smooth
edge "parallel" to a noisy edge, check if the noisy one could be smoothed
"similar" to the other one). The end result is that I can separate the
foreground from the background but I don't try or hope to recover the edge
that a human would find. My segmented images have the uncanny valley problem.
The edges are correct in some areas and slightly off in other areas. The image
looks real and fake :)

------
paulcaponetti
I had this issue for a site a few months ago and did the flood fill approach,
also making available a simpler mode. This algorithm could use a few tweaks,
but it worked quite well for images with only white backgrounds, but can
easily be tweaked to deduce background colors.

[https://github.com/PCaponetti/image-background-
removal](https://github.com/PCaponetti/image-background-removal)

------
plicense
There is nothing innovative here. And with the kind of images that they are
considering, you could simply use a Canny edge detector and take the outer-
most contour to be the region of interest. As the previous comments have said,
this is a well tackled problem. They must have done some literature survey.

------
graeham
Usually automated image processing is tough without a priori info about the
image, or some human input.

Adding to these methods, a 'holes' fill algorithm can be very helpful. This
would proably be used in place of your Alpha Mask.

The process can be run multiple times with different parameters for better
results. For example, doing edge detect->threshold->edge detect (with
different parameters) can help remove some of the smaller artifacts you see in
the images.

If you're able to have some manual input, simply clicking the object of
interest or doing a rough outline can be a great addition to an algorithm,
especially in avoiding the problem seen in Global Threshold with Complex
Background.

------
W0lf
There's a really good paper from 2011 which does quite a good job in image
segmentation with very noisy background[1]. Their algorithm is biologically
motivated and analyses contrast changes in different parts of the image as
this is exactly how we separate interesting parts in images from uninteresting
parts.

[1]
[http://vecg.cs.ucl.ac.uk/Projects/SmartGeometry/contrast_sal...](http://vecg.cs.ucl.ac.uk/Projects/SmartGeometry/contrast_saliency/contrast_saliency_cvpr11.html)

------
tlarkworthy
you need to really model what is going on at the edge boundary. For an
individual pixel, how much of the contribution is from the background, and how
much is from the object? Pixels values are the result of a smoothing operation
over each pixels field of view (which is actually probably a little bit bigger
than the image dimensions suggest).

When you zoom into an image its clear that the edges of objects can affect
pixel intensities a few pixels away (blur). So you have to reverse that which
can be tricky. Image people like Matlab so thats where the premade solution
will be found

------
taliesinb
RemoveBackground in the Wolfram Language...

[http://reference.wolfram.com/language/ref/RemoveBackground.h...](http://reference.wolfram.com/language/ref/RemoveBackground.html)

------
JanSt
For anyone interested in CV:

[http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft....](http://szeliski.org/Book/drafts/SzeliskiBook_20100903_draft.pdf)

------
josephpmay
I have conducted research in exactly this. Their results would likely be much
more accurate by applying Canny edge detection instead of the simpler Sobel
edge detection

~~~
JanSt
Canny edge detection uses Sobel in most cases as a first or second step (after
blurring) and adds a non-maximum suppression + hysteresis thresholding - You
do most probably know that, just wanted to clarify that it's not really
"instead".

------
kirkbackus
This is awesome, love to see a well thought out image processing algorithm
used for such a useful application.

~~~
mistercow
I would agree with you if the image processing algorithm were actually well
thought out. As it is, I fear that this article is encouraging some very bad
ways of thinking about problems.

First off, they don't mention searching for an off the shelf solution to the
problem. If OpenCV's built-in GrabCut or Watershed filter wouldn't work for
them, they should explain why.

Secondly, they don't examine the literature for existing approaches to the
problem. Sometimes you won't find what you're looking for in those approaches,
but in that case, the _problem_ with existing approaches will inform how you
decide to tackle the problem.

Finally, they solve the problem by building a filter chain, but they don't
seem to actually understand what the filters are doing. They say that the
Sobel operator "looks for light to dark transitions". This is completely
incorrect. The Sobel operator does nothing more than estimate the magnitude of
the gradient of the image. Furthermore, based on this misunderstanding, they
invert the image before applying the Sobel filter - a step which does
literally nothing.

------
nailer
Small suggested edit:

> it normally has moved more background than product

Should probably be:

> it normally has removed more background than product

------
sjtgraham
This seems like a great problem to tackle with deep learning.

~~~
deutronium
I'm curious what you mean by this, could you elaborate

~~~
boomzilla
It's a new game in town: whenever someone says "big data", you reply "deep
learning", someone says "mobile", you say "real time", someone says "sharing",
you say "social", etc ...

~~~
SimHacker
Hello. Greetings. Brrrrrrring!!!
[http://www.youtube.com/watch?v=KTc3PsW5ghQ](http://www.youtube.com/watch?v=KTc3PsW5ghQ)

------
bagosm
Interesting... Thanks for the disclosure of the code. Now if only NSA was
willing to open source their image/video manipulation software...

When researching for this you didn't find any good premade solution or were
they simply too highly priced?

~~~
Peroni
We didn't find any pre-made solutions which fit with our requirements, which
were: reasonable results in the general case and speed.

We used GraphicsMagick and the pgmagick package to integrate into our code
base and because GraphicsMagick is crazy fast.