Facebook use of sensitive data for advertising in Europe [pdf] - <AUTHOR> https://arxiv.org/abs/1802.05030 <comment><AUTHOR> Abstract: _The upcoming European General Data Protection Regulation (GDPR) prohibits the processing and exploitation of some categories of personal data (health, political orientation, sexual preferences, religious beliefs, ethnic origin, etc.) due to the obvious privacy risks that may be derived from a malicious use of such type of information. These categories are referred to as sensitive personal data. Facebook has been recently fined €1.2M in Spain for collecting, storing and processing sensitive personal data for advertising purposes. This paper quantifies the portion of Facebook users in the European Union (EU) who are labeled with interests linked to sensitive personal data. The results of our study reveal that Facebook labels 73% EU users with sensitive interests. This corresponds to 40% of the overall EU population. We also estimate that a malicious third-party could unveil the identity of Facebook users that have been assigned a sensitive interest at a cost as low as €0.015 per user. Finally, we propose and implement a web browser extension to inform Facebook users of the sensitive interests Facebook has assigned them._</comment>


Stand with Kerala – Flood Relief - <AUTHOR> https://donation.cmdrf.kerala.gov.in/ <comment><AUTHOR> Kerala is in the midst of an unprecedented flood havoc. The calamity has caused immeasurable misery and devastation. Many lives were lost. Hundreds of homes were totally destroyed and many more were damaged. For the first time in history, 27 dams in the State had to be opened. Never before had the State witnessed a calamity of this scale. In the fight against the flood, we have braved the odds.

Now it is our duty to help the affected rebuild their lives. The path to recovery can be long & arduous, but you can make a difference by joining the rebuilding efforts. Contribute generously to the Chief Minister's Distress Relief Fund.</comment>


Beyond Calculations: A Course in Statistical Thinking - <AUTHOR> https://www.tandfonline.com/doi/full/10.1080/00031305.2018.1505657 <comment><AUTHOR> Abstract:

Statisticians are in general agreement that there are flaws in how science is currently practiced; there is less agreement in how to make repairs. Our prescription for a Post-p < 0.05 Era is to develop and teach courses that expand our view of what constitutes the domain of statistics and thereby bridge undergraduate statistics coursework and the graduate student experience of applying statistics in research. Such courses can speed up the process of gaining statistical wisdom by giving students insight into the human propensity to make statistical errors, the meaning of a single test within a research project, ways in which p-values work and don't work as expected, the role of statistics in the lifecycle of science, and best practices for statistical communication. The course we have developed follows the story of how we use data to understand the world, leveraging simulation-based approaches to perform customized analyses and evaluate the behavior of statistical procedures. We provide ideas for expanding beyond the traditional classroom, two example activities, and a course syllabus as well as the set of statistical best practices for creating and consuming scientific information that we develop during the course.</comment>


Play old retro abandonware games and abandoned OS/interfaces online - <AUTHOR> https://classicreload.com/ <comment><AUTHOR> I've just found this site which brought to me my youth. Description taken rom the site : "ClassicReload was setup for preservation of 5,000+ old retro abandonware games and abandoned OS/interfaces that you can play online right in your web browser for education and research purposes. There is just something magical about the old DOS games that just brings back memories and takes many of us back to our childhood or a time when things were simpler and just as much fun without all the animation that is the standard these days. These games were from a time when there were so many different types of PC's on the market that all had different hardware. You will notice that many of the games will let you change sounds cards, graphics cards etc. There was so much change in the 80's and 90's with DOS games and many seem to be all but forgotten, but we have them here! (If there is a game missing shoot us a message and we'll put it up.) This website was not only setup to try to preserve these games, but also to make them playable for free on modern desktop computers, tablets and phones. Before you get started make sure that you are using the latest Google Chrome or Mozilla Firefox."</comment>


History on Why we have OK buttons in GUIs (web site slow, see article in comments) - <AUTHOR> http://folklore.org/StoryView.py?&story=Do_It.txt <comment><AUTHOR> _Do It_

Author: Andy Hertzfeld, June 1982

Many of the academic types who were involved in creating the earliest implementations of the graphical user interface at Xerox PARC and various universities sort of sneered at the first generation of personal computers when they appeared in the mid-seventies, since the early personal computers were much less powerful than the machines that they were used to programming. There wasn't that much you could do with only four kilobytes of memory and no disk drive.

But Larry Tesler, who was a key member of the Smalltalk team in the Learning Research Group at Xerox PARC, felt differently. He was really excited by the potential of personal computers, buying a Commodore PET as soon as one became available in 1977. He was one of the demonstrators at Apple's famous Xerox PARC visit in December 1979, and he was so impressed by the Apple visitors that he quit PARC and started working at Apple on July 17, 1980, as the manager of the Lisa Applications team.

Larry championed consistency between applications, and made many contributions to what eventually became the Macintosh User Interface. He was also the leading advocate and implementor at Apple of user testing: actually trying out our software out on real users and seeing what happened. Starting in the summer of 1981, Larry organized a series of user tests of the nascent Lisa software, recruiting friends and family to try out the software for the first time, while being observed by the Apple designers who recorded their reactions.

The user tests were conducted in a specially constructed room featuring a one- way mirror, so observers could watch the tests without being intrusive. The tests were conducted by a moderator who made sure the user felt comfortable and showed her the basics of using a mouse. Then, with no further instruction, users were asked to perform specific tasks, without help from the moderator, like editing some text and saving it. The moderator encouraged each user to mumble under her breath while doing the tasks, revealing her current thinking as much as possible. Each session was audio or videotaped for later analysis.

When the software required confirmation from the user, it displayed a small window called a "dialog box", that contained a question, and presented two buttons, for positive or negative confirmation. The buttons were labeled "Do It" and "Cancel". The designers observed that a few users seemed to stumble at the point that the dialog was displayed, clicking "Cancel" when they should have clicked "Do It", but it wasn't clear what they were having trouble with.

Finally, the team noticed one user that was particularly flummoxed by the dialog box, who even seemed to be getting a bit angry. The moderator interrupted the test and asked him what the problem was. He replied, "I'm not a dolt, why is the software calling me a dolt?"

It turns out he wasn't noticing the space between the 'o' and the 'I' in 'Do It'; in the sans-serif system font we were using, a capital 'I' looked very much like a lower case 'l', so he was reading 'Do It' as 'Dolt' and was therefore kind of offended.

After a bit of consideration, we switched the positive confirmation button label to 'OK' (which was initially avoided, because we thought it was too colloquial), and from that point on people seemed to have fewer problems.

<comment><USER2> A great lesson...the obvious, no matter how obvious, may in fact be the best choice.

<comment><AUTHOR> You know, when I first started using Windows 95, it felt weird to see the word OK on every single dialog or confirmation screen.</comment></comment></comment>


Ship Early, Ship Never - <AUTHOR>

This started as a "Note to Self," so please excuse me if the frustration is all too evident and the writing is less than stellar...
Three major subjects I've had at the forefront of my mind lately: 1. Motivation 2. Learning (Curiosity) 3. Doing (Making)
I've been studying motivation and incentives for months now. It seems there are an infinite number of different motivations that people might have for doing things (I realize that sounds trite but bear with me). I've been really drawn to it because I'm desperate to find out <strong>why I do the things I do and why I don't do the things that I want to do but don't do</strong>.
I'm in the midst of reading Paul Graham's excellent Hackers and Painters book. In it, he makes the case that hackers and painters are very similar because they are both "makers." Painters _make_ paintings. Hackers _make_ software. Painters don't necessarily need to understand the chemical composition of paint to make beautiful paintings. And hackers don't necessarily need to know 1's and 0's to make beautiful software.
Graham then draws the distinction between disparate computer science fields: 1. some people seem to be studying mathematics 2. some people seem to be studying the computers themselves 3. the hackers are _making_ software.
The difference is incredibly important. It seems the motivation for some is to _make_ beautiful things. And the motivation for the others is to learn out of curiosity. Certain motivations seem obvious to me, but curiosity seems a bit less obvious. I would certainly consider myself as a curious person with a seemingly unquenchable thirst to learn just about everything I can. But this is exactly where the problem comes up.
The thing that scares me so much is that I desperately want to _make_ things. I desperately want to _do_ things. I want to write a book. I want to paint a painting. I want to compose a song. I want to _do_ things like travel. But the strangeness is that I also want to _learn_ things. I want to learn to play guitar. I want to _learn_ about art history. I want to _learn_ more about philosophy and literature.
The key seems to be the balance between learning and doing... between studying and making.
While I'm not sure how much one should learn about a given thing before doing it, I know for certain that I find myself constantly on one side rather than the other. As it stands now (and as far as I can tell I've always been this way), I am a learner and not a doer. I've read great books. I've practiced guitar for years. I've spent countless hours studying programming.
But I've written 0 books. I've composed 0 songs. I've coded 0 beautiful programs. I've painted 0 beautiful paintings. I've started 0 viable businesses.
The scary part of all this is that there are probably countless unfinished works of art in the world. Is this my misanthropic revenge against society and culture to never _produce_ or _finish_ any of the works of art that I start? Perhaps the worst part (aside from this being my natural inclination), is the fact that I fucking know better. I just finished books like "Getting Things Done" and "Making Ideas Happen." I've aggregated and synthesized countless words of wisdom on how to _do_ things and how to _make_ things.
Imagine the horror of going through life without being able to do the things you want to do. If this is something you've struggled with (and hopefully overcome), please share. If not... perhaps some delicious pity would make me feel better. <comment><USER2> <http://norvig.com/21-days.html>

[http://www.selfgrowth.com/articles/10-Year_Rule_to_Become_an...](http://www.selfgrowth.com/articles/10-Year_Rule_to_Become_an_Expert.html)

So get started! You've got 10 years to go... ;-)</comment>


MobiCart - FREE m-commerce app builder for IOS & Android - <AUTHOR>

First Demonstrated at DEMO Fall 2010, MobiCart Powers the Next Generation of Custom Mobile Storefronts as Seamlessly-Updated Consumer Applications; Saves Tens of Thousands of Dollars in Application Development Costs + Incorporates a Complete Add-on Ecosystem.
November 15, 2010 – Newcastle upon Tyne, UK & San Francisco, US – MobiCart today announced that its free, mobile-optimized e-commerce storefront development solution for Apple® iOS® and Google® Android® devices is now immediately available for download and deployment from www.mobi-cart.com. First shown at the renowned DEMO Fall 2010 show in September as a public Beta, MobiCart has now been released for general use by anyone seeking to develop a native m-commerce storefront solution for their customers.
MobiCart enables retailers to easily develop customized applications for end-user download onto their mobile devices while also providing a comprehensive back-end management platform for inventory control. These work in tandem to allow users to engage in a totally customized, native mobile commerce experience that is optimized for them and their specific shopping needs via their own supported mobile device.
MobiCart works as a standalone storefront, so businesses can develop their own mobile e-commerce site without even having a website. MobiCart can of course also be fully-customized to match the look and feel of an existing website, while easily allowing for the inclusion of inventory data and new pages to the app – it's your own shop with your own brand. MobiCart simply makes it simple and free to implement.
MobiCart enables businesses to save tens of thousands of dollars that would normally be required to develop their own application and mobile storefront while also enhancing brand and customer loyalty through an app that is unique to each user. All this is accomplished without CSS or developing an encapsulated website on the mobile device – MobiCart is a native app on its supported operating systems.
With an open API, MobiCart is easily and quickly integrated into existing shopping cart solutions, while also allowing developers to build and distribute extensions to MobiCart and take part in a growing community via its online marketplace. The MobiCart marketplace is a core aspect of the solution. Developers can easily extend the storefront and develop a any manner and number of add-ons, which they will then be able to give away or sell.
Raymond Kiersey, General Manager of Retail Sales at Flybe® – Europe's largest regional airline and MobiCart's first customer – noted during the company's Beta evaluation that: "We are very excited about this revolutionary new product. Our customers are busy people and MobiCart's technology will allow them to make duty-free purchases on the go. We believe providing an easy mobile purchasing option will greatly increase sales by aiding potential customers who otherwise haven't the time."
About MobiCart:
MobiCart www.mobi-cart.com is a startup that is taking m-commerce to a new level of personalization by enabling anyone to quickly deploy a native storefront application on iOS and Android devices. MobiCart is a completely free solution that combines a customized application development platform with an online ecosystem for sharing extensions (both free and paid).
Editors' Note: MobiCart logos, screenshots and other images are available from the company press contact or at www.mobi-cart.com/press <comment><USER2> Press releases? On HN?</comment>


Uber Completes Acquisition of Careem - <AUTHOR> https://finance.yahoo.com/news/uber-completes-acquisition-careem-000000975.html <comment><AUTHOR> Uber (NYSE: UBER) today confirmed the official close of the previously announced acquisition of Careem for $3.1 billion. Careem Networks FZ-LLC has become a wholly-owned subsidiary of Uber, preserving its brand. Careem co- founder and CEO Mudassir Sheikha will continue to lead the Careem business, which will report to a board made up of three representatives from Uber and two representatives from Careem. Careem and Uber will operate their respective regional services and independent brands.

With the closing of the deal, Uber has acquired Careem's mobility, delivery, and payments businesses across the greater Middle East region, with major markets including Egypt, Jordan, Saudi Arabia, and the United Arab Emirates. The regulatory approval process in Pakistan, Qatar, and Morocco is ongoing and the transaction will not close in these territories until approvals from the legal authorities responsible are obtained.

Uber CEO Dara Khosrowshahi said: "I'm looking forward to seeing even more innovation from Careem, as they continue to operate independently under their current leadership. Working in parallel, our two platforms will be able to build upon the unique strengths of each, to the benefit of drivers, riders, and the cities we serve across the greater Middle East."

Careem co-founder and CEO Mudassir Sheikha said: "Today marks the beginning of a new chapter for Careem. The journey that we started almost a decade ago to simplify the lives of people in the greater Middle East is far from over. Joining forces with Uber accelerates that journey as we become the region's everyday super app. We are excited to take Careem to new heights alongside Uber, who appreciates the significant regional opportunity, is supportive of our values and culture, and believes in the purpose that drives us."

Both companies believe this completed acquisition will provide an opportunity to expand the variety and reliability of services offered through their applications. Similarly, for drivers and captains, the companies believe an increase in trip growth and improved services could provide better economic opportunities as well as more predictable earnings through greater utilization of drivers' time on the road.

About Uber

Uber's mission is to create opportunity through movement. We started in 2010 to solve a simple problem: how do you get access to a ride at the touch of a button? More than 15 billion trips later, we're building products to get people closer to where they want to be. By changing how people, food, and things move through cities, Uber is a platform that opens up the world to new possibilities.

About Careem

Careem is the internet platform for the greater Middle East region. A pioneer of the region's ride-hailing economy, Careem is expanding services across its platform to include mass transportation, delivery and payments to become the region's everyday super app. Careem's mission is to simplify and improve the lives of people and build an awesome organisation that inspires. Established in July 2012, Careem operates in 120 cities across 15 countries and has created more than one million employment opportunities in the region.

View source version on businesswire.com: [https://www.businesswire.com/news/home/20200102005557/en/](https://www.businesswire.com/news/home/20200102005557/en/)</comment>


Carl Icahan on Economics - <AUTHOR> http://oyc.yale.edu/economics/financial-markets/content/sessions/session-15-guest-lecture-by-carl-icahn Mr. Carl Icahn, a prominent activist investor in corporate America, talks about his career and how he became interested in finance and involved in shareholder activism. He discusses his thoughts about today's economy and American businesses and their inherent threats and opportunities. He believes that the biggest challenge facing corporate America is weak management and that today's CEOs, with exceptions, might not be the most capable of leading global companies. He sees opportunities for current, intelligent college students to succeed in the corporate world if they work hard and can identify valuable business pursuits <comment><USER2> The illustrious _corporate raider_ 's name is actually spelled "Icahn". Yes, Icahn! (sorry, could not help it)</comment>


Ask HN: How would you design a car in 2013? - <AUTHOR>

In the 80's, when steam, electric, and gasoline powered vehicles competed for dominance, gasoline powered cars won arguably because it had the greatest combination of power and convenience.
Also, the form factor of the automobile evolved over time around the gasoline engine.
In 2013 however I am almost 100% certain that the  gasoline engine will not win if the playing field were level again.
My question is: If the automobile was invented in 2013:
1. What would be the most efficient, eco-friendy and convenient way to power this car?
2.  How might the form factor of the automobile evolve around this energy source?
Lets ignore politics and bureaucracy for now, and talk only in terms of technological possibility. <comment><USER2> Counterfactuals are complicated, but I didn't realize how complicated until I wrote all this. The basics have already been invented for other reasons (internal combustion engine, battery, electric motor, etc) so any sort of car should work fine aside from the infrastructure (power distribution, roads). Which sort is best depends on e.g. whether there's cheap oil (there might be if we hadn't used so much oil already) and what people are using cars for (people would be used to another mode with different capabilities; I've read that humans regardless of transit mode tend to prefer about an hour commuting per day and the distance of an hour depends on the prevalent mode). Random thoughts follow:

If the automobile were invented in 2013, we wouldn't have range anxiety, because we'd be used to getting around on horses. (Without range anxiety, electric cars might be the cheapest.) Or we would have range anxiety if there were lots of functioning places that would exchange your horse for one that wasn't tired (and no car recharging/refueling stations, of course).

Or maybe trains would still be the way everyone gets around. The streetcar would be ubiquitous, and would be the only reason we'd have quality roads (with rails in them). The streetcars might run on electricity or natural gas, judging by modern bus and light rail. Bicycles might be popular, and suburbs would have developed around streetcar lines rather than along interstate highways (I'm US-centric).

Cars might not need charging/fueling stations, if they could piggyback on an extensive electrical streetcar network's power lines. Then they would probably be electrical and have battery for short off-network trips.

The probable lack of a good road network is going to be a problem. We'll have to design a car that's good off-road. Preferably we will avoid annoying bicyclists by wearing away their dirt roads (if bicycling is in style and horses aren't); but we'll fail, because just having an enclosure for a person (I assume an enclosure is necessary to count it as a "car") is pretty heavy, and we can't improve wheels/tires terribly much.

Which parts and energy sources would be cheap? David Archer's _Global Warming: Understanding the Forecast_ (2007 edition, p.101) tells me that about 2/3 of US oil use is in transportation. The world might not have as much oil production capacity if the car hadn't existed (thus higher prices; then again, a newly invented car wouldn't suddenly be used by everyone, so oil production could ramp up over time along with demand); or the world might still have lots of easily extractable oil (lower prices).

Who would buy/use this car: rich? middle class? People who already travel a lot? People who are content? Individuals? Businesses? Transit companies (along the lines of ZipCar or self-driving cars)? Taxi drivers (there would probably be taxis powered by horse, bicycle, or some sort of engine)? How would people learn to drive this car? Probably most people wouldn't be used to operating a dangerous piece of machinery, and there wouldn't be laws regulating it (or there would be strict industry-focused laws).

The lack of experienced drivers suggests self-driving cars are a good idea; but the lack of well-marked roads means our AI tech needs to be a bit further ahead and we might have to wait 5 years.

EDIT: crossposted this response to my blog, since why not <http://idupree.dreamwidth.org/2568.html></comment>


Seven Warning Signs of Bogus Science - <AUTHOR> http://www.quackwatch.com/01QuackeryRelatedTopics/signs.html

<comment><USER2> TLDR:

 _1\. The discoverer pitches the claim directly to the media.

2\. The discoverer says that a powerful establishment is trying to suppress his or her work.

3\. The scientific effect involved is always at the very limit of detection.

4\. Evidence for a discovery is anecdotal.

5\. The discoverer says a belief is credible because it has endured for centuries.

6\. The discoverer has worked in isolation.

7\. The discoverer must propose new laws of nature to explain an observation._</comment>


How We Found All of Optimizely's Clients - <AUTHOR> http://nerdydatablog.com/2016/12/04/how-we-found-all-of-optimizleys-clients/ <comment><USER2> Always liked this stealthy prospect finding method. Here's another few (best for B2B):

1\. If some change happens at one of your competitor's product (got acquired, changed a beloved feature, etc.), go through threads on public forums discussing it. You'll find a lot of current users (ie customers to poach) of that product. A recent example: Atlassian acquiring Trello.

1b. Also, look through the comment sections of news sites that reported this change. You'll also find tons of current users there.

2\. Go on user submitted product review sites. Lots of them out there for different types of products: Chrome extension review page, Capterra, G2 Crowd, etc. Approach the users who left reviews expressing dissatisfaction with the competitor product. Or users who use a complimentary product.

3\. If a competitor product hosts customer sites on their servers (Shopify, etc.), you can reverse lookup their IP to find all of them.

Then it's a matter of introducing your product with a semi-personalized cold email, reassuring the prospect you have the competitor product's most vital features and you also do XYZ better than them. Here's a template: [http://www.artofemails.com/cold- emails#competitor](http://www.artofemails.com/cold-emails#competitor)

<comment> martin-adams Also, if you want to find leads on Twitter, find who follows the your competitors support Twitter accounts.

</comment></comment><comment><USER3> Interesting. The biggest source of annoying search traffic at Blekko was the people who were trying to find exploitable shopping carts, wordpress blogs, and forums. It was annoying because one could be 100% certain these folks are not clicking on ads ever and so it cost to serve and generated no revenue.

On the plus side it was a great leading indicator of a vulnerability in various bits of code because we would see searches that tried to match a particular package and version increase and then shortly thereafter a story would break about some data breach and personal data being stolen.

I was always a bit conflicted by it. I developed a number of tools which could identify this traffic and automatically ban it on our search engine which was the right thing to do, but you can probably sell that information to these organizations. So as a startup it was leaving money "on the table" as it were. I expect the way extract money out of that stream would be to have a site that accepted bitcoin and would return URLs of pages that matched a particular software package pattern.

Its also a great tool for sales people who are trying to sell wordpress themes for example (or identifying you is using your non-free theme without paying).

<comment><AUTHOR> Finding unlicensed Wordpress themes is a great usecase! You can identify them as the theme name is in the stylesheet URL by default.

[https://nerdydata.com/search?regex=true&query=wp- content%2Ft...](https://nerdydata.com/search?regex=true&query=wp- content%2Fthemes%2F%5Cw%2B%2Fstyle.css)

</comment></comment><comment><USER4> I'm always amazed about how there isn't more competition in this space.

As my Master's thesis [0], I built a crawler that did similar fingerprinting (although less generic). It wasn't something breathtakingly novel, but all in all a somewhat successful project.

It detected > 100 CMS, additional features like ad networks, social embeds, CDN, industry detection, company size etc. In the end, you could run a search and get the result as an excel sheet (because apparently that's what people like.)

The whole thing took about 6 months and ended up with > 100 million domains on a single (mediocre) machine humming away at around 100 domains/s. The sales/marketing folks loved it.

Since I was just finishing university, my skills were still pretty raw, so I'd assume that an experienced engineer would be able to do this a lot faster. From what I can tell, there was a lot of demand out there and sites like builtwith sold their somewhat limited reports (at least at the time) for a good amount of money.

[0] [http://blog.marc-seeger.de/2010/12/09/my-thesis-building- blo...](http://blog.marc-seeger.de/2010/12/09/my-thesis-building-blocks-of-a- scalable-webcrawler/) Previous discussion: [https://news.ycombinator.com/item?id=2022192](https://news.ycombinator.com/item?id=2022192)

<comment><USER5> That was 2010. In 2017 this space is flooded. We all know how to write web crawlers now and this data is sold by hundreds of companies.

</comment></comment><comment><USER6> Interesting; I had never considered scraping sites looking for specific embeds as a way of sourcing potential leads.

Scraping sites looking for complementary or competitive products' customers sounds like a novel way to do market research.

<comment><USER7> BuiltWith[1] and Wappalyzer[2] offer this as a service.

For software with client-side exposure that can be discovered during scraping, BuiltWith has pretty solid coverage (at least when I used it ~1 year ago).

I don't know if those services do it, but you can also find some really useful intelligence from DNS records. From email and calendar provider data to third party services like analytics trackers and landing page service providers (such as Unbounce). If you have an app that integrates or competes with those services, it can be really useful. If you use a DNS lookup service that provides historical record changes, you can even time your outreach to coincide with their annual renewal period when they're most likely to be entertaining the idea of a switch. Or in the case of an integration, wait until _after_ the renewal period to start outreach since you know they're locked in for another year at least.

You can also use DNS records to link together entity ownership relationships. Say a company has competing product lines and doesn't overtly market them as owned by the same company. If they happen to use Salesforce Communities, the CNAME for the Salesforce community subdomain will be specific to each site but will have the same Salesforce account id in it[3], Now you know that they're operating under the same entity, which itself is useful intelligence, but you also can combine the technology usage you sniffed from both sites together.

[1][https://builtwith.com/](https://builtwith.com/) [2][https://wappalyzer.com/](https://wappalyzer.com/) [3][https://help.salesforce.com/articleView?id=000205653&type=1](https://help.salesforce.com/articleView?id=000205653&type=1)

<comment><USER8> Are these sorts of scrapers able to grab any data from the post payment side of things in any way? I imagine there are a lot of interesting tags for ad tech, remarketing, etc. firing after checkout.

<comment><USER7> There's no reason these scrapers couldn't be coded to identify fields, insert plausible but synthetic data that'd validate, and submit forms. At least in the case of lead gen forms where payment details aren't required. It's a bit skeezy, but that's never really deterred the industry before.

Back when I used BuiltWith (1-2 years ago), they didn't appear to do that. But then, it's not really necessary since their use case is just binary identification of users. With the advent of universal tags that fire on every page (and you configure in the backend which page or funnel is considered a "conversion"), you can identify a lot of the ad tech in use without any form submission. Plus a lot of conversion and remarketing tags aren't hardcoded on the post-submission page, but wrapped in javascript functions. With minification and bundling, you can get a high success rate just parsing through the javascript files included on any page.

Where automated form submission would come in really handy would be a competitive intelligence tool that scrapes an entire site (and subdomains), identifies what actions on which page trigger which tags, and stitched together entire marketing funnels. Being able to monitor a competitor's likely marketing funnels (and seeing which ones they keep over time and which change) would be incredibly valuable, and would necessitate knowing precisely which tags fired on every page, including post-submission pages.

<comment><USER8> Interesting, thanks for sharing--these are great insights.

You're right that universal tags and the event naming that you could parse from the JS could be very valuable, although it would be hard to normalize.

And you're totally right about stitching together marketing funnels for lead gen conversions, but I'm not sure how you would get that from an ecommerce setup short of making a purchase and refunding it (which might be impossible to do in many cases).

Part of me has wondered if there are any partnerships BuiltWith or others have with popular browser addons. I imagine if they snooped this somehow from users, it would be valuable to them (setting aside whether it is ok to get this level of data).

I played around with someone's open source version of BuiltWith (forgetting what it was called) a while back and it was pretty cool to see how it works. I'm not a developer (although learning to code), but I've done similar research manually as part of my job, so this is really interesting to me to see what else can be learned.

For example, if there's a publicly traded ad tech company and you know a substantial customer of theirs just removed their tag, or many customers did in a certain time frame, you could short their stock (or vice versa if you see huge growth).

</comment></comment></comment></comment></comment><comment><USER9> This is just recycled content from them....

[https://news.ycombinator.com/item?id=6363979](https://news.ycombinator.com/item?id=6363979)

</comment><comment><USER10> > Our search engine is different from search engines you've used before. > Traditional search engines are geared towards providing answers, whereas our > goal is to give you the best list of results for a query.

I kinda get what you're going for here, but I think there's probably a better way to describe it, "best list of results for a query" sounds a lot like a standard search engine to me.

</comment><comment><USER11> There is a current service - builtwith.com - that does a nice job of productizing this type of research - eg:

[https://trends.builtwith.com/analytics/Optimizely](https://trends.builtwith.com/analytics/Optimizely)

<comment><AUTHOR> It looks like Builtwith only has a few predefined technologies. NerdyData can search for any string in HTML or JS files on millions of websites and do regular expression searches as well.

Here is an example search that can extract the IDs from Google Analytics code on websites into a downloadable list [https://nerdydata.com/search?query=UA-%5Cd%2B-%5Cd%2B&regex=...](https://nerdydata.com/search?query=UA-%5Cd%2B-%5Cd%2B&regex=true)

</comment></comment><comment><USER12> On a interesting side note Optimizely was a very early customer of ours (BuiltWith) starting in 2010 which helped them find customers for their own tool based on sites using their competitors (which at the time wasn't very many), I don't think it will bother them that other businesses can do the same thing.

Glad to see all these other tools in the market now there was and is clearly a need for it.

</comment><comment><USER13> Slightly OT - Anyone else find it hard to (other than the single link to the search) navigate to their main site from the blog? Strange to have its own domain without core/nav links back to the main site.

When you do get to their main domain, it has these weird links (sitemap a b c...) to another domain in the footer of an extraordinary number of other domains (SEO?)

</comment><comment><USER14> I thought they were gonna use those client ids and get more info about them using Optimizely API or some hack.

Kind of like using the private keys found in GitHub

</comment><comment><USER15> I want to know how to do this. There are a number of apps companies use that fit into my demographic. I want to know how to find them. Do you provide this as a service? I couldnt find a link to a homepage or anything.

<comment><USER16> See [https://nerdydata.com/search](https://nerdydata.com/search)

</comment></comment><comment><USER17> Cool but they need better coverage of the internet. Looks like they are only scraping some sites. I couldn't find any sites in my niche. Top alexa sites only? ¯\\_(ツ)_/¯

<comment><AUTHOR> We crawl 200+ million websites :) Click the "Deep Web" button under the search box on the homepage.

<comment><USER17> Thanks for the tip but I tried the "Deep Web" option and still can't find any of the sites. They are small sites that get about 100-500 visitors per month. Also, there are a lot more than 200+ million websites on the web.

<comment><USER18> What is your niche?

</comment></comment></comment></comment><comment><USER19> Cool stuff. Back in the day, I worked for a website hosting company and was tasked with finding several of our major competitor's customers.

I chose a more rudimentary route than this - which was to convert the huge lists of hostnames to their IP addresses.

Everything in our world then utilized shared hosting, so a relatively small list of IP addresses would map.

We then would proceed to scrape the sites for emails, phone numbers. Which was easy because our competitors had standard templates for their clients.

</comment><comment><USER20> I wonder if you could use nmap or something like masscan([https://github.com/robertdavidgraham/masscan](https://github.com/robertdavidgraham/masscan)) to figure the IP addresses of people using a certain software(say mongodb on port 27017). And then reverse looking up those IPs to figure out which companies they belong to and then you contact the said companies to sell something.

</comment><comment><USER21> This can be achieved just by using the publicly available HTTP Archive data set and Google's Big Query to search it.

</comment><comment><USER22> How do you do market research for SaaS that don't leave an artifact in the client website? Consider project management - what ways can we find people who use Trello/Basecamp?

</comment><comment><USER23> Handy for finding employees too:

[https://nerdydata.com/search?query=humans.txt](https://nerdydata.com/search?query=humans.txt)

</comment><comment><USER24> Nice work. Can imagine tools like this would be hella useful for identifying potential new employers or users of an open-source library you're developing.

</comment><comment><USER25> love nerdydata, always wanted to build something similar myself., but would any sane business person enter the space now?

</comment><comment><USER26> Love Nerdydata.com! I used their service a while back to find optimizely clients for my sales team to go after.

Looks like they added regular expression searches and a few new data sets. sweeeeeet.</comment>


The Spirit Molecule by Graham Hancock - sahin-boydas https://www.youtube.com/watch?v=47ElVv3RG2M <comment> sahin-boydas :Transcript:

Our society is absolutely designed to shut down 90% of the potential of the human creature. Why should it be the case that society is afraid of realising human potential and what it comes down to is special interests, that there are those who run things in society and that there are those who are there to kind of serve them and serve their ends and serve their needs.

And so, what our society has created is a realm of unquestioning meat robots who will perform their daily tasks without complaining and without causing any trouble, and that may be very useful to certain small interest groups but it's extremely damaging to the rest of humanity,

Once we realise our potential we don't need elites anymore, who needs elites - who needs to be led. This is another revelation that comes from working with psychedelics is that we do not need leaders, they are not over us telling us what to do, they may imagine that that's what their doing but they have no right to do that.

And the moment we begin to wake up and start asking really profound questions about the nature of reality and about the society that we live in, their power is instantly diminished and I think that's why we've seen such vigour and real nasty unpleasantness put into this horrible wicked thing called the war on drugs which is being used as an excuse to enter the inner sanctum of peoples lives.

See, if i'm not sovereign over my own consciousness, if I can't make free choices about my own consciousness, if I can't make those free choices while doing no harm to others then I am not free, in any useful or meaningful sense.

And, the moment that I free myself then I become a danger to those who run things in the world, anybody who has seen led to question the nature of things profoundly as a result of psychedelics is a danger to the status QUO.

We risk being thrown in prison, we risk having our reputations destroyed - <AUTHOR> the powers that be enter our home and find that we're using psychedelics, I mean this is really Orwellian World that we've created,

how dare some guy in a suit sitting in government tell me to what to do with my consciousnesses - while at the same time, they're allowing the glorification and glamorisation of alcohol which is one of the most dangerous drugs on the planet.

I mean alcohol is a seriously dangerous drug, whether you're talking about cirrhosis of the liver, or whether you're talking about traffic accidents or whether you're talking about violence - alcohol, there's no doubt about it nobody can dispute, is an extremely dangerous drug.

So a society that says alcohol's okay and you can even advertise it, but psilocybin or DMT are not okay and if you use them you'll be sent to prison, that society is just riddled through and through with unbelievable hypocrisy.

And we have to bring that out, we have to show people this is the case because what's going on here is a fundamental abuse of human rights, we must be free to make decisions over our own consciousness always with the proviso that we do no harm to others and any government any state agency that attempts to limit our freedom over our own consciousness - is itself an agency of darkness and control.</comment>


Italy court rules Uber food delivery riders were exploited - <AUTHOR> https://apnews.com/62609b97fbc25575cb1fcc2573db48f1 <comment><AUTHOR> This article is pretty short, so I will quote it below in full.

> ROME (AP) — A court in Milan has ordered the appointment of a judicial > administrator to oversee Uber Italy for one year after determining the > company's food delivery service exploited workers, according to Italian news > reports.

> The decision on Friday came amid an ongoing investigation into Uber's > activities in Italy.

> Italian news agency ANSA quoted the court saying that Uber's treatment of > delivery workers was similar to the "caporalato" system used by organized > crime groups to pay desperate migrants a pittance to do farm or construction > work off the books.

> In a statement carried by ANSA, Uber said it condemned "every form of > capolarato" and complies with Italian laws. Uber said it would "continue > working to be a true, long-term partner in Italy."

> Uber has faced opposition before in Italy. The company is only allowed to > offer its higher-end Uber Black service after Italy's taxi lobby protested > the ride-sharing service in 2017.

> The Italian General Confederation of Labor vowed to stand by the food > delivery riders and said the issue of inadequate pay showed "we are in the > presence of a digital caporalato."</comment>


Diaspora is now on GitHub - <AUTHOR> https://github.com/diaspora/diaspora

<comment><AUTHOR> Dear Community,

We have been overwhelmed with your support the past week after our annoucnment of Makr.io and the opening up of signups on joindiaspora.com. This week, we are excited to share with you some important Diaspora announcements.

When we started Diaspora two years ago, the project kicked off with amazing reception and support from people that believed in our ultimate goal: giving users ownership over their data. It's a powerful idea, one that captured the imaginations of millions of people across the world. This vision has expanded and evolved over the past two years that we have been working on it as the project has grown.

Diaspora* began when we were still at NYU—just four guys trying to scratch our own itch. We had an idea about how social networks could work in a new and exciting way. We intended to be done over the course of a summer, and with an expected budget of $10,000 from our Kickstarter campaign. The reception of this idea was so good that we managed to reach 20 times the expected amount in donations, and the project expanded to cover far more than just a summer. It's been over two years now, and we are proud of what Diaspora has become.

Today, the network has grown into thousands of people using our software in hundreds of installations across the web. There are hundreds of pods that have been created by community members, and it has become one of the biggest Github projects to date. It has been translated to almost fifty languages, with hundreds of developers worldwide contributing back to the project.

Diaspora has grown into something more than just a project four guys started in their office at school. It is bigger than any one of us, the money we raised, or the code we have written. It has developed into something that people all over the world care about and are inspired by. We think the time is right to reflect this reality, and put our code where our hearts lie.

Today, we are giving control of Diaspora to the community.

As a Free Software social project, we have an obligation to take this project further, for the good of the community that revolves around it. Putting the decisions for the project's future in the hands of the community is one of the highest benefits of any FOSS project, and we'd like to bring this benefit to our users and developers. We still will remain as an important part this community as the founders, but we want to make sure we are including all of the people who care about Diaspora and want to see it succeed well into the future.

If you look around, you'll see that we've made an effort to open up to the community more to help better serve it. We've opened up our Pivotal Tracker for community developers help join in (You can sign up here), we've launched a tool that deploys one-click installations to the Heroku app hosting service, and we've updated joindiaspora.com to be more community-centric, showcasing other pods a user can join.

This will not be an immediate shift over. Many details still need to be stepped through. It is going to be a gradual process to open up more and more to community governance over time. The goal is to make this an entirely community-driven and community-run project. Sean Tilley, our Open Source Community Manager will spearhead community efforts to see that this happens. Stay tuned to our blog for a message from Sean concerning next steps, as well as ways to get involved in helping with the transition process.

This is a new opportunity for Diaspora to grow further than ever before. We can't wait to see what we can do together.

Daniel and Maxwell

PS. We also want to give special thanks to a few people who recently, and over the past few years, have shown us what a special community we have. It is by no means complete: Mr ZYX, sean tilley, David Morley, Jan-Christoph Borchardt, Joe Braun, David Morley, Hans Fase, Florian Staudacher, Movilla, Stephan Schulz, Sarah Mei, Tom Scott, kinky joe, denschub, justin thomas, Steven Hancock, Diasp, Jason Robinson

<comment><USER2> Honest question: why make a meme generator ?

</comment></comment><comment><USER3> I think I've been following the project on github for over a year. Is this new?

<comment><AUTHOR> That's correct, this announcement is what's new: <http://news.ycombinator.com/item?id=4442143>

</comment></comment><comment><USER4> I'm quite confident that I've seen it on GitHub before. It may sound harsh but no one will ever use it.

<comment><AUTHOR> That's correct, this announcement is new: <http://news.ycombinator.com/item?id=4442143></comment></comment>


Duvamis - <AUTHOR> https://www.duvamis.com/

<comment><AUTHOR> The mission of Duvamis is to build an internet communication platform – Astral Media Duvamis, which is unique in its philosophy and is intended to provide a new media for sharing, creative activities and intellectual satisfaction to free and creative users worldwide, without social, regional, individual or group barriers and restrictions.

Astral Media Duvamis is the first anonymous social network for free communication and exchange of different opinions, interests and ideas.

Duvamis protects the privacy and anonymity of its users, giving them the opportunity to express who they really are.

The name derives from the ancient Greek word "dunamis" (Δύναμις), which means "internal strength, power, ability".</comment>


Why Has Regional Income Convergence in the U.S. Declined? [pdf] - <AUTHOR> https://scholar.harvard.edu/files/shoag/files/why_has_regional_income_convergence_in_the_us_declined_01.pdf <comment><AUTHOR> > The past thirty years have seen a dramatic decline in the rate of income > convergence across states and in population flows to wealthy places. These > changes coincide with (1) an increase in housing prices in productive areas, > (2) a divergence in the skill-specific returns to living in those places, > and (3) a redirection of unskilled migration away from productive places. We > develop a model in which rising housing prices in wealthy areas deter > unskilled migration and slow income convergence. Using a new panel measure > of housing supply regulations, we demonstrate the importance of this channel > in the data. Income convergence continues in less-regulated places, while it > has mostly stopped in places with more regulation.</comment>