skysight-inc/HackerNews huggingface dataset:
Ask HN: Feedback on Techniques List for Software Performance and Scalability
Hello HN community!

In my free time I'm writing a book on "Software Performance and Scalability" and would like your feedback and input.

One section in the outline is structured around a list of patterns and techniques which can improve or harm performance and scalability. The idea is to provide a single unified document which can remind the reader of all the techniques to consider, the trade-offs, how to apply them, give before-vs-after case studies, as well as talk about useful measuring and simulation tools, benchmarks and then recommendations of specific components, services, products, packages, languages, etc.

Any things to add to my list below? Change? Improve? Disagree?

Please discuss or give feedback in the comments below, or, you can also email me privately (my address is in my HN profile.)

1. Static Not Dynamic

2. Cached

3. Distributed

4. Parallelized

5. Asynchronous

6. Incremental

7. Queues with Worker Pools

8. Step Minimization

9. Complexity Optimal Algorithms

10. Event-Driven Not Polled

11. Non-Blocking IO

12. Paginated Results

13. Web Page Component Request Minimization

14. Network Locality

15. Machine Task-to-Data Locality

16. Precompute Predicted Requests

17. Eager Init vs Lazy Init

18. Beefier Hardware

19. Do Tasks in Hardware Rather than Software

20. Leaner Languages & Runtimes

21. Tuning OS Parameters

22. Custom OS Kernel Builds

23. Non-Virtual OS Instance

24. No OS

25. Minimize Context Switching

26. Avoid Swapping, Especially Thrashing

27. Minimize Chattiness of Comm Protocols

28. Pass Smaller Messages

29. Avoid Lock Contention

30. Pass and Store Diffs Rather Than Complete Snapshots

31. Client-Server Architecture

32. Function Inlining

33. Loop Unrolling

34. Less Memory Churn and Background GC Inside Your Process

35. Local Function Calls Rather Than RPC or Web Service Requests

thanks!

Mike Kramlich
I highly suggest reading the book "Cradle to Cradle: Remaking the Way We Make Things" by Michael Braungart & William McDonough. Essentially the book explains how things currently are made in a "Cradle to Grave" style, meaning products are sourced from the earth and then deposited back in an unusable form. Cradle to cradle would be where a product is made and when it's life is up we reuse the product's materials to build a new product. This is not how recycling works. Recycling could be better described as "downcycling" in that every time say plastic is recycled it's grade is lowered and it becomes less useful due to being mixed with other plastics and contaminates. The book explains this much better than I and is also a great primer on material science.

Here's a simple example of how a consumer product can be created in a cradle to cradle fashion.

1) Raw materials are sourced from the earth.

2) Raw materials are manufactured together in a way where they can easily be seperated.

3) Product ends up with a consumer.

4) Product lifecycle ends.

5) Manufacturer pays consumer for the product.

6) Manufacturer uses product's raw materials for step 2.
Happy to share. Thank you for making it!

I'm pretty sure I've mentioned this to John Britton at one of GitHub meetups. He played some part in making that from the GitHub side, as he does education stuff with them.

That was about 3 years ago. Since then I've got a nice software engineering job in San Francisco, and I've been trying to make as many open source contributions in my free time as I can:

https://github.com/shurcooL

1,737 total in year of contributions (not counting private) hehe.

Learning to use git was an eventual step of the journey, but your work certainly played a part in enabling all that, so thank you. :)
Wow. As many others here, i was sure there wouldn't be a difference if Monty picked at random AND we ignored the cases where he picks the door with the car. But after reading "write the damn simulation!" too many times, i wrote one, being sure it would prove me right.

Of course, i was wrong :D

The simulation testing the four possible scenarios:

#!/usr/bin/env ruby

def run_games(monty_knows:, change_choice:)
options = [:goat1, :goat2, :car]
total_games = 0
wins = 0

100000.times do
choice = options.sample
monty_choice = (options - [choice, (:car if monty_knows)]).sample

# Don't consider cases where Monty shows a car.
next if monty_choice == :car

if change_choice
choice = (options - [monty_choice, choice]).first
end

total_games += 1
wins += 1 if choice == :car
end

wins / total_games.to_f
end

puts 'Monty knows & keep choice: %.2f' %
run_games(monty_knows: true, change_choice: false)
puts 'Monty knows & change choice: %.2f' %
run_games(monty_knows: true, change_choice: true)
puts 'Monty ignores & keep choice: %.2f' %
run_games(monty_knows: false, change_choice: false)
puts 'Monty ignores & change choice: %.2f' %
run_games(monty_knows: false, change_choice: true)

Sample output:

Monty knows & keep choice: 0.33
Monty knows & change choice: 0.67
Monty ignores & keep choice: 0.50
Monty ignores & change choice: 0.50

So, in the case of Monty picking at random, even if we ignore the cases where he picks the car, your chances of winning do not improve by changing your first choice or not.

After verifying that the simulation makes sense, i tried to figure out why it makes sense.

In an information theory kinda way, it makes sense that if Monty picks at random then he is not providing any new information: you are just being shown what was behind a random door you didn't choose and you can't choose now. But that explanation is not intuitive to my brain.

I found it more intuitive to actually draw a decision tree with the possibilities and their probabilities. The tree is actually very simple:

With the Always Change Door strategy:

car (1/3) -> LOSE (Monty always reveals goat,
/ we change and we always lose)
first choice
\ goat (1/2) -> WIN (we change to the
\ / car door)
goat (2/3) -> Monty choice
\
car (1/2) -> IGNORE

The probability of winning is 1/3 (2/3 * 1/2).

With the Always Keep Door strategy:

car (1/3) -> WIN (doesn't matter what Monty chooses)
/
first choice
\ goat (1/2) -> LOSE (we keep our
\ / goat-hiding door)
goat (2/3) -> Monty choice
\
car (1/2) -> IGNORE

Same tree, but with win and lose cases inverted. The probability of winning is also 1/3.

In conclusion, my intuition sucks. Thanks for showing me so!
Type phrase and get video/audio mashup from YouTube videos?
There was a site that did that but I don't remember the address. Anyone knows the one I'm talking about?
Could Google theoretically calculate the probability that some text on a webpage is extremely hard or impossible for humans to readily see based on info from the source page about the background and text?
6:24 PM PST Between 4:12 PM and 6:02 PM PST, users experienced elevated error rates when making DNS queries for CloudFront distributions. The service has recovered and is operating normally.
If you use alphabetically ordered keys (a pretty good practice anyhow), that advantage goes away. If you just develop a habit of adding stuff to the front where semantics do not matter, that advantage goes away.
Well any product with an electronic component is harder to recycle as they have many different raw materials that have been blended / melded together.
While he's right, I imagine VCs get such an unbelievable volume of cold e-mail that sorting through it is really time consuming to the point of impossibility.
I noticed this last year. Most people didn't care.
I will continue to block ads via add-ons, hosts files, and a few other methods that have worked well for me for years.

Ads are now the #1 vector for malware and I block them all, all the time. I have a right to traverse the Internet w/o being tracked and served ads. The business model is semi-broken already. I would rather see paid content than freemium any day. I pay for what matters to me, but will happily block the rest for my own safety.
XForms has a bigger scope and is connected to XHTML. Two good reasons why this might succeed where XForms failed.
Thanks for sharing it here. Actually I didn't intend to publish this tool at the beginning,because there are some already. This tool emerges from a piece of extempore code which is a regex parser implemented with DFA only. Then I give it a GUI,so becomes regulex.

Debuggex and regexper are powerful,I agree,that I also hesitate whether there's any reason to publish it.

But I'd like to point out some merits of regulex:

- Regulex is dedicated to ECMAScript RegExp syntax(So I didn't plan to add PCRE support now)

- regexper.com can not point out the syntax error index precisely.(Much JS interpreter can not do that)

- regexper and debuggex both tolerate OctalEscape.

- Regulex can detect invalid backref,which is treated as OctalEscape in Debuggex and regexper.

- Because of OctalEscape's complexity, Debuggex did not parse `(\1)\1\2` correctly ( https://www.debuggex.com/r/_U1r_NxTvLan7h9R ),also regexper.And /^\1(a)$/ will match "a",not "aa".So I treat it as an error.

So at least it can be used to find javascript regex syntax error. ;)
Sadly, I think you are correct. The Internet of the last decade has ushered in the freemium model and people now expect it. I often tell people that if you are not a paying customer, you are the product. As you so bluntly stated, most people don't care. Sad, but true. All services that I value, I pay for, as none of them are freemium. I enjoy being a customer, not the product. Ads suck, anyway, and they are now the number one vector for malware. No thank you.
I'm not sure how could be that possible without a "free pass" by apple, I mean there are rumors that says that for "big corporations" there is no need to pass through the tricky apple review system. If is not that the case I really doubt they can collect as exposed in the article so freely this kind of data, the maximum they can do is use the AI and then match it across different apps (mostly like cookies).

EDIT: AI -> Advertising Identifier (basically old UUID)
Most people just don't care about privacy or security at all. They'll say they do and get all angry when stuff like this comes up, but they don't alter their buying habits therefore they don't care.
Didn't think it was possible, but: http://www.ihasapp.com

Why is this possible?
You have the hard part down . . . building something that gives users values . . .

Are these paying users? (Just curious).

You can definitely outsource the creation of your twitter profile page and fbook page . . . and pay to have some decent articles written for your website to gain more organic traffic.

I'd recommend listening to StartUpsForTheRestOfUs.com they have some great podcasts outlining the what/how/who of marketing your product, growing an email list, getting paid signups.

Congratulations and good luck growing your product.
Not just theoretically. Very concretely.
Nice “continuity” alternative for android
After using emacs a lot I've ended up preferring screen splitting.

I used to be happy just running multiple xterms, but I've been using Linux again recently and found myself a bit unhappy with that. I think due to advancing age I'm just finding it hard to cope with lots of windows. Keeping on top of them is a pain.
> After riding with Uber you get a map sent to you, this could help in proving that a taxi driver tried to rape a woman customer in a dark ally. (this has happened, driver got fired and charged) With a regular taxi it's your word against theirs.

stop hailing strange taxis. Call a dispatch. Boom, accountability.

>With Uber you are 100% sure the driver is working for Uber, since you ordered it through the app and the app shows you the license plate. With regular taxis you have to trust that the taxi license is real if you hail a cab on the street.

that's unfair as the comparison is simply outside the scope of what Uber offers. One could also argue that Uber drivers cannot pickup a person hailing them; but no one should make that argument, because Uber does not try to emulate that particular function of the taxi service.

> With Uber you don't need to have anything representing money on you to get a ride. With regular taxis you need either cash or a credit card on you to pay for the ride, which is a safety issue before, during and after the ride.

how is a credit card a risk? Give it to the perpetrator , ensure your safety, claim the losses. If you're talking about material worth, I hope you don't carry any devices with you.

>I agree with you that there are some issues with a company like Uber just doing whatever they want. However, there are some pretty big issues with the current taxi industry and Uber is doing a prety good job at highlighting them and providing alternatives.

A third party hopping and skipping over established transportation safety regulation is not doing a good job at providing alternatives; it's simply trying to make money before government regulation radically changes the market or the IPO occurs (which will in turn cause a sudden relaxation of 'boundary-pushing' on Uber's part, relaxing government legal worries and providing customers with a worse product after brand establishment and customers begin to rely on earlier iterations)
> Peer review does not "verify results"; peer review is there to make sure there are no serious and obvious flaws. Duplication of studies and collection of additional data / use of other techniques is what verifies results.

What you're saying is true of Science than Math. There's a fundamental difference between Math research and Science research. Math research doesn't involve hypothesis and verification through experimentation. Perelman's paper is purely logical it starts with axioms and derives its conclusions from them. For research like that peer review is actually where you verify results.
You could have Lisp syntax on top of Python...

http://www.hylang.org
When did they just drop to the zip code? My memories were that they wanted full name and address for each purchase.
Me and a group of friends at NTNU spent the last three months implementing self-designed and working GPU from scratch in VHDL on a Xilinx Spartan 6 FPGA.
At the same time as this, we also implemented fully working multi-cycle and pipelined MIPS CPUs for the same FPGA.

All of us came from software backgrounds, never having touched any hardware description language.

To get started I'd recommend reading Computer organization and design by Patterson and Hennessy (http://www.amazon.com/Computer-Organization-Design-Fifth-Edi...).

For the VHDL part, all we had available was this small basic VHDL compendium (not sure if I can redistribute it), open source github repositories from friends and classmates, as well as the always helpful vhdlguru.blogspot.com

So i'll reccomend following one of the open lecture series posted above :)
1. This isn't feasible. A partner at CRI once told me they get 5,000 deals a year. It's simply impossible to sift through even more than that.

2. If you're spending your nights shipping code, then you can probably get users/customers. If you can get users/customers, getting warm intros is pretty easy. So much so, that VCs will come after you (see the WhatsApp/Sequoia story)

3. I've struggled to get funding so I can empathize with the author (among his other various random rants) but this isn't the right way to react to it.

4. VCs often say (at least in SV) their biggest mistakes are the "investments they didn't make". I know very few who said "shucks, wish I woulda have gotten a warm intro from them".

The reality is pretty simple - show growth and you'll get VC intros. If you spend your time complaining how the system is unfit for you, then building a venture-based tech company is probably not for you.
Yeah, that would be great and easier than reading through huge archives of mailing lists probably.
Can't seem to find in the app where to turn 'off' tracking! Can anyone post the steps for both iOS and Android?
nano is less confusing than vim, though that is another bad habit to form. I used it for a very long time, but switching to vim has been amazing. I think that -m is probably best for teaching a beginner, though.
What is JavaScript? (core language vs. Web APIs)
Because they use a generic email address.

Why not create a smart form, YC style, that take some time to fill and provide the most important data about a startup for them in an easy to read format?

A simple "submit your AngelList page" might go a long way.
While I agree diff should be adjusted to accomodate for whitespace diffs more easily by default (it can do that with some options), it's not just a burden on the reviewer. It is also a burden on the programmer.

If you have, say, 50 lines of assignment and you align all the values to the largest one, adding one forces you to update 50 lines. I've been faced with those very situations and that is when I understood how important it is not to align values like that.
VCs are not like your local bank with all the time and willingness to hear all the potential business plans and pitches that can be thrown at them. Also their system isn't just spawned out of some kind of elitist mindset to keep you out of the in-crowd. Just keep working at it.
Part of the fall of Radio Shack is that the market for discrete components isn't what it used to be. This is because not as many people take up electronics as a hobby. And that, in turn, is because a lot of people that would go into electronics, got sucked into computer programming instead. After all, the decline of Radio Shack coincides with the rise of affordable personal computers.
... or just hire remote workers. It will be healthier for your processes anyways.
This feels like the definition of wishful thinking. If I, a nobody, tend to dismiss cold enquiries then why on earth would someone in high demand not?
For apps that support deep linking, you could check if the device responds to a URL with a corresponding scheme (e.g. "twitter:///")
That's not working for me, even in an anonymous browser window.
If you downvoted I'm not angry but I would appreciate transparency about what you think about the role of electricity and magnetism in the Universe at large. I tried to distinguish my comment from naive ideology by cautioning that I am not comfortable with the level of scientific rigor used under the 'umbrella' of Electric Universe studies but that I find the cognitive paths they use to be on a track that probably leads to rigorous scientific experiments and provable/disprovable theories. I find it a good philosophical framework, far better than, say, Greek Astrology...
YC doesn't get more than 5,000 twice a year?
It is strange that I think this isn't a big deal.

At least, relatively speaking, this is peanuts. I mean, really think about what's being collected: <To help build a more personal Twitter experience for you, we are collecting and occasionally updating the list of apps installed on your mobile device>. That's it.

Now, think about the data Facebook has on you. Right. This is a joke in comparison to the wealth of data Facebook has on you. If you're going to express outrage over this, at least dish it out in appropriate amounts to appropriate parties.
Your story would become compelling if you share your raw revenue growth graph and tell us what market you are in. Without these, your complaints sound like hollow whining. I mean this in the most constructive way possible. You could even do this without losing your anonymity.

Lacking that information, we have no data to judge the reasonableness of your complaint. All we have is the perspective of two experts, who both said "Too early" (translation, "Zero interest. But do come back if your business does magically turn amazing").
Well I said probability because depending on how complex the background is such as photographs, video, or some weird scrolling behavior, there could be some hindrances to having a computer accurately or computationally-efficiently determine this.
I've heard that before...

https://en.wikipedia.org/wiki/XSLT
She is Jew.
"Social proof" is important to VCs. I don't blame them for not sifting through cold emails. If you're worth investing in, other people should already be talking about you.
Countdown to Apple slapping this down in 3… 2…
"Do you have at least 20 years of C experience?"

"Uh... actually, yes!"
Even in a competitive market, Comcast wants to, and can, vertically integrate with NBC then punish its competitors for the same reason Apple wants to vertically integrate content/software/hardware and push out anyone that competes with any part of that stack. There's a huge, technical, advantage to having the content company also own the network: pushing caching nodes to the edges of the network, consolidating redundant data flows on the backbone, taking advantage of multicast, etc. It's not all that different to the advantages of vertically integrating iTunes/iOS/Apple Ax-based hardware.
Entrepreneurship and Competition in Peter Thiel’s Zero to One
Certainly the review process for 'big corporations' is as strenuous if not more than for apps that affect 30 users. Sure, the BD teams and developer relations pay more attention to larger companies, but typically apps (those with tens of millions of users) submit new binaries the same way you and I do it, from Xcode, iTunes Connect (or the command line/applet tool)
With Twitter going more into app advertising, it seems reasonable. What's the point of them advertising Clash of Clans to you if you already have it installed? Or, potentially -- with their app re-engagement campaigns, this provides the correct targeting.
>You are correct. Selling other peoples work for pennies on the dollar, is a legitimate business model, and not a criminal enterprise.

So, like Spotify?
No, Because it closes out competition to newer, better information.
I lived in a country town in South Australia in the early 80s, and without the local Tandy (Oz brand of RS) I wouldn't have got into electronics or software, both of which now are my main source of income and also a source of a hell of a lot of fun. Fortunately these days the opportunity for kids to discover new stuff is provided by Internet access, but it's still a sad coda to the joy RS provided in my youth.
As said by @cwalcott should works for apps that support deeplinking so apps that enable "tel://", "skype://" etc... However if you have an app like Tinder that as no scheme (AFAIK) shouldn't be detected. Indeed is still bad that they do that but seems they're basically using an "hack".

iHasApp has a large list of URL schemes, mapped to the
iOS App that they identify. The framework essentially
runs through all of these schemes, and determines which
URL schemes are handled by the current device, and create
a list of application ids as a result of successful
queries.
"...70% of the software we use today, she had a hand in..."
Other than the weird grammar syntax, what the hell is that statement supposed to mean?

A Grace Hooper doc would be great, would even make a good biopic, but something about how lazy the Indigogo video was thrown together (I could produce something of similar quality in an afternoon) leads me to expect the worst of the result.

Wouldn't touch this campaign with a ten foot pole. Where is the appeal from the doc makers? Where is the video where they outline all the previous docs they successfully made?

Nope.
Well, I dont think most of Hindus are bothered about anyone scrutinizing the texts. In fact most of the Hindu texts themselves are scrutinization of other texts. For example Upanishads are debating Vedas and so on.

The real problem occurs when certain people like Wendy start pretending to be experts spreading falsehood initially as her "subjective opinion" and later cross referenced as "scholarly analysis".

For example Wendy's book "Hindus : An alternative history" is perfectly a find book if she would claim it to be her own understanding of Hindu texts and philosophies. When people in completely civil manner point out to the huge holes in her "scholarly work" she does not defend it but claims that right wingers are out for her head which is not simply true.
As someone making a Wikipedia competitor I could not agree more.
you contradicted yourself here - or voluntarily misunderstood the point.