Fast web PC - Page 1
Forum section: electronics / general

author: ricko_uk
author_rank: Super Contributor
timestamp: August 20, 2020, 04:30:10 pm
content: Hi,I need to buy a PC for heavy web browsing using several chrome plugins for websites pricing data comparison. The idea is to run hundreds of Chrome tabs each with its own plugin (at least 20 different types of plugins).I tried with a couple of PCs (with 16Gb of RAM) but the CPS and memory get filled up quickly and the entire PC slows down to the point of taking 10-15 seconds to open a new tab and a minute to launch something simple like MS Word or even a new browser window.All I need is just the PC, no monitor/mouse/keyboard. Preferably not more than 700-800 GBP (IF possibleby prioritising the components that make the most difference)A couple of questions:1) what specs would I need to have a ultra fast solution for the above requirements? Which processor? Would more cores make a difference? Which is best AMD or Intel? What about type of RAM and type of SSD? Which of them makes the most difference for my application?2) where is the best place to buy PCs nowadays in the UK? Amazon? Ebuyer? Others?Also, I used to assemble my own PCs years ago but not sure that by the time I spend half a day assembling one the price difference is worth it? Any feedback?Thank you

author: Marco
author_rank: Super Contributor
timestamp: August 20, 2020, 06:44:48 pm
content: A second hand HP workstation with as much memory as you can fit in your budget?

author: RenThraysk
author_rank: Regular Contributor
timestamp: August 20, 2020, 07:14:13 pm
content: Would look into using headless chrome/chromium, andpuppeteer, if needed to something more complicated than simple web scraping.Puppeteer should be able control multiple chromes running on multiple PCs rather than trying to run 100s on just one.

author: ricko_uk
author_rank: Super Contributor
timestamp: August 21, 2020, 02:45:30 am
content: Thank you RenThraysk and Marco!I looked up puppeteer but could not understand how I would use it for my application. Could you please give me a brief explanation?It looks like I would need to write some code. Is that correct? I have zero experience in writing code for web...Many thanks

author: NiHaoMike
author_rank: Super Contributor
timestamp: August 21, 2020, 03:36:40 am
content: A used server will work nicely. But before going there, have you tried upgrading the RAM and optimizing?Some easy optimizations I can think of:* Lightweight OS, for example Xubuntu.* Adblocker, for example Ublock Origin.* Disable media playback in browser.* If your Internet access is fast and unlimited, lower the browser cache setting.* Use Noscript to only enable the Javascript required for the sites to work.

author: RenThraysk
author_rank: Regular Contributor
timestamp: August 21, 2020, 03:01:17 pm
content: Quote from: ricko_uk on August 21, 2020, 02:45:30 amAh. Perhaps I don't understand what you are trying to accomplish. With puppeteer can completely automate chrome. So could go to each site and extract the pricing information, and then go onto the next. Rather than have 100s open simultaneously.But yes, need to code, so a no go solution.

author: ricko_uk
author_rank: Super Contributor
timestamp: August 21, 2020, 05:35:44 pm
content: Thank you allWhat I need to do is open the plugins on very many Chrome tabs. Main issues are:- when launching each plugin, it takes a while to load and download all data (sometimes up to a minute) so instead of waiting there I immediately open other tabs and start opening other plugins- after a while the computer slows down immensly- I have to look at the data and manually download it and often compare it so have to keep very many plugin windows/popups open- zero experience programming for web/browsers etcAny further suggestions? With those further clarifications, do I still need just tonnes of memory to make it all run smoother?Thank you again

author: MK14
author_rank: Super Contributor
timestamp: August 21, 2020, 06:57:26 pm
content: You are making it sound like the real problem is the huge/overloading plugins, you are clogging it up with.I get the impression, that for secrecy reasons, privacy or worries about embarrassment, you don't want to say what you are really trying to achieve with your web browser.Analogy: Someone is putting a trailer with 20 tons of stuff in it, behind/attached-to their normal car. Then complaining their car is way too slow. The real solutions are either to get a lorry, only take what can fit into the boot of the car, or hire a haulier to take the 20 tons of stuff to its destination for you.If your existing computers are reasonably powerful (which they sound like they are), then a faster computer and/or more memory, may not really solve your problem.Plugins can be severely security limited by the Google Chrome browser (I get the impression, they don't want plugins really), they can (presumably, I'm not 100% sure), be using a system, called scripts/interpreted languages. These can be extremely slow and take a very long time to run. In a way, which a faster computer can't really help with much. (i.e. fundamental limits on single thread performance).One or more of your plugins may be faulty and/or very poorly written and/or being used in a way, which severely overloads your computer, hence the slow downs.I'd also be worried, your computer(s) are getting infected with stuff (e.g. spy-ware), which may be making the problem considerably worse.But, more memory might help, it is not really clear what the real/exact problem is. Details about what you are really trying to do, are too vague.Most things can be achieved without using plugins, but without knowing what you are trying to do, it is difficult to help you, here. Even if you do need a better/faster computer, it is best to know what the real problem is.

author: MK14
author_rank: Super Contributor
timestamp: August 21, 2020, 07:27:37 pm
content: This seems to explain some of what I was trying to say:https://www.kaspersky.com/blog/browser-extensions-security/20886/EDIT:This one is probably too complex to easily understand, but it is another example:https://www.i-programmer.info/news/86-browsers/8491-chrome-42-outlaws-java-plugin.htmlWhat I also should have said. Is that it could well be the various plugins, insistence on sending lots of internet data via their own servers, that could be causing much of the delay. If so, a faster computer wouldn't really help, make the internet any faster, as it would be the external servers that are really slowing you down.Even a super fast connection, wouldn't necessarily help, if the real problem is because of the plugins, as regards stuff external to your computer e.g. other peoples servers, for servicing the plugins.

author: bd139
author_rank: Super Contributor
timestamp: August 21, 2020, 09:25:01 pm
content: I feel this is an "XY problem"http://xyproblem.info/I've automated similar processes using python. I've got a arbitrage scraper box that runs 24/7 throwing about 10,000 requests an hour out on a fanless celeron with 4gb of RAM and it sits about 2-5% utilisation and doesn't take any of my attention unless it finds something interesting

author: janoc
author_rank: Super Contributor
timestamp: August 21, 2020, 09:33:14 pm
content: This definitely is an XY problem - or rather "if the only tool is a hammer, treat everything as a nail".Normal web browser is going to be totally unusable and crashy with hundreds of tabs running plugins, regardless of what monstrous machine you try to run this on. Build a web scraper, that can easily run even on a RaspberryPi.

author: ricko_uk
author_rank: Super Contributor
timestamp: August 23, 2020, 04:50:55 pm
content: Thank you MK14, bd139 and janoc,MK14,maybe that's what's happening.bd139 and janoc,how do you build a web scraper? I assume you do it programmatically but as per my previous post I don't have experience in any web-based programming. Are there other ways of doing it maybe simple high level block-based systems similar to some of NI's tools? Or perhaps extremely simple scripting?

author: bd139
author_rank: Super Contributor
timestamp: August 23, 2020, 06:49:24 pm
content: Scripting. I use python + requests + BeautifulSoup:https://www.crummy.com/software/BeautifulSoup/bs4/doc/Routine is:1. get the page with requests2. parse the content with BeautifulSoup into python objects3. do whatever magic you need to do there.

author: olkipukki
author_rank: Frequent Contributor
timestamp: August 23, 2020, 07:18:37 pm
content: Quote from: ricko_uk on August 23, 2020, 04:50:55 pmhttps://en.wikipedia.org/wiki/Python_(programming_language)QuoteJust learn it once.

author: ricko_uk
author_rank: Super Contributor
timestamp: August 23, 2020, 09:30:07 pm
content: Thank you bd139 and olkipukki

author: eugenenine
author_rank: Frequent Contributor
timestamp: August 23, 2020, 09:42:23 pm
content: Quote from: ricko_uk on August 20, 2020, 04:30:10 pmMS Word = running MS Windows. You don't need a new pc you just need a new OS.

author: bd139
author_rank: Super Contributor
timestamp: August 23, 2020, 10:14:07 pm
content: But that wont run word.Hell I use word (and excel - fight me) on windows 10 and I'm a through and trough Unix/Linux dude

author: eugenenine
author_rank: Frequent Contributor
timestamp: August 23, 2020, 10:51:34 pm
content: so run windows/excel/word in a virtual machine for those few times you have to use them.

author: ebastler
author_rank: Super Contributor
timestamp: August 26, 2020, 02:19:53 pm
content: Quote from: eugenenine on August 23, 2020, 09:42:23 pmAre you implying that running a browser with a couple hundred tabs and different plugins, but under Linux, is the solution?Or did you just feel the need to start some OS dispute?

author: bd139
author_rank: Super Contributor
timestamp: August 26, 2020, 02:28:51 pm
content: One trick pony has one trick!Quote from: eugenenine on August 23, 2020, 10:51:34 pmRun linux in a virtual machine for those few times you have to use them.(I have a whole ubuntu based kubernetes cluster running on my desktop at the moment inside virtualbox, on windows)

author: eugenenine
author_rank: Frequent Contributor
timestamp: August 26, 2020, 08:54:48 pm
content: Quote from: ebastler on August 26, 2020, 02:19:53 pmI have nearly the same laptop for work and home, Dell Latitude E7250 and a Dell latitude E7550 (E72xx is 12" screen, E75xx is 15" screen). Work laptop has windows and 16G of RAM, Home laptop has Linux and 8G of RAM. I can run 10-12 tabs in Firefox on the work laptop and Chrome barely runs on it. Home laptop I have hundreds of tabs in Firefox Chrome at the same time. Home laptop has the slower emmc drive interface but work has faster SATA. Home laptop is many times faster than work laptop despite having less ram and a slower disk.There are very different ways the OS's manage memory. After Windows 2000 Microsoft removed the registry key that allowed you to change the 'swappiness' of the OS and Windows likes to leave some ram free to "speed up loading programs" as they say. Linux will let you use all available ram before it swaps if needed. The linux OS is that much more efficient.Way back when I had windows 2000 running on a work desktop I could run 4-5 vmware clients running windows 2000 server. I had a small DC, web server, app server, sql server and sometimes a win2000 or xp client to duplicate the configuration the company was using. Upgraded to XP on that box and it could barely run two of the same vmware guests.I really think MS made the change intentionally to make XP and later less efficient so they could sell their hypervisor platform.But whatever the reason Windows hasn't been the same since.I setup a dual boot on my son's gaming pc and he was always amazed at how many more 'mods' he could run in Minecraft under Linux than he could under Windows.You really should try it out, it makes a big difference in usability.

author: brucehoult
author_rank: Super Contributor
timestamp: August 26, 2020, 08:57:08 pm
content: You're pretty clearly running out of RAM. Web browsers are huge pigs with memory and 16 GB is really a very small machine now. Get 64 GB or 128 GB or more. The CPU is unlikely to matter much but anyway you can get a hugely fast 12 core AMD 3900X for under $500 now.But max out the RAM first.

author: ebastler
author_rank: Super Contributor
timestamp: August 27, 2020, 05:18:28 am
content: Quote from: eugenenine on August 26, 2020, 08:54:48 pmThanks for taking the time to elaborate. But I cannot relate to your experience: I use a modest laptop at home (i5 CPU, 8 GByte RAM, 4 or 5 years old). While I don't keep 100s of tabs open, I do use a couple dozen or so, and I have never even begun to notice any performance issues. Using Firefox mostly, but Chrome runs just as fine. Does your work laptop get dragged down by some corporate security/antivirus package, maybe?But more importantly, I think this thread had already moved on to discussing better, script-based solutions to the OP's problem -- better not just from a performance perspective, but also regarding robustness, automation and maintainability. That was the main point of my comment: Even if Linux should bring performance improvements, sticking with the browser-based scraping solution appears like a kludge.

author: Jeroen3
author_rank: Super Contributor
timestamp: August 27, 2020, 06:03:26 am
content: To answer the initial Y question:For the purpose of running many chrome tabs, just get the most core count Ryzen available in your budget.Most of the budget will go to memory anyway.Also get a fast writing nvme ssd, a consistently fast one! Many also collapse in speed after writing a few GB.Maybe even consider nvme raid for those valuable disk iops.

author: brucehoult
author_rank: Super Contributor
timestamp: August 27, 2020, 08:36:40 am
content: Quote from: Jeroen3 on August 27, 2020, 06:03:26 amNormally I suggest 4 to 8 GB of RAM per core.1 MB of RAM per MIPS of CPU has been a good rule of thumb for general purpose computing since the 1970s. Modern Out-of-Order CPUs average around 2 MIPS/MHz for each core.If it's all web browser tabs (which are memory heavy and CPU-light) I'd double that.

author: magic
author_rank: Super Contributor
timestamp: August 27, 2020, 02:42:15 pm
content: Your fundamental mistake is trying to use Chrome with more tabs than fits on the tab bar. It simply isn't meant for that, it's a strict plebs' browser.I have discovered that by the time you no longer see even a single letter of each tab's name, this hog is already using well over ten gigs. It's hopeless.If you really must, get more RAM

author: ricko_uk
author_rank: Super Contributor
timestamp: August 29, 2020, 07:45:02 pm
content: Thank you all!!Some different opinions but several interesting guidelines and useful infos!!So thank you again!!