Forum performance issues.  - Page 1
Forum section: electronics / general

author: wilfred
author_rank: Super Contributor
timestamp: June 04, 2013, 03:59:37 am
content: .

author: mariush
author_rank: Super Contributor
timestamp: June 04, 2013, 04:25:42 am
content: If you get to the point where you have to ask these questions, it may be worth more to colocate.You can get 1-2U of rack space with 10 mbps 95 percentile or 2-3 TB of bandwidth on 100mbps link for 50-75$ a month. For about 200$ a month you could get 3-4U with 100mbps unmetered port.For example, see :http://www.leaseweb.com/en/colocation/rackspacehttp://www.webhostingtalk.com/showthread.php?t=1257414&highlight=swift(i rent dedicated servers from this company, no complaints)You can easily build a server with 32-64GB of ram and a 2-3 ssd drives that would kill any shared server or bunch of shared servers and in long term cost you much less than paying monthly for ram or extra servers...

author: EEVblog
author_rank: Unknown
timestamp: June 04, 2013, 04:30:58 am
content: All I know is that this whole server hosting thing gets more and more complicated with every turn.If an extra $100/month for extra RAM will solve the problem then I will simply pay it.

author: gerrysweeney
author_rank: Regular Contributor
timestamp: June 04, 2013, 09:24:24 am
content: @Dave, there are lots of optimisations that can be made, it sounds like you have some classic symptoms of a poorly configured LAMP server. If you are getting tables locks on backups you are doing the backups wrong, you are probably using the VPS providers standard filesystem level backup which will mess with your database - assuming this is MySQL which is a common problem in this kind of setup. It would be hard to prescribe a solution without knowing more but anyone with string Linux admin skills and experience running MySQL/Apache/PHP in a production environment would be able to improve things. I don't know what your system is or the usage stats but I have a few words of guidance. My own knowledge is based on some recent experience, we have in the last two years built a cloud platform for our software (my day job) which is live and serving many customers globally.What you probably don't want to do is throw resources at the problem unless its needed, that can actually make things worse, not better.I would not suggest co-lo as an option as @mariush suggests, gets too hard to manage/look after and the hardware and OS maintenance is down to you. I would also not suggest a mysql replica and load balance which is what @wilfred suggests, this would be an active/passive setup, and the load balancer would need to know which requests from SMF were to write in order to route to the master. All doable but very complicated to make work and most importantly keep working.I would suggest three things you could do to improve things (assuming these have not already been done)1. Re-work your backup scheme, you need to schedule a database dump and back that up, ensuring that you exclude live table related files from any filesystem level backups. If your VPS provider is worth its salts they should have good documentation on how to do this. If not, you can use this resourcehttps://library.linode.comwhich is excellent.2. Optimise your SQL queries. You can turn on slow query logs and look at query planning and execution times. I have no idea how good or bad SMF is, I imagine that like most systems out of the box they are designed (and tested) with small amounts of data. Its amazing how a few index changes can transform performance.3. Look at PHP, Apache and MySQL configurations, adjust key buffers, memory buffers and limits to gain control over the use of resources4. Look at the overall operating system and stop things running that are not needed, this can make a difference in terms of memory.In a nutshell, if you want a pain free life I would start with optimising what you have. if you cannot solve the problem with a single server, things will get very complicated (and expensive) quickly. I have had similar experiences with my own blog which is wordpress running on a LAMP stack with linode.com and I have had very little issues with it. In fact I run various services as well as the web site and have probably not spent more than about 3 hours in the last two years maintaining it after I done the initial setup and optimisation.Hopefully of some use.Gerry

author: gerrysweeney
author_rank: Regular Contributor
timestamp: June 04, 2013, 09:28:19 am
content: One other thought, if you are having traffic load issues you can always throttle back traffic, to even out load on your system, this can be done with Apache configuration.Gerry

author: Psi
author_rank: Super Contributor
timestamp: June 04, 2013, 10:50:10 am
content: Quote from: EEVblog on June 04, 2013, 04:30:58 amYou could buy the ram outright every month for that price.

author: GeoffS
author_rank: Unknown
timestamp: June 04, 2013, 11:15:52 am
content: Before posting in this thread, be sure to read this one first.https://www.eevblog.com/forum/news/server-memory-upgrade-17505/msg241635/

author: justanothercanuck
author_rank: Frequent Contributor
timestamp: June 04, 2013, 12:01:32 pm
content: I too also want to apologize for being a pain in the ass. I'm just trying to understand the problem better, as I've been around this kind of stuff before. Let's hope the memory upgrade is a more permanent solution.

author: EEVblog
author_rank: Unknown
timestamp: June 04, 2013, 12:09:35 pm
content: Quote from: justanothercanuck on June 04, 2013, 12:01:32 pmIt is.8GB is now installed and working. No more lockup on database backup.

author: gnif
author_rank: Unknown
timestamp: June 04, 2013, 12:29:15 pm
content: Quote from: gerrysweeney on June 04, 2013, 09:24:24 amThat's what the past few weeks of tuning has beenQuote from: gerrysweeney on June 04, 2013, 09:24:24 amAgreed, I have configured these in the past with mysql-proxy, you have to have a very special requirement for this kind of a configuration, and this is not one of them.Quote from: gerrysweeney on June 04, 2013, 09:24:24 amWe are now using single transaction backups which prevent the locks on the tablesQuote from: gerrysweeney on June 04, 2013, 09:24:24 amYou are completely correct here, as I tried to explain in the other thread about 10 times, SMF has poorly optimised tables, missing indexes, in fact I just added another index to the smf_sessions table for the last_updated column. (DELETE FROM smf_sessions WHERE last_updated < N).Quote from: gerrysweeney on June 04, 2013, 09:24:24 amDid this but found that we were just at the limits of what we had memory wise, not enough free ram to cache key buffers or a decent portion of the InnoDB data.Quote from: gerrysweeney on June 04, 2013, 09:24:24 amHas also been done, but we can only stop so much as Dave needs a simple cPanel interface so he can still do things without an admin's assistance.We are also running a Nginx reverse proxy on the same box which handles all static content, only PHP requests are proxied through to the backend. Nginx also rate limits connections so as to prevent the server from getting overloaded, as well as caches some of the dynamic content where possible, such as dynamic thumbnails and the like.The issue was that even with Nginx heavily limited and MySQL de-tuned to use little RAM, when a backup process was run the server would run out of RAM and start swapping heavily to disk.Once the new RAM was installed I was able to bring the MySQL back up to speed, cache more, etc. Also there is enough RAM free to perform a single transaction database dump, where previously it just crashed things out when it started swapping to disk. We now have an average of 2.5-4 GB free RAM depending on server load and if the mysqldump is running.

author: gnif
author_rank: Unknown
timestamp: June 04, 2013, 12:35:50 pm
content: For the technically inclined, here are the stats from mysqltuner over the past 2 hours.Code:-------- Performance Metrics -------------------------------------------------[--] Up for: 2h 20m 35s (2M q [240.492 qps], 82K conn, TX: 4B, RX: 370M)[--] Reads / Writes: 60% / 40%[--] Total buffers: 3.1G global + 6.9M per thread (150 max threads) Maximum possible memory usage: 4.1G (53% of installed RAM) Slow queries: 0% (0/2M) Highest usage of available connections: 10% (16/150) Key buffer size / total MyISAM indexes: 512.0M/207.4M Key buffer hit rate: 99.8% (15M cached / 34K reads) Query cache efficiency: 86.5% (1M cached / 1M selects) Query cache prunes per day: 0 Sorts requiring temporary tables: 0% (0 temp sorts / 3K sorts)[!!] Joins performed without indexes: 2137 Temporary tables created on disk: 13% (606 on disk / 4K total) Thread cache hit rate: 99% (518 created / 82K connections) Table cache hit rate: 20% (940 open / 4K opened) Open file limit used: 5% (964/16K) Table locks acquired immediately: 99% (553K immediate / 554K locks) InnoDB data size / buffer pool: 1.8G/2.0G

author: xrunner
author_rank: Super Contributor
timestamp: June 04, 2013, 12:45:36 pm
content: Quote from: gnif on June 04, 2013, 12:29:15 pmGood to know it's fixed now, and I really mean this gnif - if you have improvements for SMF tell them about it and submit your ideas!

author: gnif
author_rank: Unknown
timestamp: June 04, 2013, 12:55:20 pm
content: Quote from: xrunner on June 04, 2013, 12:45:36 pmhttps://github.com/SimpleMachines/SMF2.1/pull/390

author: EEVblog
author_rank: Unknown
timestamp: June 04, 2013, 12:56:42 pm
content: Quote from: xrunner on June 04, 2013, 12:45:36 pmHe's already uploaded some code to the github repository:https://github.com/SimpleMachines/SMF2.1/pull/390

author: xrunner
author_rank: Super Contributor
timestamp: June 04, 2013, 12:59:29 pm
content: That's great, I've been testing SMF 2.1 so hope what he's got get's into the next version.