Forum Database Upgrade - Page 1
Forum section: electronics / general

author: EEVblog
author_rank: Unknown
timestamp: March 24, 2017, 02:43:33 am
content: gnif has upgraded the forum database with a new version fo MySQL that will support InnoDBPlease report any issues with performance, old posts etc.Thanks

author: Nusa
author_rank: Super Contributor
timestamp: March 24, 2017, 03:03:26 am
content: Forum was nearly useless (nonresponsive when trying to view a post) a few hours ago, and I gave up. Seems fine now, however.

author: EEVblog
author_rank: Unknown
timestamp: March 24, 2017, 04:04:10 am
content: Quote from: Nusa on March 24, 2017, 03:03:26 amThanks.Anyone else notice an improvement?

author: TheSteve
author_rank: Unknown
timestamp: March 24, 2017, 04:07:10 am
content: Speed seems fine so far, although I need to retrain my brain as to where the categories are. Moving the EEVblog specific stuff to the top totally makes sense. I feel bad for test equipment(my favorite area) being so close to the bottom though.

author: gnif
author_rank: Unknown
timestamp: March 24, 2017, 04:27:01 am
content: The re-sort was not intentional, it seems the ordering has been messed up in the upgrade, which is strange.Quote from: Nusa on March 24, 2017, 03:03:26 amSorry about that, the database was busy converting some tables we prior couldn't as the version of MySQL we were on did not support full text searches, which SMF uses quite extensively. There may be more periods like this in the coming few days as I identify other areas that could use improvement, I have already had to add several indexes to tables that SMF should have.The goal here is to prevent the 502 errors from occurring that people have been reporting.

author: BravoV
author_rank: Super Contributor
timestamp: March 24, 2017, 04:41:31 am
content: The old problem on corrupted attachment still exist, try examples below, look for user's post with photo attachments, and try to view them by clicking them.What interesting is, at some photo posts, even though they have photo thumbs, but once clicked, it wont pop out the full size photo.Examples :->https://www.eevblog.com/forum/chat/salvaging-old-electronics-parts-and-save-the-world/->https://www.eevblog.com/forum/beginners/how-to-better-wire-the-bnc-connector-and-ghz-issue/

author: gnif
author_rank: Unknown
timestamp: March 24, 2017, 05:13:27 am
content: I pulled the images in question directly from the server and verified the files on disk are corrupt, these posts are quite old and the issue that caused this may already have been corrected. If you see it on any new posts please be sure to PM me directly, this is the kind of think I want to know about ASAP before too much is corrupted.

author: BravoV
author_rank: Super Contributor
timestamp: March 24, 2017, 05:22:44 am
content: Quote from: gnif on March 24, 2017, 05:13:27 amOk, noted.Btw, just curious if you could run at background "crawling" for low impact, to check or verify attachment especially picture's files integrity, say like thru their unique header to see how much is the damaged, at least the last date when it was occurred, just an idea.

author: JPortici
author_rank: Super Contributor
timestamp: March 24, 2017, 05:38:35 am
content: did you move eevblog section 'up'?

author: Monkeh
author_rank: Super Contributor
timestamp: March 24, 2017, 05:46:35 am
content: So the forum's been running on MyISAM the whole time?Boy, does that explain a few things.

author: gnif
author_rank: Unknown
timestamp: March 24, 2017, 06:01:21 am
content: Quote from: Monkeh on March 24, 2017, 05:46:35 amNo it has not, most of the database was on InnoDB but a few of the tables contained text columns that were preventing them from being moved across to InnoDB due to the lack of full text search. The issue was the background maintenance tasks were performing a join with one of these tables, which would then cause a lock contention and stop general forum use. That lock would cause a massive backlog of traffic and a huge spike in CPU load, etc... It has involved a ton of testing to determine where the fault was before throwing "fixes" at the server.When you have a website that is as busy as this and no dev/test platform, you can't just to haphazardly upgrading packages, until now the remaining two MyISAM tables have not been causing an issue, but as load has gone up on the website it became a problem.There are a few things that are planned to improve website performance, the server is in need of some TLC when it comes to new/faster technologies, etc, it has just had to wait until I could ensure I had enough time to perform the changes and deal with any new problems that may arise. The up side is we get a faster website, the down side is there will be outages in order to perform this work.

author: Monkeh
author_rank: Super Contributor
timestamp: March 24, 2017, 06:04:09 am
content: Quote from: gnif on March 24, 2017, 06:01:21 amAh, well, that's why we have you to give the whole story. Still explains things.QuoteCalm, calm, not attacking you, just observing. You're not the only one in this line of business, and when most of the information on things goes through the Dave filter (no offence, Dave), it's hard to get a good picture.It's great you've found some time to give it the needed love, the ride's been getting a little bumpy.

author: gnif
author_rank: Unknown
timestamp: March 24, 2017, 06:05:44 am
content: Quote from: Monkeh on March 24, 2017, 06:04:09 amSorry I did not take your comments as an attack, I was just being explicit in detail for those that are not like you and do not understand why there is outages when this website is worked on.

author: EEVblog
author_rank: Unknown
timestamp: March 24, 2017, 06:24:08 am
content: Quote from: gnif on March 24, 2017, 04:27:01 amYeah, no idea why that happened!

author: TheSteve
author_rank: Unknown
timestamp: March 24, 2017, 06:29:35 am
content: It seems to have switched to alphabetic order:EEVblogElectronicsGeneralProducts

author: DimitriP
author_rank: Super Contributor
timestamp: March 24, 2017, 06:31:55 am
content: QuoteBumpy? Where ?Anytime I'm here during the last year and a half I rarely noticed anything wrong or bumpy (other than the very rare, occasional and not too often 502 bad gateway error)

author: gnif
author_rank: Unknown
timestamp: March 24, 2017, 06:44:16 am
content: Quote from: wilfred on March 24, 2017, 06:28:36 amThe issue is cost both financially and time, in an ideal world it would be setup very differently, most of my paying clients have fully redundant setups which makes this much easier, but they come with a pretty hefty price. Many people would be quite amazed at the time involved in setting this up, there are so many options, no one 'best' option, it depends on the requirements and available hardware. For example one of my larger clients (who allows me to be public about my involvement with them) have one of the largest phpBB3 forums on the internet, the forum hosting alone involves 6 servers, for redundancy and performance.

author: Richard Crowley
author_rank: Super Contributor
timestamp: March 24, 2017, 06:46:53 am
content: Wow, I have never seen a DB related problem, delay or error on this website. I wish other forums I frequent were as snappy as EEVblog. GearSlutz in particular has always been painfully slow. The only errors I see here are related to availability (mostly error messages from Cloudflare.)

author: gnif
author_rank: Unknown
timestamp: March 24, 2017, 06:48:36 am
content: Quote from: Richard Crowley on March 24, 2017, 06:46:53 amDepends on time of day, to be honest this morning at 4:00AM (ADST) was the first time I had ever seen the 502 error personally, which was a lucky break as I was able to inspect the cause while it was occurring.

author: Monkeh
author_rank: Super Contributor
timestamp: March 24, 2017, 06:57:35 am
content: Quote from: gnif on March 24, 2017, 06:48:36 amThe down(under)sides. You miss what must be much of the peak traffic down there in the future.

author: Brumby
author_rank: Unknown
timestamp: March 24, 2017, 07:14:26 am
content: Quote from: gnif on March 24, 2017, 06:44:16 amThere are some of us that do understand.I have no problems with outages - and I appreciate everything that is done behind the scenes.

author: gnif
author_rank: Unknown
timestamp: March 24, 2017, 07:15:14 am
content: You would think that since we are in the future we could stop these things before you people in the past see them

author: Brumby
author_rank: Unknown
timestamp: March 24, 2017, 07:17:33 am
content: I do love a dreamer....

author: gnif
author_rank: Unknown
timestamp: March 24, 2017, 07:40:18 am
content: It appears the ordering issue is a bug in SMF, it relies on the order of the records being returned from the database being the order they are stored, with the later version of MySQL the records are returned by the order of the first index (the primary key) by default. This bug was already fixed for the boards themselves, but not for the forum categories.I am hacking in a fix now and will submit a bug report to SMF to get this fixed.

author: EEVblog
author_rank: Unknown
timestamp: March 24, 2017, 07:44:10 am
content: Quote from: gnif on March 24, 2017, 06:44:16 am*hiding my company credit card*

author: EEVblog
author_rank: Unknown
timestamp: March 24, 2017, 07:45:18 am
content: I just got this error when going directly into the Test gear section. It only happened once.I think we have seen this recently.

author: gnif
author_rank: Unknown
timestamp: March 24, 2017, 07:46:19 am
content: Quote from: EEVblog on March 24, 2017, 07:45:18 amIt was me hacking in a order fix, a bug in the query, missing a conditional, ordering is correct now.

author: EEVblog
author_rank: Unknown
timestamp: March 24, 2017, 07:52:57 am
content: Quote from: gnif on March 24, 2017, 07:46:19 amI've always been amazed how well SMF just handled this stuff, like it's bullet proof.

author: EEVblog
author_rank: Unknown
timestamp: March 24, 2017, 11:02:16 am
content: Some major sever changes have been done which at the moment can be selectively turned on for individual users.If you'd like to help out testing this new upgrade please message or email gnif your IP address and it can be enabled.This will affect the main website and the forum.It's important to test it before it's enabled for everyone.Thanks.

author: Future soldier
author_rank: Newbie
timestamp: March 24, 2017, 03:18:42 pm
content: fitst post of this thread if quoted gives bad gateway errorhttps://www.eevblog.com/forum/contests/keysight-scope-month-feedback/idk the warning said not to report it but the problem seems to be of a specific post

author: Future soldier
author_rank: Newbie
timestamp: March 24, 2017, 03:20:46 pm
content: Quote from: EEVblog on March 24, 2017, 11:02:16 ami am willing to participate

author: Richard Crowley
author_rank: Super Contributor
timestamp: March 24, 2017, 03:25:07 pm
content: Quote from: Future soldier on March 24, 2017, 03:18:42 pmIt works fine for me (quoting it, etc.)."Bad Gateway" doesn't sound like any kind of database-related problem to me.

author: Future soldier
author_rank: Newbie
timestamp: March 24, 2017, 03:27:15 pm
content: Quote from: Richard Crowley on March 24, 2017, 03:25:07 pmso its a loclised issue i gues, i know bad gateway doesnt seem to forum issue

author: MatthewEveritt
author_rank: Unknown
timestamp: March 24, 2017, 03:58:57 pm
content: The gateway error you see isn't specifically database related, but the root cause of the issue is. It's a universal law that error messages can't be too helpful.I'm more than happy to test things. The IP I'm posting from covers 99% of my forum use.

author: PA0PBZ
author_rank: Super Contributor
timestamp: March 24, 2017, 07:37:56 pm
content: I noticed that the unread replies link takes lots of time to load, by lots I mean up to 30 secondshttps://www.eevblog.com/forum/unreadreplies/

author: T3sl4co1l
author_rank: Super Contributor
timestamp: March 24, 2017, 08:37:47 pm
content: Hmm, a bump in the road, just a moment ago (524 timeout), but it's fine now. Also had a couple 502s and outages the other day, but obviously, you Aussie bastards were doing the maintenance at local downtime, which is primetime for us... so inconsiderateDoesn't seem faster or anything, but my experience might be limited by network speed, and anyway, do whatever it is you need to do with the database to make it less resource intensive / more stable / easier to maintain.Tim

author: ProBang2
author_rank: Frequent Contributor
timestamp: March 24, 2017, 08:48:25 pm
content: I had only yesterday, for a few hours, the "Bad Gateway 502" error.Today was something strange, but with the EEVBlog main-site:Tried to load the main-site. Browser freezes for approx. 90 sec. with a blanc screen, only showing a (translated from german) "transmit data with apis.google.com". After that the site loaded as usual.What the hell is "apis.google.com"?Had anyone else this problem, too?Windows7 64BitMozilla Firefox 52.0.1 (32Bit)Adblocker deactivated onwww.wwvblog.com

author: Richard Crowley
author_rank: Super Contributor
timestamp: March 25, 2017, 03:09:23 am
content: Yes, the overall performance seems noticably slower. But that is the difference between 500ms and 1500~2000ms

author: JPortici
author_rank: Super Contributor
timestamp: March 25, 2017, 07:07:02 am
content: i noticed that tapatalk is very slow to load tooalso, the sections are in alphabetical order!

author: Nusa
author_rank: Super Contributor
timestamp: March 25, 2017, 07:09:37 am
content: Forum is feeling very sluggish at the moment.My IP is on the testing list if it matters.

author: andrewfernie
author_rank: Newbie
timestamp: March 28, 2017, 01:46:57 am
content: Quote from: JPortici on March 25, 2017, 07:07:02 amTapatalk has been unusable for me the last few days for EEvblog. Best case is that it is very slow to load, but most of the time I give up and go to a web browser. Same situation with iOS and Android.Andrew

author: gnif
author_rank: Unknown
timestamp: March 28, 2017, 05:08:24 am
content: Reports are no longer required of poor performance, we can clearly see the issue in the server's monitoring metrics. The root cause is that switching the remaining tables to InnoDB has blown out their size and they are unable to be cached in ram, as Dave stated we are getting a RAM upgrade after which there will be some tuning in order to take advantage of the additional RAM which should alleviate the issue.

author: JPortici
author_rank: Super Contributor
timestamp: March 28, 2017, 05:31:42 am
content: Quote from: andrewfernie on March 28, 2017, 01:46:57 amyesterday my new phone (sigh) arrived, it was a lot faster than the old one to load tapatalk pages.. fast enough to point the finger at the crappy wifi for slowness

author: andrewfernie
author_rank: Newbie
timestamp: March 28, 2017, 10:51:57 am
content: Quote from: andrewfernie on March 28, 2017, 01:46:57 amThat RAM upgrade last night seems to have resolved any problems I was having. All good now.

author: gnif
author_rank: Unknown
timestamp: April 03, 2017, 10:00:04 am
content: We are aware there seems to be an issue with a database query, I am investigating and will update shortly.

author: gnif
author_rank: Unknown
timestamp: April 03, 2017, 10:09:53 am
content: Quote from: gnif on April 03, 2017, 10:00:04 amProblem resolved, there was an bug in the database sort conditionals I hacked in last week.