NVidia GTX Titan Black: Anything more to check before I consider baking my card? - Page 1
Forum section: electronics / repair

author: brian27
author_rank: Newbie
timestamp: October 25, 2018, 08:37:30 pm
content: Hi all, greetings to all of you.I am new here. I have a bit troubleshooting skill and I do electrical engineering for living.I am stumbled on a dead NVidia GeForce GTX Titan Black (reference card, made by Asus, out of warranty of course, this card is almost 5 years old).This card decided to stop working without any prior sign.I never OC my card. The last time I used is just to measure worst case system temperature using OCCT stressing both CPU and GPU.I put my PC to sleep during stressing. When I resume my PC, Windows was frozen. I restarted and then the card was no longer detected. I used this in SLI configuration and only one can be detected.Tried to detect the card using NVFlash both in Windows or DOS environment, however, it says no NVidia adapter found.My quick PC spec:Intel Xeon E5-2680v2Asus P9X79 Pro2x GeForce GTX Titan Black32GB DDR3 ECCSilverstone Strider 1500W Silver SST1500The PCB of my dead GPU is reference NVidia P2083.I followed the debugging guide on this thread:https://www.eevblog.com/forum/repair/dead-graphics-card-780ti-dead-mosfet-perhaps/and found no shorted MOSFETs.Then I tried to power the card on and measure the output voltages. Surprisingly the GPU stays cool for about 35C on ambient temp of around 20C measured by a infrared thermometer.I measure the output core voltage on all the inductors: 0.85VAlso the output RAM voltage on the inductors: 1.52VAll shunts got the 11.85V powerI measure the BIOS ROM and it got 3.35V nicely.See attachment on the probing point and the voltage value.My question is, is there any chance that the BIOS got corrupted and not detected by the system? I never re-flash the BIOS or play anything with it.If everything else is correct, shall I consider baking the card or reheating with the heat gun? I found also the replacement chip in eBay with reasonable price. What's your suggestion on replacing the GPU chip?

author: janoc
author_rank: Super Contributor
timestamp: October 25, 2018, 10:09:37 pm
content: If you want to destroy the card for good, do "bake" it.Search this forum on what the "baking" does and doesn't do. If the card has died because of cracked solder balls or an underfill problem, "baking" it will "fix" it for only a short time at best. The only way to really repair that is replacing the chip but see below ...If the problem is something else - e.g. a dead power regulator or some bad capacitors then the "baking" won't do anything and you can only fry the card for good. What could have been a few bucks repair will now be an unrepairable mess and garbage bin material.TL;DR - "baking" anything except food is a stupid idea. It won't really fix anything and only can make things worse.Also this two videos explain well that this "reflowing" and "baking" is complete BS, despite geniuses like iFixit or Linus from Linus Tech Tips claiming otherwise.(warning, LOUD and a lot of swearing)Re GPU being cool - that's quite understandable because the GPU is not really doing anything, since the card isn't running. If it wasn't cool that would be a problem indicating a possible short (and fried chip).Re replacing the GPU chip - not realistically doable. You can't replace a BGA of that size with a heat gun. And having it done professionally using a BGA rework machine would cost more than any value that card may still have. Plus you have no guarantee that the chip from eBay is not counterfeit or broken, so you could end up with large labor bill and a still broken card.Re BIOS - could happen. If you have a second working card, you could try to extract the BIOS from that one and transplant it into the broken one. If it is just that it may fix it. Firmware corruption is not a very common issue but it could happen.Also, if you have a second working card, compare voltages between the two as well. The voltages you have measured look reasonable but who knows how that particular card was set up and the voltages could be higher/lower depending on how the manufacturer configured it (and some are likely software configurable too).

author: thm_w
author_rank: Super Contributor
timestamp: October 25, 2018, 11:50:36 pm
content: Quote from: janoc on October 25, 2018, 10:09:37 pmSo either it is complete BS, or it can work for some time, its one or the other not both.If all other troubleshooting options have been exhausted (eeprom, etc.), and replacing the chip means it is BER, then baking is a completely legitimate option to try.Good advice though.

author: cdev
author_rank: Super Contributor
timestamp: October 26, 2018, 12:56:03 am
content: Is this the card?https://www.videocardbenchmark.net/gpu.php?gpu=GeForce+GTX+TITAN+Black&id=2842

author: Rasz
author_rank: Super Contributor
timestamp: October 26, 2018, 03:22:52 am
content: check Q6 and that small IC next to it (afair its a 74 series logic gate array)you have 2 same cards? great, you can compare signals on all small transistors (q7 8 9 10 etc)

author: brian27
author_rank: Newbie
timestamp: October 26, 2018, 12:24:34 pm
content: Quote from: janoc on October 25, 2018, 10:09:37 pmIndeed. I am fully aware not in favor on baking my card, therefore I am looking for measurement point or things to check before baking.Problem is, the BIOS (and the card itself) is not detected by NVFlash on both Windows and DOS, thus I am considering buying a USB BIOS programmer with clip to do a standalone programming. Is this also giving a clue that assuming I got VRMs working and the card is not detected by NVFlash, then might be something bad in GPU / BIOS?Quote from: thm_w on October 25, 2018, 11:50:36 pmWhat do you mean by BER?Quote from: cdev on October 26, 2018, 12:56:03 amYes. This to be exact:https://www.techpowerup.com/gpu-specs/asus-gtx-titan-black.b2759Quote from: Rasz on October 26, 2018, 03:22:52 amMany thanks for your suggestion, I will try to measure the output voltage of each pins on q6 - q10 and the 74 logic family IC next to q6 tonight.However, I have some questions for my understanding, what does those transistor do? Is that FET or BJT? Do you know which type is it?Is the 74 IC also part of the control logic of these transistors?

author: Bud
author_rank: Super Contributor
timestamp: October 26, 2018, 01:27:53 pm
content: I baked my card once, it failed again when running a game. I baked it second time and now not running games on it but use it as a secondary videocard for general computing, as a secondary display for Altium and other CAD programs and to watch video on the TV connected to its HDMI port. It has been doing well ever since in this mode.Edit: I also used it for mining cryptocurrency after the second bake, it handled the load just fine.

author: janoc
author_rank: Super Contributor
timestamp: October 26, 2018, 07:45:32 pm
content: Quote from: thm_w on October 25, 2018, 11:50:36 pmIt is bullshit, because a fix is not something that "works for some time" (days, a few weeks tops).Baking is never a legitimate option to try. All that you will achieve if you heat the board below the melting temperature of the solder (that's what usually understood as "baking" as opposed to "reflowing") is that the underfill in the GPU melts.If you get lucky, the messed up bumps on the flip chip BGA will realign and the board will work for a short time again because the underfill will hold them place once it solidifies. That will work until the thermal stresses make it fail again because the underfill alone is not meant to ensure contact and you have done nothing with the cracked solder bumps (not solder balls under the BGA, the solder bumps that attach the flip chip BGA to the carrier!) by heating the board so low that the solder didn't even start melting. These tend to crack because of the thermal stresses the card is under during its life, together with some types of underfil, causing the problem. The only way to fix this is to replace the chip.This is what I mean when talking about the "bumps":Another thing that could happen is that you heat a failing capacitor and it will start working - until it cools down and fails again. Again, you haven't really fixed anything.Of course, while all doing this you could easily damage even an originally good GPU by prolonged exposure to heat (it is not designed to be kept this hot for that long!), so when the original problem is eventually found and fixed you will discover that the GPU is now messed up and you have artifacts on the screen or some other problem.If you actually heat the board so hard that the solder reflows, you will likely end up with parts falling off the board/knocked out of alignment - good luck fixing that.This is why "baking" anything is BS and not a fix. Of course, some people will do this and then sell a (temporarily) "working" card on eBay or Craigslist. Which is fraud, IMO, but difficult to prove unless it is obvious the board was "cooked". Or record a Youtube video showing how they baked their card and it works - but not that it has died on them again several days later ...

author: thm_w
author_rank: Super Contributor
timestamp: October 26, 2018, 08:29:54 pm
content: Quote from: brian27 on October 26, 2018, 12:24:34 pmBeyond economical repair - ie its cheaper to buy a new card than fix this one.Quote from: janoc on October 26, 2018, 07:45:32 pmYou say a few weeks tops, and yet there are hundreds if not thousands of people that baked their xbox's. Surely not all of them failed after a few weeks?Maybe there is some misunderstanding by me here, because OP said "bake card or reheat with a heat gun" which implied to me, reflowing the chip.

author: cdev
author_rank: Super Contributor
timestamp: October 26, 2018, 08:43:34 pm
content: Rather than the blunt instrument of heating the whole thing, and quite possibly destroying it, what I would do is try to find some documentation on the card (first) and then on similar cards generally, and then trouble shoot. As people here will tell you, more often than not when any piece of electronic equipment is broken, the chances of being able to repair it are often pretty good.If you can't find documentation, I would apply general rules to the situation.Trace the power, voltage regulation circuitry, etc, try to observe what it does do when power is applied, and see if there is anything that seems obviously wrong.If you have logs, look at them to see if the hardware is recognized at all at some low level. All working video cards will be recognized as a VESA card and supply a very basic functionality without any 3D acceleration. When the computer is booted, they will enumerate themselves in the logs.If its completely dead, that may indicate something simple is wrong. Part of the power circuitry for example.And so on.

author: Refrigerator
author_rank: Super Contributor
timestamp: October 26, 2018, 10:35:49 pm
content: I bake my cards, laptops, and even phones from time to time and if you do it right there's not much harm you can do.By the looks of it your card just crapped out from the stress test, happens all the time.If there are no shorted mosfets then nothing else but the GPU chip itself died.I'd say bake it, but before (if) you do check for any knocked off components, just in case you knocked one off while handling the card around.If you decide to stick it in your oven i can tell you how i do it, just a simple trick to get the temps right.BTW, i only bake my own stuff for myself, or maybe if my friend asks me to because it's a bit of a hit or miss in terms of reliability.For example i baked a SONY Vaio laptop and used it for two years for games and it's still working fine, baked HD7850 1GD5 and 2GD5 model and both work fine, baked my LG G3 about six times but that one would only come back to life for two months after the bake, there were also some laptops i baked that would come back to life but only for a short period of time, sometimes less than a week.Basically it's a bit of gamble but as a last resort it works pretty well.PS: don't heatgun your GPU, that's how i killed my G3, probably would have lasted a few more bakes in the oven had i not cooked it

author: brian27
author_rank: Newbie
timestamp: October 27, 2018, 12:12:57 am
content: Quote from: cdev on October 26, 2018, 08:43:34 pmFully agree. If at the first place I see artifacts on screen or 3D acceleration causes instability, or I cannot install the driver in Windows, I would rather say I'll bake it.However, this card is completely not detected under Windows, DOS (NVFLash) or even BIOS. It just behave like no graphics card attached to my system. Therefore I am curious either the supply circuitry or the BIOS itself gets corrupted.What do you mean by log? Do you have an idea how to log the enumeration during booting process?Quote from: Refrigerator on October 26, 2018, 10:35:49 pmInteresting, I thought heatgun will be more reliable than household oven.Quote from: Rasz on October 26, 2018, 03:22:52 amAllright, I find out that Q6, Q7, Q8, Q9 and Q10 are N-channel enhancement mode MOSFET.https://assets.nexperia.com/documents/data-sheet/2N7002.pdfI find that on my card, Q7 - Q10 has the same characteristic if I measured with the ohmmeterRds = 0.7kRgs = 10kRgd = 10.8kHowever for Q6:Rds = openRgd = openRgs = 100kIs this make sense. What to check in the 74 logic IC next to it?I have not got time yet to disassembly my other card that is working.

author: cdev
author_rank: Super Contributor
timestamp: October 27, 2018, 12:40:46 am
content: I don't know anything about Windows. No comprende Windowse! Sorry. Somebody else here can probably help. Remember that it may not complete the boot so you want to write it to some non-volatile storage medium, like a disk. Also and/or if you can log to another machine that may be useful too.

author: Rasz
author_rank: Super Contributor
timestamp: October 27, 2018, 04:11:36 am
content: Quote from: brian27 on October 26, 2018, 12:24:34 pmmonkey see monkey do - Iv seen that logic chip fail before, afaik that 74 is responsible for some enable signal, results in dead card when badQuote from: thm_w on October 26, 2018, 08:29:54 pm360 has old AMD GPU. Bad underfill was Nvidia blunder, and happened generation later. 360 was just a case of bad thermal design and new at the time rohs requirements resulting in bend pcb ripping/cracking BGA balls. It was the source of bake it bro myth.whole nvidia thing is writen up here, with some good comments from actual fab engineer in the commentshttps://www.theinquirer.net/inquirer/news/1004378/why-nvidia-chips-defective

author: wraper
author_rank: Unknown
timestamp: October 27, 2018, 05:34:23 am
content: Quote from: thm_w on October 25, 2018, 11:50:36 pmIt's BS because it's often presented as repair when it's a short term "fix" at best.

author: ebastler
author_rank: Super Contributor
timestamp: October 27, 2018, 05:41:09 am
content: Quote from: Refrigerator on October 26, 2018, 10:35:49 pmMaybe that's a hobby in its own right, but it is certainly not a repair...

author: Refrigerator
author_rank: Super Contributor
timestamp: October 27, 2018, 09:14:59 am
content: Quote from: ebastler on October 27, 2018, 05:41:09 amI never said that it is repairBut yeah, a hobby is what it is for me because i wanted to find out myself what happens when you bake stuff.And i did find out what baking is about, i also wasted a bunch of money in the processIt basically boils down to what you are trying to bake, for example, HD7850's take baking very well, the first one i baked has been working for well over a year now, handling PUBG and other games as well as it can.Haven't baked any Nvidia yet, though. Got a dead GTX550ti to try bake but that one turned out to just have a knocked off 0603 inductor and worked fine after i replaced it.

author: brian27
author_rank: Newbie
timestamp: October 27, 2018, 10:05:57 am
content: Quote from: Rasz on October 27, 2018, 04:11:36 amI've measured the Q6 and the IC next to it between good and bad card, however it is the same.The things that I noticed differs between the good and the bad card are all the Qs next to the RAm chip Q4 - Q5, Q7-Q10. In the good card, the gate has seen 3.3V. In the dead card, it stays at 0V.I have yet to find out where the control logic driving these FETs. Can it be driven by the GPU or by the logic IC?

author: cdev
author_rank: Super Contributor
timestamp: October 27, 2018, 12:58:19 pm
content: Where do you guys get your dead GPUs? Ebay? Miners? Dumpsters?