How to correctly route dual rank DDR3 chips (between the ranks)? - Page 1
Forum section: electronics / general

author: Vilius
author_rank: Contributor
timestamp: May 21, 2025, 07:04:27 am
content: Hi,It may not be the best-fitted forum branch for this question, but I could not find a better one (if there is, please excuse me and let me know)... AnywayI am designing a standard 240 pin DDR3 UDIMM ram stick (16 DRAM chips, each 256M x 8 organisation, configured in dual rank configuration) for learning purposes. I did quite a lot of research on the DDR3 architecture and PCB layout, but there are still some questions that I can not answer on my own.I know that on double-sided PCBs, it's preferred to have a via that connects the same pin on both (different rank) chips. But if you take it as a given, that you have two DRAM chips on the single side next to each other, what is the preferred routing style: fly by or T-branch?1. If the T branch is preferred, should I split the track right at the beginning of the UDIMM module, or should I split it right before the minimum inevitable track length that is required to match the time skew for both chips?2. If a fly by topology is preferred, is there any (approximate, experimental) maximum tolerable stub length with respect to the first daisy-chained chip?I add a little sketch for clarification. Thank you in advance.

author: asmi
author_rank: Super Contributor
timestamp: May 21, 2025, 01:41:29 pm
content: The short answer is - it depends on a memory controller's capabilities. DDR3 DIMMs by themselves are routed as fly-by with termination, so for 2 DPC it will have to be some sort of a tree.

author: Vilius
author_rank: Contributor
timestamp: May 21, 2025, 02:39:17 pm
content: Thank you for your reply, but can you expand a little bit on your answer?If I understand correctly (including my interpretation of JEDEC standard), there should be no termination resistors placed on the UDIMMs, because data lane signals are terminated by the ODT on the DRAM chips, and all the other daisy-chained signals, including CAS, RAS, WE, ADDRESS, etc., must be terminated on the motherboard (meaning that VTT pins on the UDIMM itself are unused). I am not arguing, but just wondering, what is the reason for branching the trace across the data lane ranks? Given that only one rank is actually used at a time (meaning that only one on die termination resistor must also be connected), in both T branch and fly by routing cases, I would get a piece of unconnected/unterminated trace beyond the termination resistor. Fly by in my case, would be considerably more convenient for routing, but I want to understand the physics behind it. What is the benefit of branching the data lane signals between the ranks over connecting them in fly by? If I am wrong about something, please explain it to me.

author: asmi
author_rank: Super Contributor
timestamp: May 21, 2025, 03:08:24 pm
content: Quote from: Vilius on May 21, 2025, 02:39:17 pmI did not work with UDIMMs specifically, but I did work with SODIMMs and I know for sure they have termination resistors for address-control signals. As for data lanes, like you mentioned, there is no need for termination due to presence of ODT. Also DDR3 memory devices contain a special ODT mode which is designed to allow multi-rank designs - you can read more about it in DDR3 memory device datasheet.

author: Vilius
author_rank: Contributor
timestamp: May 22, 2025, 08:57:58 am
content: So, I just want to clarify. All the pins (Address, Bank address, Clock, RAS,CAS,WE, Chip selects) must be terminated just like this? (Picture down below) Should the RESET, ODT and CKE be terminated?

author: asmi
author_rank: Super Contributor
timestamp: May 22, 2025, 12:54:09 pm
content: Quote from: Vilius on May 22, 2025, 08:57:58 amSODIMM has these termination resistors already, so you don't need them on your PCB. Not sure about UDIMM - you might want to check the datasheet.

author: Vilius
author_rank: Contributor
timestamp: May 22, 2025, 06:13:14 pm
content: Please correct me if I am wrong, but exactly do you mean by "has these termination resistors already, so you don't need them on your PCB"? My understanding is that termination resistors must be placed on the last daisy-chained chip pin, or very near to it, where the common signal trace ends. Where else can you possibly place termination resistor other than on PCB? ODT can only handle point-to-point connections, there is no mechanism for each DRAM chip to assess its position in the chain.

author: asmi
author_rank: Super Contributor
timestamp: May 22, 2025, 07:31:48 pm
content: Quote from: Vilius on May 22, 2025, 06:13:14 pmExactly what it sounds like - termination resistors are present on SODIMM itself.Quote from: Vilius on May 22, 2025, 06:13:14 pmAddress/control signals go from your PCB via connector to an SODIMM module, and on this module they are routed fly-by over each individual DDR3 memory device. After last memory device on SODIMM there is nowhere else for this signal to go, which is why they are terminated right on that SODIMM module. You can google SODIMM DDR3 schematics if you don't believe me. In my previous post I've attached a photo of one of PCB I designed, which has an actual SODIMM DDR3 module on it, so I'm talking from my own personal experience.Quote from: Vilius on May 22, 2025, 06:13:14 pmNot sure what ODT has to do with what we're talking about here, but in general this is not correct - there is a special mode (Micron calls it "dynamic ODT") which is designed to be used for multi-rank systems when there is more than one memory device connected to a single data lane.

author: Vilius
author_rank: Contributor
timestamp: May 22, 2025, 09:18:40 pm
content: Hmm... I am getting more and more confused after each post. Since I am designing only the UDIMM itself, it happens that saying ,,you dont need termination resistors on your PCB" isnt correct in this case - "termination resistors are present on SODIMM itself" that is my real task that I have to make it happen. So, I will terminate Address, Bank address, Clock, RAS,CAS,WE, Chip selects via 50 ohm resistors to VTT at the end of the last chip, but what about RESET, ODT and CKE, do they have to be terminated just like the mentioned signals, or they are just simple point-to-point connections?I am very grateful to you for sharing your knowledge. My replies might have seen somewhat rude to you, but that's only because of my lack of knowledge (not because of my intentions.) Majority of the information about DDR3 available online is based on way more simple cases (routing single, or max two DRAM chips to some kind of processor). Reference designs for UDIMMs virtually do not exist, at least I could not find any. I need to gather much more information and knowledge for my case, that's why I am here solving this puzzle one little piece at the time.

author: asmi
author_rank: Super Contributor
timestamp: May 22, 2025, 09:55:12 pm
content: Quote from: Vilius on May 22, 2025, 09:18:40 pmAaah OK now I understand that you are designing an actual UDIMM, and not a board which houses itAttached is a diagram of DDR3 UDIMM module with ECC, taken from Micron's datasheet. I think it should answer all your questions. Specifically, ODT and CKE needs to be terminated, RESET# does not.Also JEDEC publishes reference schematics and even PCB layouts for various DDRx modules, so you might want to check them out as well. You only need to register there, but it's free as far as I remember.https://www.jedec.org/category/technology-focus-area/memory-module-design-file-registrations

author: Vilius
author_rank: Contributor
timestamp: May 23, 2025, 01:27:30 pm
content: Thank you once again. Now I would like to get back to the original question about routing style for connecting different rank DRAMs. Earlier you said that eventually I will have to do some kind of PCB trace branching. Can you explain it in a bit more detail? Just like I said, it would make my PCB routing way more difficult, compared to fly-by style but if I am committing for it, I would like know more about the root cause of it.Also, there is one more side question that came up to me. I analyzed an old DDR3 UDIM stick with my multimeter. I found out, that every track including data lane and control bytes have a 15 Ohm resistor in series right at the beginning of the UDIMM stick. After some research I found out that the reason why series resistor might be placed on a control/byte signal is to reduce the ringing. That seems fair to me, but If I am terminating the control lines with 40-50 ohm resistor to VTT at the end of those, and I also add 15 Ohms in the beginning, I get a total series resistance of 60+ Ohms. Is this a real concern and I should not put any series resistors, or the classical Ohm's law and circuit theory does not directly apply to DDR3 speeds?

author: asmi
author_rank: Super Contributor
timestamp: May 23, 2025, 06:44:08 pm
content: Quote from: Vilius on May 23, 2025, 01:27:30 pmBranching might be required for 2DPC implementation, and it not of your concern at all. I would once again recommend looking at reference implementations.Quote from: Vilius on May 23, 2025, 01:27:30 pmThat is not how it works, and it's somewhat complex topic to explain in a single post, but basically the reason for all of this is signal integrity. Actual values for both termination resistors - parallel and serial ones can be determined by SI simulations (as first approximation), and then refined with signal measurements of actual prototype boards. Unless you have access to 6+ GHz oscilloscope, the only way to figure them out is to either rely on simulations, or to just guesstimate.

author: Vilius
author_rank: Contributor
timestamp: May 25, 2025, 09:31:54 am
content: After analyzing the provided datasheet about dual rank UDIMM, I noticed one strange thing. There are 64 DQ signal contacts in total on the UDIMM connector (this is normal and expected), but somehow there are 9 DQS pairs and DM pins, as if there should be 9 byte lanes in total (but DQ64/8 = 8 byte lanes). Chances are it's something really simple that I am missing, but I need some help or explanation. Initially I was thinking that it might be something related to ECC specifics, but as I far as I know, only the CB pins are for ECC memory....

author: asmi
author_rank: Super Contributor
timestamp: May 25, 2025, 07:09:42 pm
content: Quote from: Vilius on May 25, 2025, 09:31:54 amNope, ECC module contains 9 full byte lanes, with 9th being used for check bits, so like other lanes, it contains 8 data lines (CB0-CB7), a DQS diff pair (DQS8/DQS8#) and a DM8 line. This 9th lane is exactly the same as other 8 lanes, infact implementations are free to swap these lanes around as they see fit to facilitate layout.

author: Vilius
author_rank: Contributor
timestamp: May 25, 2025, 07:37:33 pm
content: So that means that if I use non ECC memory, I can omit (leave the UDIMM pins unconnected) one DQSx, #DQSx, DMx signal set (whichever I like) and use remaining DQS/DM freely to form the byte lanes without restricting my self to numbers?For example, I could form a byte lane from signals, that are marked on the UDIMM connector as DQ7-DQ14, DM6, DQS6, #DQS6?Thats part of the bit swapping possibility inside the lane, right?

author: asmi
author_rank: Super Contributor
timestamp: May 25, 2025, 10:43:02 pm
content: Quote from: Vilius on May 25, 2025, 07:37:33 pmNo, non-ECC DIMM contains only 8 lanes, and all of them are required to be connected to controller.Quote from: Vilius on May 25, 2025, 07:37:33 pmNo, DQ7-DQ14 belongs to lane #1, but DQS6 - to lane #6. You can't swap them like that, only entire lanes.