FPGA Clock (setup slack) issues. - Page 1
Forum section: electronics / general

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 20, 2019, 01:38:11 pm
content: Hi all,Further to the CPU I have been working on, I've moved from SRAM to SDRAM (cost being the controlling factor), If I build the CPU on it's own, all is fine, it can run at a max of 50Mhz no isses. If I build the SDRAM controller on it's own it can work at the required 133Mhz.The primary incomming clock is 133Mhz and this is dropped down to 44.333333333Mhz using a PLL for the CPU.The CPU is halted (internal clock cycles) if the SDRAM is either pending a read, pending a write or pending initialisation.I've selected all the clocks as primary clocks and defined their appropriate frequencies. Of course, the router makes it own choices based on what I suggest and comes up with:The following 3 signals are selected as primary clocks :CP1/clk0 (driver: SLICE_749, clk load #: 344; quadrants: TL/TR/BL/BR)CP1/decodeClk (driver: CP1/SLICE_963, clk load #: 51; quadrants: TL/TR/BL/BR)clock_c (driver: clock, clk load #: 64; quadrants: TL/TR/BL/BR)WARNING - The following 1 data signal has to use primary clock resource:cpuClock (driver: dc; quadrants: TL/TR/BL/BR)The following 1 signal is selected as DCS clock :cpuClock (driver: dc, clk load #: 0; quadrants: TL/TR/BL/BR)The following 3 signals are selected to use the secondary clock resources :CP1/flagsDirectRead_15__N_188 (driver: CP1/SLICE_158, clk load #: 0, sr load #: 27, ce load #: 0)CP1/ir_direct_out_0 (driver: CP1/SLICE_402, clk load #: 0, sr load #: 16, ce load #: 0)CP1/loadOperandB (driver: CP1/LCTRL/SLICE_415, clk load #: 0, sr load #: 0, ce load #: 24)Signal reset_c is selected as Global Set/Reset.I would have put cpuClock in the mainlist of clocks but it never appears as a valid clockNet to select for setup.I'm using Lattice Diamond by the way.My problem is that when routing it all out I find I have a -30us setup slack and I'm trying to figure out a way to resolve this. Do any of you have any ideas or suggestions on how to resolve this issue??Start NBR section for re-routing at 14:34:13 08/20/19Level 4, iteration 10(0.00%) conflict; 0(0.00%) untouched conn; 29721127 (nbr) score;Estimated worst slack/total negative slack<setup>: -22.216ns/-29721.128ns; real time: 1 mins 25 secsThanks for any help.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 20, 2019, 02:46:36 pm
content: There may be some info missing, and I haven't thoroughly looked at what you gave here, but just an initial thought.Your design doesn't look like having horribly long logic paths (but I haven't seen how your CPU is implemented), and with a negative slack of this order (-30µs?), this kinda smells of a clock domain crossing issue.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 20, 2019, 04:15:38 pm
content: Quote from: SiliconWizard on August 20, 2019, 02:46:36 pmFYI,The CPU:The SDRAM Controller:Each will route on their own with their respective speeds fine, it's when I connect them together with the SDRAM Controller running at 133Mhz and the CPU running at 44.33333Mhz.I'm wondering if it's somehow to do with the data/control line connections between them??? Is the CPU trying to respond at 133Mhz speeds on some of those bits?? Not sure how the router handles all this side of it.

author: mfro
author_rank: Regular Contributor
timestamp: August 20, 2019, 04:19:51 pm
content: Quote from: Joanna_H on August 20, 2019, 04:15:38 pmHow did you synchronize clock domain crossing between the two?

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 20, 2019, 04:27:01 pm
content: Quote from: mfro on August 20, 2019, 04:19:51 pmThe incomming clock is 133, the cpu clock is 44.333333, the cpu clock is basically divide by 3 and uses a PLL , it is in phase with the incomming 133 clock.The cpuClock can take a slow pulse 1hz to 10Mhz controll clock or the 44Mhz generated clock, controlled and sync'd using a DCS.The cpu will only look at data lines on it's own clock, which is 1/3rd of the SDRAM clock max.The internal cpu clocks will not toggle unless the SDRAM controller has told the CPU that is has completed the action it is performing, effectively the SDRAM controller can tell the CPU to halt as it's busy. The cpu only samples the halt line on the pos/neg edge of the incomming raw cpu clock.The cpuclock inside the CPU is divided into two, one inphase with the cpuClock and one 180deg out of phase. One controls instruction decode and setup, the other controls instruction execution.

author: mfro
author_rank: Regular Contributor
timestamp: August 20, 2019, 05:36:54 pm
content: Quote from: Joanna_H on August 20, 2019, 04:27:01 pmThat's good.I guess the failing path is a CDC from the 133 MHz clock domain to the 44 MHz clock domain?Then you need to make sure you transfer data on one of the clock edges of the fast clock where the slow clock has a chance to catch it.You could either count clock cycles on the fast clock side and only transfer data on a clock edge where the slow clock can see it (but then you might run into hold time violations on the other side) or - if possible - make sure you keep the data stable for several clock cycles and define a multicycle timing exception.As the negative slack is nearly a full cycle in the slow clock domain, it might also help to slightly offset the period by a few degrees

author: asmi
author_rank: Super Contributor
timestamp: August 20, 2019, 05:45:30 pm
content: Quote from: Joanna_H on August 20, 2019, 04:27:01 pmI'd suggest you to use FIFOs to cross clock domains - they are more reliable and they don't require any particular relationship between clocks. All SDRAM/DDRx controllers I've ever seen (and developed) use this approach - there is a command FIFO and two data FIFOs - one for read data, and another one for write data.

author: asmi
author_rank: Super Contributor
timestamp: August 20, 2019, 05:48:32 pm
content: Quote from: SiliconWizard on August 20, 2019, 02:46:36 pmThe only time I've ever seen that kind of slack was when I attempted to use HDL's builtin division operator, which Vivado happily turned into a crap ton of logic

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 20, 2019, 08:56:46 pm
content: Quote from: asmi on August 20, 2019, 05:45:30 pmYes, I've never used Lattice SDRAM controllers though, so I don't know about them. But for Xilinx ones, that certainly is true. You can even have several ports, each with command, read and write FIFOs, and all can be accessed from different clock domains.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 20, 2019, 09:02:48 pm
content: Just a note that is unrelated to your issue.Unless you implement a cache system of some kind to improve things on average, if you use the SDRAM controller to directly write individual words at random addresses as you would with a SRAM, you will get nowhere near the max possible throughput of the SDRAM chip. At 133MHz, you will probably not even get close to the CPU clock (~44MHz).

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 21, 2019, 04:06:54 am
content: I think I'll just go back to my known working SRAM version for now. These CDC issues are driving me insane.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 21, 2019, 02:41:48 pm
content: I've just taken a very quick look at Lattice's SDRAM controller IP, and it seems much simpler than what Xilinx offers. So you may indeed have to handle the CDC yourself. Not that it's very hard to do, but if you just start with Verilog (or any RTL for that matter), I can understand cumulating all that can be a bit intimidating. It will be interesting to learn how to use SDRAM in your project later on though. As I said, you will ideally have to also think about some kind of cache for RAM access. So, one thing at a time.I don't know what size you're after, but if you're looking for fast SRAM chips that are still not too atrociously expensive, you could consider this: IS61WV102416DBLL-10TLI (recent product, 1Mx16, 10ns access time). Obviously still a lot more expensive than SDRAM, but it will be much easier to use and you'll get up to 200MByte/s easily.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 21, 2019, 03:08:31 pm
content: I was actually intendeding to use their 1Mx16 10n asycn chips, I liked how they worked in the datasheets, nice and simple.

author: asmi
author_rank: Super Contributor
timestamp: August 21, 2019, 05:17:15 pm
content: Quote from: Joanna_H on August 21, 2019, 04:06:54 amYou will have to learn how to deal with them sooner or later. Pretty much all external interfaces run on their own clocks, and use some sort of FIFO to get data into and out of FPGA. Personally as a rule I always create dedicated clocks for external interfaces so that I can adjust them (or main internal system clock) without affecting external chips (some of those require very specific frequency and will not work if you provide any other one).Actually, designing your own SDRAM controller would be a good project for beginner in it's own right. Unlike some more modern memories like DDR3, SDRAM does not define minimum frequency so you can run it slower than rated speed if you want, you will just have to make sure timing for refresh command is met as otherwise you may lose memory contents - although some experiments show that in reality you can stretch this time VERY far and still have contents intact if you're not running the memory at some extreme conditions like in +70C ambient temp. If you've got logic analyzer, you can even observe signals to/from memory chip in real time if you run it slow enough - though you might want to use logic analyzer IP core for you if your FPGA vendor provides one.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 22, 2019, 06:28:39 pm
content: The worst thing about all this is.. 1) I forgot to take a backup before I modded it.. 2) I removed the SDRAM and everything stopped working.. 3) I went mad trying to work out why the CPU was failing and tweaking things... 4) I finally realised that I had an inout as an input only.... 5) I've forgotten how my orignal system even worked.... lol

author: asmi
author_rank: Super Contributor
timestamp: August 22, 2019, 06:36:54 pm
content: Quote from: Joanna_H on August 22, 2019, 06:28:39 pm"There are two kinds of IT professionals - those who are making backups, and those who will be making them"

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 22, 2019, 06:37:59 pm
content: Damn.Time to start using a version-control system. Now.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 22, 2019, 07:22:59 pm
content: Quote from: asmi on August 22, 2019, 06:36:54 pm30yrs of software dev, backup all the time... 2 weeks of playing with verilog... and ermm.. forgot... lol

author: langwadt
author_rank: Super Contributor
timestamp: August 22, 2019, 07:52:05 pm
content: why not use a single clock and a clock enable for the slower parts?

author: hamster_nz
author_rank: Super Contributor
timestamp: August 22, 2019, 10:54:30 pm
content: Quote from: langwadt on August 22, 2019, 07:52:05 pmBacause the complexity of the logic in the slow domain has to be low enough to clock at the speed of the fast domain....

author: langwadt
author_rank: Super Contributor
timestamp: August 22, 2019, 11:05:25 pm
content: Quote from: hamster_nz on August 22, 2019, 10:54:30 pmwhat do you mean? using clock enable instead of dividing a clock is totally standard

author: Someone
author_rank: Super Contributor
timestamp: August 22, 2019, 11:31:36 pm
content: Quote from: langwadt on August 22, 2019, 11:05:25 pmYes, but the multi cycle timing constraints to allow deep logic layers on the virtual "slow" clock is usually more complex and error prone than crossing domains.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 23, 2019, 04:56:40 am
content: Well, finally got it all working again... LOLDecided to focus on Microcode development for now and finish off the CPU, then come back to the memory issues. Using a formal SRAM simulation model from IDT (10ns)So, a quick test program:And the simulation results:And defining the microcode is nice and easyYeah, that PC to Internal address flag isn't needed on the step 3 of the load with index. Just hadn't removed it when I took the screenshot...Even the branch is easy to define.And Step 0 is the same on all microcode, it's just the instruction fetch routine.At least some progress was made

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 23, 2019, 02:51:57 pm
content: Nice, but are you using a VCS yet?

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 23, 2019, 04:03:53 pm
content: Quote from: SiliconWizard on August 23, 2019, 02:51:57 pmYeah, decided to just use Git as I had it installed.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 23, 2019, 06:40:37 pm
content: Still cannot remember how I was handling my clock before hand, but I had it so that I could get 40Mhz with a 10ns read for microcode and a 10ns read/write for memory.... One on the neg edge, one of the pos edge. Totally detroyed all that codeThis version crams everything into the negedge of the clock (read microcode, setup control lines, read/write memory based on setup) and then latches the result on the posedge of the clock, effectively giving half a cycle wasted. So instead of messing it all up again I just changed the duty cycle (via PLL) of the primary 40Mhz clock so that I can have 1/8's pos and 7/8's negProblem solved and a 1.875ns leeway still.Strangely enough this also gives me a higher max speed on the CPU of 160Mhz, originaly on the posedge/negedge design the theoretical maximum was 50Mhz. So if I replaced that 10ns SRAM with faster, fun timesSo turns out to actually be a happy accident.

author: asmi
author_rank: Super Contributor
timestamp: August 23, 2019, 07:57:49 pm
content: I've never seen any designs that use negedge of clock for any user logic (aside from DDR/SERDES stuff). All my designs always run off posedge, because when your signals transition from posedge-driven logic to negedge-driven one, the slack available to the P&R is only half of clock period, instead of full period in case of logic driven by the same clock edge.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 23, 2019, 10:02:46 pm
content: Quote from: asmi on August 23, 2019, 07:57:49 pmI'm setting up all the control lines/etc on the negative edge, then I'm latching any results into the registers from this on the positive edge. In the design one clock cycle will execute one micro instruction.

author: langwadt
author_rank: Super Contributor
timestamp: August 23, 2019, 10:14:31 pm
content: Quote from: Joanna_H on August 23, 2019, 10:02:46 pmbut why?

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 23, 2019, 11:03:28 pm
content: Quote from: langwadt on August 23, 2019, 10:14:31 pmIf I setup the control lines on the posedge and latch any results on the posedge I'd get nonsense in each register as what is on the databus (external or internal) would become corrupted as it tried to read the control lines in for the next microcode instruction. I wanted it to have a full microcode execute in a single clock cycle so that when it's manually stepped through in slow mode it's one press, one cycle.

author: langwadt
author_rank: Super Contributor
timestamp: August 23, 2019, 11:59:24 pm
content: Quote from: Joanna_H on August 23, 2019, 11:03:28 pmno, the whole concept of synchronous logic in an FPGA, nothing can happen instantly so while both things happen on the rising edge, the latching will always happen a fraction before the change

author: NorthGuy
author_rank: Super Contributor
timestamp: August 24, 2019, 01:57:38 am
content: Quote from: Joanna_H on August 23, 2019, 10:02:46 pmDo you mean you have something of that sort:pos-edge flops->neg-edge flops->combinatory logic->pos-edge flopsIf so, your neg-edge flops don't do anything useful other than shortening useful time for your combinatory logic. You can repace this with:pos-edge flops->combinatory logic->pos-edge flopswhich will do all the same, but will run a little bit faster.

author: mfro
author_rank: Regular Contributor
timestamp: August 24, 2019, 06:04:08 am
content: What you did with your "rather unorthodox" CPU clock is basically the same as a multicycle constraint would have done.Instead of allowing the fitter to ignore close by clock edges (which is what multicycles essentially do), you just do not generate them.I have never seen that and frankly, it astonishes me that this works at all. Congratulations for thinking out of the ordinary.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 24, 2019, 09:12:46 am
content: Quote from: mfro on August 24, 2019, 06:04:08 amlol, FPGA/etc is all new to me. I'm just finding solutions to problems, I'll look above and see if there are better solutionsAlso time to go and learn what a Multicycle constraint is.....

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 24, 2019, 09:47:37 am
content: My goal is a CPU that has the essential core components inside it, that you can program the microcode off board and then load in at powerup or reset (from an SD card, Compact Flash, etc.) You get a system where you get to define all the microcode, to create your own instructions, whatever they are, etc. You will also have the ability to either run this thing at full pelt (40+Mhz, depends on SRAM speed) or run it from a variable speed clock (from 1hz to maybe 1-5Mhz) so you can watch it do things. If you want you can even issue one clock cycle pulse at a time and just step through the microcode execution. Any initial program to be executed would again by read from the storage and placed in memory.There would be a display mechanism showing what the contents of every bus/register was in the CPU on the PCB so that you can see it do stuff as it runs. Probably just a small LED display to show the contents on screen. The CPU has a monitor output and monitor index input, so you poll through all idx numbers to get all the data, less pins required that wayThis is used in the simulation to get all those register/bus values shown on screen.I also intend later to put in some form of graphics output that will let you output something to a screen, so part of any memory will be a frame buffer that will adressable by the CPU. There also will be a programmable timer that you can use to trigger interupts, etc. etc.Note, all the SRAM is Async and one of my contraints is that an entire microcode must complete in one clock cycle to allow the step through mode to work in a sensible way, don't want to have to keep pressing the button many times to just run one micro instructionAs for the current issue, what I'm trying to have happen is:1) On the posedge, all registers, etc of the CPU need to latch in whatever the result of the microcode instruction that just finished executing was.following this:2) If the current microcode instruction had the end of microcode line set jump back to step 0, which is always fetch instruction, otherwise increment the microcode step to the next microcode instruction.3) Then push this new microcode address (just built from flags, instruction and step counter) onto the microcode address lines and wait for the control lines to be read (10ns if I store all microcode in a small SRAM.)4) Once these control lines have been loaded in they will set up control lines on registers/ALU/etc in the cpu and potentially setup a read/write to memory (10ns to complete in this case.)5) Loop back to 1.I found that if I did all this on the posedge then the new microcode control lines started to setup (well garbage as it had started a new read) for the next mc instruction and any result on the internal or external data bus got trashed as all the control lines are changing. At the same time the CPU tried to latch but there was nothing to latch as it had started to process the load for the next microcode instruction. I had latch result and load mc happening at the same time as both triggered off posedge of main clock. So I just thought.. hmm.. need a bit of time... so I told it to latch the result of the execution on the posedge and then to start the next execution on the negedge a tiny amount of time later.Perhaps, I'm completely missing something here.....This is a logical block representation of what my current cpu is:

author: hamster_nz
author_rank: Super Contributor
timestamp: August 24, 2019, 12:13:43 pm
content: Quote from: Joanna_H on August 24, 2019, 09:47:37 amUnless I am misreading your intentions, you will face quite a few problem.How wide is your off-chip SRAM? You are only going to have very limited microcode word length.Also the 10ns SRAM is the latency of the SRAM chip. You are never going to be able to run very fast if you have to wait the few ns for the address signals to get out of the FPGA I/O pads, to the SRAM, then for the 10ns SRAM latency, then you have the get resulting data back into the FPGA, (maybe 18ns so far) and then to latch the returned data and finally act on the microcode, You are looking at about a 40ns or 50ns cycle time (20 to 25MHz).But a far bigger issue is that you are confusing what is easy and convenient for you, with what is easy and convenient for the H/W to do. You want to work with the H/W unless you want to be constantly redesigning. For example:- wanting to use mixed clock edges- not using on-chip RAM for microcode- trying to do it all in one cycle rather than pipelining and stalling, as some things will take more than one cycle - bit shifting, multiplying, division, jumping to interrupts, calls/returns, conditional jumps.- writing to the regfile and immediately reading the same value from that regfile is going to be slower/more complex than you expect- the limitations imposed by memory bandwidth on overall performance.- the time required for the SRAM data bus turnaround required to go from reads to writes need to be taken into accountAll of these are stacked against you at the moment.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 24, 2019, 01:12:48 pm
content: Quote from: hamster_nz on August 24, 2019, 12:13:43 pmWill be using multiple SRAMS's to create the control line outputs, same address fed into each one, each creating a portion of the control word.Quote from: hamster_nz on August 24, 2019, 12:13:43 pmBit confused on this, I'm using the fully routed VHDL file (including delay file) created by Lattice for simulation that should take into account all I/O pin delays, etc. I'm also using a formal memory simulation file from the manufacturers that includes all the required delays for read/write/etc. I'm not seeing these extra 20+ns that you talk about, am I doing something wrong here?EDIT : Simulation was not loading the SDF file even though it was told to in the Design settings, had to force it to using $sdf_annotate, added it to the generated HDL file :/ Active-HDL being used for simulation. Yep, with that in place things have really stopped working now, just a big screen of garbage.... So yeah, everything you said is true and nothing now works.. Need to completely rethink all this.I will say that it is pretty bad bug that even though it's all setup to use the SDF file, it doesn't actually use it, giving you a false sense of confidence that your design is working when it probably is not. After doing some digging, this a known bug that they have just done nothing aboutFor a beginner, i.e me, this could be crippling, you build it all, you test it all, you push it to your chip, and bang... nothing works.Wanted to give a big thank you for bringing this up as it got me thinking, realising the simulation was wrong and figuring out that nothing is really working. Saved me from wasting weeks and weeks of wasted future effort.Quote from: hamster_nz on August 24, 2019, 12:13:43 pmI didn't use the onboard EBR as it just wasn't big enough. To get something big enough I end up having to pay silly prices for the FPGA. I'm using the SRAM (for microcode) as one big lump of virtual combinatorial logic. You give it an address (so far 16bit) and it gives you a load of control lines (so far around 34, probably end up being closer to 48+) I understand this will use more memory that really necessary, with flags being part of the address, etc.. So you end up duplicating the microcode for all the flag combinations when they are not required. But it's a simple solution for generating the control lines for each microcode step.Quote from: hamster_nz on August 24, 2019, 12:13:43 pmI understand this, I wasn't going to include MUL/DIV in the ALU as I know they take many cycles or just slow everything down to nothing.Interupts/calls/etc, yeah I get they are multi cycle as they need to push/pop the core values onto the stack, PC especially. As for the Calls (JSR) etc.. I was going to put most of that as the responsibility of the programmer, they must push all the registers to the stack before a call (or the first thing the subroutine does is this, a programmer design choice here), a return must pop all the values back before returning. The CPU will only pop/push important things like the program counter.Interupts will only push/pop the PC. If an interrupt wants to be non destructive, the first thing it should do is push registers on the stack, when it wants to return to wherever it was it must pop all the values back. Same as calling a subroutine.I was also thinking of having an onboard EBR stack just for PC push/pop so that if the programmer messed up their subroutine or interupt handler and forgot to pop the stack the PC would still be in the correct location on a seperate stack. So the CPU would continue as normal.Quote from: hamster_nz on August 24, 2019, 12:13:43 pmI do get that I'm creating myself quite a challenge.

author: NorthGuy
author_rank: Super Contributor
timestamp: August 24, 2019, 02:50:01 pm
content: Quote from: Joanna_H on August 24, 2019, 09:47:37 amFPGA simply connects hardware pieces together. They don't do things in sequence, as would be the case of CPU instructions. Everything happens at the same time.The variables are actually registers - flip-flops. They work very simple. As the clock edge comes, they get whatever it is on the input and pass it to the output. This value is then latched on the output until the next clock edge. In 'real' world, you can buy an IC which implement one or more flip-flops, and you can observe as they work.Other important element is combinatory logic. It takes one or more inputs and produces an output. It does this all the time - non stop - as inputs change, the output changes too. This is not related to any clock. An example of combinatory logic is an inverter. It has a single input. It outputs inverted values - if input is '0' it outputs '1', if input is '1' it outputs '0'. In 'real' world, you can build an inverter from a transistor and a pull-up, or you can buy an inverter IC. Inverter is the simplest. Combinatory logic may be very complex and is represented by operations in your VHDL.When you write a simple statement within a clocked block, such as:Code:a <= b;You connect the output of the 'b' flip-flop to the input of the 'a' flip-flopIf you add an operation, you also add combinatory logic, such as:Code:a <= not b;Here you connect the output of 'b' to the input of an inverter and then you connect the output of the inverter to 'a'.To see how it all works, consider a simple example with a single register and an inverter:Code:a <= not a;Note that this is within a clocked statement, so there is a clock which is used to clock the 'a' flip-flop.Here we have an inverter which is connected to both the input and output of the the 'a' flip flop. The flip flop outputs a value, which is then inverted by the inverter and goes back to the input of the same flip-flop. If the flop output is '0', the inverter sends '1' to the flop's input.Now the clock comes. The flip flop latches whatever was on its input and passes it to the output. Inverter inverts this, so the output changes as well. And it is going to stay this way until the next clock. The result is very simple - every clock edge the output inverts, so you get '0' - '1' - '0' etc (which is a square wave at half the clock frequency). The important thing, nothing changes between clocks.In the real world there are delays. A flip flop doesn't latch the input immediately, there is an ouput delay (about 300 ps on modern FPGA). Inverter also has its delays. Also FPGA interconnets and buffers add delays. Therefore, when the clock comes, it takes some time until the change propagates back to the flop input. In case of a simple inverter, the overall delay from the clock egde may be 1 ns or so. It is much longer if you have complex combinatory logic.So, here's how it transpires in time:0 ps: clock egde comes to the flop300 ps: The flip flop latches the input and produces output600 ps: The output from flip flop reaches inverter700 ps: Inverter changes its output1000 ps: The inverted signal reaches the input of flip flopFlip flop needs some time to absorb the change. This is called "setup" time. Say, it's 250 ps. Once this all has happened, the system is ready for the next clock egde. If the clock edge comes earlier, flip flop may latch a wrong value, and everything is messed up. So, we must make sure that there is at least 1250 ps between clock edges, which means the circuit works fine at clocks up to 800 MHz, but may glitch if the clock is faster.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 27, 2019, 02:38:20 am
content: Well, finally got it working againRather than fighting the hardware I just implemented a simple state machine in the control logic that does it step by step on each clock cycle. Will probably just pump in a high frequency clock and wait the required number of cycles for each operation now that I can see the correct delays, etc. I think it would make my life a lot easier for talking to SDRAM/DDR, etc... as in the control logic it can just wait for the required ack from that memory when data is ready.Currently testing it with a fixed speed per step in the state machine which is not very efficient. Especially as I know some steps require one or two nanoseconds, some 10-20ns, mem reads, etc.It's also allowed me to see that my monitor system can only poll every 15ns or whatever it get's is complete garbage.Thanks to everyone for the advice and suggestions and especially the warnings.If anyone thinks the state machine approach is still wrong, please say so ... LOL

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 28, 2019, 12:32:52 pm
content: Sometimes this stuff is so so annoying.. a 154ps hold time violation during simulation ... Buts seriously ‭0.000000000154‬ seconds! lol The simulation still seems to work correctly, but I am concerned.

author: NorthGuy
author_rank: Super Contributor
timestamp: August 28, 2019, 01:03:23 pm
content: Quote from: Joanna_H on August 28, 2019, 12:32:52 pmHold violation means that a signal arrives to a flop too soon, while the flop is still working on latching the signal from the previous clock edge. Hold violations may be cured by adding delays.ps is very typical. ns usually means you did something bad.

author: langwadt
author_rank: Super Contributor
timestamp: August 28, 2019, 01:55:38 pm
content: Quote from: NorthGuy on August 28, 2019, 01:03:23 pmistm that things are usually designed to look like zero hold so you never have to be concerned about being too fast, so if you get hold violations you are doing probably something sketchy with a clock

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 29, 2019, 12:10:58 pm
content: Well after playing for a bit with the state machine approach, I'm starting to see it's limitations, it's REAAAALLLLYY slow.....I need to stop thinking like a sequential software engineer (30yrs of engrained concepts!) and start thinking like a hardware engineer......Think I'm gonna rethink quite a few things and move to a pipeline with handshake to allow me to fully utilise everything. What is the point in sitting there reading memory, executing an instrunction, etcc. all sequentially. I could be reading in the instruction, decoding the instruction, executing the instruction, etc. And doing all those things together for different microcode steps in parrallel. Well, at least I'm learning things which was my intention.So time to get that pen and paper out again and start redesigning how the internals work, how the control logic works, etc. Makes more sense that carrying on with this currently messed up architecture.

author: NorthGuy
author_rank: Super Contributor
timestamp: August 29, 2019, 01:02:32 pm
content: Quote from: Joanna_H on August 29, 2019, 12:10:58 pmTo get a feel of what you're dealing with, take something simple, for example a 32-bit counter (counter is a great candidate for this), and create it by directly instantiating LUTs and flops (as if you're building it from ICs). This is not something you would normally do, but this will shift your mind from CPUs to hardware. Then take your counter and re-design it to run at the highest clock frequency your FPGA can support. This will give you an idea of pipelining and what it takes to make things run really fast.

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 29, 2019, 03:04:56 pm
content: Quote from: Joanna_H on August 29, 2019, 12:10:58 pmThe state machine is not really the problem. The problem is whatever it does for each state. You obviously need to reduce the length of logic paths. A keyword: pipeline.Pipelines and state machines are not mutually exclusive.When dealing with parallelism in general (also applies to software on some level), "sequential" gets a bit more complex as a concept. But even with heavily pipelined stuff, you're still going to have some kind of sequential flow (you will still need some operations to perform in a fixed order). So "sequential" per se is not the problem either here.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 29, 2019, 03:45:39 pm
content: Quote from: SiliconWizard on August 29, 2019, 03:04:56 pmI know, it's really the way I'm thinking about things. I know what I want to do, I have a concept in place, but I'm making a lot of conceptual mistakes as I keep thinking sequentially where things should be thought of in parrallel. It's my own fault. So I'm going to sit back, think about it all and then reapproach the problem and put together what I wish to achieve. I can modify and reuse some of what I have done, some will need to be changed. Still just to get this far in a week or so has been fun

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 29, 2019, 03:57:38 pm
content: Thinking parallel is not trivial. You can read this for a start:https://en.wikipedia.org/wiki/Classic_RISC_pipeline

author: asmi
author_rank: Super Contributor
timestamp: August 29, 2019, 04:17:59 pm
content: Quote from: Joanna_H on August 29, 2019, 12:10:58 pmI highly recommend you to read the book "Digital Design and Computer Architecture ARM Edition". There is a number of pitfalls that people tend to get into when they go about designing processing cores, and this book does a great job of navigating readers through them. It also explores some important concepts like cache - what are different types, how they work.Typical issues with pipelined processing cores are data hazards and control hazards.First type happens when you need some data item that it still somewhere in the pipeline and wasn't committed to destination yet - think about sequence a = 1; b = a + 2; here the value of a will still be somewhere in pipeline at the time second command comes in. Another variant of this hazard occurs when there is a command to load something from memory and use it immediately after it - again, the loaded value won't actually be known at the time second command executes.Control hazards are related to conditional branch operators - the fetch stage can't possibly know if the branch will be taken or not (as this will be determined only at execute stage), so it's got to continue fetching commands assuming one way or another, and the control unit will have to deal with consequences of that assumption turning out to be the wrong one.All of that is also covered in the book, so again - can't recommend it more.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 29, 2019, 04:50:00 pm
content: Quote from: asmi on August 29, 2019, 04:17:59 pmThanks, I was just reading all that from the link that Silicon just posted, lots to think aboutI shall look for that book, again thank you

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 29, 2019, 04:53:07 pm
content: I'd also like to say thanks to everyone, only joined a few weeks ago. I'm really appreciative of how amazingly helpful everyone is. A great forum.

author: asmi
author_rank: Super Contributor
timestamp: August 29, 2019, 05:17:33 pm
content: Quote from: Joanna_H on August 29, 2019, 04:50:00 pmYea, there is a lot of things to think about. For example, consider the situation when you have a command that reads from or writes to the memory, and your fetch stage needs to fetch the next command from the same memory in the same cycle

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 30, 2019, 08:38:49 pm
content: Quote from: asmi on August 29, 2019, 05:17:33 pmLOL.. Evilllllll....

author: SiliconWizard
author_rank: Super Contributor
timestamp: August 30, 2019, 08:44:36 pm
content: Quote from: asmi on August 29, 2019, 05:17:33 pmJust don't mix code and data memories. Done.

author: Joanna_H
author_rank: Regular Contributor
timestamp: August 30, 2019, 10:31:54 pm
content: Quote from: SiliconWizard on August 30, 2019, 08:44:36 pmNon mixed code and data, that's Harvard if I remember correctly...

author: Joanna_H
author_rank: Regular Contributor
timestamp: September 01, 2019, 11:07:07 am
content: Well, that recommended book has been ordered, looking forward to reading that. In the meatime I've been working on better core internals. So far, managed to created a configurable register file that uses <10% of the resources I did previously and runs at a max speed of 833Mhz. Optimised the ALU so again it uses a fraction of the resources while keeping the existing functionality, it's speed has increased from 140Mhz to 833Mhz. So little steps in preperation for what is to come.

author: langwadt
author_rank: Super Contributor
timestamp: September 01, 2019, 11:28:30 am
content: Quote from: Joanna_H on September 01, 2019, 11:07:07 amuntil you get the book this might interesting,http://www.fpgacpu.org/xsoc/xr16.html

author: asmi
author_rank: Super Contributor
timestamp: September 01, 2019, 10:17:34 pm
content: Quote from: Joanna_H on August 30, 2019, 10:31:54 pmThat's one way of doing it. But there are other ways, for example implementing a cache - either separate I/D ones with single port access to each, or implement a shared cache with dual-port access. Most FPGAs have native dual port BRAM, so you can implement either approach.

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 01, 2019, 10:21:21 pm
content: Quote from: Joanna_H on September 01, 2019, 11:07:07 am833MHz on an XP2? That sounds kind of fishy, even for just a few registers.Don't get too excited over this figure as it will not mean much in your final design.

author: Joanna_H
author_rank: Regular Contributor
timestamp: September 02, 2019, 07:05:53 am
content: Quote from: SiliconWizard on September 01, 2019, 10:21:21 pmNot too worried, just making sure that the timings for each component are not going to be an issue down the line. In the end I can only use the 5, 8 or 17 versions as the worst thing I'll be able to handle is a PQFP208. I'm assuming I'll end up on a 17 (speedgrade, not sure yet) Using the BGA's is not really an option for me.I did look at Xilinx. But all their new stuff is BGA only.I'm just going through each part individually, making sure it's not going to be a bottleneck, reducing critical paths, etc. Thinking about long-term design issues such as how I'm going to divide the pipeline stages, to keep them fairly balanced (where possible), etc. How I'm going to deal with interlocking. Am I going to use scoreboarding or combinatorial (or something I haven't learnt yet) for RW/WR/WW detection, etc. Am I going to stall for the entire execution or use feed forward, etc. Looking into caching, instruction/data seperation, etc. Basically just planning ahead...Yep, I've been doing a lot of research/learning over the last few days.I'm also thinking about memory control, I/O mapping, how to simplify the control logic, how simple (or complex) an instruction set do I want to support, how much I want to implement in hardware and how much I want the microcode to handle, what extra support the microcode would require if it does complex instructions at that level (counters, branches, etc [such as required for the rep prefix on the x86]), etc.The only major decision I have made, which is a change from the original design, is to move to a 3 address format instead of 2 address in my ISA. So I need to make sure the hardware can handle this. My core goal of a CPU that can support simple and complex microcode levels is still there, the user implements their microcode and the hardware supports it.As for the registers running at that speed:Report Summary------------------------------------------------------------------------------------------Preference | Constraint| Actual|Levels----------------------------------------------------------------------------| | |FREQUENCY NET "clk_c" 300.000000 MHz ; | 300.000 MHz| 869.565 MHz| 0| | |----------------------------------------------------------------------------Code:module RegisterFileV2(clk,d,q_a,q_b,w_idx,q_a_idx,q_b_idx,w_en);parameter DATA_WIDTH = 32;parameter R_IDX_BITS = 5;localparam REGISTER_COUNT = 2 ** R_IDX_BITS;input clk;input [DATA_WIDTH-1:0]d;output [DATA_WIDTH-1:0]q_a;output [DATA_WIDTH-1:0]q_b;input [R_IDX_BITS-1:0]w_idx;input [R_IDX_BITS-1:0]q_a_idx;input [R_IDX_BITS-1:0]q_b_idx;input w_en;reg [DATA_WIDTH-1:0]register[REGISTER_COUNT-1:0];assign q_a = register;assign q_b = register;always@(posedge clk)beginif(w_en)beginregister <= d;endendendmodule

author: Joanna_H
author_rank: Regular Contributor
timestamp: September 02, 2019, 09:42:34 pm
content: One thing I am slowly figuring out is that trying to create a CPU that has a multistage microcode (CISC) that doesn't map directly to the (effectively RISC) pipleline stages is going to be a nightmare. So my goal of having user customisable microcode that allows CISC/RISC implementations on the same core may not actually be cleanly (or efficiently) possible

author: Bassman59
author_rank: Super Contributor
timestamp: September 03, 2019, 06:52:57 pm
content: Quote from: Joanna_H on September 02, 2019, 07:05:53 am0 logic levels might mean that everything was optimized away, and there is no logic left at all!

author: asmi
author_rank: Super Contributor
timestamp: September 03, 2019, 09:07:25 pm
content: Quote from: Joanna_H on September 02, 2019, 07:05:53 amThis is starting to feel like some kind of phobia. BGAs with reasonable pinch are nice and easy to work with, there are absolutely no rational reasons to be afraid of them. Take Spartan-7 in FBGA-196 package - this package was specifically designed to be routed out on 4 layer boards, which are very cheap nowadays. The FPGA chip itself will likely cost more than the board for it

author: Joanna_H
author_rank: Regular Contributor
timestamp: September 03, 2019, 09:25:17 pm
content: Quote from: asmi on September 03, 2019, 09:07:25 pmOh.. nah, just don't have hot air gear or the stuff to reball if I mess up.Hence going for pin based choices.

author: asmi
author_rank: Super Contributor
timestamp: September 03, 2019, 09:29:13 pm
content: Quote from: Joanna_H on September 03, 2019, 09:25:17 pmI don't have any reballing equipment either - never stopped me from using BGAs. Will buy one when I need it, but that time is yet to come. Didn't I say that BGAs are easy?I'm yet to encounter any soldering defect with them, while QFNs, for example have much worse track record... I'd take reasonable-pitch BGA over QFN any time.Hot air gun is extremely useful for desoldering of anything (even simple SMD resistors are much easier to remove with it than soldering iron), and they are fairly cheap as well.

author: Joanna_H
author_rank: Regular Contributor
timestamp: September 03, 2019, 09:35:08 pm
content: Quote from: asmi on September 03, 2019, 09:29:13 pmThanks for the advice, I'll look into it further then and do more research.

author: asmi
author_rank: Super Contributor
timestamp: September 03, 2019, 10:28:06 pm
content: Quote from: Joanna_H on September 03, 2019, 09:35:08 pmI've been itching to design a simple breakout board for that FPGA specifically for beginners like you so that you'd have something to work off of. Something with just a bare minimum components required to run the chip, plus perhaps some kind of external memory (SRAM or SDRAM - these are two options available in non-BGA packages), and the rest of pins just broken out on a 2.54mm header. I would even do it in KiCAD to make modifying the board easier if someone wants to add some other peripherals. Would you be interested in that kind of project, and if so - which memory would you prefer?The bare minimum required to run S7 FPGA is:- 2 DC-DC converters (1.0V/2.5A for the core, 1.8V/0.5A for PLLs and such) plus 3.3V for broken out pins - that one is not required to run FPGA, but most users would prefer to use 3.3V logic for external pins- decoupling caps- QSPI flash for bitstream- connector for external programmer. I use the one from Digilent which is like $60, but it's a one time purchase that you can use for all Xilinx FPGAs and SoCs.That is not a lot of parts, so it should be fairly accessible to a lot of hobbyists.

author: langwadt
author_rank: Super Contributor
timestamp: September 03, 2019, 10:43:46 pm
content: Quote from: asmi on September 03, 2019, 10:28:06 pmreadily available,http://shop.lambdaconcept.com/home/15-xilinx-fpga-spartan-6-xc6slx16-sdram.htmlalso many on ebay

author: asmi
author_rank: Super Contributor
timestamp: September 03, 2019, 10:47:32 pm
content: Quote from: langwadt on September 03, 2019, 10:43:46 pmThis isn'tSpartan-7part that I was talking about. And the whole purpose of a project is to have source files available for anyone to modify and make their own boards with whatever additional peripherals they want or need. Non of pre-made boards have that.

author: Joanna_H
author_rank: Regular Contributor
timestamp: September 04, 2019, 09:08:45 am
content: Quote from: Bassman59 on September 03, 2019, 06:52:57 pmIt's just using the RAM in each slice, not much logic needed

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 04, 2019, 02:56:26 pm
content: Quote from: Joanna_H on September 04, 2019, 09:08:45 amThis is why the given max frequency doesn't mean much here: it's basically just the inverse of the min delay to access the RAM itself, independently of any additional logic (obviously), but also of ANY internal routing...

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 04, 2019, 03:01:36 pm
content: Quote from: langwadt on September 03, 2019, 10:43:46 pmNote that, even though I don't quite know who is the original author of those boards, it appears to me that it's QMTECH:http://www.chinaqmtech.com/PRODUCTSThey got cloned A LOT all over the place (but the original ones are marked QMTECH).I've bought quite a few QMTECH boards on Aliexpress, they are pretty cheap and reasonably designed.They don't have just Spartan-6 boards; they also have Spartan-7, Artix-7, Cyclone IV, V and 10 boards. The Artix-7 (35T) board has 256MiB DDR3 RAM and is like $50. Great deal for prototyping stuff.

author: langwadt
author_rank: Super Contributor
timestamp: September 04, 2019, 03:21:48 pm
content: Quote from: SiliconWizard on September 04, 2019, 03:01:36 pmyeh, on digikey the fpga alone is $35