Embedded System with cache, how to know if read returns actual data or cache? - Page 1
Forum section: electronics / general

author: matrixofdynamism
author_rank: Regular Contributor
timestamp: June 13, 2022, 10:29:12 pm
content: This question is specifically about writing programs for Microblaze softprocessor and the Nios II soft processor. However, it applies to other processors as well. A cache memory is not present in low end microcontrollers. However, it will be present in 32-bit processors and can be added into systems built from soft processors.Embedded Systems contain peripherals connected to the main processor and these can be controlled by accessing specific registers present inside the address space of the processor.Now the question is, when we perform read, how do we know if the data is coming from cache or it is coming from the actual register of the peripheral we are trying to access?

author: langwadt
author_rank: Super Contributor
timestamp: June 13, 2022, 10:48:54 pm
content: isn't NIOS only instruction cache? anyway registers shouldn't be cached

author: Benta
author_rank: Super Contributor
timestamp: June 13, 2022, 10:50:16 pm
content: The cache controllers I've seen, there are memory areas that can be excluded from caching.

author: ataradov
author_rank: Super Contributor
timestamp: June 13, 2022, 10:51:22 pm
content: You should never set peripheral address range as cacheable. There is no way to know that value came from the cache, but if you register value ever does, you are screwed and need to redesign your system.

author: brucehoult
author_rank: Super Contributor
timestamp: June 13, 2022, 11:26:40 pm
content: The only way to know is by how long it takes. Cache exists because it is many times faster. But you must only enable caching for things that are not changed by external agents -- or where if an external agent changes the value then it tells your cache to refresh the value or (more likely) forget it.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 13, 2022, 11:38:57 pm
content: I find the question odd. Peripheral registers are usually exluded from caching for good reasons, so you actually know if an access is going to be cached or not from the address itself.The memory areas being cached may be fixed or configurable, depending on the implementation.But an access either goes through a cache or does not, and it only depends on the address as far as I can think of.And the access time is unfortunately not even a reliable indication of an access going through a cache or not. You just can't know. If the access is cached and the data is already in cache, the access time will be at full bus speed usually. If it isn't in cache, there will be a significant latency. If you're accessing a peripheral, the access time will depend on other factors. The peripheral bus may not be clocked at the same rate as the data cache, in which case a peripheral register access is often slower than accessing cached memory.But peripheral registers are almost NEVER cached anyway, so the original question sounds moot, unless it was purely rhetorical. If anyone can think of a valid case for caching peripheral registers, don't hesitate to chime in.

author: brucehoult
author_rank: Super Contributor
timestamp: June 14, 2022, 01:39:20 am
content: Quote from: SiliconWizard on June 13, 2022, 11:38:57 pmSure. I've seen I/O space cached so that the CPU can read or write buffers in an I/O device 32 or 64 bytes per cycle instead of a single word or byte each time. The driver code just has to make sure to explicitly flush each cache block when writing, and invalidate those addresses in the cache before reading.

author: ajb
author_rank: Super Contributor
timestamp: June 14, 2022, 03:00:58 pm
content: In addition to IO space being marked as noncached, some architectures may not even put the cache in the path between the IO space and the CPU. DMA controllers would typically be on the far side of the cache, so if data was pulled from IO space by DMA then it could not have been pulled from the cache. But if you attempt to read from the area the DMA controller moved that data *to*, then the CPU could be seeing cached data instead of what the DMA controller just wrote to that location. You'd have to look at the bus matrix for the part in question to spot these potential traps.Some devices have multiply mapped areas of memory, where you can access the same physical location from two or more different logical addresses, and those different addresses may be tagged differently such that one is cacheable and one is not--this is very useful for external buses that may connect to external memory or may connect to parallel peripherals. On other devices you may need to configure an MPU to ensure those areas are accessed properly.When you get into the higher complexity/performance levels of embedded processors it becomes increasingly important to understand how the architecture handles all of these things, and what guarantees it does or does not provide. You'll want the architecture reference manual on hand along with the datasheet and the MCU reference manual when you get past a certain depth.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 14, 2022, 05:40:51 pm
content: Quote from: brucehoult on June 14, 2022, 01:39:20 amI've never seen that myself and using a DMA for that almost always wins in terms of performance and less chances of shooting yourself in the foot. But who knows. I'm sure there are tons of weird (or creative, depending on the context) architectures out there.Also, depending on the architecture, the cache controller(s) may have access to the whole address space, or only a chunk of it. While on some CPUs, you have to explicitely exclude the IO area (when you don't want it cached, which again is the most usual) in the controller's configuration, on some others (which is what I personally do when using soft cores in FPGAs), the IO area (and possibly others) can't physically be accessed by the cache controller, and that's it.So anyway, again back to the OP's question, whatever area of memory is cached or not, that's something you know from the architecture and possibly the configuration of the cache controller (when it's configurable), so there is no need to guess. And again, if the question is how to detect that at run-time directly from the accesses themselves (and I don't see the point but that may be a rhetorical question), it's pretty much impossible generally speaking.

author: woofy
author_rank: Frequent Contributor
timestamp: June 14, 2022, 08:41:14 pm
content: I don't see any way to sensibly cache I/O space. If I want to read a peripheral, say an I/O port, or a UART Rx register, or wait for a bit in a status register, then the actual register needs to be read.Caching I/O space seems pointless. None of the cached chips I've used cache I/O space, only instruction and data spaces.PIC32MX (MIPS) for example places SFR's in KSEG1, the non-cached segment.

author: matrixofdynamism
author_rank: Regular Contributor
timestamp: June 14, 2022, 11:11:43 pm
content: Thanks everyone, the main part of my question is clear now. However, ajb has raised an interesting point.QuoteNow in a scenario where DMA has moved data to another location in the memory, and then we perform read from that location but end up getting data from (outdated) cache memory instead, how would we know that such a thing has happened and what is one supposed to do then?

author: westfw
author_rank: Super Contributor
timestamp: June 15, 2022, 01:13:20 am
content: Lots of microPROCESSOR systems will put IO devices in the memory space, and require some significant care to access them properly in spite of caching (and sometimes paging, too.) This can be done (sometimes) by marking regions/pages as unchached/unmapped, or by explicitly bypassing the caching with explicit "flush" or "disable caching" operations, depending on the whole chip architecture (MPUs tend to have more explicit cache/pager manipulation capabilities than MCUs.)The point is that you don't "know" if your reads return cached data, you TELL the system that you want fresh data and real reads.

author: SiliconWizard
author_rank: Super Contributor
timestamp: June 15, 2022, 01:41:21 am
content: Quote from: matrixofdynamism on June 14, 2022, 11:11:43 pmThat would largely depend on the architecture. In many cases, the operation is, thankfully, transparent to the user: the DMA controller will instruct the cache controller that the area has become invalid and must be read again. On more advanced architectures, this could be even more transparent performance-wise: the cache may get updated transparently while the DMA is operating, so that the cache controller wouldn't need to waste time later on reading the same area in cache again.

author: ajb
author_rank: Super Contributor
timestamp: June 15, 2022, 04:33:15 am
content: Quote from: matrixofdynamism on June 14, 2022, 11:11:43 pmYou would know that's happened if you look at the reference material for the device or architecture and see that the DMA controller is not connected to the cache... possibly after scratching your head for a while over why your software isn't working.There are generally two ways to avoid this problem. One is to invalidate the cache after the DMA transaction has finished but before the CPU reads the result. This can be tricky if the cache--or at least the lines that need to be invalidated--is dirty, containing data that has not yet been written back to memory. The other way is to use a non-cacheable area of memory as the target of the DMA transaction. If the device has a configurable MPU, you can use it to define that area at whatever size and with whatever tags you need. You can either use a direct pointer to that address space to access it, or add a matching section in your linker script and assign the required objects to it when you declare them.

author: brucehoult
author_rank: Super Contributor
timestamp: June 15, 2022, 04:59:36 am
content: Quote from: ajb on June 15, 2022, 04:33:15 amThe 3rd way is to use a cache-coherent DMA engine.