I am lost with the PCI bar size - Page 1
Forum section: electronics / general

author: DiTBho
author_rank: Super Contributor
timestamp: February 05, 2023, 09:58:17 pm
content: The Intel' PCI specs says you can discover the size of a BAR by writing all ones to it and reading it back, which tells us how many bits of the BAR are hard-wired to zero.Right?So when the firmware does it and you see size=0 ... is it likely a hardware defect? of the PCI card? of the PCI controller?And, can it be safely ignored?

author: magic
author_rank: Super Contributor
timestamp: February 06, 2023, 11:25:57 am
content: Prolly one of:1. everything reads as zeros - the card is badly inserted, controller bug, software bug, etc.2. you are again dealing with some proprietary obsolete shit which was never meant to work with anything but its corresponding proprietary obsolete shit drivers.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 06, 2023, 06:08:35 pm
content: If this is the Commel MP-6421 module (1106:3249), PCI SATA+CF, based on VIA VT6421A (datasheet), it behaves per PCI 2.2, and runs (both PCI and SATA) at 3.3V. It uses a 128-byte (64-word) SPI EEPROM, and to configure PCI properly, the PCI Base and Limit registers at offsets 0x70 and 0x74 need to be programmed, as they default to – no surprises here – zero.Section 6.2.5 of the PCI Local Bus Specification Revision 2.2 describes how the BIOS (power-up software) should set up the BAR(s). Quite likely your BIOS or its equivalent does not do that, so you'll have to do it in software. In Linux, that would be very early, between base PCI init (because you want PCI bridges to be initialized) but before device probing; so most likely in arch-dependent code.If you run Linux on a machine that does have hardware 3.3V PCI support, but no BIOS that initializes PCI devices correctly, and have the MP-6421 module installed, then the VIA SATA driver should error out indrivers/ata/sata_via.c:svia_init_one()with error "invalid PCI BAR # (sz 0, val 0)", or similar (if not earlier). Again, the fix is to set those registers somewhere very early in PCI init, most likely in arch-specific code.However, I could be completely wrong here. I do not have much experience with the hardware/pre-driver side of PCI (or PCIe) devices, so I'm basing this analysis partly on guesses (based on the MP-6421 we discussed in PMs), and the rest on what kind of scenario could lead to PCI base and limit being detected as zeroes in the Linux driver.

author: DiTBho
author_rank: Super Contributor
timestamp: February 07, 2023, 12:02:56 am
content: Quote from: Nominal Animal on February 06, 2023, 06:08:35 pmyeah, that kind of hack, but at the moment I'm fighting with a close-source firmware called "RouterBOOT v2.8" since I don't know how it initializes the PCI, and it looks * whatever it's doing * it's doing it wrong.I did something similar for a PowerPC embedded card, but I was able to replace IBM's proprietary firmware with u-boot, so I was able to write all the pre-initialization of the specific device.(I prefer to have that part inside the bootloader, so I don't have to fight with the Linux Kernel when I have to "move" my code form kernel v4.* to kernel v6.*)Anyway I'm not a PCI expert, what I wrote was a simple pre-initialization for a pATA controller that bootstraps in "legacy-LBA" mode and needs a couple of commands to switch to "PCI" mode.I mean, I know how to write machine-specific code in the first section of the Linux kernel, but I still don't know exactly what I have to write, and I wish I could reprogram the FLASH with u-boot instead of RouterBOOTAll of this, in order to add a bloody sATA disk to a router

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 07, 2023, 04:48:04 am
content: Quote from: DiTBho on February 07, 2023, 12:02:56 amYep, that is a real pain in the butt. It should be something like corebootsrc/device/pci_device.c:pci_get_resource(), but isn't.Latest working RouterOS version for RB532A is 6.32.4, so you might wish to download that and update RouterBOOT to that version.Looking atOpenWRT bootlog on RB532A (kernel 3.10.34):- pci 0000:00:00.0 is the IDT PCI root- pci 0000:00:02.0 is VIA Rhine VT6106S Ethernet- pci 0000:00:03.0 is VIA Rhine VT6106S Ethernet- pci 0000:00:04.0 is Lite-On WLAN card (mini-PCI)- pci 0000:00:05.0 is Lite-On WLAN card (mini-PCI)Only the IO BARs get assigned, none of the memory BARs do. (The third Ethernet port "korina", PCI 011d:0214, is integrated on the SoC.)The SoC itself is IDT (Renesas) 79RC32H435 (Mouser datasheet). It seems to have a perfectly functional 32-bit PCI 2.2 bus, but the datasheet mentions its local memory/IO controller has a 26-bit address bus.Augh. I only realized when writing this post, thatarch/mips/pci/*rc32434.calso belong to the RB532A kernels, and italreadyhas PCI fixups for the korina ethernet port, writing the korina configuration registers (similar to PCI V2X registers on VT6421A)! So, essentially, you need to add your own PCI fixups toarch/mips/pci/pci-rc32434.c:rc32434_pcibridge_init(), similar to theforloop that copies the 25korina_cnfg_regs[]entries to the corresponding PCI addresses. I am not 100% sure, but looking at the drivers/ata/sata_via.c for VT6421A, it looks like it only needs IO bars for the driver to work.In other words, this board does not have a "BIOS" to properly initialize anything PCI, and it has to be done in the Linux kernel (or whatever you want to run).(It looks like VT6421A is able to store its configuration defaults (including PCI V2X registers) in EEPROM, up to 63 8-bit changes to the defaults, in words/byte pairs where upper byte indicates the register offset, and lower byte the new value. If the EEPROM includes pairs 0x70xx and 0x71xx, it defines the PCI V2X base address; if the EEPROM includes pairs 0x72xx and 0x73xx, it defines the limit, and so on. See the VT6421A datasheet for details.)

author: DiTBho
author_rank: Super Contributor
timestamp: February 07, 2023, 09:54:07 am
content: Quote from: Nominal Animal on February 07, 2023, 04:48:04 amumm, "RouterBoot" is the "bootloader", while "RouterOS" is a ... dunno-what-it-is you need to buy ... perhaps a license to use it? it's a "close-source OS" of which I don't even have the admin passwd, all I know is that commercial binary blob goes into the onboard flash, and here we have an other problem with the bootloader:Code:trying bootp protocol..... OKGot IP address: 192.168.1.4resolved mac address 00:02:8A:26:B2:1Ctransfer started .......................................................... transfer ok, time=2.80ssetting up elf image... kernel out of rangekernel loading failedboooooom, "kernel out of range", means you cannot bootstrap a kernel bigger than 7700000 bytenand-bootcf-boottftp-bootthree methods to boot, all fail the same wayI have already updated my builder accordinglyCode:/src/machine/mips/rb532/kernel/5.4.128 # ./mybuild-2023 2kernel-5.4.128(mips/rb532)toolchain(mipsel-unknown-linux-gnu:2.34/9.3.0)app_name = kernel-5.4.128-32bit-rb532machine_note = revB-experimental-ath9kwifi-korina-rhine-pciserialqualified_host = { macmini-intel, LaptopT220 }-----------------------------------------------gadget_to_do(build) ... overlay/buildgadget_to_do(clean) ... kernel/cleanapp_machine=rb532 ... [MIPS32R2/LE]building kernel ... success-----------------------------------------------app_size =7840728 byteapp_size_max=7700000 bytechecking size ... panicmodule=profile/do/kernel/cleanreason="kernel size too big for the firmware"kernel out of range, it's tooooooooooooo big for the firmwareSo, as emergency-plan, I am developing a kind of elf application, just to- initialize the CPU (minimal)- initialize tlb and ram (with a window of 16Mbyte)- initialize the eth0 (korina)- load a kernel from udp/ip into ram- flush cache and prepare tlb- jump into kernel_startThat crazy stuff is less than 1Mbyte and it will go into the flash "OS" partitionflash partition#0 RouterBOOT (don't touch it, or you will brick the router)flash partition#1 RouterOS <------------ my app will go here@reset -> RouterBOOTRouterBOOT -> load app.elf from Flashapp.elf (my bootloader) -> load kernel.elf from net, then jump into $start___ OMG, why life is so complex?I would most likely throw the router away and buy a modern one ...... anyway, this is the plan for 2023

author: DiTBho
author_rank: Super Contributor
timestamp: February 07, 2023, 09:57:59 am
content: p.s.I am back to kernel-v4.4.197.yup, older, but it's smaller and more stable

author: DiTBho
author_rank: Super Contributor
timestamp: February 07, 2023, 10:14:46 am
content: kernel-v4.4.197Code:PCI: Initializing PCIPCI host bridge to bus 0000:00pci_bus 0000:00: root bus resource~ [mem 0x50000000-0x5fffffff]pci_bus 0000:00: root bus resource~ [io 0x18800000-0x188fffff]pci_bus 0000:00: root bus resource~ [??? 0x00000000 flags 0x0]pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]pci 0000:00:00.0: reg0x14'size is zero, drivers/pci/probepci 0000:00:00.0: reg0x18'size is zero, drivers/pci/probepci 0000:00:04.0: BAR 0: assigned [mem 0x50000000-0x5001ffff]pci 0000:00:04.0: BAR 1: assigned [mem 0x50020000-0x5003ffff]pci 0000:00:04.0: BAR 6: assigned [mem 0x50040000-0x5005ffff pref]pci 0000:00:02.0: BAR 0: assigned [io 0x18800000-0x188000ff]pci 0000:00:02.0: BAR 1: assigned [mem 0x50060000-0x500600ff]pci 0000:00:03.0: BAR 0: assigned [io 0x18800400-0x188004ff]pci 0000:00:03.0: BAR 1: assigned [mem 0x50060100-0x500601ff]pci 0000:00:04.0: BAR 2: assigned [io 0x18800800-0x1880083f](stripped out to minimal)

author: DiTBho
author_rank: Super Contributor
timestamp: February 07, 2023, 12:28:35 pm
content: Code:rb532 / # lspci00:00.0 Non-VGA unclassified device: Integrated Device Technology, Inc. Device 000000:02.0 Ethernet controller: VIA Technologies, Inc. VT6105/VT6106S [Rhine-III] (rev 86)00:03.0 Ethernet controller: VIA Technologies, Inc. VT6105/VT6106S [Rhine-III] (rev 86)00:04.0 Ethernet controller: Intel Corporation 82541GI Gigabit Ethernet Controller (rev 05)00:04.0 is a miniPCI i82541 single-channel ethernet card, using both PCI_IO and PCI_memCode:00:04.0 Ethernet controller: Intel Corporation 82541GI Gigabit Ethernet Controller (rev 05)Subsystem: Intel Corporation PRO/1000 MT Network ConnectionFlags: bus master, 66MHz, medium devsel, latency 64, IRQ 142Memory at 50000000 (32-bit, non-prefetchable) [size=128K]Memory at 50020000 (32-bit, non-prefetchable) [size=128K]I/O ports at 18800800 [size=64] Expansion ROM at 50040000  [size=128K]Capabilities:  Power Management version 2Capabilities:  PCI-X non-bridge device

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 07, 2023, 05:01:47 pm
content: Quote from: DiTBho on February 07, 2023, 09:54:07 amYes, I know. A RouterOS license isincluded with the RB532A board. The bootloader ispaired with the RouterOS.Again, the latest version supported without issues by RB532A is 6.32.4, so I don't believe it would cost you anything (except time and internet connection).Essentially, you'd have to installrouteros-mipsle-6.32.4for the duration of upgrading the bootloader, and then reinstall your preferred OS. (The documentation says there is a way to upgrade only the bootloader via a serial cable ––"Note: RouterOS cannot be upgraded through serial cable. Using this method only RouterBOOT can be upgraded. "–– but the link does not exist in their Wiki. Might be worth investigating, though; the bootloader serial is 38400,8,N,1 as you know, and the option in bootloader config is 'g'. The bootloader should be able to boot a Linux kernel over Ethernet (TFTP), too.)I am not sure if it would help with the issues you have, though. It's just what I'd do (update the bootloader to the latest good version) to eliminate possible sources of problems, one by one.Quote from: DiTBho on February 07, 2023, 09:54:07 amNow you know first-hand why I claim software engineering hasn't really progressed in the last two or three decades: we have justmorecode now, requiring more resources than before.As to the elf application, you could also use a minimal kernel (and minimal userspace on initrd or mounted read-only), and usekexecto load and switch to the actual kernel.Quote from: DiTBho on February 07, 2023, 10:14:46 amAside from the harmless "pci_bus 0000:00: root bus resource~ [??? 0x00000000 flags 0x0]", it seems to me the hardware is now detected and mapped correctly, and you have solved all the PCI issues you had.

author: DiTBho
author_rank: Super Contributor
timestamp: February 07, 2023, 06:17:49 pm
content: nah, it works with the miniPCI 82541GI Gigabit Ethernet card, but not with the VIA-VT6421A card nor with the SIL3112 card.Unlucky with any tested sATA cardThese two cause such strange behavior during PCI probing that you need to do a hardware reset (or unplug the power supply) to resume.I would like to find and buy an EHCI miniPCI with a NEC chip. I'm 99% sure it will work without any problems with the rb532A!

author: DiTBho
author_rank: Super Contributor
timestamp: February 07, 2023, 07:26:11 pm
content: Despite what's written in the doc, my board is brand new but doesn't have RooterOS installed.The FLASH is empty.I have already downloaded the "routeros-mipsle-6.32.4.npk" file, but not yet understood how to use it.I tried a simple "firmware upgrade via ethernet", the tftpboot server sends the file to the router, but it doesn't work because the router complains the file-ID doesn't match something.Code:trying bootp protocol... OKGot IP address: 192.168.1.41resolved mac address 00:02:8A:26:B2:1Ctransfer started ......................................................... transfer ok, time=2.70sinvalid upgrade file idThe other methods require RooterOS installed and running.So, I will contact Mikrotik, and ask what to do, and the change log.Meanwhile, found and bought a new rb532A board, this time it comes with RouterBOOT-v2.9.We make progress, from RouterBOOT-v2.8 to v2.9

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 07, 2023, 08:49:14 pm
content: Quote from: DiTBho on February 07, 2023, 06:17:49 pmRight; so we know the RB532A works, just not with the two SATA bridge cards you have.In my opinion (which may not be worth that much!), that indicates that the VT6421A and SIL3112 cards require early PCI initialization to work. Both support EEPROM to initialize configuration register values. If you can remove the EEPROM chip, the two datasheet has enough information for us to (read and) reprogram the contents so that the card itself initializes its PCI bus settings correctly. Right now, they're probably preset to some x86-compatible settings, which may not work for the RB532A.Would you be willing to try a patch toarch/mips/pci/pci-rc32434.c, to see if the VT6421A or SIL3112 could be initialized to get one of them working?Before that, right now, I would suggest adding two (early?) PCI fixup handlers inarch/mips/pci/fixup-rc32434.c, that do the same thing: dump the PCI configuration space register values,Code:static void dump_sil3112a(struct pci_dev *pdev){u32 v;int i;for (i = 0; i <= 0xC4; i += 4) {pci_read_config_dword(pdev, i, &v);printk(KERN_INFO "SIL3112A [%x] = %x\n", i, v);}}DECLARE_PCI_FIXUP_HEADER(0x1095, 0x3112, dump_sil3112a);static void dump_vt6421a(struct pci_dev *pdev){u32 v;int i;for (i = 0; i <= 0xD2; i += 4) {pci_read_config_dword(pdev, i, &v);printk(KERN_INFO "VT6421A [%x] = %x\n", i, v);}}DECLARE_PCI_FIXUP_HEADER(0x1106, 0x3249, dump_vt6421a);If the kernel locks up before you get the output, switch both toDECLARE_PCI_FIXUP_EARLY–– although that might interfere withprintk().For the SIL3112A, 10,14,18,1C,20 are the five IO BARs (and must have LSB set); 24 is the mem BAR the linux driver uses and should have low 10 bits clear; and 30 is the ROM address, and should have LSB clear (disabled).For the VT6421A, 60 is boot ROM address, 70 shows PCI V2X base and limit, 88 primary SG base address, 8C secondary SG base address.All the rest are useful too, because they can be compared to the default values shown in the datasheets. Knowing the values of all these, since the Linux kernel drivers only use a subset of the features so the rest can/should be left to chip defaults, a PCI fixup/early fixup can use a very similar loop to reconfigure the cards.Another approach is to move from PCI fixups toarch/mips/pci/pci-rc32434.c:rc32434_pcibridge_init(), and use a loop to detect and then initialize the cards' PCI registers as soon as the PCI bridge itself is initialized, just like thekorina PCI config registersfor the SoC built-in ethernet.I'd prefer the fixups myself, though. But, without knowing what the cards' PCI registers, without dumping the register contents as in the above addition, it's pretty much impossible to do because we don't yet know which registers get initialized to invalid values. (I suspect the ROM apertures, myself. For RB532A they most likely need to be disabled.)

author: DiTBho
author_rank: Super Contributor
timestamp: February 08, 2023, 07:55:07 am
content: Quote from: Nominal Animal on February 07, 2023, 08:49:14 pmYup, it's exactly what I did with u-boot on the PowerPC-embedded SBC to make the PCI-pATA card work.That bloody pATA controller boots in "legacy mode", which is like "ISA16" bus, which is a common IBM-PC design choice, crappy design because not supported in HPPA workstations and PowerMacs, which expect a PCI-pATA-devide working like a PCI-device/class-storage."legacy mode" means the card doesn't respond to the PCI-probing and instead it assumes it can operate with a predefined setup that is only OK on PCs, and what if that setup, in a non-PC machine, conflicts with other devices? ..I spent two years to fix it, because it was not documented.

author: DiTBho
author_rank: Super Contributor
timestamp: February 08, 2023, 08:01:22 am
content: Quote from: Nominal Animal on February 07, 2023, 08:49:14 pmI suspect too. Bought a second SIL card because (it's cheap (5UKP) and) you can easily remove the Flash and unsolder the serial EEPROM.On the VIA card the flash is high-density. I will try to unsoder the /CE pin, it should disable the chip