yashsingh1/xdaforums huggingface dataset:
[Barnes & Noble Nook Tablet] hkvc:
First baby step - try and understand the default memory map :-)

Hi,

Did try the 1st baby step towards this, by trying to go thro the running systems memory map.

LBHKVC driver v04Jan2012_2110

INFO: Total memory is iTotalMem 0x40000000

INFO: Page Size ??? PAGE_SHIFT 12 ie 4096

INFO: Begining of Platform ram PHYS_OFFSET 0x80000000(0x40000000)

INFO: End of userspace mapping TASK_SIZE 0xbf000000(0x7f000000)

INFO: Start address for Modules Space MODULES_VADDR 0xbf000000(0x7f000000)

INFO: End address for Modules Space MODULES_END 0xbfe00000(0x7fe00000)

INFO: Permanent kernal mappings PKMAP_BASE 0xbfe00000(0x7fe00000)

INFO: Kernel direct 1:1 map ??? platform RamBeg PAGE_OFFSET 0xc0000000(0x80000000)

INFO: Kernel direct 1:1 map ??? platform RamEnd high_memory 0xf0000000(0xb0000000)

INFO: vmalloc/ioremap space Begin VMALLOC_START 0xf0800000(0xb0800000)

INFO: vmalloc/ioremap space End VMALLOC_END 0xf8000000(0xb8000000)

Now one thing which is potentially true and easy is, the Physical Memory from 0x8000.0000 is 1:1 mapped to 0xc000.0000 in a linear way (Have to validate, but should be). There are certain virtual to physical maps which seem bit odd, most probably I am using the wrong function to do the conversion and or traversal of pagetable. Also some of them are not necessarily meant to have a mapping other than act has markers conceptually (Have to verify later).

Having something same to same mapped would have kept things to a minimal, and made this too trivial now - why not ooh linux gods ;-(. Now this is forcing some hijacking or forcing of page table entries or experimenting and seeing if this is good enough to me/any one else interested.

Also there seems to be something called SAR_RAM, have to see if this is of any use. As well as some of the regions reserved thro kernel command line and see if anyone is using it, or if it can be evicted if required, there are only for funny experimentations.

Otherwise I think, I should be able to get even my current modules virtual address translated to physical address and may be same-to-same mapped if required. Or forcefully take some region beyond my current running code and any exception logic I require.

Also I have to check how critical is the same-to-same map if any when switching off mmu, or is 1:1 map good enough, have been away from lowlevel arm also for too long now.
[Barnes & Noble Nook Tablet] Loglud:
Wow hkvc you respond quicker then I do.

First off it is not a 1 to 1 mapping if you look at the arm B&Ndefconfig youll see that they utilize a different way of mapping (not looking at the code right now).

Second off kexec with the kernel module that does KEXEC_LOADED is essentially this. I would look at the kexec.c code, and you will see that you can comment out the sanity checks in the find hole function and it will find a valid hole, then with injection you could make this work. However, while i have done embedded design and some hacking, kernel modules are not my specialty. We need a high priority kernel module that removes interrupts, so that the kexec code can load.
[Barnes & Noble Nook Tablet] hkvc:
BabyStep3 basic implementation done - but results say long fight ahead

Hi All,

Over yesterday and today, I have implemented a basic logic to test minimal kexec equivalent logic using kernel module.

Rather had to dig thro the kernel source code to

a) refresh thro my kernel basics and to understand atleast some of the changes in the newer kernel versions and MORE importantly

b) to OVER COME the tendency of core kernel developers to DISABLE EXPORT of some of the useful functions to external kernel modules.

Eitherway most of the disabled symbols I could pickup and hardcode the address in my code to still access them indirectly using function pointers - what would world be with out function pointers or rather pointers in general.

Also because of the SMP nature of the SOC, had to dig thro some of those stuffs also. Then for now decided to use some of the support mechanism already available within kernel to help with kexec logic like fin, reset, etc to try and see if I could use the easy path into it for now initially

However what I seem to have realised/found is that

a) Either these support routines haven't been fully implemented in the used kernel version on NookTab currently and or

b) I am missing few more additional steps wrt SMP (Have already killed the 2nd Processor using proper api in kernel - Have to cross check the CP15 to verify for sure once again later).

It seems to be mostly (a) and inturn related to cache cleanup and may be mmu switching, have to debug further.

Otherwise the same logic seems to be working in BeagleboardXM (rather within qemu -M beaglexm) except for some reason the uart messages seem to disappear once I switch over even thou the code seems to be running in the new Same-to-Same map with the physical memory address part, with proper UART address - checked using info registers in qemu (Have to debug this part of qemu related to how it decides which uart to show in Ctl-Alt-3 etc bit more and or try on a physical board sometime next week).

In few days time I will upload the code I have come up with (even thou useless from using/achieving kexec logic perspective currently, still may provide some ideas or act as a base platform for someone wanting to experiment, but having initial inertia :-), if I am not able to spend more time on it.
[Barnes & Noble Nook Tablet] hkvc:
Initial Pre-Alpha source for linboothkvc kernel module and utils

Hi All,

As promised, I am uploading the initial version of the kernel module source code (with lot more updates compared to last weekend, when I mentioned about it) for achieving kexec like functionality even if kexec is disabled in kernel. As I always told, kernel modules are equivalent to kernel, so you can do what ever you want in kernel module that can be done in kernel, provided one is bit patient ;-).

Note: This is pre - alpha version of code for people with initial inertia/starting trouble to experiment. This has a known bug with cache handling which I have to fix as well as some restructuring and cleanup to do.

This will currently only run in Qemu, because it doesn't bother much with Cache. But on ACTUAL targets it will fail Randomly (there is a very very small one in a million chance that it can succeed if all the stars line up and the cache gods support you :-)

This is no longer as critical for NookTab has it was 1 week back, because Now some people have already released a exploit which uses a uboot bug ;-(, which ideally we could have with held for a future product (because NookTab was already sufficiently open and didn't need any more exploits to use its full potential, as I had mentioned last week). But either way I realise that people are very impatient these days. And congrats to those people for their work however.

For people who want to experiment, the initial skeleton is available in this release :-)
[Barnes & Noble Nook Tablet] hkvc:
Alpha release with embedded x-loader for BeagleXM Qemu

Hi,

NOTE: This release successfully bootstraps a new linux from with in linux in Qemu, my yesterdays release would have also done the same, but would have required additional code to handle the final nitty grities, this is taken care of now in this release. So that it is easier for people wanting to experiment.

I have updated the kernel module to allow two images to be embedded into it.

a) The initial boot strap loader called Nirvana (Examples like bloop0.S, bloop1.S, ... and now a full fledged omap3callbootrom1.S for BeagleXM on Qemu - Rather it in itself is independent of Qemu-BeagleXm or Hw-BeagleXM)

and

b) The actual bootloader/???? to run in the new prestine environment called Nirvana's Child (NChild). Currenlty it is a version of the x-loader for BeagleXM (Either Qemu or Actual BeagleXM).

In turn once the kernel module is done with its job. It passes control to Nirvana. Also it passes the physical address and length of the NChild image to Nirvana thro r0 and r1.

The Nirvana code in turn can decide what it wants to do with the system. The default Nirvana code i.e omap3callbootrom1.S takes care of copying the NChild(by default x-load.bin) to 0x4020.0800 and then pass control to 0x4001.4000 so that it loads x-loader properly (ie setup stack for all modes etc).

NOTE: As of now it works on Qemu only. My earlier release yesterday would have just printed 1s on the screen, while this version actually boots into a new linux kernel + system in Qemu from with in a already running Linux system in qemu.

However it won't run outside Qemu successfully, as I haven't yet had time to look at fixing the cache issue, because I had to add support for NChild image logic either way for doing anything useful with the code.

NOTE: As this is either way no longer critical for NookTab, I will take a stab at it based on my hack-vs-life balance also. Depending on what and all come up over the next few days in life.

HOWEVER this latest release has the FULL REQUIRED INFRASTRUCTURE/LOGIC for running this on a actual h/w expect for the missing proper cache handling. It also includes a x-load.bin by default for BeagleXM which can be used either with Qemu-BeagleXm or Hw-BeagleXm.
[Barnes & Noble Nook Tablet] hkvc:
Beta Source - Success on a Hardware (beaglexm for now) should work on NT also ...

Hi All,

I have finally identified the stupid cache issue which was frustrating me and eating my head for long and stalling this project unnecessarily :-). However it gave a nice oppurtunity for me to dig thro the kernel code as well as Arm documentations - which is what I am after either way So ALL IS WELL in the end :-)

The problem was related to kernel's normal code using MVA based cache operations for flushing, which in the newer architectures stops mostly at L2 cache rather than hitting the memory. This is fine for Normal linux kernel operations because they don't disable cache and so the proper data will get used. But for us as we want to disable cache to give a true pristine NIRVANA environment, this doesn't work, as memory contains old/stale/wrong data. In our code flow This even affects the KERNEL MODULE CODE itself, leave alone Nirvana or NChild code.

Once I realised this by digging thro documents as well as seeing the strange behaviour (rather unbelieavble, initially for me) of my code(kernel module as well as Nirvana and NChild)/logic after cache disable, I was able to get it RUNNING SUCCESSFULLY on BeagleXM actual hardware and not the qemu emulation which I was previously using.

As of now it is restricted to Omap3/BeagleXM, because the Nirvana code Omap3callbootrom2 makes use of the bootrom memory map usage knowledge to setup NChild appropriately and pass control to it thro Bootrom (so that stack is setup for all modes). With small effort the same can be changed or updated for Omap4 and NT should be in the fold (Unless SMP creeps up, inspite of me shutting down the 2nd processor, for some crazy reason in the worst case ;-)

NOTE: Also my last release (alpha) for Qemu beagle had a bug which was a blessing in disguise, in that it was not disabling the MMU from with in kernel module(which I had added just for the heck of it and is actually not required). However it was still doing it once it hit Nirvana code, as required. If I had not done the mistake of using mrc instead of mcr with in my kernel module code, it would have failed immidiately, because BeagleXM has only 512MB ram and the kernel module code space is at virtual address 0xbf00.0000 or around it, which is WELL BEYOND the 512 MB of physical memory, so there would not have been any 1-to-1 memory map for it and disabling MMU would have made things go crazy. Note that Nirvana and NChild are kmalloced into linear mapped kernel address space which is with in physical memory limits normally ;-).

NOTE: May be there is a watchdog timer or coprocessor or so which I have to disable, haven't looked into this yet, which seems to mess things up, if I stay in Nirvana code for too long. However as by default there is no need to remain in Nirvana code for long, as it is required to pass control to NChild as quickly as possible, this is not a immidiate issue to worry about now.

Let the experimentations begin :-)
[Barnes & Noble Nook Tablet] hkvc:
For H/w BeagleXM boot, bypass stupid SMI based L2 Cache maintaince rot in uboot ;-)

Hi All,

If anyone has tried running my yesterdays Beta release on BeagleXM h/w there is one update and one other IMPORTANT things to keep in mind.

a) Update the omap3callbootrom2.S to directly call into NChild rather than thro BootRom, that call into BootRom is not required. i.e jump to 0x4020.0800 instead of 0x4001.4000 at the end.

NOTE: This also makes the code more generic and usable with minimal love across Beagle and NookTab.

b) In U-Boot remember to DISABLE/BYPASS the calls to ROM Support routines thro SMI to setup L2 cache as well as to invalidate cache. This is no longer required in newer Omap3 chips as well as there is actually few bugs in u-boot code itself related to this, as one wants to clear full cache but SMI rom routine only clears L2 (also the full cache walking for invalidate is there following the SMI call, so bypassing doesn't lose functionality), if I get rom code description in TRM correctly, plus few other bugs atleast in rowboat version. The files board.c and cache.S contain the calls to SMI which has to be bypassed.

I have attached the minimal patch required to uboot to allow it to work with linboothkvc with BeagleXM.

With the above two changes, linboothkvc will always succeed in BeagleXM, enjoy :-)

NOTE: In my setup today, I have modified the NChild x-loader to load u-bootk.bin rather than u-boot.bin. Thus I can have both the Normal u-boot.bin for Normal booting and u-bootk.bin for linboothkvc based booting :-). My Beta release doesn't contain this modified x-loader NChild, the next release will contain the modified one. But with source of x-loader, you can do it yourself and copy over as NChild into linboothkvc
[Barnes & Noble Nook Tablet] hkvc:
Success on nook tab :-)

Hi All,

LinBootHKVC has successfully booted into Nirvana code in NookTab :-)

:D:D:D

NOTE: No major change required to my beta release other than one mentioned in my last post to make my released Nirvana code more generic and the address update in this post for NookTab.

As I had mentioned yesterday, even thou I hadn't got the time yesterday to check it on NookTab yet, it should work 90% except for any crazy SMP issues, inspite of me disabling the 2nd Processor. WELL IT turns out that what I had done already was sufficient for NookTab also.

Only in omap3callbootrom2.S you have to change the sram address to which things are copied from 0x4020.0800 to 0x4030.0800 for NChild code. However I haven't crosschecked the x-loader based NChild on NookTab as of now, but DONT SEE ANY REASON why it should fail, other than for any issues with stupid code, like the SMI calls to manage L2 cache in u-boot for beaglexm :-) and for some reason if 0x4030.0000 space has some issues I haven't thought of in Omap4 (I am relatively new to Omap4 started with NookTab only).

Will upload the Release Candidate version 1 of the code in a day or two. It is late here and I have been up on this NookTab project for few weeks now

a) starting from idea of linboothkvc

b) then moving to 2ndihkvc

c) followed by NOPBypass with UART access

0) The uboot loop hole (Oh my my my :-) (But not released from my side hoping to keep it for future, but alas, if only people have/had patience ..., that is partly a dream now :-)

d) followed by MenuK for 2ndihkvc - haven't released yet, time got sucked back

into linboothkvc :-)

and back to

e) linboothkvc

all of the above work on NookTab successfully as of today and all can be used to achieve custom Roms and in case of NOPBypass and linboothkvc even custom kernels :-)

So enjoy everyone. Some sweet rest for me atlast ;-)

NOTE: The POWER of Kernel MODULE and inturn linboothkvc goes well beyond NookTab for the adventures people out there :-)
[Barnes & Noble Nook Tablet] hkvc:
Oh My MURPHY - For now keep away from 0x4030.xxxx

Hi All,

Now as Murphys law would have it :D:D:D:D, 0x4030.4350 (the default address used by x-loader in Omap4) has some issue with it (which I have to debug later). So if trying on NookTab remember to use some other address (what other I will leave as a exercise to the interested :-)

Also the x-loader from BN and or inturn from Ti expects a special meta data structure to be passed along thro r0, which inturn tells it about boot device and boot mode. So you will require to take care of this or better still simplify the structure to handle in cpu/omap4/start.S

Also I did the cardinal sin of doing too many changes when working/debugging on a new unknown device (from my perspective i.e NookTab) with no jtag access (atleast at my end for now).

So ended up digging into Arm L2X0 cache and inturn PL310 trm and writing a L2X0 cache flush logic (obviously also cross checking the equivalent logic in Linux kernel) when none was required in reality as I had already found and mentioned in my last post few days back. I forgot (rather got too lazy to checkout my own old code, just for the fun of exploring further ;-) my own Nirvana code which I had tried towards the beginning of the week which had successfully run and printed stuff on screen.

Also dug into the Address translation support registers in CP15 to be 100% sure that the 1-to-1 map was in fact there (again lazy to dig thro the linux kernel setup_mm code, when arm walks the page table tree for you and on top when I had not tried this arm instruction method before :-).

But ended up finally realizing that me using the 0x4030.4350 was the culprit atleast for now.

So keep away from that address for now (or debug further on this on your own for now, I will be looking at it only later, could be something to do with HS code using that region or ..., there is more interesting stuff to do for now) and remember to patch x-loader appropriately and you should be able to get it running on NookTab; again with my last code release with hardly any changes other than what I have already mentioned in the last 2 to 3 posts.

A updated source with the now useless :-) l2x0 cache flush and va2pa address translation verification logic and a newer nirvana code with panda board support also (verified works perfectly - only slight changes required between Panda and NookTab as far as Nirvana code is concerned) I will release after some more experimentation and code stabilisation at my end.
[Barnes & Noble Nook Tablet] Loglud:
hkvc:

Thanks for all of your work. Even though the bootloader bypass has been found, you do not know how many others you are helping. This could be used as an indefinite alternative to locked bootloaders for ALL DEVICES! Keep chugging on this, and I'm sure you'll find your solution to the l2 cache flush. Also i have been looking around, and I was curious if this could be another explotable boot flaw,

/*
* The SAR RAM is maintained during Device OFF mode.
* It is split into 4 banks with different privilege accesses
*
* ---------------------------------------------------------------------
* Access mode Bank Address Range
* ---------------------------------------------------------------------
* HS/GP : Public 1 0x4A32_6000 - 0x4A32_6FFF (4kB)
* HS/GP : Public, Secured
* if padconfaccdisable=1 2 0x4A32_7000 - 0x4A32_73FF (1kB)
* HS/EMU : Secured
* GP : Public 3 0x4A32_8000 - 0x4A32_87FF (2kB)
* HS/GP :
* Secure Priviledge,
* write once. 4 0x4A32_9000 - 0x4A32_93FF (1kB)
* ---------------------------------------------------------------------
* The SAR RAM save regiter layout is fixed since restore is done by hardware.
*/
[Barnes & Noble Nook Tablet] hkvc:
Release v1.1 source with binary for NookTab

Hi All,

Being bit lazy I had not called cache flush once again after disabling the cache, because I was calling it just a wee bit before disabling it and was also expecting any body who enables Cache in Nirvana environment to do a thorough job of cache invalidation before enabling it again.

However found that the linux kernel 2.6.35 used by Nook firmware has this problem, so now I take care of calling cache disable once again after disabling the Cache to be on safe side and it does help for NookTab (while on Beagle and Pandboard I didn't have this issue, it was a newer kernel which I had tried there so may be it has some additional effort put in wrt cache invalidation or it was pure luck, either way haven't debugged that aspect now).

So with this release, linboothkvc provides a better Nirvana environment, except for the SMC related stuff, which I am ignoring for now :-). So related to that there is some cleanup or bypassing required to be done in Kernel otherwise linux kernel should be pretty much ok in linboothkvc Nirvana environment, haven't had time to look at it fully yet. Will spend sometime in a day or two when a holiday is coming my way.

Check out the README file some of the same info as above and may be a few more things here and there. Also the new binary for NookTab is there in the source package within the release directory.
[Barnes & Noble Nook Tablet] dodgepot:
I've read every one of these posts and have no idea what is going on but I'm glad that it's all working. Great job!
[Barnes & Noble Nook Tablet] hkvc:
Getting Linux kernel for NookTab up in LinBootHKVC - Step2 and 3

Hi All,

Step2

------------

There seems to be few race conditions in the linux kernel source and or some initialisations not being done properly in the kernel code, because of which by default the BN kernel source will fail in LinBootHKVC environment.

The below patch fixes 1 such race issue I have noticed wrt initial clock event handler for ipi timer (Have to debug this further later).

NOTE: A initialisation issue with the Cache, in that it not getting invalidated properly during booting was fixed in my v1.1 linboothkvc kernel module, by doing a additional flush before switching to Nirvana. Because that was also a appropriate stuff for linboothkvc also to do, independent of the kernel initialisation issue.

static void ipi_timer(void)

{

struct clock_event_device *evt = &__get_cpu_var(percpu_clockevent);

irq_enter();

- evt->event_handler(evt);

+ if(evt) {

+ if(evt->event_handler) {

+ evt->event_handler(evt);

+ }

+ else

+ printk("WARN:ipi_timer: event_handler missing\n");

+ }

+ else

+ printk("WARN:ipi_timer: evt not set\n");

irq_exit();

}

Beyond this any additional issues, I have to check yet. Also the reason for this race, I haven't debugged for now.

UPDATE1 (Step3)

----------------------

I have further cross checked that CONFIG_OMAP_RESET_CLOCKS causes some problem in linboothkvc Nirvana environment for few specific clocks, either way for now Disabling this config in BN NookTab kernel before compilation will allow the resultant kernel to run SUCCESSFULLY on NOOKTAB.

NOTE: As of now, even thou SMP is enabled, one of the Processors doesn't come up live. That has to be debugged later. But otherwise the Linux is FULLY running in NookTab from with in the LinBoothKVC Nirvana Environment and in turn the Linux User space is also running fine.
[Barnes & Noble Nook Tablet] howcansheslap:
That sounds like a big step. Awesome work!
[Barnes & Noble Nook Tablet] Shandry:
Really great work hkvc! Thank you very much for this. It's really outstanding. Trying to port this for the Bionic. No luck however, lack of knowledge.
[Barnes & Noble Nook Tablet] XVilka:
You can see reversed omap3430 bootrom for the reference http://hg.droid-developers.org/reverse_engineering/src/0d4804225ad5/asm/omap3430_boot_rom_14.57.asm (not yet reversed OMAP44xx bootrom, but must be similar)

Also, which branch of qemu you're using for testing purposes? official qemu have poor support of beagle/omap and CP15/CP16 coprocessors.
[HD2 Windows Phone 7 Q&A, Help & Troubleshooting an] YZ:
Great thread! :)
[HD2 Windows Phone 7 Q&A, Help & Troubleshooting an] walidhazem:
thank you :D