yashsingh1/xdaforums huggingface dataset:
[Galaxy 3 Android Development] hillbeast:
Things to be aware of

I feel these things should be mentioned as they are important changes that may disrupt peoples user experience on the phone.

Odin will not work anymore for flashing ROMs. It's as simple as that. We will no longer be using STL/BML/TFSR for interfacing with the NAND, and that is what ODIN expects to be working with: partitions in STL format. We simply will not support STL when this kernel is complete as it is closed source and will not function on kernel 3.x. We will be moving over to MTD which is mainline, and regularly gets updates to improve performance, reliability and in general to make it better.

Kernels will however be flashable from ODIN. This WILL NOT change. Kernels, modem.bin, Sbl and boot.bin will all still work from Odin. Data, system and cache will not.

I know this will annoy a few people, as it will disrupt me too as a developer and a Windows user, it's nice to be able to flash a ROM this way, but we're going to have to move towards distributing ROMs in system.imgs, or more preferably in update.zips.

There will be no support for Samsung Froyo/Eclair ROMs in kernel 3.x. The reasoning behind this is because Samsung ROMs want Samsung drivers and we won't be using any of them. We will be aiming to support CM7 and CM9, or generally any AOSP ROM, but not Samsungs proprietary ROM. You may think 'well why not make the drivers work for both', but the problem is not with making the drivers work, the problem is with the fact that a lot of these drivers were written for a kernel over 10 major versions out of date by now, and will not function on the new kernels APIs. If we try to make them work, it will make the kernel unstable and it's going to degrade the experience for the end user. As well as that, we don't know how half these drivers actually work because they are either coded so badly it's impossible to make heads or tails of it, or it's closed source and we can't find sources for it.

There are plenty of good custom kernels for Froyo, and I have heard word in the grapevine a few people want to make a custom Eclair kernel (not confirmed, and that is just rumours), but I will not be adding support for Froyo to kernel 3.x

Kernel 3.x will not allow us to work magic with the phone. Having a newer kernel won't allow us to make the CPU itself faster, we can't give it more RAM (or find more RAM in there). We won't be able to unlock more performance from x part. This is not a magical rebirth of the phone, this is just a major fix up of all the current issues we face with AOSP based ROMs on the phone. It will most likely make the phone faster, but we can't magically make it into a dual core or something like that. It just isn't possible.
[Galaxy 3 Android Development] hillbeast:
I see a reserved post there!
[Galaxy 3 Android Development] hillbeast:
And one more for good luck. Now you may post.
[Galaxy 3 Android Development] cdesai:
So many reserved posts WHOA!

could possibly add a guide to clone and build it :)

EDIT:

Btw, also add a note about the two branches on github

android-3.0 = android-common kernel from android.googlesource.com

linux-samsung = kgene/linux-samsung on kernel.org / github
[Galaxy 3 Android Development] pkatsaras:
+1! Yeah guys well done! Finally! Is it only me that I think that may we have a beta or final version of Gingerbread or (cross fingers) ICS if you guys port the 3.0 kernel for our phone?
[Galaxy 3 Android Development] raja4sure:
Good to have a new thread now.....

All the best Mark & all the Devs in this project..
[Galaxy 3 Android Development] Jason-EX:
+1 :D:D Good luck!!
[Galaxy 3 Android Development] GuestK0091:
Good luck with this project :)
[Galaxy 3 Android Development] ahsanaimad:
All the best.. Although this is gonna take much time still waiting patiently :D
[Galaxy 3 Android Development] ak700:
I may sound like an ultra-noob here : But what is Kernel 3.0 ?

Google isn't help here
[Galaxy 3 Android Development] Smonic:
You could add this to the first post but only computer programmers will understand it, unlike me :p
[Galaxy 3 Android Development] hillbeast:
Newer version of the kernel that will be completely built from scratch. tom3q did this on the Spica and it worked wonders, now it's time to do it to our phone.

I don't like kernel.org. They're hopeless, shown by their hacking last year and their slow recovery time, and then made worse for the fact that Linus Torvalds himself switched to Github.
[Galaxy 3 Android Development] andreapier:
This is what i wanted to see since long times now. I just wanto say thank you to all the devs for just trying
[Galaxy 3 Android Development] hillbeast:
Frig. I was doing so well and then I hit a brick wall with bloody assembler...

hillbeast@powermacg5:~/devel/galaxy3/kernel-3.0$ ./build.sh Test
Setting kernel name to (Test)
Compiling the kernel
CHK include/linux/version.h
CHK include/generated/utsrelease.h
UPD include/generated/utsrelease.h
make[1]: `include/generated/mach-types.h' is up to date.
CALL scripts/checksyscalls.sh
CHK include/generated/compile.h
CC init/version.o
AS arch/arm/kernel/debug.o
arch/arm/kernel/debug.S: Assembler messages:
arch/arm/kernel/debug.S:157: Error: too many positional arguments
arch/arm/kernel/debug.S:173: Error: too many positional arguments
make[1]: *** [arch/arm/kernel/debug.o] Error 1
make: *** [arch/arm/kernel] Error 2
make: *** Waiting for unfinished jobs....

Anyone smarter than me with assember know what that means?

EDIT: It's to do with ASCII and UART O.o

Line 156-170 debug.S

ENTRY(printascii)
addruart_current r3, r1, r2
b	2f
1:	waituart r2, r3
senduart r1, r3
busyuart r2, r3
teq	r1, #'\n'
moveq	r1, #'\r'
beq	1b
2:	teq	r0, #0
ldrneb	r1, [r0], #1
teqne	r1, #0
bne	1b
mov	pc, lr
ENDPROC(printascii)
[Galaxy 3 Android Development] moikop:
The error is not on /arch/arm/kernel/debug.S, it's on /arch/arm/mach-s5p6442/include/mach/debug-macro.S (Error caused by just putting Samsung's .32 code into 3.x I guess...).

Check out that addruart_current is defined a few lines before, and it refers to "adduart" which is defined at debug-macro.S (that's why before defining addruart_current, debug-macro.S is included).

And if you check debug-macro.S, you'll see that adduart is defined only with one parameter, when in debug.S it's used with two parameters. That's where the error comes from.

In order to fix this, you'll have to redo the debug-macro.S file so that adduart takes two parameters (IMO should be based in S5PC110 or Spica's (or maybe s5p64x0?) debug-macro.S).
[Galaxy 3 Android Development] hillbeast:
Brilliant. Now we're back to good ol' fixing up references.
[Galaxy 3 Android Development] ajay.d96:
you're using homebrew right? if yes, does it compile faster? in a word, ill grab my sis macbook pro !

BTW, which baseband are you gonna use for this kernel ?
[Galaxy 3 Android Development] Spitlebug:
Did I read that correctly?

"Division by zero in kernel."
[Galaxy 3 Android Development] Arakmar:
I can't promise but I'll try to help during August, one month of vacation :)

Anyway, very good work !
[Galaxy 3 Android Development] tom3q:
It depends what you mean by memory map. Basically those 256 MB of RAM is not the entire DRAM memory on G3. There is also a 128 MB OneDRAM chip that has three banks, 80 MB of memory dedicated for application processor, 16 MB of memory shared between baseband processor and application processor and 32 MB for exclusive use of baseband processor. Those 80 megabytes can be (and is, on current kernels) used for hardware buffers, for example through Android PMEM allocator.
[Galaxy 3 Android Development] hillbeast:
Oh okay I was not aware of the purpose of the OneDRAM chip. I always assumed there was the OneNAND sitting on top of the CPU which had 512MB of NAND and 256MB of RAM in it. So you say it also has another 128MB in there?
[Galaxy 3 Android Development] tom3q:
Yes, although the top 32 MB isn't accessible to the CPU at all and the next lower 16 MB only if a switch register called OneDRAM semaphore is set to give access to application processor. Basically only those bottom 80 MB can be considered as a general-purpose memory. The shared 16 MB is used inside OneDRAM driver for communication with baseband processor (firmware loading, message queues, etc.) like a standard MMIO.

It's still to be decided which approach we should use for managing this extra block of memory. For Linux 3.4 it should be possible to apply CMA patchset and manage the memory dynamically, extending the total amount available for the system to 336 MB and allocate hardware buffers dynamically, although I don't know how Android support for this method looks at the moment. (Making a wrapper with the same interface as PMEM would be pretty straightforward, though.)

EDIT: There is already a CMA back-end available for the new ION memory allocator used in Android 4.0 and newer.
[Galaxy 3 Android Development] sarthak.killedar:
So with ION memory allocator we can define those 336 MB of RAM without creating the wrapper but if we use this then Android 2.3 won't be able to use this kernel advantages of extra RAM, right?
[Galaxy 3 Android Development] artifex21:
@up Well,if ION is for 4.0+ logic says that extra RAM wont be able in 2.3 .That's my opinion,let;s hear the experts :)
[Galaxy 3 Android Development] tom3q:
Android 2.3 supports only PMEM, so it isn't possible to run it on ION. For Android 2.3 PMEM would have to be modified or wrapped to work on top of CMA.
[Galaxy 3 Android Development] sarthak.killedar:
Rather than wrapper, we create this kernel only for above Android 4.0 versions? As wrapper may not work 100% for every instances/event.
[Galaxy 3 Android Development] hillbeast:
I aim to have this kernel work on Android 4.0+. I can't be bothered developing it for Gingerbread. Maybe someone else can but I can't. Gingerbread is too old.
[Galaxy 3 Android Development] sarthak.killedar:
I doubt upon our device hardware capability to run this newer versions!
[Galaxy 3 Android Development] TheCreat:
Just because it's newer doesn't mean it needs more resources. There have been many optimizations in the newer versions making the whole thing smoother even on slow phones. This isn't windows!

Swyped on my Galaxy S2
[Galaxy 3 Android Development] tom3q:
Yes, this would be the best choice. From my experience in low level development for Android 2.3 I can say that it contains many interfaces that haven't been properly designed, making them problematic to implement on some (or even most) hardware. Fortunately Google has reworked most of them in newer version and Android 4.0 seems much better in this matter.
[Galaxy 3 Android Development] aarrun:
i think the project must be focused at 1 thing only.... working on android 4.0 only is good.... i think tom and hill are right.... focusing only on android 4.0 will be good for the projects pace and development....
[Galaxy 3 Android Development] z3c0:
why would people want to stay with 2.3 when there's a kernel that supports 4.0+ along with all of the hardware features?
[Galaxy 3 Android Development] DiGaTTo:
Guys on my opinion we are all right now starting discussing offtopic things. My sujesion is concentrate only on importaten tihngs related directly to subject, If it will be kernell for android 4+ it;s good, if some one will one to port it to 2.3 it will be more then good. I and many others waiting for this kernel, trying to ask or push devs to make a lot of aditional work will move finishing of the kernel far far away.
[Galaxy 3 Android Development] Keinbockwurst:
Because 4.0 is not a performance beast at the galaxy 3 ;)
[Galaxy 3 Android Development] hillbeast:
Enough off topic. Android 4.0 is a million tomes faster than 2.3 and that is end of discussion. Anyone who discusses this further will be reported for off topicing in a development thread. If you want to discuss it, make a thread in GD.

Sent from my Serenity Galaxy S2
[Galaxy 3 Android Development] Spitlebug:
Is a wrapper the best implemetation for using all availible memory? Isn't doing that slightly slower?
[Galaxy 3 Android Development] tom3q:
Allocation of hardware buffers is done rarely (usually at the beginning, before starting the main operation), so it wouldn't matter. Another thing is that allocation of physically contiguous memory is always a costly process and the minimal overhead of a wrapper done properly would be insignificant.

However, for Android 4.0+ we can use ION instead of PMEM and it has a CMA back-end, so no wrapper would be needed.
[Galaxy 3 Android Development] hillbeast:
@tom3q Currently the kernel is stuck on calculating the delay loop (calculating the jiffies) and as far as I can tell, it seems to be because it can't find a source for the clock ticks. Do you know where it gets the ticks from?
[Galaxy 3 Android Development] tom3q:
PWM timers are used for timekeeping. The implementation resides in arch/arm/plat-s5p/s5p-time.c and is common for the whole s5p platform. (You can see that at the bottom of mach-apollo.c, in machine descriptor there is a field timer set to s5p_timer struct, which is defined in s5p-time.c.)

Possibly there is something wrong with memory mapping of the PWM unit (different memory base on 6442) or unmasking of required clocks.

P.S. If you need a good way of looking through kernel sources, I recommend using an editor with support for CTAGS (e.g. Kate), that allows to index and then quickly search the sources for symbols or a cross reference like http://lxr.linux.no/#linux+v3.4/ .
[Galaxy 3 Android Development] Spitlebug:
I gotcha, thanks. :)
[Galaxy 3 Android Development] amarendar94:
@Hillbeast.....anything to share about the progress.

Sent from my only primary phone GT i5801
[Galaxy 3 Android Development] hillbeast:
Moikop and I were discussing while I was at work (feeding ideas to each other and him testing them) and I'm just waiting for him to commit them so I can try on my end. I won't say what it is yet as I don't want to get hopes up over nothing, and I feel moikop should take credit seeing he did the coding of this while I was giving him assistance.

So yes, there is definite progress. I'll just give a sneak:

<3>fsr: version magic '2.6.32.9 mod_unload ARMv6 ' should be '3.4.0-rc6-g8943f8'
<3>fsr_stl: version magic '2.6.32.9 mod_unload ARMv6 ' should be '3.4.0-rc6-g89'
<3>rfs_glue: version magic '2.6.32.9 mod_unload ARMv6 ' should be '3.4.0-rc6-g8'
<3>rfs_fat: version magic '2.6.32.9 mod_unload ARMv6 ' should be '3.4.0-rc6-g89'
<3>tun: version magic '2.6.32.9 mod_unload ARMv6 ' should be '3.4.0-rc6-g8943f8'
[Galaxy 3 Android Development] moikop:
I'm committing them right now to the new branch you made. Shouldn't take me more than an hour...

Samsung, Kernel 3.4, UART and GIT. What could go wrong? :rolleyes:
[Galaxy 3 Android Development] tom3q:
Not that I want to negate the progress, as there is definitely some looking at git commits (although not perfectly conforming to what we decided before, but it's another story), but you won't be able to load binary modules compiled for Linux 2.6.32 on anything significantly newer than 2.6.32, even after disabling module versioning.

It's just because the in-kernel interfaces constantly evolve and even a single structure modification will break existing binaries. That's why it's important (or even a must) for kernel modules to be compiled for kernel with which they will be used.
[Galaxy 3 Android Development] moikop:
That's not what he's trying to show with that snippet (and that's not what we're trying to do, no samsung crap). What I think he's trying to show there is that /init is actually being executed (given that init is the one who loads those modules). And those modules are there only because the initramfs that was being used at that trial was a stock one (with all samsung crap)..
[Galaxy 3 Android Development] tom3q:
Fine, if that's so. Sorry for my misunderstanding.
[Galaxy 3 Android Development] hillbeast:
I'm not showing that these modules aren't working, just showing they are trying to load at all meaning we are deadly close to getting busybox shell working. We just need the UART drivers working 100% and we will have a shell. A basic one yes, but that is filling in our goal of getting a shell working first and then building up to recovery followed by Android itself.

I still think we are on target to have a console by the end of the month.

Also yes I know moikop and I have taken an approach that is less than ideal but I think if we get it working with the current setup then it will be easier to bring in components from S5P64x0 and S5PC210. Its a round about method but it will work. I'll get it to start linking in the other code once we have a shell.

Also while I wait upon moikops tarball to be sent my way, I'll make a simple initramfs so we don't try loading crap modules.

Sent from my Serenity Galaxy S2
[Galaxy 3 Android Development] MmegaMac:
Good works !!

Envoyé depuis mon GT-I5800
[Galaxy 3 Android Development] Spitlebug:
Man, that is positive! Fast turn around too.

I take it you have a strategy to get UART going?
[Galaxy 3 Android Development] hillbeast:
Oh good. I fixed the 'Calibrating delay loop' hang. Basically this is a function of Linux which tells it how powerful the processor is, and how long the delay is between instructions (well that's what I can tell by reading about it), but before we didn't have the right IRQ set up for telling it the timing, but I put in the right details and away it went.

Also, interestingly:

[B]G3MOD:[/B] Calibrating delay loop... 665.19 BogoMIPS (lpj=1662976)
[B]Kernel 3.4.0-rc6:[/B] Calibrating delay loop... 1327.10 BogoMIPS (lpj=3317760)

According to this information, the processor is twice as fast as before while still keeping the same CPU clock thanks to kernel 3.4s improvements over 2.6, adding preemption, and various other changes. I will take that information with a pinch of salt though, but it's good to see a new lease of life for our Galaxy 3.
[Galaxy 3 Android Development] hillbeast:
Just find where it's got missing information by comparing it to kernel 2.6.38.8. Moikop and I are working from kernel 2.6.38.8 to find what we need to change because 2.6.32 is such a mess. Thanks Samsung...
[Galaxy 3 Android Development] i_rnb:
whoohooohoo.. thats wicked man!
[Galaxy 3 Android Development] tom3q:
It looks like there is no driver data provided to the driver. Most of SoC drivers use the platform driver model, where the driver itself doesn't contain any board-specific (or sometimes even SoC-specific, if it supports multiple SoCs) data, but instead any remaining parameters are provided in a structure called platform data, which has to be set up in board/SoC-specific code, i.e. somewhere in mach-s5p6442 directory. You're probably missing this structure or it isn't set up properly.
[Galaxy 3 Android Development] tom3q:
I'm afraid that it's just because of wrong clock settings/detection or incorrect timer configuration. Correct BogoMIPS value on ARM should be roughly equal to CPU frequency.
[Galaxy 3 Android Development] hillbeast:
I'll take a look through to see where it should be defined by referring to S5P64x0 and other processors.

It could be because PCLKD0 and D1 are wrong. I need to fix them....
[Galaxy 3 Android Development] tom3q:
Adding

|| defined(CONFIG_CPU_S5P6442)

to https://github.com/sg3/android_kern...5p6442-3.4/drivers/tty/serial/samsung.c#L1539 might do the thing.
[Galaxy 3 Android Development] hillbeast:
Yep that did the trick. Now it's glitching out though, looks to me like a bad clock. I'm pretty sure UART is clocked by PCLK, so I'll fix that before I go any further.

This is what I mean by 'glitching out':

<6>Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
s3c24xx_serial_probe(c06a74f0) 0
s3c24xx_serial_probe: initialising port c0691f2c...
s3c24xx_serial_init_port: port=c0691f4c, platdev=c06a74f0
resource c0670284 (ec000000..ec000100)
port: map=ec000000, mem=f7000000, irq=74 (74,75), clock=1
s3c24xx_serial_probe: adding port
<6>s3c6400-uart.0: ttySAC0 at MMIO 0xec000000 (irq = 74) is a S3C6400/10
s3c24xx_serial_console_setup: co=c0692248 (1), 115200
s3c24xx_serial_probe(c06726d4) 1
s3c24xx_serial_probe: initialising port c0691ff4...
s3c24xx_serial_init_port: port=c0692014, platdev=c06726d4
resource c06702bc (ec000400..ec000500)
port: map=ec000400, mem=f7000400, irq=75 (75,76), clock=1
. de e
0,1e: adding port
<6>s3c6400-uart.1: ttySAC1 atr1[c.26
m0r]a )on3 <[s3c24xppn[ 0h1sem[ 0 robce[[]
6>brd: module loaded
[ 0.375000] brd: module loaded
46>loop: module loaded
[ 0.375000] loop: module loaded
<6>bonding:D<6>eql: Equalizer2002: Simon Janes (simon@ncm.com[
6>Fixed MDIO Bus: probed
<6>t<<6>PPPr<6>PPPP<6>PPP MPPE Compression module r[<6>mousedec<6>device-map <6>device-e<<6>ashmem: initialized
<6>lo[<6>lg<6>lo[<6>lo<6>TCP: cubic reg<6>NET: Registere <6>NET: Registered protocol family 1<[<6>registered taskst
6>console [netcon0] enabled
[ 0.400000] console [netcon0] ena s3c64xx_4<6[a<6>md: Autodetect[00VFS: C P" [ku
[x0 ro[ e0t

But by seeing [ 0.400000] in there, it tells us that the UART driver has started and it's attempting to output to it. That's timing information there and that only shows on console, not early_printk.

EDIT: Yes, UART is fed by PCLKD1. Need to find out why it's not dividing that clock right. Most likely it has the wrong divider or something/
[Galaxy 3 Android Development] tom3q:
Where would you want to add it? If you're talking about the .clocks and .clocks_size fields of s3c2410_uartcfg struct, then if those fields are zero a default config is used, with PCLK (clk_p) as clock source.

Additional clock sources can be defined to achieve some special baud rates, but for 115200 the 83 MHz of PCLKD0 is perfectly fine, with almost integral divisor value.

EDIT: Disabling early printk might be worth trying as it might somehow interfere with the full UART driver.
[Galaxy 3 Android Development] hillbeast:
Okay I'll try disable early_printk and look at what has been set in the uart_cfg.

Sent from my Serenity Galaxy S2
[Galaxy 3 Android Development] moikop:
Already tried that (disabling early_printk, NOT debug_ll) and changes nothing..

Disabling debug_ll only makes it output nothing until the uart driver is initialized (and the output is screw'd anyway, so...)

Is it possible, because the way clocks and devices are initialized changed quite a bit between .39 and 3.4, that's causing this problem?
[Galaxy 3 Android Development] hillbeast:
I just added the OneDRAM to the memory so now the phone has 336MB of RAM. Of course some of that will be allocated to certain hardware like the GPU and all that, but at

least it can be allocated by Android and Ion and all that crap rather than limiting us to a set amount.

<6>Memory: 256MB 80MB = 336MB total
<5>Memory: 328112k/328112k available, 15952k reserved, 0K highmem
<5>Virtual kernel memory layout:
vector : 0xffff0000 - 0xffff1000 ( 4 kB)
fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB)
vmalloc : 0xe5800000 - 0xff000000 ( 408 MB)
lowmem : 0xc0000000 - 0xe5000000 ( 592 MB)
modules : 0xbf000000 - 0xc0000000 ( 16 MB)
.text : 0xc0008000 - 0xc08b1000 (8868 kB)
.init : 0xc08b1000 - 0xc08fb000 ( 296 kB)
.data : 0xc08fc000 - 0xc0971d28 ( 472 kB)
.bss : 0xc0971d4c - 0xc0b50ee8 (1917 kB)

In other news, I'm still looking at the UART, and noticed an error come up. Unfortunately I can't see all the text of it so I had to find it in the source, and I tracked it down to this line: drivers/tty/serial/samsung.c:263.

I'll find out why it's coming up with this error.