[General Omni Discussion] Lanchon:
mitigation

it doesn't look like we are making enough progress; maybe it's time to implement mitigation.

performance:

1) linux:

the situation in ARM linux is this: every time a time-slice of a userland process ends and the process is switched out of a core, the CPU context is saved. if the FPU is in use by the process, this will also include the FPU context, which is *huge* (and thus very expensive to save and restore). but when a userland context is switched in, its FPU state is left out to avoid the high cost of moving the FPU context around, and the FPU is instead simply disabled by the kernel. if userland ever touches the FPU during this time-slice, the CPU will trap and the kernel will restore the FPU context only then, once it was shown to be necessary for this time-slice. it will also schedule an FPU context save for when this slice ends. this mode of operation is called "eager save, lazy restore" and is used in multicore ARM linux.

2) gcc:

gcc will happily use the FPU to generate more efficient code, if allowed to do so. it will use the FPU's huge register bank to spill CPU registers there instead of into memory, which should be faster. it can emit NEON instructions to make certain operations faster, it can vectorize loops, etc.

3) C runtime lib:

some library implementations can be built to use NEON-optimized versions of some functions. typically memcpy memclr strcpy strcmp etc have alternate NEON implementations than execute noticeably faster. (i ignore if these versions kick in conditionally at runtime for larger data only, or are always used.)

the result:

gcc and the C runtime seek top performance in a monotasking world. in reality, if gcc only saves a temporary value to an FPU register, it will cause an additional trap and a full FPU context restore, and later a save too, which is *extremely* more expensive than a memory access. for some processes these 'optimizations' might be causing great performance loss.

take the case of the sdcard.c fuse process: when a userland process reads a fuse file, the kernel will have to do a full switch to fuse userland (assuming other cores are in use). fuse will then read the 'real' file and block while the hardware fulfills the read. presumably the kernel will do another switch to an unrelated userland process that is waiting and then back into fuse when the hardware's response is ready. finally after fuse packages the hardware's response into its own response, the kernel will switch back to the userland that read the file.

in all, for a simple read call, the kernel did 2 extra userland context switches because of fuse compared to what would have been if the the target file system was implemented in the kernel. now fuse doesn't need to make FPU calculations, and touching the FPU just to spill registers (as it does) potentially causes 4 extra FPU context switches and 2 lazy restore traps. all this to avoid the 'cost' of a couple memory accesses!

granted, fuse copies memory and might benefit from a NEON-optimized memcpy. but still, for such a shortly executing, often switched in and out background process that doesn't do FP math, i suppose touching the FPU just causes loss of performance and battery drain.

the general case might be that background userland processes that don't do math and don't do memory blitting are best forbidden to touch the FPU for silly stuff like register spilling or the odd high efficiency instruction.

in fact, i believe GCC should support a new mode of compilation in which it will touch the FPU *only* for math, and avoid it for anything else that represents small gains.

in short: compiling parts of android (for all phones) without FPU support might increase performance and battery life.

bug mitigation:

as we know, compiling parts of android without FPU support will make these parts immune to the FP bug in the 4210 platform.

what can be done:

disabling FPU support in processes that actually do math can be a huge performance hit and should be avoided. how can we know? there can be others, but one way is this: gcc can compile code without FPU support, but it will emit FP library calls if there are any real FP operations in the source. by excluding the software FP library from the link stage, the modules that actually do FP will fail to link and can thus be identified. the rest can be compiled without FP, probably increasing performance and battery life, and in the case of the 4210 platform, working around the FP bug (for those processes).

the library is more difficult. code does a lot of memcpy and memclr, and most of it are very short chunks of memory and paying the price of a couple FPU context switches for the 'optimization' might not be reasonable. on the other hand, a minority of calls are probably very big and would clearly benefit from NEON.

in an ideal world, string functions (strcpy strcmp etc) would be non-NEON, and known-length calls (memcpy memclr etc) would dynamically select the NEON version for chunks bigger than some experimentally derived constant(s). in this world, androids with NEON fully on and fully off in the library can be compared in real life, and one of the two options selected. for the 4210 platform, strong bias towards non-NEON should be applied, given that NEON use in the library makes all code potentially brittle.