Please help me refort this documentation in full.

START OF DOCUMENTATION TO REWRITE {
Neon technology is an advanced SIMD (Single Instruction, Multiple Data) architecture for the Arm Cortex-A series processors. It can accelerate multimedia and signal processing algorithms such as video encoder/decoder, 2D/3D graphics, gaming, audio and speech processing, image processing, telephony, and sound.

Neon instructions perform "Packed SIMD" processing:

Registers are considered as vectors of elements of the same data type
Data types can be: signed/unsigned 8-bit, 16-bit, 32-bit, 64-bit, single-precision floating-point on ARM 32-bit platform, both single-precision floating-point and double-precision floating-point on ARM 64-bit platform.
Instructions perform the same operation in all lanes
History of Arm Adv SIMD
Armv6
SIMD extension Armv7-A
Neon Armv8-A AArch64
Neon
• Operates on 32-bit general purpose ARM registers
• 8-bit/16-bit integer
• 2x16-bit/4x8-bit operations per instruction • Separate register bank, 32x64-bit Neon registers
• 8/16/32/64-bit integer
• Single precision floating point
• Up to 16x8-bit operations per instruction • Separate register bank, 32x128-bit Neon registers
• 8/16/32/64-bit integer
• Single precision floating point
• Double precision floating point, both of them are IEEE compliance
• Up to 16x8-bit operations per instruction
Why use Neon
Neon provides:

Support for both integer and floating point operations ensures the adaptability of a broad range of applications, from codecs to High Performance Computing to 3D graphics.
Tight coupling to the Arm processor provides a single instruction stream and a unified view of memory, presenting a single development platform target with a simpler tool flow
Armv7/v8 comparison
Armv8-A is a fundamental change to the Arm architecture. It supports the 64-bit Execution state called “AArch64”, and a new 64-bit instruction set “A64”. To provide compatibility with the Armv7-A (32-bit architecture) instruction set, a 32-bit variant of Armv8-A “AArch32” is provided. Most of existing Armv7-A code can be run in the AArch32 execution state of Armv8-A.

This section compares the Neon-related features of both the Armv7-A and Armv8-A architectures. In addition, general purpose Arm registers and Arm instructions, which are used often for Neon programming, will also be mentioned. However, the focus is still on the Neon technology.

Register
Armv7-A and AArch32 have the same general purpose Arm registers – 16 x 32-bit general purpose Arm registers (R0-R15).

Armv7-A and AArch32 have 32 x 64-bit Neon registers (D0-D31). These registers can also be viewed as 16x128-bit registers (Q0-Q15). Each of the Q0-Q15 registers maps to a pair of D registers
AArch64 by comparison, has 31 x 64-bit general purpose Arm registers and 1 special register having different names, depending on the context in which it is used. These registers can be viewed as either 31 x 64-bit registers (X0-X30) or as 31 x 32-bit registers (W0-W30).

AArch64 has 32 x 128-bit Neon registers (V0-V31). These registers can also be viewed as 32-bit Sn registers or 64-bit Dn registers.The Armv8-A AArch32 instruction set consists of A32 (Arm instruction set, a 32-bit fixed length instruction set) and T32 (Thumb instruction set, a 16-bit fixed length instruction set; Thumb2 instruction set, 16 or 32-bit length instruction set). It is a superset of the Armv7-A instruction set, so that it retains the backwards compatibility necessary to run existing software. There are some additions to A32 and T32 to maintain alignment with the A64 instruction set, including Neon division, and the Cryptographic Extension instructions. Neon double precision floating point (IEEE compliance) is also supported.Armv7-A/AArch32 instruction syntax
All mnemonics for Armv7-A/AAArch32 Neon instructions (as with VFP) begin with the letter “V”. Instructions are generally able to operate on different data types, with this being specified in the instruction encoding. The size is indicated with a suffix to the instruction. The number of elements is indicated by the specified register size and data type of operation. Instructions have the following general format:

V{<mod>}<op>{<shape>}{<cond>}{.<dt>}{<dest>}, src1, src2

Where:

<mod> - modifiers

Q: The instruction uses saturating arithmetic, so that the result is saturated within the range of the specified data type, such as VQABS, VQSHL etc.
H: The instruction will halve the result. It does this by shifting right by one place (effectively a divide by two with truncation), such as VHADD, VHSUB.
D: The instruction doubles the result, such as VQDMULL, VQDMLAL, VQDMLSL and VQ{R}DMULH
R: The instruction will perform rounding on the result, equivalent to adding 0.5 to the result before truncating, such as VRHADD, VRSHR.
<op> - the operation (for example, ADD, SUB, MUL).

<shape> - Shape.

Neon data processing instructions are typically available in Normal, Long, Wide and Narrow variants.

Long (L): instructions operate on double-word vector operands and produce a quad-word vector result. The result elements are twice the width of the operands, and of the same type. Lengthening instructions are specified using an L appended to the instruction.Wide (W): instructions operate on a double-word vector operand and a quad-word vector operand, producing a quad-word vector result. The result elements and the first operand are twice the width of the second operand elements. Widening instructions have a W appended to the instruction.Narrow (N): instructions operate on quad-word vector operands, and produce a double-word vector result. The result elements are half the width of the operand elements. Narrowing instructions are specified using an N appended to the instruction.<cond> - Condition, used with IT instruction

<.dt> - Data type, such as s8, u8, f32 etc.

<dest> - Destination

<src1> - Source operand 1

<src2> - Source operand 2
Note: {} represents and optional parameter.

For example:

VADD.I8 D0, D1, D2

VMULL.S16 Q2, D8, D9
AArch64 Neon instruction syntax
In the AArch64 execution state, the syntax of Neon instruction has changed. It can be described as follows:

{<prefix>}<op>{<suffix>} Vd.<T>, Vn.<T>, Vm.<T>

Where:

<prefix> - prefix, such as using S/U/F/P to represent signed/unsigned/float/bool data type.

<op> – operation, such as ADD, AND etc.

<suffix> - suffix

P: “pairwise” operations, such as ADDP.
V: the new reduction (across-all-lanes) operations, such as FMAXV.
2：new widening/narrowing “second part” instructions, such as ADDHN2, SADDL2.
ADDHN2: add two 128-bit vectors and produce a 64-bit vector result which is stored as high 64-bit part of Neon register.

SADDL2: add two high 64-bit vectors of Neon register and produce a 128-bit vector result.

<T> - data type, 8B/16B/4H/8H/2S/4S/2D. B represents byte (8-bit). H represents half-word (16-bit). S represents word (32-bit). D represents a double-word (64-bit).

For example:
UADDLP V0.8H, V0.16B

FADD V0.4S, V0.4S, V0.4S
Neon instructions
The following table compares the Armv7-A, AArch32 and AArch64 Neon instruction set.

“√” indicates that the AArch32 Neon instruction has the same format as Armv7-A Neon instruction.

“Y” indicates that the AArch64 Neon instruction has the same functionality as Armv7-A Neon instructions, but the format is different. Please check the Armv8-A ISA document.

If you are familiar with the Armv7-A Neon instructions, there is a simple way to map the Neon instructions of Armv7-A and AArch64. It is to check the Neon intrinsics document, so that you can find the AArch64 Neon instruction according to the intrinsics instruction.

New or changed functionality is highlighted.

Armv7-A AArch32 AArch64
logical and compare VAND, VBIC, VEOR, VORN, and VORR (register)
√

Y
VBIC and VORR (immediate)
√

Y
VBIF, VBIT, and VBSL
√

Y
VMOV, VMVN (register)
√

Y
VACGE and VACGT
√

Y
VCEQ, VCGE, VCGT, VCLE, and VCLT
√

Y
VTST
√

Y
general data processing VCVT (between fixed-point or integer, and floating-point)
√

Y
VCVT (between half-precision and single-precision floating-point)
√

Y
n/a n/a FCVTXN(double)
to single-precision)
VDUP
√

Y
VEXT
√

Y
VMOV, VMVN (immediate)
√

Y
VMOVL, V{Q}MOVN, VQMOVUN
√

Y
VREV
√

Y
VSWP
√

n/a
VTBL, VTBX
√

Y
VTRN
√

TRN1, TRN2
VUZP, VZIP
√

UZP1,UZP2, ZIP, ZIP2
n/a n/a INS
n/a VRINTA, VRINM,
VRINTN, VRINTP,
VRINTR, VRINTX,
VRINTZ FRINTA, FRINTI, FRINTM, FRINTN, FRINTP, FRINTX, FRINTZ
shift VSHL, VQSHL, VQSHLU, and VSHLL (by immediate)
√

Y
V{Q}{R}SHL (by signed variable)
√

Y
V{R}SHR
√

Y
V{R}SHRN
√

Y
V{R}SRA
√

Y
VQ{R}SHR{U}N
√

Y
VSLI and VSRI
√

Y
general arithmetic VABA{L} and VABD{L}
√

Y
V{Q}ABS and V{Q}NEG
√

Y
V{Q}ADD, VADDL, VADDW, V{Q}SUB, VSUBL, and VSUBW
√

Y
n/a n/a SUQADD, USQADD
V{R}ADDHN and V{R}SUBHN
√

Y
V{R}HADD and VHSUB
√

Y
VPADD{L}, VPADAL
√

Y
VMAX, VMIN, VPMAX, and VPMIN
√

Y
n/a n/a FMAXNMP, FMINNMP
VCLS, VCLZ, and VCNT
√

Y
VRECPE and VRSQRTE
√

Y
VRECPS and VRSQRTS
√

Y
n/a n/a FRECPX
RBIT
FSQRT
ADDV
SADDLV, UADDLV
SMAXV,UMAXV,FMAXV
FMAXNMV
SMINV,UMINV,FMINV
FMINNMV
multiply VMUL{L}, VMLA{L}, and VMLS{L}
√

There isn’t float MLA/MLS
VMUL{L}, VMLA{L}, and VMLS{L} (by scalar)
√

Y
VFMA, VFMS
√

Y
VQDMULL, VQDMLAL, and VQDMLSL (by vector or by scalar)
√

Y
VQ{R}DMULH (by vector or by scalar)
√

Y
n/a n/a FMULX
n/a n/a FDIV
load and store VLDn/VSTn(n=1, 2, 3, 4)
√

Y
VPUSH/VPOP
√

n/a
Crypto Extension n/a PMULL, PMULL2 PMULL, PMULL2
AESD, AESE AESD, AESE
AESIMC, AESMC AESIMC, AESMC
SHA1C, SHA1H, SHA1M, SHA1P SHA1C, SHA1H, SHA1M, SHA1P
SHA1SU0,
SHA1SU1 SHA1SU0,
SHA1SU1
SHA256H,
SHA256H2 SHA256H,
SHA256H2
SHA256SU0,
SHA256SU1 SHA256SU0,
SHA256SU1
Neon programming basics
There are four ways of using Neon

Neon optimized libraries
Vectorizing compilers
Neon intrinsics
Neon assembly
Libraries
The users can call the Neon optimized libraries directly in their program. Currently, you can use the following libraries:

OpenMax DL
This provides the recommended approach for accelerating AV codecs and supports signal processing and color space conversions.

Ne10
It is Arm’s open source project. Currently, the Ne10 library provides some math, image processing and FFT function. The FFT implementation is faster than other open source FFT implementations.

Vectorizing compilers
Adding vectorizing options in GCC can help C code to generate Neon code. GNU GCC gives you a wide range of options that aim to increase the speed, or reduce the size of the executable files they generate. For each line in your source code there are generally many possible choices of assembly instructions that could be used. The compiler must trade-off a number of resources, such as registers, stack and heap space, code size (number of instructions), compilation time, ease of debug, and number of cycles per instruction in order to produce an optimized image file.

Neon intrinsics
Neon intrinsics provides a C function call interface to Neon operations, and the compiler will automatically generate relevant Neon instructions allowing you to program once and run on either an Armv7-A or Armv8-A platform. If you intend to use the AArch64 specific Neon instructions, you can use the (aarch64) macro definition to separate these codes, as in the following example.

Neon intrinsics example:
//add for float array. assumed that count is multiple of 4

#include<arm_neon.h>

void add_float_c(float* dst, float* src1, float* src2, int count)

{
text
 int i;

 for (i = 0; i < count; i++)

     dst[i] = src1[i] + src2[i];

}

void add_float_neon1(float* dst, float* src1, float* src2, int count)

{
text
 int i;

 for (i = 0; i < count; i += 4)

 {

     float32x4_t in1, in2, out;

     in1 = vld1q_f32(src1);

     src1 += 4;

     in2 = vld1q_f32(src2);

     src2 += 4;

     out = vaddq_f32(in1, in2);

     vst1q_f32(dst, out);

     dst += 4;

     // The following is only an example describing how to use AArch64 specific NEON

     // instructions.                             

#if defined (aarch64)
text
     float32_t tmp = vaddvq_f32(in1);

#endif
text
 }

}
Checking disassembly, you can find vld1/vadd/vst1 Neon instruction on Armv7-A platform and ldr/fadd/str Neon instruction on Armv8-A platform.

Neon assembly
There are two ways to write Neon assembly:

Assembly files
Inline assembly
Assembly files
You can use ".S" or “.s” as the file suffix. The only difference is that C/C ++ preprocessor will process .S files first. C language features such as macro definitions can be used.

When writing Neon assembly in a separate file, you need to pay attention to saving the registers. For both Armv7 and Armv8, the following registers must be saved:

Armv7-A/AArch32 AArch64
General purpose registers R0-R3 parameters
R4-R11 need to be saved
R12 IP
R13(SP)
R14(LR) need to be saved
R0 for return value X0-X7 parameters
X8-X18
X19-X28 need to be saved
X29(FP) need to be saved
X30(LR)
X0, X1 for return value
Neon registers D8-D15 need to be saved D part of V8-V15 need to be saved
Stack alignment 64-bit alignment 128-bit alignment
Stack push/pop PUSH/POP Rn list
VPUSH/VPOP Dn list LDP/STP register pair
The following is an example of ARM v7-A and ARM v8-A Neon assembly.

//header
void add_float_Neon2(float* dst, float* src1, float* src2, int count);
//assembly code in .S file
Armv7-A/AArch32 AArch64
.text .syntax unified .align 4 .global add_float_Neon2 .type add_float_Neon2, %function .thumb .thumb_func add_float_Neon2: .L_loop: vld1.32 {q0}, [r1]!
vld1.32 {q1}, [r2]!
vadd.f32 q0, q0, q1
subs r3, r3, #4
vst1.32 {q0}, [r0]!
bgt .L_loop
bx lr .text
.align 4
.global add_float_Neon2
.type add_float_Neon2, %function
add_float_Neon2:
.L_loop:
ld1 {v0.4s}, [x1], #16
ld1 {v1.4s}, [x2], #16
fadd v0.4s, v0.4s, v1.4s
subs x3, x3, #4
st1 {v0.4s}, [x0], #16
bgt .L_loop
ret
For more examples, see GitHub.

Inline assembly
You can use Neon inline assembly directly in C/C++ code.

Pros:

The procedure call standard is simple. You do not need to save registers manually.
You can use C / C ++ variables and functions, so it can be easily integrated into C / C ++ code.
Cons:

Inline assembly has a complex syntax.
Neon assembly code is embedded in C/C ++ code, and it’s not easily ported to other platforms.
Example:
// Armv7-A/AArch32
void add_float_neon3(float* dst, float* src1, float* src2, int count)
{
asm volatile (
"1: \n"
"vld1.32 {q0}, [%[src1]]! \n"
"vld1.32 {q1}, [%[src2]]! \n"
"vadd.f32 q0, q0, q1 \n"
"subs %[count], %[count], #4 \n"
"vst1.32 {q0}, [%[dst]]! \n"
"bgt 1b \n"
: [dst] "+r" (dst)
: [src1] "r" (src1), [src2] "r" (src2), [count] "r" (count)
: "memory", "q0", "q1"
);
} // AArch64
void add_float_neon3(float* dst, float* src1, float* src2, int count)
{
asm volatile (
"1: \n"
"ld1 {v0.4s}, [%[src1]], #16 \n"
"ld1 {v1.4s}, [%[src2]], #16 \n"
"fadd v0.4s, v0.4s, v1.4s \n"
"subs %[count], %[count], #4 \n"
"st1 {v0.4s}, [%[dst]], #16 \n"
"bgt 1b \n"
: [dst] "+r" (dst)
: [src1] "r" (src1), [src2] "r" (src2), [count] "r" (count)
: "memory", "v0", "v1"
);
}
Neon intrinsics and assembly
Neon intrinsics and assembly are the commonly used Neon. The following table describes the pros and cons of these two approaches:

Neon assembly Neon intrinsic
Performance Always shows the best performance for the specified platform for an experienced developer. Depends heavily on the toolchain used
Portability The different ISAs (ARMv7-A/AArch32 and AArch64) have different assembly implementations. Even for the same ISA, the assembly might need to be fine-tuned to achieve ideal performance between different micro architectures. Program once and run on different ISA’s. The compiler may also grant performance fine-tuning for different micro-architectures.
Maintainability Hard to read/write compared to C. Similar to C code, it’s easy to read/write.
With the above information, you can choose a Neon implementation and start your Neon programming journey.

ARM NEON optimization guide!
1. Introduction After reading the article ARM NEON programming quick reference, I believe you have a basic understanding of ARM NEON programming. But when applying ARM NEON to a real-world applications, there are many programming skills to observe.This article aims to introduce some common NEON optimization skills which come from development practice. The issue of NEON assembly and intrinsics will also be discussed.
2. NEON optimization skills When using NEON to optimize applications, there are some commonly used optimization skills as follows.
2.1. Remove data dependencies
On the ARMv7-A platform, NEON instructions usually take more cycles than ARM instructions. To reduce instruction latency, it’s better to avoid using the destination register of current instruction as the source register of next instruction.

Example:
C code:
float SumSquareError_C(const float* src_a, const float* src_b, int count)

{

float sse = 0u;

int i;

for (i = 0; i < count; ++i) {
text
float diff = src_a[i] - src_b[i];

sse += (float)(diff * diff);

}

return sse;

}

NEON implementation 1

float SumSquareError_NEON1(const float* src_a, const float* src_b, int count)

{

float sse;

asm volatile (
text
// Clear q8, q9, q10, q11

"veor    q8, q8, q8                            \n"

"veor    q9, q9, q9                            \n"

"veor    q10, q10, q10                     \n"

"veor    q11, q11, q11                     \n"

"1: \n"
text
"vld1.32     {q0, q1}, [%[src_a]]!       \n"

"vld1.32     {q2, q3}, [%[src_a]]!       \n"

"vld1.32     {q12, q13}, [%[src_b]]!  \n"

"vld1.32     {q14, q15}, [%[src_b]]!  \n"

"subs %[count], %[count], #16 \n"

// q0, q1, q2, q3 are the destination of vsub.

// they are also the source of vmla.
text
"vsub.f32 q0, q0, q12                      \n"

"vmla.f32   q8, q0, q0                        \n"

"vsub.f32   q1, q1, q13                      \n"

"vmla.f32   q9, q1, q1                       \n"

"vsub.f32   q2, q2, q14                    \n"

"vmla.f32   q10, q2, q2                    \n"

"vsub.f32   q3, q3, q15                    \n"

"vmla.f32   q11, q3, q3                    \n"

"bgt        1b                                        \n"

"vadd.f32   q8, q8, q9                      \n"

"vadd.f32   q10, q10, q11               \n"

"vadd.f32   q11, q8, q10                 \n"

"vpadd.f32  d2, d22, d23                \n"

"vpadd.f32  d0, d2, d2                     \n"

"vmov.32    %3, d0[0]                      \n"

: "+r"(src_a),

  "+r"(src_b),

  "+r"(count),

  "=r"(sse)

:

: "memory", "cc", "q0", "q1", "q2", "q3", "q8", "q9", "q10", "q11",

  "q12", "q13","q14", "q15");

return sse;

}

NEON implementation 2

float SumSquareError_NEON2(const float* src_a, const float* src_b, int count)

{

float sse;

asm volatile (
text
// Clear q8, q9, q10, q11

"veor    q8, q8, q8                            \n"

"veor    q9, q9, q9                            \n"

"veor    q10, q10, q10                     \n"

"veor    q11, q11, q11                     \n"

"1: \n"
text
"vld1.32     {q0, q1}, [%[src_a]]!       \n"

"vld1.32     {q2, q3}, [%[src_a]]!       \n"

"vld1.32     {q12, q13}, [%[src_b]]!  \n"

"vld1.32     {q14, q15}, [%[src_b]]!  \n"

"subs       %[count], %[count], #16  \n"

"vsub.f32 q0, q0, q12                      \n"

"vsub.f32   q1, q1, q13                     \n"

"vsub.f32   q2, q2, q14                     \n"

"vsub.f32   q3, q3, q15                     \n"

"vmla.f32   q8, q0, q0                      \n"

"vmla.f32   q9, q1, q1                      \n"

"vmla.f32   q10, q2, q2                    \n"

"vmla.f32   q11, q3, q3                    \n"

"bgt        1b                                         \n"

"vadd.f32   q8, q8, q9                      \n"

"vadd.f32   q10, q10, q11                \n"

"vadd.f32   q11, q8, q10                  \n"

"vpadd.f32  d2, d22, d23                 \n"

"vpadd.f32  d0, d2, d2                      \n"

"vmov.32    %3, d0[0]                       \n"

: "+r"(src_a),

  "+r"(src_b),

  "+r"(count),

  "=r"(sse)

:

: "memory", "cc", "q0", "q1", "q2", "q3", "q8", "q9", "q10", "q11",

  "q12", "q13","q14", "q15");

return sse;

}
In NEON implementation 1, the destination register is used as source register immediately; In NEON implementation 2, instructions are rescheduled and given the latency as much as possible. The test result indicates that implementation 2 is ~30% faster than implementation 1. Thus, reducing data dependencies can improve performance significantly. A good news is that compiler can fine-tune NEON intrinsics automatically to avoid data dependencies which is really one of the big advantages.

Note: this test runs on Cortex-A9. The result may be different on other platforms.

2.2 Reduce branches
There isn’t branch jump instruction in NEON instruction set. When the branch jump is needed, jump instructions of ARM are used. In ARM processors, branch prediction techniques are widely used. But once the branch prediction fails, the punishment is rather high. So it’s better to avoid the using jump instructions. In fact, logical operations can be used to replace branch in some cases.

Example:

C implementation

if( flag )

{
text
    dst[x * 4]       = a;

    dst[x * 4 + 1] = a;

    dst[x * 4 + 2] = a;

    dst[x * 4 + 3] = a;

}

else

{
text
    dst[x * 4]       = b;

    dst[x * 4 + 1] = b;

    dst[x * 4 + 2] = b;

    dst[x * 4 + 3] = b;

}
NEON implementation
//dst[x * 4] = (a&Eflag) | (b&~Eflag);

//dst[x * 4 + 1] = (a&Eflag) | (b&~Eflag);

//dst[x * 4 + 2] = (a&Eflag) | (b&~Eflag);

//dst[x * 4 + 3] = (a&Eflag) | (b&~Eflag);

VBSL qFlag, qA, qB
ARM NEON instruction set provides the instructions as follows to help users implement the logical operation above:

VCEQ, VCGE, VCGT, VCLE, VCLT
VBIT, VBIF, VBSL
Reducing branches is not specific to NEON only. It is a commonly used trick. Even in a C program, this trick is also worth the effort.
2.3 Preload data-PLD[i]
ARM processors are a load/store system. Except load/store instructions, all operations perform on registers. Therefore increasing the efficiency of load/store instructions is very important for optimizing application.

Preload instruction allows the processor to signal the memory system that a data load from an address is likely in the near future. If the data is preloaded into cache correctly, it would be helpful to improve the rate of cache hit which can boost performance significantly. But the preload is not a panacea. It’s very hard to use on recent processors and it can be harmful too. A bad preload will reduce performance.

PLD syntax:
PLD{cond} [Rn {, #offset}]
text
PLD{cond} [Rn, +/-Rm {, shift}]

PLD{cond} label

Where:

Cond - is an optional condition code.
Rn - is the register on which the memory address is based.
Offset - is an immediate offset. If offset is omitted, the address is the value in Rn.
Rm - contains an offset value and must not be PC (or SP, in Thumb state).
Shift - is an optional shift.
Label - is a PC-relative expression.

The PLD operation features:

Independent of load and store instruction execution
Happens in the background while the processor continues to execute other instructions.
The offset is specified to real cases.
2.4 Misc
In ARM NEON programming, Different instruction sequences can be used to perform the same operation. But fewer instructions do not always produce better performance. It depends on benchmark and profiling result of specific cases. Below listed are some special cases in development practice.

2.4.1 Floating-point VMLA/VMLS instruction
This example is specific for Cortex-A9. For other platforms, the result needs to be verified again.

Usually, VMUL+VADD/VMUL+VSUB can be replaced by VMLA/VMLS because fewer instructions are used. But compared to floating-point VMUL, floating-point VMLA/VMLS has a longer instruction delay. If there aren’t other instructions that can be inserted into delay slot, using floating-point VMUL+VADD/VMUL+VSUB will show a better performance.

A real-world example is floating-point FIR function in Ne10. The code snippets are as follows:

Implementation 1: there is only one instruction “VEXT” between two “VMLA” which needs 9 execution cycles according to the table of NEON floating-point instructions timing.
VEXT qTemp1,qInp,qTemp,#1

VMLA qAcc0,qInp,dCoeff_0[0]

VEXT qTemp2,qInp,qTemp,#2

VMLA qAcc0,qTemp1,dCoeff_0[1]

VEXT qTemp3,qInp,qTemp,#3

VMLA qAcc0,qTemp2,dCoeff_1[0]

VMLA qAcc0,qTemp3,dCoeff_1[1]
Implementation 2: there is still data dependency on qAcc0. But VADD/VMUL needs 5 execution cycles only.
VEXT qTemp1,qInp,qTemp,#1

VMLA qAcc0,qInp,dCoeff_0[0] ]

VMUL qAcc1,qTemp1,dCoeff_0[1]

VEXT qTemp2,qInp,qTemp,#2

VMUL qAcc2,qTemp2,dCoeff_1[0]

VADD qAcc0, qAcc0, qAcc1

VEXT qTemp3,qInp,qTemp,#3

VMUL qAcc3,qTemp3,dCoeff_1[1]

VADD qAcc0, qAcc0, qAcc2

VADD qAcc0, qAcc0, qAcc3
The benchmark shows that implementation 2 has a better performance.

For more code details, see GitHub.

NEON floating-point instructions timing:

Name Format Cycles Result
VADDVSUBVMUL Dd,Dn,Dm 1 5
Qd,Qn,Dm 2 5
VMLAVMLS Dd,Dn,Dm 1 9
Qd,Qn,Dm 2 9
The table is from appendix[ii]。

Where:

Cycles: instruction issue
Result: instruction execute
2.5 Summary
NEON optimization techniques are summarized as follows:

Utilize the delay slot of instruction as much as possible.
Avoid branches.
Pay attention to cache hit.
3. NEON assembly and intrinsics
In “ARM NEON programming quick guide”, there is a simple comparison of the pros and cons of NEON assembly and intrinsics:

NEON assembly NEON intrinsic
Performance Always shows the best performance for the specified platform for an experienced developer. Depends heavily on the toolchain that is used
Portability The different ISA (i.e. ARMv7-A/v8-A AArch32 and ARMv8-A AArch64) has different assembly implementation. Even for the same ISA, the assembly might need to be fine-tuned to achieve ideal performance between different micro architectures. Program once and run on different ISA’s. The compiler may also grant performance fine-tuning for different micro-architectures.
Maintainability Hard to read/write compared with C. Similar to C code, it’s easy to read/write.
But the reality is far more complex than many of these, especially when it comes to ARMv7-A/v8-A cross-platform. In the following sections, this issue will be analyzed further with some examples.

3.1 Programming
For NEON beginners, intrinsics are easier to use than assembly. But experienced developers may be more familiar with NEON assembly programming. They need time to adapt to the intrinsics coding. Some issues that may occur in real development are described in the following.

3.1.1 Flexibility of instruction
From the perspective of using instructions, assembly instruction is more flexible than intrinsics. It is mainly reflected in the data load / store.

Example:

Intrinsics instruction Load data into a 64-bit register, vld1_s8/u8/s16/u16/s32…etc
Load data into a 128-bit register, vld1q_s8/u8/s16/u16/s32…etc
ARMv7-A assembly VLD1 { Dd}, [Rn]
VLD1 { Dd, D(d+1) }, [Rn]
VLD1 { Dd, D(d+1), D(d+2)}, [Rn]
VLD1 { Dd, D(d+1), D(d+2), D(d+3) }, [Rn]
ARMv8-A assembly LD1 { .<T}, [<Xn|SP>]
LD1 { ., .}, [<Xn|SP>]
LD1 { ., ., . }, [<Xn|SP>]
LD1 { ., ., ., . }, [<Xn|SP>]
This issue will be fixed with upgrading the compiler in the future. Sometimes, the compiler has been able to translate two intrinsics instructions into one assembly instruction, such as:

More info here:https://github.com/projectNe10/Ne10/blob/97c162781c83584851ea3758203f9d2aa46772d5/modules/dsp/NE10_cfft.neon.s
Therefore, it’s expected that intrinsics instruction will have the same flexibility with assembly instructions, with the upgrade of the ARMv8 toolchain.

3.1.2 Register allocation
When programming in NEON assembly, registers have to be allocated by users. It must be known clearly which registers are occupied. One of benefits of programming in intrinsics is that users only need to define variables. The compiler will allocate registers automatically. This is an advantage, but it might be a weakness in some cases. Practice has proved that using too many NEON registers simultaneously in the intrinsics programming will bring gcc compiler register allocation issue. When this issue happens, many data are pushed into the stack, which will greatly affect performance of the program. Therefore, users should pay attention to this issue when intrinsics programming. When there is performance exception (such as the performance of C is better than it of NEON), you need to check the disassembly to see if there is register allocation issue firstly. For ARMv8-A AArch64, there are more registers (32 128-bit NEON registers). The impact of this issue is significantly reduced.

3.2 Performance and compiler
On one platform, the performance of NEON assembly is only decided by implementation, and has nothing to do with the compiler. The benefit is that you can predict and control the performance of program when you hand-tune the codes, but there isn’t surprise.

Conversely, the performance of NEON intrinsics is greatly dependent on the compiler used. Different compilers may bring very different performance. In general, the older the compiler, the worse the performance. When compatibility with older compilers is needed, it must be considered carefully whether the intrinsics will fit the need. In addition, when fine-tuning the code, it’s hard for user to predict the change of performance with the intervention of the compiler. But there may be surprise. Sometimes the intrinsics might bring better performance than assembly. This is very rare, but does occur.

Compiler will have an impact on the process of NEON optimization. The following figure describes the general process of NEON implementation and optimization.

NEON assembly and intrinsics have the same process of implementation, coding - debugging – performance test. But they have different process of optimization step.The methods of assembly fine-tuning are:

Change implementations, such as changing the instructions, adjusting the parallelism.
Adjust the instruction sequence to reduce data dependency.
Try the skills described in section 2.
When fine-tuning assembly, a sophisticated way is that:

Know the number of used instructions exactly
Get the execution cycles of program by using the PMU (Performance Monitoring Unit).
Adjust the sequence of instructions based on the timing of used instructions. And try to minimize the instruction delay as possible as you can
The disadvantage of this approach is that the changes are specific to one micro-architecture. When the platform is switched, performance improvement achieved might be lost. This is also very time-consuming to do for often comparatively small gains.Fine-tuning of NEON intrinsics is more difficult.

Try the methods used in NEON assembly optimization.
Look at the generated assembly and check the data dependencies and register usage.
Check whether the performance meets the expectation. If yes, the work of optimization is done. Then the performance with other compilers needs to be verified again.
When porting the assembly code of ARMv7-A with intrinsics for ARMv7-A/v8-A compatibility, performance of assembly can be used as a reference of performance. So it is easy to check whether the work is done. However, when intrinsics are used to optimize ARMv8-A code, there is not a performance reference. It is difficult to determine whether the performance is optimal. Based on the experience on ARMv7-A, there might be a doubt whether the assembly has the better performance. I think the impact of this issue will become smaller and smaller with the maturity of the ARMv8-A environment.

3.3 Cross-platform and portability
Now, most of the existing NEON assembly codes can only run on the platforms of ARMv7-A/ARMv8-A AArch32 mode. If you want to run them on platforms of ARMv8-A AArch64 mode, you must rewrite these codes, which take a lot of work. In such situation, if the codes are programmed with NEON intrinsics, they can be run directly on platforms of ARMv8-A AArch64 mode. Cross-platform is one of great advantages. Meanwhile, you just need to maintain on set of code for different platform by using intrinsics, which also significantly reduces the maintenance effort. However, due to the different hardware resources on ARMv7-A/ARMv8-A platform, sometimes there still might be two sets of code even with intrinsics. The FFT implementation in Ne10 project is an example:
// radix 4 butterfly with twiddlesscratch[0].r = scratch_in[0].r;scratch[0].i = scratch_in[0].i;scratch[1].r = scratch_in[1].r * scratch_tw[0].r - scratch_in[1].i * scratch_tw[0].i;scratch[1].i = scratch_in[1].i * scratch_tw[0].r + scratch_in[1].r * scratch_tw[0].i;scratch[2].r = scratch_in[2].r * scratch_tw[1].r - scratch_in[2].i * scratch_tw[1].i;scratch[2].i = scratch_in[2].i * scratch_tw[1].r + scratch_in[2].r * scratch_tw[1].i;scratch[3].r = scratch_in[3].r * scratch_tw[2].r - scratch_in[3].i * scratch_tw[2].i;scratch[3].i = scratch_in[3].i * scratch_tw[2].r + scratch_in[3].r * scratch_tw[2].i;
The above code snippet lists the basic element of FFT---- radix4 butterfly. From the code, the following can be concluded:

20 64-bit NEON registers are needed if 2 radix4 butterflies are executed in one loop.
20 128-bit NEON registers are needed if 4 radix4 butterflies are executed in one loop.
And, for ARMv7-A/v8-A AArch32 and v8-A AArch64,

There are 32 64-bit or 16 128-bit NEON registers for ARMv7-A/v8-A AArch32.
There are 32 128-bit NEON registers for ARMv8-A AArch64.
Considering the above factors, the FFT implementation of Ne10 eventually has an assembly version, in which 2 radix4 butterflies are executed in one loop, for ARMv7-A/v8-A AAch32, and an intrinsic version, in which 4 radix4 butterflies are executed in one loop, for ARMv8-A AArch64.The above example can illustrate that you need to pay attention to some exceptions when maintaining one set of code across ARMv7-A/v8-A platform,

3.4 Future
Many issues about using NEON assembly and intrinsics have been discussed. But these issues are temporary. In the long term, intrinsics will be better. By using intrinsics, you can reap the benefits of hardware and compiler upgrade without reprogramming. That means some classical algorithms just need to be implemented once.

The compiler will help to adjust these codes for new hardware, which reduces the workload significantly. Pffft is an example.The following figure describes the performance of pffft and Ne10 real FFT on the ARM Cortex-A9 platform with gcc. X-axis represents the length of FFT. Y-axis represents the time of execution, the smaller the better. Pffft is implemented with NEON intrinsics. Ne10 real FFT is implemented with NEON assembly. They don’t use the same algorithm, but they have similar performance. In the ARMv8-A AArch64 mode, the Ne10 real FFT is rewritten with both NEON assembly and intrinsics. Section 3.3 has explained that ARMv8-A can process 4 butterflies in parallel, but ARMv7-A can only process 2 butterflies in parallel. So theoretically, the effect of FFT optimization on ARMv8-A would be better than that on ARMv7-A. However, based on the following figure, it’s clear that pffft has the best performance.
From this example, it is concluded that: pffft, the performance of which isn’t the best on the ARMv7-A, shows a very good performance in the ARMv8-A AArch64 mode. That proves the point: compiler can adjust intrinsics automatically for ARMv8-A to achieve a good performance.In the long run, existing NEON assembly code have to be rewritten for ARMv8. If NEON is upgraded in the future, the code has to be rewritten again and again. But for NEON intrinsics code, it is expected that it may show good performance on ARMv8-A with the help of compilers. Even if NEON is upgraded, you can also look forward to the upgrade of compilers.
3.5 Summary
In this section, the pros and cons of NEON assembly and intrinsics have been analyzed with some examples. The benefits of intrinsics far outweigh drawbacks. Compared to assembly, Intrinsics are more easily programmed, and have better compatibility between ARMv7 and ARMv8.Some tips for NEON intrinsics are summarized as follows:
The number of registers used
Compiler
Do look at the generated assembly
References
Cortex -A Series Version: 2.0 Programmer’s Guide: 17-12 and 17-20
Cortex -A9 NEON Media Processing Engine Revision: r4p1 Technical Reference Manual: 3.4.8  } END OF DOCUMENTATION TO REWRITE  REWRITE THE DOCUMENTS EXACTLY AS ASKED TO PLEASE. DO NOT PARAPHRASE. NEVER SAY I. JUST FIX THE FORMATING VERBATIM. CONTINUE WORKING FOR AS LONG AS IT TAKES.

again just rewrite it in a much more concatenated way WITHOUT deleting any code or factual technical examples. just delete the redundant "talk" and remove the double newlines. Delete enough stuff to make it about 7000tokens long overall as a guide, try to remove newlines in general without making it look messy, include all the code in the new guide however, and all the technical stuff from programmers, only remove "fluff" not substance.