Research task (STEM):
Write a technical report describing how to implement a high-performance double-precision General Matrix-Matrix Multiplication (GEMM) tuned for AVX2, Fused Multiply-Add (FMA), and multi-core CPUs. Make it understandable for an audience with technical competency but not adept at low-level systems programming. Include intuitive explanations and formulas of the optimization techniques, including but not limited to blocking, intrinsics, loop unrolling, and annotated code snippets in C for key components (e.g., microkernel, blocking strategy, parallelization, etc).

Evaluation rubric:
criterion: The response ensures all acronyms (e.g., (1) GEMM as General Matrix-Matrix Multiplication, (2) AVX2 as Advanced Vector Extensions 2, (3) FMA as Fused Multiply-Add) are expanded on it during their first appearance and provides clear definitions for all specialized terms and mathematical symbols (e.g., (1) instruction-level parallelism, (2) loop unrolling), (3) vectorization, (4) blocking upon first use, accompanied by explanatory prose or examples.
weight: 4
axis: Instruction Following

criterion: The response includes at least two high-level analogy or intuitive explanation for the core concepts (i.e. blocking, cache, SIMD vectorization, parallelization) (e.g., (1) compares blocking to "tiling a floor, so only a small area is exposed at a time", (2) Instead of delivering the mail one at a time and going back to the post office to pick up the next, the mail person carries some of the mail for the region in a truck (caching), (3) Parallel execution is like have multiple chefs cooking together rather than just one chef, (4) pipelining is like an assembly line where each component of an item get assembled piecewise by a servicing worker as the item goes down the line).
weight: 4
axis: Instruction Following

criterion: The response includes diagrams and figures to explain important concepts (e.g., (1) vectorization visualized by a single instruction executed on an array of items, (2) A diagram of how GEMM is computed, (3) A diagram of blocking where the matrix is divided up into tiles, (4) A diagram of data packing where the data is copied into a contiguous section of memory, (5) graphs and charts of speedup comparisons).
weight: 3
axis: Communication Quality

criterion: The response is clearly targeted for an audience with technical competency but not adept at low-level systems programming (e.g., (1) avoids assembly listings or overly low-level register diagrams, (2) Overly explaining basic technical terms such as what is a CPU, (3) Skips intermediate optimization steps and jumps to the final advanced optimization with several techniques applied, (4) assumes that the audience knows the details of x86 architecture, (5) proper sequencing of optimization techniques from easy to advanced (e.g., naive implementation -> cache blocking and packing -> microkernel -> multicore or multithreading).).
weight: 5
axis: Instruction Following

criterion: The response introduces unrelated concepts and technologies (e.g., GPU CUDA kernels, FPGAs, Systolic arrays, High Bandwidth Memory(HBM), threading synchronization, and barrier techniques).
weight: -4
axis: Instruction Following

criterion: The response provides at least one cautionary note about portability (e.g., AVX2 code will run slowly or crash on CPUs without AVX2 support).
weight: 2
axis: Implicit Criteria

criterion: The response is formatted as a technical report with a title page, abstract, introduction, body, conclusion, and references, and each optimization technique (e.g., blocking, intrinsics/FMA, loop unrolling, parallelization) into its own clearly titled section.
weight: 3
axis: Communication Quality

criterion: The response provides at least one annotated code snippet in C for each key component and optimization of the GEMM implementation (e.g., microkernel, blocking strategy, parallelization, loop unrolling, loop ordering) and presents the final optimized code that contains all the optimizations combined at the end.
weight: 5
axis: Explicit Criteria

criterion: The response explains GEMM (i.e., its mathematical definition C := αAB + βC (or the simplified α = β = 1 form) using matrix notation where C is m x n, A is m x k, B is k x n) and tailors the report for double precision (i.e., a floating-point number format also known as float64, usually 64 bits in computer memory) (e.g., using doubles in the code snippets, AVX2 for double precision, explaining and the use of 64 bits).
weight: 5
axis: Explicit Criteria

criterion: The response explains what Fused Multiply-Add (FMA) is and its benefits (i.e., Fused Multiply-Add is a fused operation of multiply and add with single rounding at the end, which allows better performance by combining two operations into a single instruction and better numerical accuracy due to single rounding).
weight: 4
axis: Explicit Criteria

criterion: The response explains Single Instruction, Multiple Data (SIMD) vectorization, in particular AVX2 (Advanced Vector Extensions 2), and why it is important for GEMM optimization (i.e., Advanced Vector Extensions is the SIMD extensions to the x86-64 processors from Intel and Advanced Micro Devices (AMD) while Advanced Vector Extensions 2 adds new instructions and expands most integer commands to 256 bits (YMM registers)).
weight: 5
axis: Explicit Criteria

criterion: The response explains data packing and its benefits and costs (i.e., data that is required for computation is copied to a buffer to ensure contiguous memory access for vectorization and remove Translation Lookaside Buffer (TLB) misses. The tradeoff is the extra memory copy, but the benefit can be significant in terms of performance).
weight: 4
axis: Implicit Criteria

criterion: The response explains microkernels (i.e., the smallest level of optimization where the computation fits inside the registers of the CPU, such as loop unrolling, vectorization, software prefetching and intrinsics (AVX2)) and provides details on at least one technique such as register blocking (e.g. The block sizes for MR and NR in register blocking can be determined by: (MR * NR) / VW + MR / VW + 1 <= N_reg; 2 * MR * NR <= 2 * L * T * VW where N_reg is the number of architectural vector registers, VW is the SIMD vector width in doubles, L is the pipeline latency in cycles, and T is the FMA throughput in results per cycle).
weight: 5
axis: Explicit Criteria

criterion: The response explains cache blocking, stating that a cache-aware blocking strategy minimizes compulsory, capacity, and conflict misses by keeping working sub-matrices in the L1/L2 cache, and that the sub-matrices should fit in the targeted level cache. (e.g., the correct cache-blocking formula for determining block sizes for MC, KC, and NC: (MC * KC + KC * NC * MC * NC) * sizeof(double) < L2 cache size).
weight: 5
axis: Explicit Criteria

criterion: The response briefly compares performance to a vendor BLAS (e.g., BLIS, OpenBLAS, Intel MKL) to contextualize achieved GFLOPS.
weight: 3
axis: Synthesis of Information

criterion: The response explains loop ordering with performance considerations (e.g., j-p-i, improves performance because of the contiguous memory access pattern, improving temporal and spatial locality, and reducing cache misses, adjusting row-major order and column-major order for contiguous memory access).
weight: 3
axis: Explicit Criteria

criterion: The response explains loop unrolling and its benefits and costs in terms of performance, specifically for GEMM (i.e., reduce loop overhead by expanding out the loop, reducing branch penalty but increasing the number of branches, meaning more missed branch predictions, optimized register usage).
weight: 5
axis: Explicit Criteria

criterion: The response mentions alignment or padding guidance (e.g., aligning packing buffers to 32- or 64-byte boundaries for AVX2).
weight: 4
axis: Implicit Criteria

criterion: The response addresses multi-core scalability limits such as memory bandwidth or false sharing and other issues like load balancing (e.g., (1) Explain that even if multiple cores can scale theoretical speedup, the memory availability will be a limiter at a much earlier point, (2) false sharing occurs in multiple cores tries to read and write the same cache line, (3) choosing the number of threads, and block sizes for balancing work).
weight: 4
axis: Synthesis of Information

criterion: The response explains how to achieve multi-core or multi-threaded parallelism (e.g., using OpenMP, C++ standard library, Boost, oneAPI Threading Building Blocks (oneTBB)) and choosing and justifying what part of the code to parallelize.
weight: 5
axis: Explicit Criteria

criterion: The response contains messy, unnecessary formulas, code, or tables. (e.g., (1) Unrendered latex, (2) messy code snippets with improperly formatted structure or messy comments, (3) broken tables that are unrendered, (4) missing figures or formulas or sections that are referenced in the text).
weight: -4
axis: Communication Quality

criterion: The response explains why the naive approach is very slow on the high conceptual level near the start of the report (e.g., (1) The algorithm itself has theoretical limits, (2) Memory bandwidth limits how much data can be processed at a time, (3) Multicore issues such as false sharing, (4) Computer architecture is laid out with levels of cache with different speeds and sizes (smaller caches are faster)).
weight: 3
axis: Synthesis of Information