Repository: open-telemetry/opentelemetry-java
File: sdk/all/src/jmh/java/io/opentelemetry/sdk/MetricRecordBenchmark.java (Java)

Code before:
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.data.AggregationTemporality;
import io.opentelemetry.sdk.metrics.export.DefaultAggregationSelector;
import io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.samplers.Sampler;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Random;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Group;
import org.openjdk.jmh.annotations.GroupThreads;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.TearDown;
import org.openjdk.jmh.annotations.Warmup;

/**
 * This benchmark measures the performance of recording metrics and includes the following
 * dimensions:
 *
 * <ul>
 *   <li>{@link BenchmarkState#instrumentTypeAndAggregation} composite of {@link InstrumentType} and
 *       {@link Aggregation}, including all relevant combinations for synchronous instruments.
 *   <li>{@link BenchmarkState#aggregationTemporality}
 *   <li>{@link BenchmarkState#cardinality}
 *   <li>thread count
 *   <li>{@link BenchmarkState#instrumentValueType}, {@link BenchmarkState#memoryMode}, and {@link
 *       BenchmarkState#exemplars} are disabled to reduce combinatorial explosion.
 * </ul>
 *
 * <p>Each operation consists of recording {@link MetricRecordBenchmark#RECORDS_PER_INVOCATION}
 * measurements.
 *
 * <p>The cardinality and thread count dimensions partially overlap. Cardinality dictates how many
 * unique attribute sets (i.e. series) are recorded to, and thread count dictates how many threads
 * are simultaneously recording to those series. In all cases, the record path needs to look up an
 * aggregation handle for the series corresponding to the measurement's {@link Attributes} in a
 * {@link java.util.concurrent.ConcurrentHashMap}. That will be the case until otel adds support for
 * <a href="https://github.com/open-telemetry/opentelemetry-specification/issues/4126">bound
 * instruments</a>. The cardinality dictates the size of this map, which has some impact on
 * performance. However, by far the dominant bottleneck is contention. That is, the number of
 * threads simultaneously trying to record to the same series. Increasing the threads increases
 * contention. Increasing cardinality decreases contention, as the threads are now spreading their
 * record activities over more distinct series. The highest contention scenario is cardinality=1,
 * threads=4. Any scenario with threads=1 has zero contention.

Review comment (suggestion):
One of my initial concerns with public benchmarks was that they need to be contextualized. 

To address this, I'd like to:

- Put some effort into making the javadoc for our public benchmarks up to date and useful
- Update the [benchmark static](https://open-telemetry.github.io/opentelemetry-java/benchmarks/) webpage to link to the relevant javadoc for each benchmark

Code after:
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.data.AggregationTemporality;
import io.opentelemetry.sdk.metrics.export.DefaultAggregationSelector;
import io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.samplers.Sampler;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Random;
import org.openjdk.jmh.annotations.Benchmark;
import org.openjdk.jmh.annotations.Fork;
import org.openjdk.jmh.annotations.Group;
import org.openjdk.jmh.annotations.GroupThreads;
import org.openjdk.jmh.annotations.Measurement;
import org.openjdk.jmh.annotations.OperationsPerInvocation;
import org.openjdk.jmh.annotations.Param;
import org.openjdk.jmh.annotations.Scope;
import org.openjdk.jmh.annotations.Setup;
import org.openjdk.jmh.annotations.State;
import org.openjdk.jmh.annotations.TearDown;
import org.openjdk.jmh.annotations.Warmup;

/**
 * This benchmark measures the performance of recording metrics. It includes the following
 * dimensions:
 *
 * <ul>
 *   <li>{@link BenchmarkState#instrumentTypeAndAggregation} composite of {@link InstrumentType} and
 *       {@link Aggregation}, including all relevant combinations for synchronous instruments.
 *   <li>{@link BenchmarkState#aggregationTemporality}
 *   <li>{@link BenchmarkState#cardinality}
 *   <li>thread count
 *   <li>{@link BenchmarkState#instrumentValueType}, {@link BenchmarkState#memoryMode}, and {@link
 *       BenchmarkState#exemplars} are disabled to reduce combinatorial explosion.
 * </ul>
 *
 * <p>Each operation consists of recording {@link MetricRecordBenchmark#RECORDS_PER_INVOCATION}
 * measurements.
 *
 * <p>The cardinality and thread count dimensions partially overlap. Cardinality dictates how many
 * unique attribute sets (i.e. series) are recorded to, and thread count dictates how many threads
 * are simultaneously recording to those series. In all cases, the record path needs to look up an
 * aggregation handle for the series corresponding to the measurement's {@link Attributes} in a
 * {@link java.util.concurrent.ConcurrentHashMap}. That will be the case until otel adds support for
 * <a href="https://github.com/open-telemetry/opentelemetry-specification/issues/4126">bound
 * instruments</a>. The cardinality dictates the size of this map, which has some impact on
 * performance. However, by far the dominant bottleneck is contention. That is, the number of
 * threads simultaneously trying to record to the same series. Increasing the threads increases
 * contention. Increasing cardinality decreases contention, as the threads are now spreading their
 * record activities over more distinct series. The highest contention scenario is cardinality=1,