auphong2707/dl4se-code-review-dataset huggingface dataset:
Source:
public InputRow nextRow() throws IOException { if (!hasMore()) { throw new NoSuchElementException(); } if (!parsedInputRows.isEmpty()) { START return parsedInputRows.remove(0); END } return getNextRow(); } `ArrayList.remove()` internally moves the positions of remaining elements and this will happen for every row in this case which could be expensive. Since `parsedInputRows` won't be large in general, I suggest to use an iterator on the list instead of modifying the list. ```java public E remove(int index) { rangeCheck(index); modCount++; E oldValue = elementData(index); int numMoved = size - index - 1; if (numMoved > 0) System.arraycopy(elementData, index+1, elementData, index, numMoved); elementData[--size] = null; // clear to let GC do its work return oldValue; } ```

Target:
public InputRow nextRow() throws IOException { if (!hasMore()) { throw new NoSuchElementException(); } if (parsedInputRows.hasNext()) { return parsedInputRows.next(); } return getNextRow(); }
Source:
START public Map<String, ImmutableWorkerInfo> getWorkersEligibleToRunTasks() END { return Maps.transformEntries( Maps.filterEntries( workers, input -> !lazyWorkers.containsKey(input.getKey()) && !workersWithUnacknowledgedTask.containsKey(input.getKey()) && !blackListedWorkers.containsKey(input.getKey()) && input.getValue().isInitialized() && input.getValue().isEnabled() ), (String key, WorkerHolder value) -> value.toImmutable() ); } Looks better to be package-private if it's not supposed to be used in general.

Target:
Map<String, ImmutableWorkerInfo> getWorkersEligibleToRunTasks() { // In this class, this method is called with statusLock held. // writes to workersWithUnacknowledgedTask are always guarded by statusLock. // however writes to lazyWorker/blacklistedWorkers aren't necessarily guarded by same lock, so technically there // could be races in that a task could get assigned to a worker which in another thread is concurrently being // marked lazy/blacklisted , but that is ok because that is equivalent to this worker being picked for task and // being assigned lazy/blacklisted right after even when the two threads hold a mutually exclusive lock. return Maps.transformEntries( Maps.filterEntries( workers, input -> !lazyWorkers.containsKey(input.getKey()) && !workersWithUnacknowledgedTask.containsKey(input.getKey()) && !blackListedWorkers.containsKey(input.getKey()) && input.getValue().isInitialized() && input.getValue().isEnabled() ), (String key, WorkerHolder value) -> value.toImmutable() ); }
Source:
protected Long computeResult() { long result = 0; for (int i = 0; i < numBuckets; i += cycleSize) { if (buckets[(i + startFrom) % numBuckets] != null) { result += (buckets[(i + startFrom) % numBuckets]).longValue(); } else { START result += 0.0; END } } startFrom++; return result; } LGTM.com has a warning about this line: https://lgtm.com/projects/g/apache/incubator-druid/snapshot/9d76bbc14139a88dc90787309253de7c438a1e9c/files/extensions-contrib/moving-average-query/src/main/java/org/apache/druid/query/movingaverage/averagers/LongSumAverager.java?sort=name&dir=ASC&mode=heatmap#L38

Target:
protected Long computeResult() { long result = 0; for (int i = 0; i < numBuckets; i += cycleSize) { if (buckets[(i + startFrom) % numBuckets] != null) { result += (buckets[(i + startFrom) % numBuckets]).longValue(); } else { result += 0; } } startFrom++; return result; }
Source:
public byte[] getCacheKey() { return new CacheKeyBuilder(PostAggregatorIds.HLL_SKETCH_TO_ESTIMATE_CACHE_TYPE_ID) START .appendString(name) END .appendCacheable(field) .build(); } I think `name` shouldn't be in the cache key?

Target:
public byte[] getCacheKey() { return new CacheKeyBuilder(PostAggregatorIds.HLL_SKETCH_TO_ESTIMATE_CACHE_TYPE_ID) .appendCacheable(field) .build(); }
Source:
public int getAndIncrementCounter() { int oldVal = counter; counter++; START return oldVal; END } nit: can be simplified to `return counter++;`

Target:
public int getAndIncrementCounter() { return counter++; }
Source:
private void shutdown(Lifecycle lifecycle) { //noinspection finally (not completing the 'finally' block normally is intentional) try { lifecycle.stop(); } catch (Throwable t) { START log.warn(t, "Exception when stopping druid lifecycle"); END } finally { System.exit(1); } } This feels like it should be error log i think

Target:
private void shutdown(Lifecycle lifecycle) { //noinspection finally (not completing the 'finally' block normally is intentional) try { lifecycle.stop(); } catch (Throwable t) { log.error(t, "Exception when stopping druid lifecycle"); } finally { System.exit(1); } }
Source:
public void addAppenderator(String taskId, Appenderator appenderator) { taskAppenderatorMap.computeIfAbsent( taskId, (myTaskId) -> { return new ArrayList<>(); } ); START taskAppenderatorMap.get(taskId).add(appenderator); END } I think the below is better to avoid an extra lookup on the map. ``` taskAppenderatorMap.computeIfAbsent(taskId, myTaskId -> new ArrayList<>()).add(appenderator); ```

Target:
public void addAppenderator(String taskId, Appenderator appenderator) { taskAppenderatorMap.computeIfAbsent( taskId, (myTaskId) -> { return new ArrayList<>(); } ).add(appenderator); }
Source:
public HashSet<String> getRequiredColumns() { return Sets.newHashSet(dimensions.stream() .map(DimensionSpec::getDimension) START .collect(Collectors.toSet()) END ); } Please fix `Sets.newHashSet(...collect(...))`. I've also opened #8241 about fixing this generically.

Target:
public Set<String> getRequiredColumns() { return dimensions.stream().map(DimensionSpec::getDimension).collect(Collectors.toSet()); }
Source:
public boolean shouldUseBitmapIndex(BitmapIndexSelector selector) { START return Filters.shouldUseBitmapIndex(this, selector, filterTuning); END } It's confusing that this method doesn't just return `false` while `supportsBitmapIndex()` does.

Target:
public boolean shouldUseBitmapIndex(BitmapIndexSelector selector) { return false; }
Source:
public void shutdown() { if (mergeExecutor != null) { START mergeExecutor.shutdownNow(); END } } After shutting down the `Executor`, we should explicitly set it to `null` which is more friendly to the GC.

Target:
public void shutdown() { if (mergeExecutor != null) { mergeExecutor.shutdownNow(); mergeExecutor = null; } }
Source:
public Comparator getComparator() { if (errorBoundsStdDev == null) { return Ordering.natural(); } else { START return (o1, o2) -> Doubles.compare( END ((SketchEstimateWithErrorBounds) o1).getEstimate(), ((SketchEstimateWithErrorBounds) o2).getEstimate() ); } } It's `Comparator.comparingDouble(SketchEstimateWithErrorBounds::getEstimate)`

Target:
public Comparator getComparator() { if (errorBoundsStdDev == null) { return Ordering.natural(); } else { return new Comparator() { @Override public int compare(Object o1, Object o2) { return Doubles.compare( ((SketchEstimateWithErrorBounds) o1).getEstimate(), ((SketchEstimateWithErrorBounds) o2).getEstimate() ); } }; } }
Source:
public void stop() throws InterruptedException { if (supervisorTaskChecker != null) { supervisorTaskChecker.shutdownNow(); supervisorTaskChecker.awaitTermination(10, TimeUnit.SECONDS); START supervisorTaskChecker = null; END } supervisorTaskCheckTimes.clear(); } what is the use of setting it null here ?

Target:
public void stop() throws InterruptedException { if (supervisorTaskChecker != null) { supervisorTaskChecker.shutdownNow(); supervisorTaskChecker.awaitTermination(10, TimeUnit.SECONDS); } supervisorTaskCheckTimes.clear(); }
Source:
public Set<Expr> getScalarInputs(List<Expr> args) { START return ImmutableSet.of(args.get(1), args.get(2)); END } this should probably check if args length is 2 or 3?

Target:
public Set<Expr> getScalarInputs(List<Expr> args) { if (args.size() == 3) { return ImmutableSet.of(args.get(1), args.get(2)); } else { return ImmutableSet.of(args.get(1)); } }
Source:
START public void setDisableAcl(boolean disableAcl) END { this.disableAcl = disableAcl; } This (and `setS3Bucket()`) don't strictly need to be public, but could be package-private (remove the modifier). Since they're currently only being used by tests, annotating them with `@VisibleForTesting` would also be a good practice.

Target:
void setDisableAcl(boolean disableAcl) { this.disableAcl = disableAcl; }
Source:
START protected static String sanitize(String namespace) END { Pattern DOT_OR_WHITESPACE = Pattern.compile("[\\s]+|[.]+"); return DOT_OR_WHITESPACE.matcher(namespace).replaceAll("_"); } ```suggestion private static String sanitize(String namespace) ```

Target:
private static String sanitize(String namespace) { return DOT_OR_WHITESPACE.matcher(namespace).replaceAll("_"); }
Source:
START public boolean isValid() END { return interval == null ^ segmentIds == null; } This could just be a preconditions check in the constructor, and I would use a different error message that just tells the user that `intervals` and `segmentIds` cannot both be set. The current error message I don't think is accurate ("Request payload contains invalid interval[%s] or segmentIds[%s], atmost one valid value must be provided"), since the check is not really for validity of the supplied values but for the presence of both.

Target:
public boolean isValid() { return (interval == null ^ segmentIds == null) && (segmentIds == null || !segmentIds.isEmpty()); }
Source:
public byte[] getCacheKey() { return new CacheKeyBuilder(HavingSpecUtil.CACHE_TYPE_ID_GREATER_THAN) .appendString(aggregationName) .appendByteArray(StringUtils.toUtf8(String.valueOf(value))) .appendStrings(aggregators.keySet()) START .appendCacheables(aggregators.values()) END .build(); } You don't need to add them to the cache key -- only the `aggregation` and `value` matter. The idea is that anything that changes the meaning or effect of the operator should be part of the cache key, but not other stuff. `aggregators` is just some extra information that the HavingSpec uses to do its job.

Target:
public byte[] getCacheKey() { return new CacheKeyBuilder(HavingSpecUtil.CACHE_TYPE_ID_GREATER_THAN) .appendString(aggregationName) .appendByteArray(StringUtils.toUtf8(String.valueOf(value))) .build(); }
Source:
START protected ExprEval eval(ExprEval param) END { return eval(param, 0); } Could probably be private instead of protected, which if so, would be preferred.

Target:
private ExprEval eval(ExprEval param) { return eval(param, 0); }
Source:
private Segment getAdapter(final DataSegment segment) throws SegmentLoadingException { final Segment adapter; START synchronized (segmentLoader) { END try { adapter = segmentLoader.getSegment(segment); } catch (SegmentLoadingException e) { segmentLoader.cleanup(segment); throw e; } } if (adapter == null) { throw new SegmentLoadingException("Null adapter from loadSpec[%s]", segment.getLoadSpec()); } return adapter; } This is too much synchronization - it makes parallel segment loading impossible.

Target:
private Segment getAdapter(final DataSegment segment) throws SegmentLoadingException { final Segment adapter; try { adapter = segmentLoader.getSegment(segment); } catch (SegmentLoadingException e) { segmentLoader.cleanup(segment); throw e; } if (adapter == null) { throw new SegmentLoadingException("Null adapter from loadSpec[%s]", segment.getLoadSpec()); } return adapter; }
Source:
static <T> T getRandomElementFromList(List<T> list) { START if (list.size() == 0) { END return null; } return list.get(ThreadLocalRandom.current().nextInt(list.size())); } Highly suggest `isEmpty`

Target:
static <T> T getRandomElementFromList(List<T> list) { if (list.isEmpty()) { return null; } return list.get(ThreadLocalRandom.current().nextInt(list.size())); }
Source:
public static QueryRunner makeMMappedQueryRunner( SegmentId segmentId, boolean rollup, boolean bitmaps, QueryRunnerFactory factory ) { QueryableIndex index = bitmaps START ? rollup END ? TestIndex.getMMappedTestIndex() : TestIndex.getNoRollupMMappedTestIndex() : TestIndex.getNoBitmapMMappedTestIndex(); return QueryRunnerTestHelper.makeQueryRunner( factory, segmentId, new QueryableIndexSegment(index, segmentId), null ); } Nested ternaries = 😭. Please consider using normal `if`s.

Target:
public static QueryRunner makeMMappedQueryRunner( SegmentId segmentId, boolean rollup, boolean bitmaps, QueryRunnerFactory factory ) { QueryableIndex index; if (bitmaps) { index = rollup ? TestIndex.getMMappedTestIndex() : TestIndex.getNoRollupMMappedTestIndex(); } else { index = TestIndex.getNoBitmapMMappedTestIndex(); } return QueryRunnerTestHelper.makeQueryRunner( factory, segmentId, new QueryableIndexSegment(index, segmentId), null ); }
Source:
public static IncrementalIndex getNoBitmapIncrementalTestIndex() { synchronized (log) { if (noBitmapRealtimeIndex != null) { return noBitmapRealtimeIndex; } START } END return noBitmapRealtimeIndex = makeRealtimeIndex("druid.sample.numeric.tsv", false, false); } I know this synchronization pattern was before the change, I'm just confused what it tries to accomplish? ie, if `noBitmapRealtimeIndex` is not created then all threads can initialize `noBitmapRealtimeIndex` simultaneously, doesn't it look strange?

Target:
public static IncrementalIndex getNoBitmapIncrementalTestIndex() { return noBitmapRealtimeIndex.get(); }
Source:
public Map<String, Object> baseline(final Blackhole blackhole) { Map<String, Object> parsed = flatParser.parseToMap(flatInputs.get(flatCounter)); START for (Map.Entry<String, Object> entry : parsed.entrySet()) { END String s = entry.getKey(); blackhole.consume(parsed.get(s)); } flatCounter = (flatCounter + 1) % numEvents; return parsed; } This can be just an iteration over `parsed.values()`. Similar in many other places in this PR.

Target:
public Map<String, Object> baseline(final Blackhole blackhole) { Map<String, Object> parsed = flatParser.parseToMap(flatInputs.get(flatCounter)); for (Object s : parsed.values()) { blackhole.consume(s); } flatCounter = (flatCounter + 1) % numEvents; return parsed; }
Source:
public void printStuff() { System.out.println(); for (Map<Object, Integer> valueMap : dimensionMap.values()) { START System.out.println("DIMENSION " + valueMap + "\n============"); END new TreeMap<>(valueMap).forEach((val, cnt) -> System.out.println(" VAL: " + val + " CNT: " + cnt)); } } Now it prints `valueMap` twice. It should be `entrySet()` iteration (or better `dimensionMap.forEach()` call for readability).

Target:
public void printStuff() { System.out.println(); dimensionMap.forEach((key, valueMap) -> new TreeMap<>(valueMap).forEach((val, cnt) -> System.out.println(" VAL: " + val + " CNT: " + cnt))); }
Source:
START private static @Nullable String findVersion(Map<Interval, String> versions, Interval interval) END { return versions.entrySet().stream() .filter(entry -> entry.getKey().contains(interval)) .map(Entry::getValue) .findFirst() .orElse(null); } nit: we usually add the annotation above the method.

Target:
private static String findVersion(Map<Interval, String> versions, Interval interval) { return versions.entrySet().stream() .filter(entry -> entry.getKey().contains(interval)) .map(Entry::getValue) .findFirst() .orElse(null); }
Source:
public synchronized void close() { if (closed) { return; } START closed = true; END log.info("Firehose closing."); // Critical to add the poison pill to the queue, don't allow interruption. Uninterruptibles.putUninterruptibly(buffer, FIREHOSE_CLOSED); eventReceiverFirehoseRegister.unregister(serviceName); if (chatHandlerProvider != null) { chatHandlerProvider.unregister(serviceName); } Thread delayedCloseExecutor = this.delayedCloseExecutor; if (delayedCloseExecutor != null && !delayedCloseExecutor.equals(Thread.currentThread())) { // Interrupt delayedCloseExecutor to let it discover that closed flag is already set and exit. delayedCloseExecutor.interrupt(); } } Should we do `closed = true` right before the synchronized block end (in case of exception)?

Target:
public synchronized void close() { if (closed) { return; } closed = true; log.info("Firehose closing."); // Critical to add the poison pill to the queue, don't allow interruption. Uninterruptibles.putUninterruptibly(buffer, FIREHOSE_CLOSED); eventReceiverFirehoseRegister.unregister(serviceName); if (chatHandlerProvider != null) { chatHandlerProvider.unregister(serviceName); } if (delayedCloseExecutor != null && !delayedCloseExecutor.equals(Thread.currentThread())) { // Interrupt delayedCloseExecutor to let it discover that closed flag is already set and exit. delayedCloseExecutor.interrupt(); } }
Source:
public RelDataType rowType() { synchronized (lock) { Preconditions.checkState(prepareResult != null || plannerResult != null, START "must be called after sql has been prepared"); END return plannerResult != null ? plannerResult.rowType() : prepareResult.getRowType(); } } The message is not consistent with the condition, since if the SQL has been planned, then the precondition is met. It looks like the only place where `SqlLifecycle#rowType()` is called without a preceding call to `SqlLifecycle#plan()` is in `DruidStatement#prepare()`. However, in that method, it is possible to directly call `PrepareResult#getRowType()` on the return value of the earlier call to `SqlLifecycle#prepare()`. If that is done then the changes here are not needed.

Target:
public RelDataType rowType() { synchronized (lock) { return plannerResult != null ? plannerResult.rowType() : prepareResult.getRowType(); } }
Source:
public CoordinatorBasicAuthenticatorCacheNotifier( AuthenticatorMapper authenticatorMapper, DruidNodeDiscoveryProvider discoveryProvider, @EscalatedClient HttpClient httpClient ) { userCacheNotifier = new CommonCacheNotifier( START initAuthenticatorConfigMap(authenticatorMapper), END discoveryProvider, httpClient, "/druid-ext/basic-security/authentication/listen/%s", "CoordinatorBasicAuthenticatorCacheNotifier" ); configCacheNotifier = new CommonCacheNotifier( initAuthenticatorConfigMap(authenticatorMapper), discoveryProvider, httpClient, "/druid-ext/basic-security/authentication/listen/config/%s", "CoordinatorBasicAuthenticatorCacheNotifier" ); } should this be initAuthenticatorUserMap instead of configMap ?

Target:
public CoordinatorBasicAuthenticatorCacheNotifier( AuthenticatorMapper authenticatorMapper, DruidNodeDiscoveryProvider discoveryProvider, @EscalatedClient HttpClient httpClient ) { userCacheNotifier = new CommonCacheNotifier( initAuthenticatorConfigMap(authenticatorMapper), discoveryProvider, httpClient, "/druid-ext/basic-security/authentication/listen/%s", "CoordinatorBasicAuthenticatorCacheNotifier" ); }
Source:
public static DateTime of(String instant) { try { START Integer seconds = Integer.valueOf(instant); END return new DateTime(seconds * 1000L, ISOChronology.getInstanceUTC()); } catch (NumberFormatException ex) { try { return new DateTime(Long.valueOf(instant), ISOChronology.getInstanceUTC()); } catch (IllegalArgumentException ex2) { try { return new DateTime(instant, ISOChronology.getInstanceUTC()); } catch (Exception ex3) { throw ex; } } } } This code won't work for timestamps that are near the epoch. I think it's a bit too clever and we should stick to only working on milliseconds timestamps and ISO8601 strings.

Target:
public static DateTime of(String instant) { try { return new DateTime(instant, ISOChronology.getInstanceUTC()); } catch (IllegalArgumentException ex) { try { return new DateTime(Long.valueOf(instant), ISOChronology.getInstanceUTC()); } catch (IllegalArgumentException ex2) { throw ex; } } }
Source:
START protected void setSegmentMetadataHolder(final DataSegment segment, final SegmentMetadataHolder segmentMetadataHolder) END { synchronized (lock) { TreeMap<DataSegment, SegmentMetadataHolder> dataSourceSegments = segmentMetadataInfo.computeIfAbsent( segment.getDataSource(), x -> new TreeMap<>(SEGMENT_ORDER) ); if (dataSourceSegments.put(segment, segmentMetadataHolder) == null) { totalSegments++; } } } Could this also be package-private?

Target:
void setSegmentMetadataHolder(final DataSegment segment, final SegmentMetadataHolder segmentMetadataHolder) { synchronized (lock) { TreeMap<DataSegment, SegmentMetadataHolder> dataSourceSegments = segmentMetadataInfo.computeIfAbsent( segment.getDataSource(), x -> new TreeMap<>(SEGMENT_ORDER) ); if (dataSourceSegments.put(segment, segmentMetadataHolder) == null) { totalSegments++; } } }
Source:
public void setTerminateDruidProcessOnConnectFail(Boolean terminateDruidProcessOnConnectFail) { START this.terminateDruidProcessOnConnectFail = terminateDruidProcessOnConnectFail == null ? false : terminateDruidProcessOnConnectFail; END } Line longer than 120 cols. Please use if-else.

Target:
public void setTerminateDruidProcessOnConnectFail(Boolean terminateDruidProcessOnConnectFail) { if (terminateDruidProcessOnConnectFail == null) { this.terminateDruidProcessOnConnectFail = false; } else { this.terminateDruidProcessOnConnectFail = terminateDruidProcessOnConnectFail; } }
Source:
private void emitThreadAllocationMetrics(ServiceEmitter emitter) { final ServiceMetricEvent.Builder builder = builder(); MonitorUtils.addDimensionsToBuilder(builder, dimensions); Optional<Long> delta = collector.calculateDelta(); START delta.ifPresent((value) -> emitter.emit(builder.build("jvm/heapAlloc/bytes", value))); END } Maybe `jvm/heapAlloc/bytes/delta`

Target:
private void emitThreadAllocationMetrics(ServiceEmitter emitter) { final ServiceMetricEvent.Builder builder = builder(); MonitorUtils.addDimensionsToBuilder(builder, dimensions); if (collector != null) { long delta = collector.calculateDelta(); emitter.emit(builder.build("jvm/heapAlloc/bytes", delta)); } }
Source:
public int available() { final ReentrantLock lock = this.lock; START lock.lock(); END try { return objects.size(); } finally { lock.unlock(); } } Why bother locking if the size returned is immediately going to be invalid? aka, some other thread may have already changed the size.

Target:
public int available() { return objects.size(); }
Source:
START public synchronized void close() END { } Doesn't need to be synchronized

Target:
public void close() { }
Source:
public ComplexMetricExtractor getExtractor() { return new ComplexMetricExtractor() { @Override public Class<?> extractedClass() { return MomentSketchWrapper.class; } @Override public Object extractValue(final InputRow inputRow, final String metricName) { Object rawValue = inputRow.getRaw(metricName); START if (rawValue instanceof MomentSketchWrapper) { END return (MomentSketchWrapper) rawValue; } else { throw new IAE("Not a momentSketch"); } } }; } This explicitness doesn't add a lot of value, just cast and let it throw ClassCastException. It will also log the wrong class, unlike the current code.

Target:
public ComplexMetricExtractor getExtractor() { return new ComplexMetricExtractor() { @Override public Class<?> extractedClass() { return MomentSketchWrapper.class; } @Override public Object extractValue(final InputRow inputRow, final String metricName) { return (MomentSketchWrapper) inputRow.getRaw(metricName); } }; }
Source:
public void configure(Binder binder) { String typeName = MomentSketchAggregatorFactory.TYPE_NAME; START if (ComplexMetrics.getSerdeForType(typeName) == null) { END ComplexMetrics.registerSerde(typeName, new MomentSketchComplexMetricSerde()); } } Could you please replace this boilerplate pattern with a single method `registerSerde(String, Supplier<ComplexMetricSerde>)` thoughout the code?

Target:
public void configure(Binder binder) { ComplexMetrics.registerSerde( MomentSketchAggregatorFactory.TYPE_NAME, MomentSketchComplexMetricSerde::new ); }
Source:
public MomentSketchWrapper fromByteBuffer(ByteBuffer buffer, int numBytes) { if (numBytes == 0) { START return EMPTY_SKETCH; END } buffer.limit(buffer.position() + numBytes); return MomentSketchWrapper.fromBytes(buffer); } Does `EMPTY_SKETCH.toByteArray()` result in an empty array? Currently there is a discrepancy between `fromByteBuffer()` and `toBytes()` that looks suspicious when just reading the code.

Target:
public MomentSketchWrapper fromByteBuffer(ByteBuffer buffer, int numBytes) { if (numBytes == 0) { return null; } buffer.limit(buffer.position() + numBytes); return MomentSketchWrapper.fromBytes(buffer); }
Source:
public List<Task> getActiveTasks() { giant.lock(); try { final ImmutableList.Builder<Task> listBuilder = ImmutableList.builder(); for (final TaskStuff taskStuff : tasks.values()) { START if (taskStuff.getStatus().isRunnable() && taskStuff.getTask() != null) { END listBuilder.add(taskStuff.getTask()); } } return listBuilder.build(); } finally { giant.unlock(); } } I guess `task` is never null in HeapMemoryTaskStorage?

Target:
public List<Task> getActiveTasks() { giant.lock(); try { final ImmutableList.Builder<Task> listBuilder = ImmutableList.builder(); for (final TaskStuff taskStuff : tasks.values()) { if (taskStuff.getStatus().isRunnable()) { listBuilder.add(taskStuff.getTask()); } } return listBuilder.build(); } finally { giant.unlock(); } }
Source:
public Set<StreamPartition<Integer>> getAssignment() { Set<TopicPartition> topicPartitions = consumer.assignment(); return topicPartitions .stream() START .map((TopicPartition e) -> new StreamPartition<>(e.topic(), e.partition())) END .collect(Collectors.toSet()); } Don't need (TopicPartition e) here, just 'e' should be sufficient

Target:
public Set<StreamPartition<Integer>> getAssignment() { Set<TopicPartition> topicPartitions = consumer.assignment(); return topicPartitions .stream() .map(e -> new StreamPartition<>(e.topic(), e.partition())) .collect(Collectors.toSet()); }
Source:
public void close() { if (this.closed) { return; } assign(ImmutableSet.of()); scheduledExec.shutdown(); try { if (!scheduledExec.awaitTermination(EXCEPTION_RETRY_DELAY_MS, TimeUnit.MILLISECONDS)) { scheduledExec.shutdownNow(); } } catch (InterruptedException e) { START log.info(e, "InterruptedException while shutting down"); END } this.closed = true; } Should be warn at least.

Target:
public void close() { if (this.closed) { return; } assign(ImmutableSet.of()); scheduledExec.shutdown(); try { if (!scheduledExec.awaitTermination(EXCEPTION_RETRY_DELAY_MS, TimeUnit.MILLISECONDS)) { scheduledExec.shutdownNow(); } } catch (InterruptedException e) { log.warn(e, "InterruptedException while shutting down"); throw new RuntimeException(e); } this.closed = true; }
Source:
private KinesisSequenceNumber(@NotNull String sequenceNumber, boolean isExclusive) { super(sequenceNumber, isExclusive); if (OrderedPartitionableRecord.END_OF_SHARD_MARKER.equals(sequenceNumber) || SeekableStreamPartitions.NO_END_SEQUENCE_NUMBER.equals(sequenceNumber)) { isMaxSequenceNumber = true; this.intSequence = null; } else { isMaxSequenceNumber = false; START this.intSequence = new BigInteger(sequenceNumber); END } } Is it safe to assume that all sequenceNumbers are in the string format BigInteger allows? If so, why don't we use it instead of String?

Target:
private KinesisSequenceNumber(@NotNull String sequenceNumber, boolean isExclusive) { super(sequenceNumber, isExclusive); if (END_OF_SHARD_MARKER.equals(sequenceNumber) || SeekableStreamPartitions.NO_END_SEQUENCE_NUMBER.equals(sequenceNumber)) { isMaxSequenceNumber = true; this.intSequence = null; } else { isMaxSequenceNumber = false; this.intSequence = new BigInteger(sequenceNumber); } }
Source:
public void close() { if (this.closed) { return; } assign(ImmutableSet.of()); scheduledExec.shutdown(); try { if (!scheduledExec.awaitTermination(EXCEPTION_RETRY_DELAY_MS, TimeUnit.MILLISECONDS)) { scheduledExec.shutdownNow(); } } catch (InterruptedException e) { START log.warn(e, "InterruptedException while shutting down"); END } this.closed = true; } On `InterruptedException`, you should rethrow the exception or interrupt the thread properly.

Target:
public void close() { if (this.closed) { return; } assign(ImmutableSet.of()); scheduledExec.shutdown(); try { if (!scheduledExec.awaitTermination(EXCEPTION_RETRY_DELAY_MS, TimeUnit.MILLISECONDS)) { scheduledExec.shutdownNow(); } } catch (InterruptedException e) { log.warn(e, "InterruptedException while shutting down"); throw new RuntimeException(e); } this.closed = true; }
Source:
private void rescheduleRunnable(long delayMillis) { if (started && !stopRequested) { try { scheduledExec.schedule(getRecordRunnable(), delayMillis, TimeUnit.MILLISECONDS); } catch (RejectedExecutionException e) { START log.warn( END "Caught RejectedExecutionException, KinesisRecordSupplier for partition[%s] has likely temporarily shutdown the ExecutorService." + "This is expected behavior after calling seek(), seekToEarliest() and seekToLatest()", streamPartition.getPartitionId() ); } } else { log.info("Worker for partition[%s] has been stopped", streamPartition.getPartitionId()); } } Please add the exception like this. ``` log.warn( e, "Caught RejectedExecutionException, KinesisRecordSupplier for partition[%s] has likely temporarily shutdown the ExecutorService." + "This is expected behavior after calling seek(), seekToEarliest() and seekToLatest()", streamPartition.getPartitionId() ); ```

Target:
private void rescheduleRunnable(long delayMillis) { if (started && !stopRequested) { try { scheduledExec.schedule(getRecordRunnable(), delayMillis, TimeUnit.MILLISECONDS); } catch (RejectedExecutionException e) { log.warn( e, "Caught RejectedExecutionException, KinesisRecordSupplier for partition[%s] has likely temporarily shutdown the ExecutorService." + "This is expected behavior after calling seek(), seekToEarliest() and seekToLatest()", streamPartition.getPartitionId() ); } } else { log.info("Worker for partition[%s] has been stopped", streamPartition.getPartitionId()); } }
Source:
START public String getStream() END { return id; } Missing @JsonProperty ?

Target:
public String getStream() { return stream; }
Source:
START public void testGetDependentFields() throws Exception END { List<String> dependentFields = fac.getDependentFields(); assertEquals(1, dependentFields.size()); assertEquals("field", dependentFields.get(0)); } I think teamcity is complaining about the `Exception` which is never thrown.

Target:
public void testGetDependentFields() { List<String> dependentFields = fac.getDependentFields(); assertEquals(1, dependentFields.size()); assertEquals("field", dependentFields.get(0)); }
Source:
protected I[] getBuckets() { START return (I[]) buckets; END } Unnecessary cast.

Target:
protected I[] getBuckets() { return buckets; }
Source:
public boolean equals(Object o) { if (this == o) { return true; } if (o == null || o.getClass() != getClass()) { return false; } PeriodBroadcastDistributionRule that = (PeriodBroadcastDistributionRule) o; if (!Objects.equals(period, that.period)) { return false; } START if (!Objects.equals(includeFuture, that.includeFuture)) { END return false; } return Objects.equals(colocatedDataSources, that.colocatedDataSources); } `includeFuture` can't be null. `if (includeFuture != that.includeFuture)`?

Target:
public boolean equals(Object o) { if (this == o) { return true; } if (o == null || o.getClass() != getClass()) { return false; } PeriodBroadcastDistributionRule that = (PeriodBroadcastDistributionRule) o; if (!Objects.equals(period, that.period)) { return false; } if (includeFuture != that.includeFuture) { return false; } return Objects.equals(colocatedDataSources, that.colocatedDataSources); }
Source:
public Object finalizeComputation(Object object) { try { if (object instanceof ByteBuffer) { return BloomKFilter.deserialize((ByteBuffer) object); } else if (object instanceof byte[]) { return BloomKFilter.deserialize(ByteBuffer.wrap((byte[]) object)); } else { return object; } } catch (IOException ioe) { START throw new RuntimeException("Failed to deserialize BloomKFilter"); END } } `, ioe`

Target:
public Object finalizeComputation(Object object) { try { if (object instanceof ByteBuffer) { return BloomKFilter.deserialize((ByteBuffer) object); } else if (object instanceof byte[]) { return BloomKFilter.deserialize(ByteBuffer.wrap((byte[]) object)); } else { return object; } } catch (IOException ioe) { throw new RuntimeException("Failed to deserialize BloomKFilter", ioe); } }
Source:
public void testSerDeser() throws IOException { ObjectMapper mapper = new DefaultObjectMapper(); Assert.assertEquals(loadingCache, mapper.readerFor(LoadingCache.class).readValue(mapper.writeValueAsString(loadingCache))); START Assert.assertTrue(loadingCache.hashCode() == mapper.readerFor(LoadingCache.class).readValue(mapper.writeValueAsString(loadingCache)).hashCode()); END } Could use assertEquals?

Target:
public void testSerDeser() throws IOException { ObjectMapper mapper = new DefaultObjectMapper(); Assert.assertEquals(loadingCache, mapper.readerFor(LoadingCache.class).readValue(mapper.writeValueAsString(loadingCache))); Assert.assertEquals(loadingCache.hashCode(), mapper.readerFor(LoadingCache.class).readValue(mapper.writeValueAsString(loadingCache)).hashCode()); }
Source:
public Parser getFlatParser() { JSONParseSpec spec = new JSONParseSpec( START new TimestampSpec("ts", "iso", null, null), END new DimensionsSpec(null, null, null), null, null ); return spec.makeParser(); } It might have been worth considering adding an additional constructor to `TimestampSpec` with `@VisibleForTesting` annotation so that this PR could have a bit lighter of a footprint and not have to modify all of these tests since they are all just passing in `null` anyway.

Target:
public Parser getFlatParser() { JSONParseSpec spec = new JSONParseSpec( new TimestampSpec("ts", "iso", null), new DimensionsSpec(null, null, null), null, null ); return spec.makeParser(); }
Source:
public boolean isArray(final Object o) { START if (o instanceof List) { END return true; } return false; } this can just be `return (o instanceof List)`

Target:
public boolean isArray(final Object o) { return o instanceof List; }
Source:
public boolean isMap(final Object o) { START if (o instanceof Map) { END return true; } else { return o instanceof Group; } } could be `(o instanceof Map) || (o instanceof Group)`

Target:
public boolean isMap(final Object o) { return o instanceof Map || o instanceof Group; }
Source:
public String toString() { return "PrefixFilteredDimensionSpec{" + START "Prefix='" + NullHandling.nullToEmptyIfNeeded(prefix) + '\'' + END '}'; } "prefix" has a null check in the constructor, so the nullToEmpty call is not needed here

Target:
public String toString() { return "PrefixFilteredDimensionSpec{" + "Prefix='" + prefix + '\'' + '}'; }
Source:
public DerivativeDataSourceMetadata( @JsonProperty("baseDataSource") String baseDataSource, @JsonProperty("dimensions") Set<String> dimensions, @JsonProperty("metrics") Set<String> metrics ) { this.dimensions = Preconditions.checkNotNull(dimensions, "dimensions cannot be null. This is not a valid DerivativeDataSourceMetadata."); this.metrics = Preconditions.checkNotNull(metrics, "metrics cannot be null. This is not a valid DerivativeDataSourceMetadata."); Preconditions.checkArgument(!Strings.isNullOrEmpty(baseDataSource), "baseDataSource cannot be null or empty. Please provide a baseDataSource."); START this.baseDataSource = baseDataSource; END } nit: Please don't change the initialization order. It would be good if they're aligned with their definition order.

Target:
public DerivativeDataSourceMetadata( @JsonProperty("baseDataSource") String baseDataSource, @JsonProperty("dimensions") Set<String> dimensions, @JsonProperty("metrics") Set<String> metrics ) { Preconditions.checkArgument(!Strings.isNullOrEmpty(baseDataSource), "baseDataSource cannot be null or empty. Please provide a baseDataSource."); this.baseDataSource = baseDataSource; this.dimensions = Preconditions.checkNotNull(dimensions, "dimensions cannot be null. This is not a valid DerivativeDataSourceMetadata."); this.metrics = Preconditions.checkNotNull(metrics, "metrics cannot be null. This is not a valid DerivativeDataSourceMetadata."); }
Source:
public void limit(TimeseriesQuery query) { START setDimension("limit", toString().valueOf(query.getLimit())); END } `toString().valueOf(query.getLimit())` is just a weird way of writing `String.valueOf(query.getLimit())`, but using a strange Java feature that lets you call static methods (like valueOf) as if they were methods of specific objects. So please replace it with the latter.

Target:
public void limit(TimeseriesQuery query) { setDimension("limit", String.valueOf(query.getLimit())); }
Source:
public AnonymousAuthenticator( @JsonProperty("name") String name, @JsonProperty("authorizerName") String authorizerName, @JsonProperty("identity") String identity ) { this.name = name; this.authorizerName = authorizerName; this.identity = identity == null ? DEFAULT_IDENTITY : identity; START this.anonymousResult = new AuthenticationResult( END this.identity, this.authorizerName, this.name, null ); } nit : It seems only the anonymousResult needs to be a field in the class, rest can be removed ? In case you meant this class to be used in JSON serde, please add @JsonProperty annotations to the fields and create their getter methods.

Target:
public AnonymousAuthenticator( @JsonProperty("name") String name, @JsonProperty("authorizerName") String authorizerName, @JsonProperty("identity") String identity ) { this.anonymousResult = new AuthenticationResult( identity == null ? DEFAULT_IDENTITY : identity, authorizerName, name, null ); }
Source:
public AppenderatorDriverAddResult add( InputRow row, String sequenceName ) throws IOException { START return append(row, sequenceName, null, true, true); END } Why is the BatchAppenderatorDriver skipping the lineage check now? I thought it could still make more than one segment per interval if it's running in non-incremental-publishing mode.

Target:
public AppenderatorDriverAddResult add( InputRow row, String sequenceName ) throws IOException { return append(row, sequenceName, null, false, true); }
Source:
public static long getMaxBytesInMemoryOrDefault(final long maxBytesInMemory) { // In the main tuningConfig class constructor, we set the maxBytes to 0 if null to avoid setting // maxBytes to max jvm memory of the process that starts first. Instead we set the default based on // the actual task node's jvm memory. long newMaxBytesInMemory = maxBytesInMemory; if (maxBytesInMemory == 0) { newMaxBytesInMemory = TuningConfig.DEFAULT_MAX_BYTES_IN_MEMORY; START } else if (maxBytesInMemory == -1) { END newMaxBytesInMemory = Long.MAX_VALUE; } return newMaxBytesInMemory; } `else if maxBytesInMemory < 0`?

Target:
public static long getMaxBytesInMemoryOrDefault(final long maxBytesInMemory) { // In the main tuningConfig class constructor, we set the maxBytes to 0 if null to avoid setting // maxBytes to max jvm memory of the process that starts first. Instead we set the default based on // the actual task node's jvm memory. long newMaxBytesInMemory = maxBytesInMemory; if (maxBytesInMemory == 0) { newMaxBytesInMemory = TuningConfig.DEFAULT_MAX_BYTES_IN_MEMORY; } else if (maxBytesInMemory < 0) { newMaxBytesInMemory = Long.MAX_VALUE; } return newMaxBytesInMemory; }
Source:
public Map<String, Object> getStats() { try { return getCurrentTotalStats(); } catch (InterruptedException ie) { Thread.currentThread().interrupt(); START log.error("getStats() interrupted."); END throw new RuntimeException(ie); } catch (ExecutionException | TimeoutException eete) { throw new RuntimeException(eete); } } Please add `ie` to the log.

Target:
public Map<String, Map<String, Object>> getStats() { try { return getCurrentTotalStats(); } catch (InterruptedException ie) { Thread.currentThread().interrupt(); log.error(ie, "getStats() interrupted."); throw new RuntimeException(ie); } catch (ExecutionException | TimeoutException eete) { throw new RuntimeException(eete); } }
Source:
public HllSketch combine(final Object objectA, final Object objectB) { final Union union = new Union(lgK); union.update((HllSketch) objectA); START union.update((HllSketch) objectA); END return union.getResult(tgtHllType); } Why duplicate line?

Target:
public HllSketch combine(final Object objectA, final Object objectB) { final Union union = new Union(lgK); union.update((HllSketch) objectA); union.update((HllSketch) objectB); return union.getResult(tgtHllType); }
Source:
public AuthenticationResult createEscalatedAuthenticationResult() { START return new AuthenticationResult(internalClientUsername, authorizerName, BASIC, null); END } > If i am getting this right there is one to one mapping between escalator type and authenticator name/type. Authenticator name/type are not necessarily correlated (I can name my kerberos authenticator "George" if I wanted). There's also no restriction on having two Authenticators of the same type (maybe that makes sense for some hypothetical implementation where running two instances with a different "namespace" or similar concept makes sense). So there is only a mapping between a specific escalator instance and an authenticator instance, which is why I suggest adding the property.

Target:
public AuthenticationResult createEscalatedAuthenticationResult() { // if you found your self asking why the authenticatedBy field is set to null please read this: // https://github.com/druid-io/druid/pull/5706#discussion_r185940889 return new AuthenticationResult(internalClientUsername, authorizerName, null, null); }
Source:
public void close() { Throwable thrown = null; for (MappedByteBuffer mappedByteBuffer : buffersList) { if (mappedByteBuffer == null) { continue; } try { ByteBufferUtils.unmap(mappedByteBuffer); } catch (Throwable t) { if (thrown == null) { thrown = t; } else { thrown.addSuppressed(t); } } } buffersList.clear(); START Throwables.propagateIfPossible(thrown); END } This doesn't seem right. `Throwables.propagateIfPossible` will suppress any exception that is not a RuntimeException or Error. I think there's no reason here to only throw specific error types and suppress others, so I assume it's a bug. Instead, it should probably be, ```java Throwables.propagateIfPossible(thrown); throw new RuntimeException(thrown); ``` I know this is unrelated to your patch, but it's something that looks wrong in nearby code and we could fix that too.

Target:
public void close() { Throwable thrown = null; for (MappedByteBuffer mappedByteBuffer : buffersList) { if (mappedByteBuffer == null) { continue; } try { ByteBufferUtils.unmap(mappedByteBuffer); } catch (Throwable t) { if (thrown == null) { thrown = t; } else { thrown.addSuppressed(t); } } } buffersList.clear(); if (thrown != null) { throw new RuntimeException(thrown); } }
Source:
public List<ReferenceCountingResourceHolder<T>> takeBatch(final int elementNum) { checkInitialized(); try { START return takeObjects(elementNum).stream().map(this::wrapObject).collect(Collectors.toList()); END } catch (InterruptedException e) { throw Throwables.propagate(e); } } I think this has a buffer leak (and it looks like the old code had the leak too, and so does `pollObjects`). If either `pollObjects` or `takeObjects` is interrupted while it's waiting for more objects to become available, then the objects popped from `objects` are not returned to the pool - they are lost.

Target:
public List<ReferenceCountingResourceHolder<T>> takeBatch(final int elementNum) { checkInitialized(); try { return takeObjects(elementNum).stream().map(this::wrapObject).collect(Collectors.toList()); } catch (InterruptedException e) { throw new RuntimeException(e); } }
Source:
public static boolean eligibleForLoad(Period period, Interval interval, DateTime referenceTimestamp) { final Interval currInterval = new Interval(period, referenceTimestamp); START return currInterval.overlaps(interval); END } nit: better to call `eligibleForLoad(currInterval, interval)` for easier code maintenance.

Target:
public static boolean eligibleForLoad(Period period, Interval interval, DateTime referenceTimestamp) { final Interval currInterval = new Interval(period, referenceTimestamp); return eligibleForLoad(currInterval, interval); }
Source:
OnheapIncrementalIndex( IncrementalIndexSchema incrementalIndexSchema, boolean deserializeComplexMetrics, boolean reportParseExceptions, boolean concurrentEventAdd, boolean sortFacts, int maxRowCount, long maxBytesInMemory ) { super(incrementalIndexSchema, deserializeComplexMetrics, reportParseExceptions, concurrentEventAdd); this.maxRowCount = maxRowCount; START this.maxBytesInMemory = maxBytesInMemory == 0 ? defaultMaxBytesInMemory : maxBytesInMemory; END this.facts = incrementalIndexSchema.isRollup() ? new RollupFactsHolder(sortFacts, dimsComparator(), getDimensions()) : new PlainFactsHolder(sortFacts); if (maxBytesInMemory != -1) { maxBytesPerRowForAggregators = getMaxBytesPerRowForAggregators(incrementalIndexSchema); } } Looks like this cannot be 0 in the current implementation. Suggest to throw an exception if it's 0 and remove `defaultMaxBytesInMemory`.

Target:
OnheapIncrementalIndex( IncrementalIndexSchema incrementalIndexSchema, boolean deserializeComplexMetrics, boolean reportParseExceptions, boolean concurrentEventAdd, boolean sortFacts, int maxRowCount, long maxBytesInMemory ) { super(incrementalIndexSchema, deserializeComplexMetrics, reportParseExceptions, concurrentEventAdd); this.maxRowCount = maxRowCount; this.maxBytesInMemory = maxBytesInMemory == 0 ? -1 : maxBytesInMemory; this.facts = incrementalIndexSchema.isRollup() ? new RollupFactsHolder(sortFacts, dimsComparator(), getDimensions()) : new PlainFactsHolder(sortFacts); maxBytesPerRowForAggregators = getMaxBytesPerRowForAggregators(incrementalIndexSchema); }
Source:
OnheapIncrementalIndex( IncrementalIndexSchema incrementalIndexSchema, boolean deserializeComplexMetrics, boolean reportParseExceptions, boolean concurrentEventAdd, boolean sortFacts, int maxRowCount, long maxBytesInMemory ) { super(incrementalIndexSchema, deserializeComplexMetrics, reportParseExceptions, concurrentEventAdd); this.maxRowCount = maxRowCount; this.maxBytesInMemory = maxBytesInMemory == 0 ? defaultMaxBytesInMemory : maxBytesInMemory; this.facts = incrementalIndexSchema.isRollup() ? new RollupFactsHolder(sortFacts, dimsComparator(), getDimensions()) : new PlainFactsHolder(sortFacts); START maxBytesPerRowForAggregators = getMaxBytesPerRowForAggregators(incrementalIndexSchema); END } Conventionally, we include "this", like `this.maxBytesPerRowForAggregators =` in constructors when initializing fields. It doesn't matter for correctness, but it's conventional style in the codebase.

Target:
OnheapIncrementalIndex( IncrementalIndexSchema incrementalIndexSchema, boolean deserializeComplexMetrics, boolean reportParseExceptions, boolean concurrentEventAdd, boolean sortFacts, int maxRowCount, long maxBytesInMemory ) { super(incrementalIndexSchema, deserializeComplexMetrics, reportParseExceptions, concurrentEventAdd); this.maxRowCount = maxRowCount; this.maxBytesInMemory = maxBytesInMemory == 0 ? -1 : maxBytesInMemory; this.facts = incrementalIndexSchema.isRollup() ? new RollupFactsHolder(sortFacts, dimsComparator(), getDimensions()) : new PlainFactsHolder(sortFacts); maxBytesPerRowForAggregators = getMaxBytesPerRowForAggregators(incrementalIndexSchema); }
Source:
public DerivativeDataSourceMetadata( @JsonProperty("baseDataSource") String baseDataSource, @JsonProperty("dimensions") Set<String> dimensions, @JsonProperty("metrics") Set<String> metrics ) { Preconditions.checkNotNull(baseDataSource, "baseDataSource cannot be null. This is not a valid DerivativeDataSourceMetadata."); Preconditions.checkNotNull(dimensions, "dimensions cannot be null. This is not a valid DerivativeDataSourceMetadata."); Preconditions.checkNotNull(metrics, "metrics cannot be null. This is not a valid DerivativeDataSourceMetadata."); START this.baseDataSource = baseDataSource; END this.dimensions = dimensions; this.metrics = metrics; } nit: Can be simplified to `this.baseDataSource = Preconditions.checkNotNull(baseDataSource, "baseDataSource cannot be null. This is not a valid DerivativeDataSourceMetadata.");`

Target:
public DerivativeDataSourceMetadata( @JsonProperty("baseDataSource") String baseDataSource, @JsonProperty("dimensions") Set<String> dimensions, @JsonProperty("metrics") Set<String> metrics ) { this.baseDataSource = Preconditions.checkNotNull(baseDataSource, "baseDataSource cannot be null. This is not a valid DerivativeDataSourceMetadata."); this.dimensions = Preconditions.checkNotNull(dimensions, "dimensions cannot be null. This is not a valid DerivativeDataSourceMetadata."); this.metrics = Preconditions.checkNotNull(metrics, "metrics cannot be null. This is not a valid DerivativeDataSourceMetadata."); }