auphong2707/dl4se-code-review-dataset huggingface dataset:
Source:
private boolean isNotDuplicateMessage(String message) { for (String currentMessage : getMessages()) { if (StringUtils.equals(currentMessage, message)) { return false; } } return true; }

Target:
This function is presuming that the method caller is verifying "messages" is not null prior to being called which is ok for these changes as that is happening but if this private method is used elsewhere in this class in the future that may pose an issue.
Source:
public void verifyAddEmptyCollectionMessage() { validationResult.addMessage(TEST_STRING_ONE); validationResult.addMessages(new ArrayList()); assertEquals("There should only be one line", 1, validationResult.getMessages().size()); }

Target:
Missing generics on ArrayList. Adding at least an empty "<>" should allow the compiler to derive the correct generic type, provided that the addMessages() method signature has enough information.
Source:
public boolean buildCollectorFile(ConcurStandardAccountingExtractFile saeFileContents) { ConcurStandardAccountingExtractCollectorBatchBuilder builder = new ConcurStandardAccountingExtractCollectorBatchBuilder( universityDateService::getFiscalYear, dateTimeService::toString, concurSAEValidationService::validateConcurStandardAccountingExtractDetailLine, this::getConcurParameterValueAsString); CollectorBatch collectorBatch = builder.buildCollectorBatchFromStandardAccountingExtract(0, saeFileContents); if (collectorBatch == null) { LOG.error("There was a problem preparing the data for the Collector file; will not create a file."); return false; } return true; }

Target:
would it be better to define these lamba functions as a variables to help with understanding what is going on?
Source:
private Object loadFile(String fileName) { byte[] fileByteContent = safelyLoadFileBytes(fileName); Object parsedObject; parsedObject = getBatchInputFileService().parse(getBatchInputFileType(), fileByteContent); return parsedObject; }

Target:
these two lines could be combined in one. no need for a declaration on a separate line.
Source:
public boolean fileApprovedAmountsMatchHeaderApprovedAmount(ConcurRequestExtractFile requestExtractFile) { KualiDecimal detailLinesAmountSumKualiDecimal = getTotalRequestFileRequestAmount(requestExtractFile); if (detailLinesAmountSumKualiDecimal.doubleValue() == requestExtractFile.getTotalApprovedAmount().doubleValue()) { return true; } else { LOG.error("Header amount validation failed. Header amount was =" + requestExtractFile.getTotalApprovedAmount().toString() + "= while file calculated amount was =" + detailLinesAmountSumKualiDecimal.toString() + "="); return false; } }

Target:
I am not sure why the comparison between the doubleValue is needed here. You can use the KualiDecimal equals or compareTo methods for this test. (equals is calling the compareTo)
Source:
public void createPdpFeedsFromRequestExtracts() { List<String> filesToProcess = getUnprocessedRequestExtractFiles(); if (filesToProcess.isEmpty()) { LOG.error("No Request Extract files found to process."); } else { for (String requestExtractFileName : filesToProcess) { createPdpFeedFromRequestExtract(requestExtractFileName); LOG.info("Request Extract File " + requestExtractFileName + " was processed."); } } }

Target:
I am not sure this is necessarily an error. Should this log be at info level?
Source:
public static boolean isExpenseReportURI(String URI) { if (StringUtils.isNotBlank(URI) && URI.contains(ConcurConstants.EXPENSE_REPORT_URI_INDICATOR)) { return true; } return false; }

Target:
Why not just return the condtional?
Source:
public ValidationResult validateConcurAccountInfo(ConcurAccountInfo concurAccountInfo) { ValidationResult validationResult = checkAccountingString( concurAccountInfo.getChart(), concurAccountInfo.getAccountNumber(), concurAccountInfo.getSubAccountNumber(), concurAccountInfo.getObjectCode(), concurAccountInfo.getSubObjectCode(), concurAccountInfo.getProjectCode()); return validationResult; }

Target:
I would consider eliminating this intermediate variable, validationResult, and just returning the result of the checkAccountingString() call directly.
Source:
public JobExecutionListener loggingListener() { return new LoggingListener(); }

Target:
You should make these the concrete classes as the return types. You don't need to downcast them for them to be picked up in collection injections and using the concrete classes lets them be recognized easier for conditionals.
Source:
protected void addParametersForScript( final Map<String, Object> parameters, final Set<Command> resources, final JobRequest jobRequest ) { parameters.put(COMMANDS_BINDING, resources); }

Target:
How come the job request is added to the parameters automatically, but the list of resources is not?
Source:
protected void attemptTransition( final ExecutionContext executionContext ) throws RetryableTransitionException, FatalTransitionException { final File jobDirectory = executionContext.getJobDirectory(); @NotNull final CleanupStrategy cleanupStrategy = executionContext.getCleanupStrategy(); if (jobDirectory != null) { log.info("Cleaning up job directory (strategy: {})", cleanupStrategy); try { this.jobSetupService.cleanupJobDirectory( jobDirectory.toPath(), cleanupStrategy ); } catch (final IOException e) { throw new RetryableTransitionException("Failed to cleanup job directory", e); } } }

Target:
FYI I don't think putting these inside the method contents has any effect as there is no proxy to intercept and evaluate them.
Source:
public void testExpectedContextWhenSNSDisabled() { this.contextRunner .withPropertyValues( "genie.notifications.sns.enabled=false" ) .run( (context) -> { System.out.println(context); Assertions.assertThat(context).doesNotHaveBean(AWSAutoConfiguration.SNS_CLIENT_BEAN_NAME); } ); }

Target:
Probably didn't mean to leave this in here?
Source:
Loader( final TaskScheduler taskScheduler, final Environment environment, final GenieFileTransferService fileTransferService, final MeterRegistry registry ) { this.environment = environment; this.registry = registry; this.fileTransferService = fileTransferService; // Schedule the task to run with the configured refresh rate // Task will be stopped when the system stops final long refreshRate = environment.getProperty( ScriptLoadBalancerProperties.REFRESH_RATE_PROPERTY, Long.class, 300_000L ); taskScheduler.scheduleWithFixedDelay(this::refresh, refreshRate); }

Target:
Not to increase the scope creep here but we'll probably want to get rid of this service soon so if you were so inclined to flip this to use the `ResourceLoader` I wouldn't object
Source:
CompiledScript get() { return script.get(); }

Target:
Is this one of those `@VisibleForTesting` things? Seems unnecessary
Source:
public void jobUpdate(final JobEntity jobEntity) { final JobStatus currentState = jobEntity.getStatus(); final JobStatus previouslyNotifiedState = jobEntity.getNotifiedJobStatus(); final String jobId = jobEntity.getUniqueId(); if (currentState != previouslyNotifiedState) { log.info( "Detected state change for job: from: to: {}", jobId, previouslyNotifiedState, currentState ); // Notify observer if not null if (this.persistedJobStatusObserver != null) { this.persistedJobStatusObserver.notify(jobId, previouslyNotifiedState, currentState); } // Save this as the latest published state jobEntity.setNotifiedJobStatus(currentState); } }

Target:
If the observer is null why bother doing anything? There doesn't appear to be any way to change it from being null as the constructor is the only modifier so why not short circuit all the other logic and just return.
Source:
public Server gRpcServer( @Value("${grpc.server.port:0}") final int port, final Set<BindableService> services, final List<ServerInterceptor> serverInterceptors ) { final NettyServerBuilder builder = NettyServerBuilder.forPort( port == RANDOM_PORT ? SocketUtils.findAvailableTcpPort() : port ); // Add Service interceptors and add services to the server services .stream() .map(BindableService::bindService) .map(serviceDefinition -> ServerInterceptors.intercept(serviceDefinition, serverInterceptors)) .forEach(builder::addService); return builder.build(); }

Target:
I'd prefer to set the port to 0 and let the socket bind to an ephemeral port. There's no chance of collision that way.
Source:
public static String[] getOptionArguments(final String[] args) { final List<String> temporary = Arrays.asList(args); final int index = temporary.indexOf(BARE_DOUBLE_DASH); if (index == -1) { // All arguments are options if no explicit double dash is passed return args; } else if (index == 0) { // Double dash as first argument means all arguments are parameters return new String[0]; } else { // All argument up to (and not including) the double dash return temporary.subList(0, index).toArray(new String[index]); } }

Target:
It's late and I'm tired so I could be totally off on this but I feel like this could be done in a single pass and save all the conversions and checks. Maybe just iterate through the array and write each item into a resultant array until you encounter the double dash and then return the result array?
Source:
public void canSetRequestedArchiveLocationPrefix() { Assert.assertFalse(this.jobEntity.getRequestedArchiveLocationPrefix().isPresent()); final String requestedArchiveLocationPrefix = UUID.randomUUID().toString(); this.jobEntity.setRequestedArchiveLocationPrefix(requestedArchiveLocationPrefix); Assert.assertThat( this.jobEntity.getRequestedArchiveLocationPrefix().orElse(UUID.randomUUID().toString()), Matchers.is(requestedArchiveLocationPrefix) ); }

Target:
I find the `orElse` here a bit confusing... how about just `.get`?
Source:
private <ResponseType> ResponseType handleResponseFuture( final ListenableFuture<ResponseType> responseListenableFuture ) { final ResponseType response; try { response = responseListenableFuture.get(); } catch (final ExecutionException | InterruptedException e) { throw new GenieRuntimeException("Failed to perform request"); } return response; }

Target:
you should add `e` as the cause here so that it trickles up properly
Source:
private void createDirectoryStructureIfNotExists(final File dir) { if (!dir.exists()) { try { Files.createParentDirs(new File(dir, DUMMY_FILE_NAME)); } catch (final IOException e) { throw new RuntimeException("Failed to create directory: " + dir.getAbsolutePath(), e); } } else if (!dir.isDirectory()) { throw new RuntimeException("This location is not a directory: " + dir.getAbsolutePath()); } }

Target:
Is it a conscious decision to throw an unchecked here?
Source:
public PingServiceGrpc.PingServiceBlockingStub getBlockingPingClient( final String serverHost, final int serverPort ) { final Channel channel = channelFactory.getSharedManagedChannel(serverHost, serverPort); return PingServiceGrpc.newBlockingStub(channel); }

Target:
combine into one return line?
Source:
public void setUp() throws IOException { final Registry registry = Mockito.mock(Registry.class); this.initialSetupTask = new InitialSetupTask(registry); this.temporaryTestDirectory = Files.createTempDirectory(InitialSetupTaskUnitTest.class.getSimpleName() + "-"); }

Target:
Is there a reason you didn't use [this](http://junit.org/junit4/javadoc/4.12/org/junit/rules/TemporaryFolder.html)?
Source:
public void setUp() throws IOException { final Registry registry = Mockito.mock(Registry.class); this.initialSetupTask = new InitialSetupTask(registry); this.temporaryTestDirectory = Files.createTempDirectory(this.getClass().getSimpleName() + "-"); }

Target:
I thought you were going to switch this to use the `@Rule` from JUnit for temp directories?
Source:
public JobKickoffTask( final boolean runAsUserEnabled, final boolean userCreationEnabled, @NotNull final Executor executor, @NotNull final String hostname, @NotNull final Registry registry ) { this.isRunAsUserEnabled = runAsUserEnabled; this.isUserCreationEnabled = userCreationEnabled; this.executor = executor; this.hostname = hostname; this.timer = registry.timer("genie.jobs.tasks.jobKickoffTask.timer"); retryTemplate = new RetryTemplate(); retryTemplate.setBackOffPolicy(new ExponentialBackOffPolicy()); }

Target:
Don't we already have a retry template as a bean somewhere? why isn't this passed into the constructor?
Source:
private void validateHostAndUpdateErrorCount(final String host) { final boolean isNodeHealthy = isNodeHealthy(host); // // If node is healthy, remove the entry from the errorCounts. // If node is not healthy, update the entry in errorCounts // if (isNodeHealthy) { if (errorCounts.containsKey(host)) { errorCounts.remove(host); } } else { if (this.errorCounts.containsKey(host)) { this.errorCounts.put(host, this.errorCounts.get(host) + 1); } else { this.errorCounts.put(host, 1); } } }

Target:
Just call the method within the if statement, no need to save it to a local variable
Source:
public void executeTask( @NotNull final Map<String, Object> context ) throws GenieException { log.debug("Executing JobKillLogic Task in the workflow."); super.executeTask(context); // Append logic for handling job kill signal Utils.appendToWriter(writer, JobConstants.JOB_KILL_HANDLER_LOGIC.toString()); }

Target:
Instead of calling toString over and over here you should just call it once in the constant class when it is created.
Source:
public List<TopologySummary> getTopologySummaries() throws AuthorizationException, TException { return getNimbus().getTopologySummaries(); }

Target:
AuthorizationException can be removed
Source:
public TopologySummary getTopologySummaryByName(String name) throws AuthorizationException, TException { return getNimbus().getTopologySummaryByName(name); }

Target:
AuthorizationException can be removed
Source:
public TopologyInfo getTopologyInfoWithOpts(String id, GetInfoOptions options) throws NotAliveException, AuthorizationException, TException { return nimbus.getTopologyInfoWithOpts(id, options); }

Target:
NotAliveException, AuthorizationException can be removed
Source:
public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws NotAliveException, AuthorizationException, TException { return nimbus.getTopologyInfoByNameWithOpts(name, options); }

Target:
NotAliveException, AuthorizationException can be removed
Source:
public TopologyInfo getTopologyInfoByName(String name) throws NotAliveException, AuthorizationException, TException { try { getTopologyInfoByNameCalls.mark(); GetInfoOptions options = new GetInfoOptions(); options.set_num_err_choice(NumErrorsChoice.ALL); return getTopologyInfoByNameImpl(name, options); } catch (Exception e) { LOG.warn("get topology info exception. (topology name={})", name, e); if (e instanceof TException) { throw (TException) e; } throw new RuntimeException(e); } }

Target:
Looks like we don't need `NotAliveException, AuthorizationException`
Source:
public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws NotAliveException, AuthorizationException, TException { try { getTopologyInfoByNameWithOptsCalls.mark(); return getTopologyInfoByNameImpl(name, options); } catch (Exception e) { LOG.warn("get topology info withOpts by name exception. (topology name={})", name, e); if (e instanceof TException) { throw (TException) e; } throw new RuntimeException(e); } }

Target:
looks like we don't need `NotAliveException, AuthorizationException`
Source:
public List<TopologySummary> getTopologySummaries() throws AuthorizationException, TException { try { getTopologySummariesCalls.mark(); checkAuthorization(null, null, "getTopologySummaries"); return getTopologySummariesImpl(); } catch (Exception e) { LOG.warn("Get TopologySummary info exception.", e); if (e instanceof TException) { throw (TException) e; } throw new RuntimeException(e); } }

Target:
looks like we don't need AuthorizationException
Source:
private static void oneProducer1Consumer(int prodBatchSz) { JCQueue q1 = new JCQueue("q1", null, 50_000, 0, prodBatchSz, new WaitStrategyPark(100), "test", "test", Collections.singletonList(1000), 1000, new StormMetricRegistry()); final Producer prod1 = new Producer(q1); final Consumer cons1 = new Consumer(q1); runAllThds(prod1, cons1); }

Target:
why is this `null` (Although I guess it doesn't matter)
Source:
private static JCQueue createQueue(String name, int queueSize) { return new JCQueue(name, null, queueSize, 0, 1, new WaitStrategyPark(0), "test", "test", Collections.singletonList(1000), 1000, new StormMetricRegistry()); }

Target:
is it better to use `name` instead of `null`?
Source:
private JCQueue createQueue(String name, int batchSize, int queueSize) { return new JCQueue(name, null, queueSize, 0, batchSize, waitStrategy, "test", "test", Collections.singletonList(1000), 1000, new StormMetricRegistry()); }

Target:
is it better to use name instead of null?
Source:
public void nodeAssignmentSent(String node, boolean successful) { if (!blacklistSendAssignentFailures) { return; } if (!successful) { int failCount = assignmentFailures.getOrDefault(node, 0) + 1; assignmentFailures.put(node, failCount); } }

Target:
Synchronization is needed otherwise there can be a race condition between getting the current count and updating the count (line 296 and line297) Also between `this.assignmentFailures.clear();` and `assignmentFailures.put(node, failCount);
Source:
private void trackAssignmentFailures() { if (!blacklistSendAssignentFailures) { return; } Map<String, Integer> assignmentFailureWindow = new HashMap<>(); assignmentFailureWindow.putAll(this.assignmentFailures); synchronized (assignmentFailures) { this.assignmentFailures.clear(); } this.sendAssignmentFailureCount.add(assignmentFailureWindow); }

Target:
Since we already `synchronized` on it, maybe should just as well put `assignmentFailureWindow.putAll(this.assignmentFailures);` inside the block and then it doesn't need to be a `concurrentHashMap`. To reduce two synchronization mechanisms to one
Source:
public TestConstraintSolverStrategy(boolean consolidatedConfigFlag) { this.consolidatedConfigFlag = consolidatedConfigFlag; LOG.info("Running tests with consolidatedConfigFlag={}", ""+consolidatedConfigFlag); }

Target:
`""+` doesn't seem necessary.
Source:
private Set<String> collectMapValues(Map<String, Set<String>> map) { return map.values() .stream() .flatMap(set -> set.stream()) .collect(Collectors.toCollection(HashSet::new)); }

Target:
This looks overly complicated. Maybe ``` Set<String> set = new HashSet<>(); map.values().forEach((s) -> set.addAll(s)); return set; ```
Source:
public boolean blobExists(String key, Subject who) throws AuthorizationException, KeyNotFoundException { who = checkAndGetSubject(who); validateKey(key); SettableBlobMeta meta = getStoredBlobMeta(key); aclHandler.hasPermissions(meta.get_acl(), READ, who, key); try { BlobStoreFile blobStoreFile = hbs.read(DATA_PREFIX + key); return blobStoreFile.exists(); } catch (IOException e) { throw new RuntimeException(e); } }

Target:
We don't need this. For every key, there will be two files `meta_<key>` and `data_<key>`. Every operation to the blob will first check and get `meta_<key>`. If the blob is deleted, it throws `KeyNotFoundException`. If it exists, we don't need to continue to check `data_<key>` because we can simply assume `data_<key>` exists. Because if `meta_<key>`exists but `data_<key>` doesn't, there will be RuntimeException with the client tries to download or access to the blob. In this way, we can reduce the hdfs operation further.
Source:
public static Map<String, Map<String, Object>> getTopologyLag(StormTopology userTopology, Map<String,Object> config) { Boolean disableLagMonitoring = ObjectReader.getBoolean(config.get(DaemonConfig.UI_DISABLE_SPOUT_LAG_MONITORING), false); return disableLagMonitoring ? Collections.EMPTY_MAP : TopologySpoutLag.lag(userTopology, config); }

Target:
Nit: Consider adding the default to defaults.yaml instead of here. That way it's obvious to users what the default is.
Source:
private void seekOffsetByStartTimeStamp(TopicPartition tp) { long startTimeStampOffset; Map<TopicPartition, OffsetAndTimestamp> offsetsForTimes = consumer.offsetsForTimes(Collections.singletonMap(tp, startTimeStamp)); OffsetAndTimestamp startOffsetAndTimeStamp = offsetsForTimes.get(tp); startTimeStampOffset = startOffsetAndTimeStamp.offset(); LOG.debug("First poll for topic partition [{}], seeking to partition from startTimeStamp [{}]", tp, startTimeStamp); consumer.seek(tp, startTimeStampOffset); }

Target:
Nit: No reason to not declare and assign on the same line.
Source:
public boolean remove(NormalizedResourcesWithMemory other) { boolean negativeResources = normalizedResources.remove(other.getNormalizedResources(), null); totalMemoryMb -= other.getTotalMemoryMb(); if (totalMemoryMb < 0.0) { negativeResources = true; totalMemoryMb = 0.0; } return negativeResources; }

Target:
Maybe change `public boolean remove(NormalizedResourcesWithMemory other, ResourceMetrics resourceMetrics)` and then just call `remove(other, null)`
Source:
private void mkAssignments() throws Exception { try { mkAssignments(null); } catch(Exception e) { this.mkAssignmentsErrors.mark(); throw e; } }

Target:
Do we want to make the change inside `mkAssignments(String scratchTopoId)` so we can count errors during the rebalance command as well?
Source:
public void removeAllPrivateWorkerKeys(String topologyId) { for (WorkerTokenServiceType type : WorkerTokenServiceType.values()) { String path = ClusterUtils.secretKeysPath(type, topologyId); try { LOG.debug("Removing worker keys under {}", path); stateStorage.delete_node(path); } catch (RuntimeException e) { //This should never happen because only the primary nimbus is active, but just in case // declare the race safe, even if we lose it. if (!Utils.exceptionCauseIsInstanceOf(KeeperException.NoNodeException.class, e)) { throw e; } } } }

Target:
If this does not hit often, I would vote for making this info.
Source:
List<DirectoryCleaner.DeletionMeta> perWorkerDirCleanup(long size) { return workerLogs.getAllWorkerDirs().stream() .map(Unchecked.function(dir -> directoryCleaner.deleteOldestWhileTooLarge(Collections.singletonList(dir), size, true, null))) .collect(toList()); }

Target:
If DeletionMeta is used outside DirectoryCleaner, I think it belongs in its own file.
Source:
public boolean recordBackPressure(Integer taskId) { return tasks.get(taskId).backpressure.getAndSet(true); }

Target:
The return value of this method has switched. getAndSet is going to return the previous value, which is true if it already was under backpressure and false if it was not. This is the opposite of ``` * @return true if an update was recorded, false if taskId is already under BP ``` This is used to control when the server sends messages to the client about backpressure, and could make it happen not frequently enough and the beginning and then way too frequently after that.
Source:
public Values(Object... vals) { super(vals != null ? vals.length : 0); if (vals != null && vals.length != 0 ) { for (Object o : vals) { add(o); } } else { add(null); } }

Target:
`vals.length != 0` should be removed in this condition, cause condition for else statement would be `vals == null || vals.length == 0`, which `Values(List())` goes into else statement and comes out of `Values(null)` instead of no element.
Source:
public Values(Object... vals) { super(vals != null ? vals.length : 0); if (vals != null ) { for (Object o : vals) { add(o); } } else { add(null); } }

Target:
I think this can be `super(vals != null ? vals.length : 1);` since `if (vals==null)` we are going to add `null` to it. The length will be 1.
Source:
public void close() throws Exception { super.close(); getMeasured().close(); }

Target:
Could we please change this a bit? Stopping the timer is not nearly as important as actually closing the resource, and if stopping the timer throws any kind of exception it will not stop the resource. Could we do something more like. ``` try { super.close(); } finally { getMeasured().close(); } ``` This should guarantee that we close the resource always, and if an exception is thrown in both cases the resource exception takes precedent and in newer versions of java the other exception will show up as a Suppressed exception.
Source:
private TopologyContext createTopologyContext(Map<String, Object> topoConf) { CustomIndexArray<String> taskToComponent = new CustomIndexArray<>(0,8); taskToComponent.set(7, "Xcom"); return new TopologyContext(null, topoConf, taskToComponent, null, null, null, null, null, null, 7, 6703, null, null, null, null, null, null); }

Target:
Why 0,8 if all we are setting is 7?
Source:
public static WorkerMetricsProcessor configureMetricProcessor(Map conf) throws MetricException { try { String processorClass = (String)conf.get(DaemonConfig.STORM_METRIC_PROCESSOR_CLASS); WorkerMetricsProcessor processor = (WorkerMetricsProcessor) (Class.forName(processorClass)).newInstance(); processor.prepare(conf); return processor; } catch (Throwable t) { throw new MetricException("Failed to create metric processor", t); } }

Target:
Can we just make this an Exception instead of a Throwable? Most Errors are not things that we want to try and recover from.
Source:
public void run() { while (true) { if (shutdown) { return; } try { Thread.sleep(sleepMs); } catch (InterruptedException e) { LOG.error("Sleep interrupted", e); continue; } try { purgeMetrics(); } catch (MetricException e) { LOG.error("Failed to purge metrics", e); if (this.failureMeter != null) { this.failureMeter.mark(); } } } }

Target:
Is this a reason to have separate if statement instead of `while (!shutdown)`?
Source:
private void dropMessage(AddressedTuple tuple, JCQueue queue) { ++dropCount; queue.recordMsgDrop(); LOG.warn("Dropping message as overflow threshold has reached for Q = {}. OverflowCount = {}. Total Drop Count= {}, Dropped Message : {}", queue.getName(), queue.getOverflowCount(), dropCount, tuple.toString()); }

Target:
nit: `tuple.toString()` is not needed, as the logging will do it for you, but since this is a warn log that should almost never be called it really does not matter.
Source:
private void closeFlushScheduler() { if (flushScheduler != null) { flushScheduler.shutdown(); try { if (!flushScheduler.awaitTermination(1, TimeUnit.SECONDS)) { flushScheduler.shutdownNow(); if (!flushScheduler.awaitTermination(1, TimeUnit.SECONDS)) { LOG.error("Flush scheduler did not terminate after trials of shutdown."); } } } catch (InterruptedException ie) { // (Re-)Cancel if current thread also interrupted flushScheduler.shutdownNow(); // Preserve interrupt status Thread.currentThread().interrupt(); } } }

Target:
nit: You may want to have multiple iterations(which can be 2 or more) with while loop instead of nested if conditionals.
Source:
public void addToEmitMsgs(long offset) { this.emittedOffsets.add(offset); // O(Log N) this.latestEmittedOffset = offset; }

Target:
I don't think this works. This will just set the latest emitted offset to the offset that was most recently emitted. I think you want the highest offset that has been emitted.
Source:
public void activate() { try { subscribeKafkaConsumer(); metricsConsumer = new KafkaConsumer<>(kafkaSpoutConfig.getKafkaProps()); } catch (InterruptException e) { throwKafkaConsumerInterruptedException(); } }

Target:
Why do we need a separate consumer for the metric? Also if we really need two consumers, you should create this via the factory. Otherwise you prevent us from stubbing/spying on this consumer.
Source:
public void deactivate() { try { shutdown(); metricsConsumer.close(); } catch (InterruptException e) { throwKafkaConsumerInterruptedException(); } }

Target:
I think this belongs in shutdown.
Source:
protected void process(Tuple tuple) { try { LOG.debug("Processing Tuple: {}", tuple); SolrRequest request = solrMapper.toSolrRequest(tuple); solrClient.request(request, solrMapper.getCollection()); ack(tuple); LOG.debug("Acked Tuple: {}", tuple); } catch (Exception e) { fail(tuple, e); LOG.debug("Failed Tuple: {}", tuple, e); } }

Target:
nit: may want to push log statement to `fail` method.
Source:
protected void process(Tuple tuple) { try { LOG.debug("Processing Tuple: {}", tuple); SolrRequest request = solrMapper.toSolrRequest(tuple); solrClient.request(request, solrMapper.getCollection()); ack(tuple); LOG.debug("Acked Tuple: {}", tuple); } catch (Exception e) { fail(tuple, e); } }

Target:
Storm provides these debug log messages when the Config has setDebug(true). Therefore we don't usually add these types of logging.
Source:
public void cleanup() { if (solrClient != null) { try { solrClient.close(); } catch (IOException e) { LOG.debug("Error while closing solrClient", e); } } }

Target:
should it be error level?
Source:
public static TopologyDef parseResource(String resource, boolean dumpYaml, boolean processIncludes, String propertiesFile, boolean envSub) throws IOException { InputStream in = FluxParser.class.getResourceAsStream(resource); InputStream propertiesIn = null; if (propertiesFile != null) { propertiesIn = FluxParser.class.getResourceAsStream(propertiesFile); } TopologyDef topology = parseInputStream(in, dumpYaml, processIncludes, propertiesIn, propertiesFile, envSub); in.close(); return topology; }

Target:
There's a corner case here: 1. `--resource` option is provided hence this method called. 2. yaml file has an include statement which sets resource as `false`. 3. `parseFile` is called which doesn't handle propertiesFile as resource. So it needs more fine-grained approach regarding dealing with propertiesFile. If my understanding is right, given that we only read and don't modify properties file after loading it, we could just create Properties instance from here and pass it instead of passing properties file name. Makes sense?
Source:
public void disconnect() { stateStorage.unregister(stateId); if (solo) { stateStorage.close(); this.assignmentsBackend.dispose(); } }

Target:
nit: spacing.
Source:
private static String checkInitResource(Map<String, Double> topologyResources, Map topologyConf, String resourceName) { StringBuilder msgBuilder = new StringBuilder(); if (topologyResources.containsKey(resourceName)) { Double resourceValue = (Double) topologyConf.getOrDefault(resourceName, null); topologyResources.put(resourceName, resourceValue); msgBuilder.append(resourceName.substring(resourceName.lastIndexOf(".")) + " has been set to " + resourceValue); } return msgBuilder.toString(); }

Target:
I think you missed something in the translation. I think it should be more like ``` if (topologyResources.containsKey(resourceName)) { Double resourceValue = ObjectReader.getDouble(topologyConf.get(resourceName), null); if (resourceValue != null) { topologyResources.put(resourceName, resourceValue); } ... ```
Source:
public void tearDown() { if (waterMarkEventGenerator != null) { waterMarkEventGenerator.shutdown(); } }

Target:
Nit: I don't think this can be null
Source:
public void shutdown() { LOG.debug("Shutting down WaterMarkEventGenerator"); if (this.executorService != null) { executorService.shutdown(); try { if (!executorService.awaitTermination(2, TimeUnit.SECONDS)) { executorService.shutdownNow(); } } catch (InterruptedException ie) { executorService.shutdownNow(); Thread.currentThread().interrupt(); } } }

Target:
Nit: This doesn't seem like it can be null?
Source:
public JedisClusterConfig(Set<InetSocketAddress> nodes, int timeout, int maxRedirections) { Preconditions.checkNotNull(nodes, "Node information should be presented"); this.nodes = nodes; this.timeout = timeout; this.maxRedirections = maxRedirections; }

Target:
Minor: Maybe better to call `this(nodes, timeout, maxRedirections, null);` to remove duplicate codes.
Source:
public LinearDRPCInputDeclarer addSharedMemory(SharedMemory request) { _component.sharedMemory.add(request); return null; }

Target:
Should this return null?
Source:
public WorkerSlot(String nodeId, Number port) { if (port == null) throw new NullPointerException("port cannot be null"); if (nodeId == null) throw new NullPointerException("node id cannot be null"); this.nodeId = nodeId; this.port = port.intValue(); }

Target:
I think checkstyle will complain about this due to missing braces. Maybe you can use http://google.github.io/guava/releases/snapshot/api/docs/com/google/common/base/Preconditions.html#checkNotNull(T) instead?
Source:
public void start() { // initial delay is slightly less than the duration so that the initial tuples wont't expire due to time drift long initialDelay = duration - Math.min((long) (duration * .1), 100); executorFuture = executor.scheduleAtFixedRate(newTriggerTask(), initialDelay, duration, TimeUnit.MILLISECONDS); }

Target:
If the initial delay is slightly less than the given duration then its subsequent scheduling will happen after the given period of initialDelay which can be earlier than expected. Is not it?
Source:
public void getAutoCredentialsTest() { Map<String, Object> emptyMap = new HashMap<>(); Map<String, Object> map = new HashMap<>(); map.put(Config.TOPOLOGY_AUTO_CREDENTIALS, Arrays.asList(new String[]{"org.apache.storm.security.auth.AuthUtilsTestMock"})); Assert.assertTrue(AuthUtils.GetAutoCredentials(emptyMap).isEmpty()); Assert.assertEquals(AuthUtils.GetAutoCredentials(map).size(), 1); }

Target:
Nit: Consider replacing this with Collections.emptyMap() in L107.
Source:
public static byte[] serialize(Object obj) throws IOException { try (ByteArrayOutputStream b = new ByteArrayOutputStream()) { try (ObjectOutputStream o = new ObjectOutputStream(b)) { o.writeObject(obj); o.close(); } return b.toByteArray(); } }

Target:
>o.close(); [](start = 16, length = 10) not needed...
Source:
public List<Object> deserialize(EventData eventData) { final List<Object> fieldContents = new ArrayList<Object>(); String messageData = ""; messageData = new String (eventData.getBody(),eventData.getBodyOffset(),eventData.getBodyLength(),Charset.defaultCharset()); Map metaDataMap = eventData.getProperties(); fieldContents.add(messageData); fieldContents.add(metaDataMap); return fieldContents; }

Target:
The old code handles the case when AmqpValue is species as payload - if you need that support here - you will need to wait until 0.12.0 Refer to this pr for details : https://github.com/Azure/azure-event-hubs-java/pull/66
Source:
public void fail(Object msgId) { KafkaMessageId id = (KafkaMessageId) msgId; PartitionManager m = _coordinator.getManager(id.partition); if (m != null) { m.fail(id.offset); } { // managers for partitions changed - try to find new manager responsible for that partition PartitionManager newManager = getManagerForPartition(id.partition.partition); if (newManager != null) { newManager.fail(id.offset); } } }

Target:
missing `else` between `} {`
Source:
public KafkaSpoutConfig<K,V> build() { Number maxPollRecords = (Number)kafkaProps.get(ConsumerConfig.MAX_POLL_RECORDS_CONFIG); if(maxPollRecords == null) { setMaxPollRecords(maxUncommittedOffsets); } else { setMaxPollRecords(Math.min(maxPollRecords.intValue(), maxUncommittedOffsets)); } return new KafkaSpoutConfig<>(this); }

Target:
@srdo I don't agree with this. If the user does not set MAX_POLL_RECORDS_CONFIG, this value should keep its Kafka default. It should not silently be set to maxUncommittedOffsets. They are two unrelated things, although I agree that MAX_POLL_RECORDS_CONFIG should not exceed maxUncommittedOffsets
Source:
public static PartitionAssignmentChangeListener listenerOf(final ConsumerRebalanceListener consumerRebalanceListener) { return new PartitionAssignmentChangeListener() { @Override public void onPartitionAssignmentChange(Set<TopicPartition> oldPartitions, Set<TopicPartition> newPartitions) { consumerRebalanceListener.onPartitionsRevoked(oldPartitions); consumerRebalanceListener.onPartitionsAssigned(newPartitions); } }; }

Target:
You don't have to change it, but on master we're targeting Java 8, so lambdas are an option for this kind of thing :)
Source:
public void activate() { openReader(); }

Target:
seems every time the spout is deactivated/activated it starts reading the file from the beginning which may not be desirable.
Source:
public static Builder<String, String> builder(String bootstrapServers, Collection<String> topics) { return new Builder<>(bootstrapServers, new StringDeserializer(), new StringDeserializer(), topics); }

Target:
I don't think this works when the topology is serialized. See https://github.com/apache/storm/pull/1696/files#diff-d2c28214a421176fceab415ffe4c95c0R24
Source:
private void subscribeKafkaConsumer() { kafkaConsumer = kafkaConsumerFactory.createConsumer(kafkaSpoutConfig); kafkaSpoutConfig.getSubscription().subscribe(kafkaConsumer, new KafkaSpoutConsumerRebalanceListener(), context); // Initial poll to get the consumer registration process going. // KafkaSpoutConsumerRebalanceListener will be called following this poll, upon partition registration kafkaConsumer.poll(0); }

Target:
Doesn't this belong in the automatic subscriptions? It's not necessary if using manual partition assignment.
Source:
public void prepare(Map stormConfig, TopologyContext topologyContext, OutputCollector outputCollector) { this.outputCollector = outputCollector; this.stormConfig = stormConfig; Map<String, Object> cassandraClientConfig = cassandraConfig != null ? stormConfig : stormConfig; this.cassandraConf = new CassandraConf(cassandraClientConfig); this.client = clientProvider.getClient(cassandraClientConfig); try { session = client.connect(); } catch (NoHostAvailableException e) { outputCollector.reportError(e); } }

Target:
This always pick stormConfig which seems not intended.
Source:
private static Set<String> getListOfKeysFromBlobStore(Map<String, Object> stormConf) { NimbusBlobStore client = new NimbusBlobStore(); client.prepare(stormConf); return Sets.newHashSet(client.listKeys()); }

Target:
Need to guarantee calling client.shutdown().
Source:
private void emit() { //Keep trying to send a tuple when requested while(!emitTupleIfNotEmitted(waitingToEmit.next()) && waitingToEmit.hasNext()) waitingToEmit.remove(); }

Target:
While I don't think it hurts, I don't see why this is necessary. Storm should call nextTuple repeatedly until something is emitted.
Source:
void populateTopicData(String topicName, int msgCount) { kafkaUnitRule.getKafkaUnit().createTopic(topicName); IntStream.range(0, msgCount).forEach(value -> { KeyedMessage<String, String> keyedMessage = new KeyedMessage<>( topicName, ((Integer)value).toString(), ((Integer)value).toString()); kafkaUnitRule.getKafkaUnit().sendMessages(keyedMessage); }); }

Target:
The Integer cast seems unnecessary?
Source:
static public Config getConfig() { Config config = new Config(); config.setDebug(true); return config; }

Target:
Nitpick: In all of the methods declared in this class, can you put the method modifiers in the order public static. That is the most standard way and recommended by the [JLS](https://docs.oracle.com/javase/specs/jls/se8/html/jls-8.html#jls-8.4.3). I also believe it makes the code a bit easier to read.
Source:
private boolean poll() { final int maxUncommittedOffsets = kafkaSpoutConfig.getMaxUncommittedOffsets(); final boolean poll = !waitingToEmit() && numUncommittedOffsets < maxUncommittedOffsets; if (!poll) { final boolean isLoggable = logTimer.isExpiredResetOnTrue(); if (isLoggable) { if (waitingToEmit()) { LOG.debug("Not polling. Tuples waiting to be emitted. uncommitted offsets across all topic partitions", numUncommittedOffsets); } if (numUncommittedOffsets >= maxUncommittedOffsets) { LOG.debug("Not polling. uncommitted offsets across all topic partitions has reached the threshold of [{}]", numUncommittedOffsets, maxUncommittedOffsets); } } } return poll; }

Target:
do we need this check since these are debug logs.
Source:
public LSWorkerHeartbeat readHeartbeat() throws IOException { LocalState localState = ConfigUtils.workerState(_conf, _workerId); LSWorkerHeartbeat hb = localState.getWorkerHeartBeat(); LOG.warn("{}: Reading heartbeat {}", _workerId, hb); return hb; }

Target:
debug
Source:
public boolean areAllProcessesDead() throws IOException { Set<Long> pids = getAllPids(); String user = getWorkerUser(); boolean allDead = true; for (Long pid: pids) { if (!isProcessAlive(pid, user)) { LOG.warn("{}: PID is dead", _workerId, pid); } else { allDead = false; break; } } return allDead; }

Target:
debug?
Source:
public static String workerPidPath(Map<String, Object> conf, String id, long pid) { return (workerPidsRoot(conf, id) + FILE_SEPARATOR + pid); }

Target:
Nit: might be better to call workerPidPath(conf, id, String.valueOf(pid)) or vice versa.
Source:
public KinesisConnectionInfo (AWSCredentialsProvider credentialsProvider, ClientConfiguration clientConfiguration, Regions region, Integer recordsLimit) { if (recordsLimit == null || recordsLimit <= 0) { throw new IllegalArgumentException("recordsLimit has to be a positive integer"); } if (region == null) { throw new IllegalArgumentException("region cannot be null"); } serializedKinesisCredsProvider = getKryoSerializedBytes(credentialsProvider); serializedkinesisClientConfig = getKryoSerializedBytes(clientConfiguration); this.recordsLimit = recordsLimit; this.region = region; this.credentialsProvider = null; this.clientConfiguration = null; }

Target:
we don't need to explicitly assign a null to these.
Source:
public void testDefaults() { SimpleFileNameFormat format = new SimpleFileNameFormat(); format.prepare(null, createTopologyContext()); String path = format.getPath(); String name = format.getName(1, System.currentTimeMillis()); Assert.assertEquals("/storm", path); String now = new SimpleDateFormat("yyyyMMddHHmmss").format(System.currentTimeMillis()); Assert.assertEquals(now+".1.txt", name); }

Target:
This is a nit, but this test might occasionally fail since the time is taken at two different points which might resolve to different seconds.
Source:
public BaseWindowedBolt withLateTupleStream(String streamId) { if (!windowConfiguration.containsKey(Config.TOPOLOGY_BOLTS_TUPLE_TIMESTAMP_FIELD_NAME)) { throw new IllegalArgumentException("Timestamp field should be specified before specifying stream for late tuples"); } windowConfiguration.put(Config.TOPOLOGY_BOLTS_LATE_TUPLE_STREAM, streamId); return this; }

Target:
On re-thinking, can this validation be moved to `initWindowManager` or `validate` so that we don't impose an ordering in the builder?
Source:
public void declareOutputFields(OutputFieldsDeclarer declarer) { bolt.declareOutputFields(declarer); super.declareCheckpointStream(declarer); }

Target:
Nit: It doesn't need to be called with `super.`
Source:
private boolean isFilteredIn(String metricName) { if (noneSpecified) { return true; } if (!whitelistPattern.isEmpty()) { return checkMatching(metricName, whitelistPattern, true); } else if (!blacklistPattern.isEmpty()) { return checkMatching(metricName, blacklistPattern, false); } throw new IllegalStateException("Shouldn't reach here"); }

Target:
should this be moved/copied to apply function?
Source:
public boolean isWorkerHbTimedOutImpl(int now, LSWorkerHeartbeat whb, Map conf) { boolean result = false; if ((now - whb.get_time_secs()) > Utils.getInt(conf.get(Config.SUPERVISOR_WORKER_TIMEOUT_SECS))) { result = true; } return result; }

Target:
Minor: This all could just be `return (now - whb.get_time_secs()) > Utils.getInt(conf.get(Config.SUPERVISOR_WORKER_TIMEOUT_SECS));`
Source:
public final static List<ACL> supervisorZkAcls() { final List<ACL> acls = new ArrayList<>(); acls.add(ZooDefs.Ids.CREATOR_ALL_ACL.get(0)); acls.add(new ACL((ZooDefs.Perms.READ ^ ZooDefs.Perms.CREATE), ZooDefs.Ids.ANYONE_ID_UNSAFE)); return acls; }

Target:
minor nit: static implies final for functions/methods, we don't need final.
Source:
private void launchChannelAliveThread() { // netty TimerTask is already defined and hence a fully // qualified name timer.schedule(new java.util.TimerTask() { public void run() { try { LOG.debug("running timer task, address {}", dstAddress); if(closing) { this.cancel(); } getConnectedChannel(); } catch (Exception exp) { LOG.error("channel connection error {}", exp); } } }, 0, CHANNEL_ALIVE_INTERVAL_MS); }

Target:
Let's move the code in the static block above down here at the beginning of `launchChannelAliveThread`, so that as @revans2 suggests, the stack trace will not be so confusing. ``` Java if (timer == null) { synchronized (Client.class) { if (timer == null) { timer = new Timer("Netty-ChannelAlive-Timer", true); } } } ```
Source:
protected Map<String, StormBase> topologyBasesImpl(IStormClusterState stormClusterState) { List<String> activeTopologys = stormClusterState.activeStorms(); Map<String, StormBase> stormBases = new HashMap<String, StormBase>(); if (activeTopologys != null) { for (String topologyId : activeTopologys) { StormBase base = stormClusterState.stormBase(topologyId, null); if (base != null) { stormBases.put(topologyId, base); } } } return stormBases; }

Target:
The original code didn't check for null here. Is this needed? Extra checks that fix things up like this make me nervous that be broke something somewhere and we are using this to work around it.
Source:
public static Map componentConf(Object component) { Map<Object, Object> conf = new HashMap<Object, Object>(); ComponentCommon common = getComponentCommon(component); if (common != null) { String jconf = common.get_json_conf(); if (jconf != null) { conf.putAll((Map<Object, Object>) JSONValue.parse(jconf)); } } return conf; }

Target:
The original code didn't check for null and would throw an NPE if we passed in a null. We should do the same.
Source:
private static boolean isEmptyInputs(ComponentCommon common) { if (common == null) { return true; } else if (common.get_inputs() == null) { return true; } else { return common.get_inputs().isEmpty(); } }

Target:
The original code would throw an NPE if this was null. In this case I can see it being a minor nit, but I still would prefer to keep the code the same if possible.
Source:
private static Set<String> getStreamOutputFields(Map<String, StreamInfo> streams) { Set<String> outputFields = new HashSet<String>(); if (streams != null) { for (StreamInfo streamInfo : streams.values()) { outputFields.addAll(streamInfo.get_output_fields()); } } return outputFields; }

Target:
In the original code if streams was null the return value was null.
Source:
public static ComponentCommon getComponentCommon(Object component) { if (component == null) { return null; } ComponentCommon common = null; if (component instanceof StateSpoutSpec) { common = ((StateSpoutSpec) component).get_common(); } else if (component instanceof SpoutSpec) { common = ((SpoutSpec) component).get_common(); } else if (component instanceof Bolt) { common = ((Bolt) component).get_common(); } return common; }

Target:
This null check was not in the original code, but it is a minor nit here.
Source:
public static void addMetricStreams(StormTopology topology) { for (Object component : allComponents(topology).values()) { ComponentCommon common = getComponentCommon(component); if (common != null) { StreamInfo streamInfo = Thrift.outputFields(Arrays.asList("task-info", "data-points")); common.put_to_streams(Constants.METRICS_STREAM_ID, streamInfo); } } }

Target:
This null check was not in the original code.
Source:
public static void addSystemStreams(StormTopology topology) { for (Object component : allComponents(topology).values()) { ComponentCommon common = getComponentCommon(component); if (common != null) { StreamInfo streamInfo = Thrift.outputFields(Arrays.asList("event")); common.put_to_streams(SYSTEM_STREAM_ID, streamInfo); } } }

Target:
Again no null check in original code.
Source:
public static int numStartExecutors(Object component) throws InvalidTopologyException { ComponentCommon common = getComponentCommon(component); if (common == null) { throw new InvalidTopologyException("unknown component type " + component.getClass().getName()); } int parallelismHintNum = Thrift.getParallelismHint(common); return parallelismHintNum; }

Target:
This null check didn't exist in the original code.
Source:
public void setCurrAssignment(Map<Long, LocalAssignment> currAssignment) { this.currAssignment.clear(); this.currAssignment.putAll(currAssignment); }

Target:
The clear with the add should probably be atomic. The race is very small, but I would rather see us have an AtomicReference<Map<Long, LocalAssignment>> instead of a ConcurrentHashMap.
Source:
public void setAssignmentVersions(Map<String, Map<String, Object>> assignmentVersions) { this.assignmentVersions.clear(); this.assignmentVersions.putAll(assignmentVersions); }

Target:
Here too this operation should be atomic and I would rather see assignmentVersions be an `AtomicReference<Map<String, Map<String, Object>>>`