auphong2707/dl4se-code-review-dataset huggingface dataset:
Source:
public boolean buildCollectorFile(ConcurStandardAccountingExtractFile saeFileContents) { ConcurStandardAccountingExtractCollectorBatchBuilder builder = new ConcurStandardAccountingExtractCollectorBatchBuilder( START universityDateService::getFiscalYear, dateTimeService::toString, END concurSAEValidationService::validateConcurStandardAccountingExtractDetailLine, this::getConcurParameterValueAsString); CollectorBatch collectorBatch = builder.buildCollectorBatchFromStandardAccountingExtract(0, saeFileContents); if (collectorBatch == null) { LOG.error("There was a problem preparing the data for the Collector file; will not create a file."); return false; } return true; } would it be better to define these lamba functions as a variables to help with understanding what is going on?

Target:
public boolean buildCollectorFile(ConcurStandardAccountingExtractFile saeFileContents) { ConcurStandardAccountingExtractCollectorBatchBuilder builder = new ConcurStandardAccountingExtractCollectorBatchBuilder( universityDateService, dateTimeService, concurSAEValidationService, this::getConcurParameterValueAsString); CollectorBatch collectorBatch = builder.buildCollectorBatchFromStandardAccountingExtract(0, saeFileContents); if (collectorBatch == null) { LOG.error("There was a problem preparing the data for the Collector file; will not create a file."); return false; } return true; }
Source:
private Object loadFile(String fileName) { byte[] fileByteContent = safelyLoadFileBytes(fileName); START Object parsedObject; END parsedObject = getBatchInputFileService().parse(getBatchInputFileType(), fileByteContent); return parsedObject; } these two lines could be combined in one. no need for a declaration on a separate line.

Target:
private Object loadFile(String fileName) { byte[] fileByteContent = safelyLoadFileBytes(fileName); Object parsedObject = getBatchInputFileService().parse(getBatchInputFileType(), fileByteContent); return parsedObject; }
Source:
public boolean fileApprovedAmountsMatchHeaderApprovedAmount(ConcurRequestExtractFile requestExtractFile) { KualiDecimal detailLinesAmountSumKualiDecimal = getTotalRequestFileRequestAmount(requestExtractFile); START if (detailLinesAmountSumKualiDecimal.doubleValue() == requestExtractFile.getTotalApprovedAmount().doubleValue()) { END return true; } else { LOG.error("Header amount validation failed. Header amount was =" + requestExtractFile.getTotalApprovedAmount().toString() + "= while file calculated amount was =" + detailLinesAmountSumKualiDecimal.toString() + "="); return false; } } I am not sure why the comparison between the doubleValue is needed here. You can use the KualiDecimal equals or compareTo methods for this test. (equals is calling the compareTo)

Target:
private boolean fileApprovedAmountsMatchHeaderApprovedAmount(ConcurRequestExtractFile requestExtractFile) { KualiDecimal detailLinesAmountSumKualiDecimal = getTotalRequestFileRequestAmount(requestExtractFile); if (detailLinesAmountSumKualiDecimal.equals(requestExtractFile.getTotalApprovedAmount())) { return true; } else { LOG.error("Header amount validation failed. Header amount was =" + requestExtractFile.getTotalApprovedAmount().toString() + "= while file calculated amount was =" + detailLinesAmountSumKualiDecimal.toString() + "="); return false; } }
Source:
public void createPdpFeedsFromRequestExtracts() { List<String> filesToProcess = getUnprocessedRequestExtractFiles(); if (filesToProcess.isEmpty()) { START LOG.error("No Request Extract files found to process."); END } else { for (String requestExtractFileName : filesToProcess) { createPdpFeedFromRequestExtract(requestExtractFileName); LOG.info("Request Extract File " + requestExtractFileName + " was processed."); } } } I am not sure this is necessarily an error. Should this log be at info level?

Target:
public void createPdpFeedsFromRequestExtracts() { List<String> filesToProcess = getUnprocessedRequestExtractFiles(); if (filesToProcess.isEmpty()) { LOG.info("No Request Extract files found to process."); } else { for (String requestExtractFileName : filesToProcess) { createPdpFeedFromRequestExtract(requestExtractFileName); LOG.info("Request Extract File " + requestExtractFileName + " was processed."); } } }
Source:
public static boolean isExpenseReportURI(String URI) { START if (StringUtils.isNotBlank(URI) && URI.contains(ConcurConstants.EXPENSE_REPORT_URI_INDICATOR)) { END return true; } return false; } Why not just return the condtional?

Target:
public static boolean isExpenseReportURI(String URI) { return StringUtils.isNotBlank(URI) && URI.contains(ConcurConstants.EXPENSE_REPORT_URI_INDICATOR); }
Source:
public ValidationResult validateConcurAccountInfo(ConcurAccountInfo concurAccountInfo) { START ValidationResult validationResult = checkAccountingString( END concurAccountInfo.getChart(), concurAccountInfo.getAccountNumber(), concurAccountInfo.getSubAccountNumber(), concurAccountInfo.getObjectCode(), concurAccountInfo.getSubObjectCode(), concurAccountInfo.getProjectCode()); return validationResult; } I would consider eliminating this intermediate variable, validationResult, and just returning the result of the checkAccountingString() call directly.

Target:
public ValidationResult validateConcurAccountInfo(ConcurAccountInfo concurAccountInfo) { return checkAccountingString( concurAccountInfo.getChart(), concurAccountInfo.getAccountNumber(), concurAccountInfo.getSubAccountNumber(), concurAccountInfo.getObjectCode(), concurAccountInfo.getSubObjectCode(), concurAccountInfo.getProjectCode()); }
Source:
START public JobExecutionListener loggingListener() { END return new LoggingListener(); } You should make these the concrete classes as the return types. You don't need to downcast them for them to be picked up in collection injections and using the concrete classes lets them be recognized easier for conditionals.

Target:
public LoggingListener loggingListener() { return new LoggingListener(); }
Source:
protected void addParametersForScript( final Map<String, Object> parameters, final Set<Command> resources, final JobRequest jobRequest ) { START parameters.put(COMMANDS_BINDING, resources); END } How come the job request is added to the parameters automatically, but the list of resources is not?

Target:
protected void addParametersForScript( final Map<String, Object> parameters, final Set<Command> resources, final JobRequest jobRequest ) { super.addParametersForScript(parameters, resources, jobRequest); parameters.put(COMMANDS_BINDING, resources); }
Source:
protected void attemptTransition( final ExecutionContext executionContext ) throws RetryableTransitionException, FatalTransitionException { final File jobDirectory = executionContext.getJobDirectory(); START @NotNull final CleanupStrategy cleanupStrategy = executionContext.getCleanupStrategy(); END if (jobDirectory != null) { log.info("Cleaning up job directory (strategy: {})", cleanupStrategy); try { this.jobSetupService.cleanupJobDirectory( jobDirectory.toPath(), cleanupStrategy ); } catch (final IOException e) { throw new RetryableTransitionException("Failed to cleanup job directory", e); } } } FYI I don't think putting these inside the method contents has any effect as there is no proxy to intercept and evaluate them.

Target:
protected void attemptTransition( final ExecutionContext executionContext ) throws RetryableTransitionException, FatalTransitionException { final File jobDirectory = executionContext.getJobDirectory(); final CleanupStrategy cleanupStrategy = executionContext.getCleanupStrategy(); if (jobDirectory != null) { log.info("Cleaning up job directory (strategy: {})", cleanupStrategy); try { this.jobSetupService.cleanupJobDirectory( jobDirectory.toPath(), cleanupStrategy ); } catch (final IOException e) { throw new RetryableTransitionException("Failed to cleanup job directory", e); } } }
Source:
public void testExpectedContextWhenSNSDisabled() { this.contextRunner .withPropertyValues( "genie.notifications.sns.enabled=false" ) .run( (context) -> { START System.out.println(context); END Assertions.assertThat(context).doesNotHaveBean(AWSAutoConfiguration.SNS_CLIENT_BEAN_NAME); } ); } Probably didn't mean to leave this in here?

Target:
public void testExpectedContextWhenSNSDisabled() { this.contextRunner .withPropertyValues( "genie.notifications.sns.enabled=false" ).run( (context) -> { Assertions.assertThat(context).doesNotHaveBean(AWSAutoConfiguration.SNS_CLIENT_BEAN_NAME); } ); }
Source:
Loader( final TaskScheduler taskScheduler, final Environment environment, final GenieFileTransferService fileTransferService, START final MeterRegistry registry END ) { this.environment = environment; this.registry = registry; this.fileTransferService = fileTransferService; // Schedule the task to run with the configured refresh rate // Task will be stopped when the system stops final long refreshRate = environment.getProperty( ScriptLoadBalancerProperties.REFRESH_RATE_PROPERTY, Long.class, 300_000L ); taskScheduler.scheduleWithFixedDelay(this::refresh, refreshRate); } Not to increase the scope creep here but we'll probably want to get rid of this service soon so if you were so inclined to flip this to use the `ResourceLoader` I wouldn't object

Target:
protected Loader( final TaskScheduler taskScheduler, final Environment environment, final GenieFileTransferService fileTransferService, final MeterRegistry registry ) { this.environment = environment; this.registry = registry; this.fileTransferService = fileTransferService; // Schedule the task to run with the configured refresh rate // Task will be stopped when the system stops final long refreshRate = environment.getProperty( ScriptLoadBalancerProperties.REFRESH_RATE_PROPERTY, Long.class, 300_000L ); taskScheduler.scheduleWithFixedDelay(this::refresh, refreshRate); }
Source:
START CompiledScript get() { END return script.get(); } Is this one of those `@VisibleForTesting` things? Seems unnecessary

Target:
protected CompiledScript get() { return script.get(); }
Source:
public void jobUpdate(final JobEntity jobEntity) { final JobStatus currentState = jobEntity.getStatus(); final JobStatus previouslyNotifiedState = jobEntity.getNotifiedJobStatus(); final String jobId = jobEntity.getUniqueId(); if (currentState != previouslyNotifiedState) { log.info( "Detected state change for job: from: to: {}", jobId, previouslyNotifiedState, currentState ); // Notify observer if not null START if (this.persistedJobStatusObserver != null) { END this.persistedJobStatusObserver.notify(jobId, previouslyNotifiedState, currentState); } // Save this as the latest published state jobEntity.setNotifiedJobStatus(currentState); } } If the observer is null why bother doing anything? There doesn't appear to be any way to change it from being null as the constructor is the only modifier so why not short circuit all the other logic and just return.

Target:
public void jobUpdate(final JobEntity jobEntity) { final JobStatus currentState = jobEntity.getStatus(); final JobStatus previouslyNotifiedState = jobEntity.getNotifiedJobStatus(); final String jobId = jobEntity.getUniqueId(); if (currentState != previouslyNotifiedState) { log.info( "Detected state change for job: from: to: {}", jobId, previouslyNotifiedState, currentState ); // Notify observer this.persistedJobStatusObserver.notify(jobId, previouslyNotifiedState, currentState); // Save this as the latest published state jobEntity.setNotifiedJobStatus(currentState); } }
Source:
public Server gRpcServer( @Value("${grpc.server.port:0}") final int port, final Set<BindableService> services, final List<ServerInterceptor> serverInterceptors ) { final NettyServerBuilder builder = NettyServerBuilder.forPort( port == RANDOM_PORT START ? SocketUtils.findAvailableTcpPort() END : port ); // Add Service interceptors and add services to the server services .stream() .map(BindableService::bindService) .map(serviceDefinition -> ServerInterceptors.intercept(serviceDefinition, serverInterceptors)) .forEach(builder::addService); return builder.build(); } I'd prefer to set the port to 0 and let the socket bind to an ephemeral port. There's no chance of collision that way.

Target:
public Server gRpcServer( @Value("${grpc.server.port:0}") final int port, // TODO: finalize how to get configure this property final Set<BindableService> services, final List<ServerInterceptor> serverInterceptors ) { final NettyServerBuilder builder = NettyServerBuilder.forPort(port); // Add Service interceptors and add services to the server services .stream() .map(BindableService::bindService) .map(serviceDefinition -> ServerInterceptors.intercept(serviceDefinition, serverInterceptors)) .forEach(builder::addService); return builder.build(); }
Source:
public static String[] getOptionArguments(final String[] args) { final List<String> temporary = Arrays.asList(args); START final int index = temporary.indexOf(BARE_DOUBLE_DASH); END if (index == -1) { // All arguments are options if no explicit double dash is passed return args; } else if (index == 0) { // Double dash as first argument means all arguments are parameters return new String[0]; } else { // All argument up to (and not including) the double dash return temporary.subList(0, index).toArray(new String[index]); } } It's late and I'm tired so I could be totally off on this but I feel like this could be done in a single pass and save all the conversions and checks. Maybe just iterate through the array and write each item into a resultant array until you encounter the double dash and then return the result array?

Target:
public static String[] getOptionArguments(final String[] args) { final List<String> temporary = new ArrayList<>(args.length); for (int i = 0; i < args.length; i++) { if (BARE_DOUBLE_DASH.equals(args[i])) { break; } temporary.add(args[i]); } return temporary.toArray(new String[temporary.size()]); }
Source:
public void canSetRequestedArchiveLocationPrefix() { Assert.assertFalse(this.jobEntity.getRequestedArchiveLocationPrefix().isPresent()); final String requestedArchiveLocationPrefix = UUID.randomUUID().toString(); this.jobEntity.setRequestedArchiveLocationPrefix(requestedArchiveLocationPrefix); Assert.assertThat( START this.jobEntity.getRequestedArchiveLocationPrefix().orElse(UUID.randomUUID().toString()), END Matchers.is(requestedArchiveLocationPrefix) ); } I find the `orElse` here a bit confusing... how about just `.get`?

Target:
public void canSetRequestedArchiveLocationPrefix() { Assert.assertFalse(this.jobEntity.getRequestedArchiveLocationPrefix().isPresent()); final String requestedArchiveLocationPrefix = UUID.randomUUID().toString(); this.jobEntity.setRequestedArchiveLocationPrefix(requestedArchiveLocationPrefix); Assert.assertThat( this.jobEntity.getRequestedArchiveLocationPrefix().get(), Matchers.is(requestedArchiveLocationPrefix) ); }
Source:
private <ResponseType> ResponseType handleResponseFuture( final ListenableFuture<ResponseType> responseListenableFuture ) { final ResponseType response; try { response = responseListenableFuture.get(); } catch (final ExecutionException | InterruptedException e) { START throw new GenieRuntimeException("Failed to perform request"); END } return response; } you should add `e` as the cause here so that it trickles up properly

Target:
private <ResponseType> ResponseType handleResponseFuture( final ListenableFuture<ResponseType> responseListenableFuture ) { final ResponseType response; try { response = responseListenableFuture.get(); } catch (final ExecutionException | InterruptedException e) { throw new GenieRuntimeException("Failed to perform request", e); } return response; }
Source:
private void createDirectoryStructureIfNotExists(final File dir) { if (!dir.exists()) { try { Files.createParentDirs(new File(dir, DUMMY_FILE_NAME)); } catch (final IOException e) { START throw new RuntimeException("Failed to create directory: " + dir.getAbsolutePath(), e); END } } else if (!dir.isDirectory()) { throw new RuntimeException("This location is not a directory: " + dir.getAbsolutePath()); } } Is it a conscious decision to throw an unchecked here?

Target:
private void createDirectoryStructureIfNotExists(final File dir) throws IOException { if (!dir.exists()) { try { Files.createParentDirs(new File(dir, DUMMY_FILE_NAME)); } catch (final IOException e) { throw new IOException("Failed to create directory: " + dir.getAbsolutePath(), e); } } else if (!dir.isDirectory()) { throw new IOException("This location is not a directory: " + dir.getAbsolutePath()); } }
Source:
public PingServiceGrpc.PingServiceBlockingStub getBlockingPingClient( final String serverHost, final int serverPort ) { START final Channel channel = channelFactory.getSharedManagedChannel(serverHost, serverPort); END return PingServiceGrpc.newBlockingStub(channel); } combine into one return line?

Target:
public PingServiceGrpc.PingServiceBlockingStub getBlockingPingClient( final String serverHost, final int serverPort ) { return PingServiceGrpc.newBlockingStub( channelFactory.getSharedManagedChannel(serverHost, serverPort) ); }
Source:
public void setUp() throws IOException { final Registry registry = Mockito.mock(Registry.class); this.initialSetupTask = new InitialSetupTask(registry); START this.temporaryTestDirectory = Files.createTempDirectory(InitialSetupTaskUnitTest.class.getSimpleName() + "-"); END } Is there a reason you didn't use [this](http://junit.org/junit4/javadoc/4.12/org/junit/rules/TemporaryFolder.html)?

Target:
public void setUp() throws IOException { final Registry registry = Mockito.mock(Registry.class); this.initialSetupTask = new InitialSetupTask(registry); }
Source:
public void setUp() throws IOException { final Registry registry = Mockito.mock(Registry.class); this.initialSetupTask = new InitialSetupTask(registry); START this.temporaryTestDirectory = Files.createTempDirectory(this.getClass().getSimpleName() + "-"); END } I thought you were going to switch this to use the `@Rule` from JUnit for temp directories?

Target:
public void setUp() throws IOException { final Registry registry = Mockito.mock(Registry.class); this.initialSetupTask = new InitialSetupTask(registry); }
Source:
public JobKickoffTask( final boolean runAsUserEnabled, final boolean userCreationEnabled, @NotNull final Executor executor, @NotNull final String hostname, @NotNull final Registry registry ) { this.isRunAsUserEnabled = runAsUserEnabled; this.isUserCreationEnabled = userCreationEnabled; this.executor = executor; this.hostname = hostname; this.timer = registry.timer("genie.jobs.tasks.jobKickoffTask.timer"); retryTemplate = new RetryTemplate(); START retryTemplate.setBackOffPolicy(new ExponentialBackOffPolicy()); END } Don't we already have a retry template as a bean somewhere? why isn't this passed into the constructor?

Target:
public JobKickoffTask( final boolean runAsUserEnabled, final boolean userCreationEnabled, @NotNull final Executor executor, @NotNull final String hostname, @NotNull final Registry registry ) { this.isRunAsUserEnabled = runAsUserEnabled; this.isUserCreationEnabled = userCreationEnabled; this.executor = executor; this.hostname = hostname; this.registry = registry; this.timer = registry.timer("genie.jobs.tasks.jobKickoffTask.timer"); retryTemplate = new RetryTemplate(); retryTemplate.setBackOffPolicy(new ExponentialBackOffPolicy()); }
Source:
private void validateHostAndUpdateErrorCount(final String host) { final boolean isNodeHealthy = isNodeHealthy(host); // // If node is healthy, remove the entry from the errorCounts. // If node is not healthy, update the entry in errorCounts // START if (isNodeHealthy) { END if (errorCounts.containsKey(host)) { errorCounts.remove(host); } } else { if (this.errorCounts.containsKey(host)) { this.errorCounts.put(host, this.errorCounts.get(host) + 1); } else { this.errorCounts.put(host, 1); } } } Just call the method within the if statement, no need to save it to a local variable

Target:
private void validateHostAndUpdateErrorCount(final String host) { // // If node is healthy, remove the entry from the errorCounts. // If node is not healthy, update the entry in errorCounts // if (isNodeHealthy(host)) { if (errorCounts.containsKey(host)) { errorCounts.remove(host); } } else { if (this.errorCounts.containsKey(host)) { this.errorCounts.put(host, this.errorCounts.get(host) + 1); } else { this.errorCounts.put(host, 1); } } }
Source:
public void executeTask( @NotNull final Map<String, Object> context ) throws GenieException { log.debug("Executing JobKillLogic Task in the workflow."); super.executeTask(context); // Append logic for handling job kill signal START Utils.appendToWriter(writer, JobConstants.JOB_KILL_HANDLER_LOGIC.toString()); END } Instead of calling toString over and over here you should just call it once in the constant class when it is created.

Target:
public void executeTask( @NotNull final Map<String, Object> context ) throws GenieException { log.debug("Executing JobKillLogic Task in the workflow."); super.executeTask(context); // Append logic for handling job kill signal Utils.appendToWriter(writer, JobConstants.JOB_KILL_HANDLER_LOGIC); }
Source:
START public List<TopologySummary> getTopologySummaries() throws AuthorizationException, TException { END return getNimbus().getTopologySummaries(); } AuthorizationException can be removed

Target:
public List<TopologySummary> getTopologySummaries() throws TException { return getNimbus().getTopologySummaries(); }
Source:
START public TopologySummary getTopologySummaryByName(String name) throws AuthorizationException, TException { END return getNimbus().getTopologySummaryByName(name); } AuthorizationException can be removed

Target:
public TopologySummary getTopologySummaryByName(String name) throws TException { return getNimbus().getTopologySummaryByName(name); }
Source:
START public TopologyInfo getTopologyInfoWithOpts(String id, GetInfoOptions options) throws NotAliveException, AuthorizationException, END TException { return nimbus.getTopologyInfoWithOpts(id, options); } NotAliveException, AuthorizationException can be removed

Target:
public TopologyInfo getTopologyInfoWithOpts(String id, GetInfoOptions options) throws TException { return nimbus.getTopologyInfoWithOpts(id, options); }
Source:
START public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws NotAliveException, AuthorizationException, END TException { return nimbus.getTopologyInfoByNameWithOpts(name, options); } NotAliveException, AuthorizationException can be removed

Target:
public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws TException { return nimbus.getTopologyInfoByNameWithOpts(name, options); }
Source:
START public TopologyInfo getTopologyInfoByName(String name) throws NotAliveException, AuthorizationException, TException { END try { getTopologyInfoByNameCalls.mark(); GetInfoOptions options = new GetInfoOptions(); options.set_num_err_choice(NumErrorsChoice.ALL); return getTopologyInfoByNameImpl(name, options); } catch (Exception e) { LOG.warn("get topology info exception. (topology name={})", name, e); if (e instanceof TException) { throw (TException) e; } throw new RuntimeException(e); } } Looks like we don't need `NotAliveException, AuthorizationException`

Target:
public TopologyInfo getTopologyInfoByName(String name) throws TException { try { getTopologyInfoByNameCalls.mark(); GetInfoOptions options = new GetInfoOptions(); options.set_num_err_choice(NumErrorsChoice.ALL); return getTopologyInfoByNameImpl(name, options); } catch (Exception e) { LOG.warn("get topology info exception. (topology name={})", name, e); if (e instanceof TException) { throw (TException) e; } throw new RuntimeException(e); } }
Source:
public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws START NotAliveException, AuthorizationException, TException { END try { getTopologyInfoByNameWithOptsCalls.mark(); return getTopologyInfoByNameImpl(name, options); } catch (Exception e) { LOG.warn("get topology info withOpts by name exception. (topology name={})", name, e); if (e instanceof TException) { throw (TException) e; } throw new RuntimeException(e); } } looks like we don't need `NotAliveException, AuthorizationException`

Target:
public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws TException { try { getTopologyInfoByNameWithOptsCalls.mark(); return getTopologyInfoByNameImpl(name, options); } catch (Exception e) { LOG.warn("get topology info withOpts by name exception. (topology name={})", name, e); if (e instanceof TException) { throw (TException) e; } throw new RuntimeException(e); } }
Source:
START public List<TopologySummary> getTopologySummaries() throws AuthorizationException, TException { END try { getTopologySummariesCalls.mark(); checkAuthorization(null, null, "getTopologySummaries"); return getTopologySummariesImpl(); } catch (Exception e) { LOG.warn("Get TopologySummary info exception.", e); if (e instanceof TException) { throw (TException) e; } throw new RuntimeException(e); } } looks like we don't need AuthorizationException

Target:
public List<TopologySummary> getTopologySummaries() throws TException { try { getTopologySummariesCalls.mark(); checkAuthorization(null, null, "getTopologySummaries"); return getTopologySummariesImpl(); } catch (Exception e) { LOG.warn("Get TopologySummary info exception.", e); if (e instanceof TException) { throw (TException) e; } throw new RuntimeException(e); } }
Source:
private static void oneProducer1Consumer(int prodBatchSz) { START JCQueue q1 = new JCQueue("q1", null, 50_000, 0, prodBatchSz, new WaitStrategyPark(100), "test", "test", END Collections.singletonList(1000), 1000, new StormMetricRegistry()); final Producer prod1 = new Producer(q1); final Consumer cons1 = new Consumer(q1); runAllThds(prod1, cons1); } why is this `null` (Although I guess it doesn't matter)

Target:
private static void oneProducer1Consumer(int prodBatchSz) { JCQueue q1 = new JCQueue("q1", "q1", 50_000, 0, prodBatchSz, new WaitStrategyPark(100), "test", "test", Collections.singletonList(1000), 1000, new StormMetricRegistry()); final Producer prod1 = new Producer(q1); final Consumer cons1 = new Consumer(q1); runAllThds(prod1, cons1); }
Source:
private static JCQueue createQueue(String name, int queueSize) { START return new JCQueue(name, null, queueSize, 0, 1, new WaitStrategyPark(0), "test", "test", Collections.singletonList(1000), 1000, new StormMetricRegistry()); END } is it better to use `name` instead of `null`?

Target:
private static JCQueue createQueue(String name, int queueSize) { return new JCQueue(name, name, queueSize, 0, 1, new WaitStrategyPark(0), "test", "test", Collections.singletonList(1000), 1000, new StormMetricRegistry()); }
Source:
private JCQueue createQueue(String name, int batchSize, int queueSize) { START return new JCQueue(name, null, queueSize, 0, batchSize, waitStrategy, "test", "test", Collections.singletonList(1000), 1000, new StormMetricRegistry()); END } is it better to use name instead of null?

Target:
private JCQueue createQueue(String name, int batchSize, int queueSize) { return new JCQueue(name, name, queueSize, 0, batchSize, waitStrategy, "test", "test", Collections.singletonList(1000), 1000, new StormMetricRegistry()); }
Source:
public void nodeAssignmentSent(String node, boolean successful) { if (!blacklistSendAssignentFailures) { return; } if (!successful) { int failCount = assignmentFailures.getOrDefault(node, 0) + 1; START assignmentFailures.put(node, failCount); END } } Synchronization is needed otherwise there can be a race condition between getting the current count and updating the count (line 296 and line297) Also between `this.assignmentFailures.clear();` and `assignmentFailures.put(node, failCount);

Target:
public void nodeAssignmentSent(String node, boolean successful) { if (!blacklistSendAssignentFailures) { return; } if (!successful) { synchronized (assignmentFailures) { int failCount = assignmentFailures.getOrDefault(node, 0) + 1; assignmentFailures.put(node, failCount); } } }
Source:
private void trackAssignmentFailures() { if (!blacklistSendAssignentFailures) { return; } START Map<String, Integer> assignmentFailureWindow = new HashMap<>(); END assignmentFailureWindow.putAll(this.assignmentFailures); synchronized (assignmentFailures) { this.assignmentFailures.clear(); } this.sendAssignmentFailureCount.add(assignmentFailureWindow); } Since we already `synchronized` on it, maybe should just as well put `assignmentFailureWindow.putAll(this.assignmentFailures);` inside the block and then it doesn't need to be a `concurrentHashMap`. To reduce two synchronization mechanisms to one

Target:
private void trackAssignmentFailures() { if (!blacklistSendAssignentFailures) { return; } Map<String, Integer> assignmentFailureWindow = new HashMap<>(); synchronized (assignmentFailures) { assignmentFailureWindow.putAll(this.assignmentFailures); this.assignmentFailures.clear(); } this.sendAssignmentFailureCount.add(assignmentFailureWindow); }
Source:
public TestConstraintSolverStrategy(boolean consolidatedConfigFlag) { this.consolidatedConfigFlag = consolidatedConfigFlag; START LOG.info("Running tests with consolidatedConfigFlag={}", ""+consolidatedConfigFlag); END } `""+` doesn't seem necessary.

Target:
public TestConstraintSolverStrategy(boolean consolidatedConfigFlag) { this.consolidatedConfigFlag = consolidatedConfigFlag; LOG.info("Running tests with consolidatedConfigFlag={}", consolidatedConfigFlag); }
Source:
private Set<String> collectMapValues(Map<String, Set<String>> map) { START return map.values() END .stream() .flatMap(set -> set.stream()) .collect(Collectors.toCollection(HashSet::new)); } This looks overly complicated. Maybe ``` Set<String> set = new HashSet<>(); map.values().forEach((s) -> set.addAll(s)); return set; ```

Target:
private Set<String> collectMapValues(Map<String, Set<String>> map) { Set<String> set = new HashSet<>(); map.values().forEach((s) -> set.addAll(s)); return set; }
Source:
public boolean blobExists(String key, Subject who) throws AuthorizationException, KeyNotFoundException { who = checkAndGetSubject(who); validateKey(key); SettableBlobMeta meta = getStoredBlobMeta(key); aclHandler.hasPermissions(meta.get_acl(), READ, who, key); try { BlobStoreFile blobStoreFile = hbs.read(DATA_PREFIX + key); START return blobStoreFile.exists(); END } catch (IOException e) { throw new RuntimeException(e); } } We don't need this. For every key, there will be two files `meta_<key>` and `data_<key>`. Every operation to the blob will first check and get `meta_<key>`. If the blob is deleted, it throws `KeyNotFoundException`. If it exists, we don't need to continue to check `data_<key>` because we can simply assume `data_<key>` exists. Because if `meta_<key>`exists but `data_<key>` doesn't, there will be RuntimeException with the client tries to download or access to the blob. In this way, we can reduce the hdfs operation further.

Target:
public boolean blobExists(String key, Subject who) throws AuthorizationException { try { who = checkAndGetSubject(who); validateKey(key); SettableBlobMeta meta = getStoredBlobMeta(key); aclHandler.hasPermissions(meta.get_acl(), READ, who, key); } catch (KeyNotFoundException e) { return false; } return true; }
Source:
public static Map<String, Map<String, Object>> getTopologyLag(StormTopology userTopology, Map<String,Object> config) { START Boolean disableLagMonitoring = ObjectReader.getBoolean(config.get(DaemonConfig.UI_DISABLE_SPOUT_LAG_MONITORING), false); END return disableLagMonitoring ? Collections.EMPTY_MAP : TopologySpoutLag.lag(userTopology, config); } Nit: Consider adding the default to defaults.yaml instead of here. That way it's obvious to users what the default is.

Target:
public static Map<String, Map<String, Object>> getTopologyLag(StormTopology userTopology, Map<String,Object> config) { Boolean disableLagMonitoring = (Boolean)(config.get(DaemonConfig.UI_DISABLE_SPOUT_LAG_MONITORING)); return disableLagMonitoring ? Collections.EMPTY_MAP : TopologySpoutLag.lag(userTopology, config); }
Source:
private void seekOffsetByStartTimeStamp(TopicPartition tp) { START long startTimeStampOffset; END Map<TopicPartition, OffsetAndTimestamp> offsetsForTimes = consumer.offsetsForTimes(Collections.singletonMap(tp, startTimeStamp)); OffsetAndTimestamp startOffsetAndTimeStamp = offsetsForTimes.get(tp); startTimeStampOffset = startOffsetAndTimeStamp.offset(); LOG.debug("First poll for topic partition [{}], seeking to partition from startTimeStamp [{}]", tp, startTimeStamp); consumer.seek(tp, startTimeStampOffset); } Nit: No reason to not declare and assign on the same line.

Target:
private void seekOffsetByStartTimeStamp(TopicPartition tp) { Map<TopicPartition, OffsetAndTimestamp> offsetsForTimes = consumer.offsetsForTimes(Collections.singletonMap(tp, startTimeStamp)); OffsetAndTimestamp startOffsetAndTimeStamp = offsetsForTimes.get(tp); long startTimeStampOffset = startOffsetAndTimeStamp.offset(); LOG.debug("First poll for topic partition [{}], seeking to partition from startTimeStamp [{}]", tp, startTimeStamp); consumer.seek(tp, startTimeStampOffset); }
Source:
START public boolean remove(NormalizedResourcesWithMemory other) { END boolean negativeResources = normalizedResources.remove(other.getNormalizedResources(), null); totalMemoryMb -= other.getTotalMemoryMb(); if (totalMemoryMb < 0.0) { negativeResources = true; totalMemoryMb = 0.0; } return negativeResources; } Maybe change `public boolean remove(NormalizedResourcesWithMemory other, ResourceMetrics resourceMetrics)` and then just call `remove(other, null)`

Target:
public boolean remove(NormalizedResourcesWithMemory other) { return remove(other, null); }
Source:
private void mkAssignments() throws Exception { try { START mkAssignments(null); END } catch(Exception e) { this.mkAssignmentsErrors.mark(); throw e; } } Do we want to make the change inside `mkAssignments(String scratchTopoId)` so we can count errors during the rebalance command as well?

Target:
private void mkAssignments() throws Exception { mkAssignments(null); }
Source:
public void removeAllPrivateWorkerKeys(String topologyId) { for (WorkerTokenServiceType type : WorkerTokenServiceType.values()) { String path = ClusterUtils.secretKeysPath(type, topologyId); try { START LOG.debug("Removing worker keys under {}", path); END stateStorage.delete_node(path); } catch (RuntimeException e) { //This should never happen because only the primary nimbus is active, but just in case // declare the race safe, even if we lose it. if (!Utils.exceptionCauseIsInstanceOf(KeeperException.NoNodeException.class, e)) { throw e; } } } } If this does not hit often, I would vote for making this info.

Target:
public void removeAllPrivateWorkerKeys(String topologyId) { for (WorkerTokenServiceType type : WorkerTokenServiceType.values()) { String path = ClusterUtils.secretKeysPath(type, topologyId); try { LOG.info("Removing worker keys under {}", path); stateStorage.delete_node(path); } catch (RuntimeException e) { //This should never happen because only the primary nimbus is active, but just in case // declare the race safe, even if we lose it. if (!Utils.exceptionCauseIsInstanceOf(KeeperException.NoNodeException.class, e)) { throw e; } } } }
Source:
START List<DirectoryCleaner.DeletionMeta> perWorkerDirCleanup(long size) { END return workerLogs.getAllWorkerDirs().stream() .map(Unchecked.function(dir -> directoryCleaner.deleteOldestWhileTooLarge(Collections.singletonList(dir), size, true, null))) .collect(toList()); } If DeletionMeta is used outside DirectoryCleaner, I think it belongs in its own file.

Target:
List<DeletionMeta> perWorkerDirCleanup(long size) { return workerLogs.getAllWorkerDirs().stream() .map(Unchecked.function(dir -> directoryCleaner.deleteOldestWhileTooLarge(Collections.singletonList(dir), size, true, null))) .collect(toList()); }
Source:
public boolean recordBackPressure(Integer taskId) { START return tasks.get(taskId).backpressure.getAndSet(true); END } The return value of this method has switched. getAndSet is going to return the previous value, which is true if it already was under backpressure and false if it was not. This is the opposite of ``` * @return true if an update was recorded, false if taskId is already under BP ``` This is used to control when the server sends messages to the client about backpressure, and could make it happen not frequently enough and the beginning and then way too frequently after that.

Target:
public boolean recordBackPressure(Integer taskId) { return tasks.get(taskId).backpressure.getAndSet(true) == false; }
Source:
public Values(Object... vals) { super(vals != null ? vals.length : 0); START if (vals != null && vals.length != 0 ) { END for (Object o : vals) { add(o); } } else { add(null); } } `vals.length != 0` should be removed in this condition, cause condition for else statement would be `vals == null || vals.length == 0`, which `Values(List())` goes into else statement and comes out of `Values(null)` instead of no element.

Target:
public Values(Object... vals) { super(vals != null ? vals.length : 1); if (vals != null ) { for (Object o : vals) { add(o); } } else { add(null); } }
Source:
public Values(Object... vals) { START super(vals != null ? vals.length : 0); END if (vals != null ) { for (Object o : vals) { add(o); } } else { add(null); } } I think this can be `super(vals != null ? vals.length : 1);` since `if (vals==null)` we are going to add `null` to it. The length will be 1.

Target:
public Values(Object... vals) { super(vals != null ? vals.length : 1); if (vals != null ) { for (Object o : vals) { add(o); } } else { add(null); } }
Source:
public void close() throws Exception { START super.close(); END getMeasured().close(); } Could we please change this a bit? Stopping the timer is not nearly as important as actually closing the resource, and if stopping the timer throws any kind of exception it will not stop the resource. Could we do something more like. ``` try { super.close(); } finally { getMeasured().close(); } ``` This should guarantee that we close the resource always, and if an exception is thrown in both cases the resource exception takes precedent and in newer versions of java the other exception will show up as a Suppressed exception.

Target:
public void close() throws Exception { try { super.close(); } finally { getMeasured().close(); } }
Source:
private TopologyContext createTopologyContext(Map<String, Object> topoConf) { START CustomIndexArray<String> taskToComponent = new CustomIndexArray<>(0,8); END taskToComponent.set(7, "Xcom"); return new TopologyContext(null, topoConf, taskToComponent, null, null, null, null, null, null, 7, 6703, null, null, null, null, null, null); } Why 0,8 if all we are setting is 7?

Target:
private TopologyContext createTopologyContext(Map<String, Object> topoConf) { CustomIndexArray<String> taskToComponent = new CustomIndexArray<>(7,7); taskToComponent.set(7, "Xcom"); return new TopologyContext(null, topoConf, taskToComponent, null, null, null, null, null, null, 7, 6703, null, null, null, null, null, null); }
Source:
public static WorkerMetricsProcessor configureMetricProcessor(Map conf) throws MetricException { try { String processorClass = (String)conf.get(DaemonConfig.STORM_METRIC_PROCESSOR_CLASS); WorkerMetricsProcessor processor = (WorkerMetricsProcessor) (Class.forName(processorClass)).newInstance(); processor.prepare(conf); return processor; START } catch (Throwable t) { END throw new MetricException("Failed to create metric processor", t); } } Can we just make this an Exception instead of a Throwable? Most Errors are not things that we want to try and recover from.

Target:
public static WorkerMetricsProcessor configureMetricProcessor(Map conf) throws MetricException { try { String processorClass = (String)conf.get(DaemonConfig.STORM_METRIC_PROCESSOR_CLASS); WorkerMetricsProcessor processor = (WorkerMetricsProcessor) (Class.forName(processorClass)).newInstance(); processor.prepare(conf); return processor; } catch (Exception e) { throw new MetricException("Failed to create metric processor", e); } }
Source:
public void run() { while (true) { START if (shutdown) { END return; } try { Thread.sleep(sleepMs); } catch (InterruptedException e) { LOG.error("Sleep interrupted", e); continue; } try { purgeMetrics(); } catch (MetricException e) { LOG.error("Failed to purge metrics", e); if (this.failureMeter != null) { this.failureMeter.mark(); } } } } Is this a reason to have separate if statement instead of `while (!shutdown)`?

Target:
public void run() { while (!shutdown) { try { Thread.sleep(sleepMs); } catch (InterruptedException e) { LOG.error("Sleep interrupted", e); continue; } try { purgeMetrics(); } catch (MetricException e) { LOG.error("Failed to purge metrics", e); if (this.failureMeter != null) { this.failureMeter.mark(); } } } }
Source:
private void dropMessage(AddressedTuple tuple, JCQueue queue) { ++dropCount; queue.recordMsgDrop(); START LOG.warn("Dropping message as overflow threshold has reached for Q = {}. OverflowCount = {}. Total Drop Count= {}, Dropped Message : {}", queue.getName(), queue.getOverflowCount(), dropCount, tuple.toString()); END } nit: `tuple.toString()` is not needed, as the logging will do it for you, but since this is a warn log that should almost never be called it really does not matter.

Target:
private void dropMessage(AddressedTuple tuple, JCQueue queue) { ++dropCount; queue.recordMsgDrop(); LOG.warn("Dropping message as overflow threshold has reached for Q = {}. OverflowCount = {}. Total Drop Count= {}, Dropped Message : {}", queue.getName(), queue.getOverflowCount(), dropCount, tuple); }
Source:
private void closeFlushScheduler() { if (flushScheduler != null) { flushScheduler.shutdown(); try { START if (!flushScheduler.awaitTermination(1, TimeUnit.SECONDS)) { END flushScheduler.shutdownNow(); if (!flushScheduler.awaitTermination(1, TimeUnit.SECONDS)) { LOG.error("Flush scheduler did not terminate after trials of shutdown."); } } } catch (InterruptedException ie) { // (Re-)Cancel if current thread also interrupted flushScheduler.shutdownNow(); // Preserve interrupt status Thread.currentThread().interrupt(); } } } nit: You may want to have multiple iterations(which can be 2 or more) with while loop instead of nested if conditionals.

Target:
private void closeFlushScheduler() { if (flushScheduler != null) { flushScheduler.shutdown(); try { if (!flushScheduler.awaitTermination(2, TimeUnit.SECONDS)) { flushScheduler.shutdownNow(); } } catch (InterruptedException ie) { // (Re-)Cancel if current thread also interrupted flushScheduler.shutdownNow(); // Preserve interrupt status Thread.currentThread().interrupt(); } } }
Source:
public void addToEmitMsgs(long offset) { this.emittedOffsets.add(offset); // O(Log N) START this.latestEmittedOffset = offset; END } I don't think this works. This will just set the latest emitted offset to the offset that was most recently emitted. I think you want the highest offset that has been emitted.

Target:
public void addToEmitMsgs(long offset) { this.emittedOffsets.add(offset); // O(Log N) this.latestEmittedOffset = Math.max(latestEmittedOffset, offset); }
Source:
public void activate() { try { subscribeKafkaConsumer(); START metricsConsumer = new KafkaConsumer<>(kafkaSpoutConfig.getKafkaProps()); END } catch (InterruptException e) { throwKafkaConsumerInterruptedException(); } } Why do we need a separate consumer for the metric? Also if we really need two consumers, you should create this via the factory. Otherwise you prevent us from stubbing/spying on this consumer.

Target:
public void activate() { try { subscribeKafkaConsumer(); } catch (InterruptException e) { throwKafkaConsumerInterruptedException(); } }
Source:
public void deactivate() { try { shutdown(); START metricsConsumer.close(); END } catch (InterruptException e) { throwKafkaConsumerInterruptedException(); } } I think this belongs in shutdown.

Target:
public void deactivate() { try { shutdown(); } catch (InterruptException e) { throwKafkaConsumerInterruptedException(); } }
Source:
protected void process(Tuple tuple) { try { LOG.debug("Processing Tuple: {}", tuple); SolrRequest request = solrMapper.toSolrRequest(tuple); solrClient.request(request, solrMapper.getCollection()); ack(tuple); LOG.debug("Acked Tuple: {}", tuple); } catch (Exception e) { fail(tuple, e); START LOG.debug("Failed Tuple: {}", tuple, e); END } } nit: may want to push log statement to `fail` method.

Target:
protected void process(Tuple tuple) { try { SolrRequest request = solrMapper.toSolrRequest(tuple); solrClient.request(request, solrMapper.getCollection()); ack(tuple); } catch (Exception e) { fail(tuple, e); } }
Source:
protected void process(Tuple tuple) { try { START LOG.debug("Processing Tuple: {}", tuple); END SolrRequest request = solrMapper.toSolrRequest(tuple); solrClient.request(request, solrMapper.getCollection()); ack(tuple); LOG.debug("Acked Tuple: {}", tuple); } catch (Exception e) { fail(tuple, e); } } Storm provides these debug log messages when the Config has setDebug(true). Therefore we don't usually add these types of logging.

Target:
protected void process(Tuple tuple) { try { SolrRequest request = solrMapper.toSolrRequest(tuple); solrClient.request(request, solrMapper.getCollection()); ack(tuple); } catch (Exception e) { fail(tuple, e); } }
Source:
public void cleanup() { if (solrClient != null) { try { solrClient.close(); } catch (IOException e) { START LOG.debug("Error while closing solrClient", e); END } } } should it be error level?

Target:
public void cleanup() { if (solrClient != null) { try { solrClient.close(); } catch (IOException e) { LOG.error("Error while closing solrClient", e); } } }
Source:
public static TopologyDef parseResource(String resource, boolean dumpYaml, boolean processIncludes, String propertiesFile, boolean envSub) throws IOException { InputStream in = FluxParser.class.getResourceAsStream(resource); InputStream propertiesIn = null; if (propertiesFile != null) { propertiesIn = FluxParser.class.getResourceAsStream(propertiesFile); } START TopologyDef topology = parseInputStream(in, dumpYaml, processIncludes, propertiesIn, propertiesFile, envSub); END in.close(); return topology; } There's a corner case here: 1. `--resource` option is provided hence this method called. 2. yaml file has an include statement which sets resource as `false`. 3. `parseFile` is called which doesn't handle propertiesFile as resource. So it needs more fine-grained approach regarding dealing with propertiesFile. If my understanding is right, given that we only read and don't modify properties file after loading it, we could just create Properties instance from here and pass it instead of passing properties file name. Makes sense?

Target:
public static TopologyDef parseResource(String resource, boolean dumpYaml, boolean processIncludes, String propertiesFile, boolean envSub) throws IOException { InputStream in = FluxParser.class.getResourceAsStream(resource); TopologyDef topology = parseInputStream(in, dumpYaml, processIncludes, propertiesFile, envSub); in.close(); return topology; }
Source:
public void disconnect() { stateStorage.unregister(stateId); if (solo) { stateStorage.close(); START this.assignmentsBackend.dispose(); END } } nit: spacing.

Target:
public void disconnect() { stateStorage.unregister(stateId); if (solo) { stateStorage.close(); this.assignmentsBackend.close(); } }
Source:
private static String checkInitResource(Map<String, Double> topologyResources, Map topologyConf, String resourceName) { StringBuilder msgBuilder = new StringBuilder(); if (topologyResources.containsKey(resourceName)) { Double resourceValue = (Double) topologyConf.getOrDefault(resourceName, null); START topologyResources.put(resourceName, resourceValue); END msgBuilder.append(resourceName.substring(resourceName.lastIndexOf(".")) + " has been set to " + resourceValue); } return msgBuilder.toString(); } I think you missed something in the translation. I think it should be more like ``` if (topologyResources.containsKey(resourceName)) { Double resourceValue = ObjectReader.getDouble(topologyConf.get(resourceName), null); if (resourceValue != null) { topologyResources.put(resourceName, resourceValue); } ... ```

Target:
private static String checkInitResource(Map<String, Double> topologyResources, Map topologyConf, String resourceName) { StringBuilder msgBuilder = new StringBuilder(); if (topologyResources.containsKey(resourceName)) { Double resourceValue = (Double) topologyConf.getOrDefault(resourceName, null); if (resourceValue != null) { topologyResources.put(resourceName, resourceValue); msgBuilder.append(resourceName.substring(resourceName.lastIndexOf(".")) + " has been set to " + resourceValue); } } return msgBuilder.toString(); }