muttakin2301/ccr2000 huggingface dataset:
Code review comment: You should also test that the exception is retryable or not, and if they have hit the maximum number of attempts. These are checks that are agnostic of the retry algorithm.

Code under review:
public Duration getNextRetryInterval(Exception lastException, Duration remainingTime) {
        int baseWaitTime = 0;

        if (lastException == null || !(lastException instanceof AmqpException)) {
            return this.onGetNextRetryInterval(lastException, remainingTime, baseWaitTime, this.getRetryCount());
        }

        if (((AmqpException) lastException).getErrorCondition() == ErrorCondition.SERVER_BUSY_ERROR) {
            baseWaitTime += ClientConstants.SERVER_BUSY_BASE_SLEEP_TIME_IN_SECS;
        }
        return this.onGetNextRetryInterval(lastException, remainingTime, baseWaitTime, this.getRetryCount());
    }

Line commented on: if (lastException == null || !(lastException instanceof AmqpException)) {

Revised code:
public Duration getNextRetryInterval(Exception lastException, Duration remainingTime) {
        int baseWaitTime = 0;

        if (!isRetriableException(lastException)) {
            return null;
        }

        if (retryCount.get() >= maxRetryCount) {
            return null;
        }

        if (((AmqpException) lastException).getErrorCondition() == ErrorCondition.SERVER_BUSY_ERROR) {
            baseWaitTime += ClientConstants.SERVER_BUSY_BASE_SLEEP_TIME_IN_SECS;
        }

        return this.calculateNextRetryInterval(lastException, remainingTime, baseWaitTime, this.getRetryCount());
    }
Code review comment: The `SyntheticBean` constructor takes a `List<SomeBean>`, so the type argument here is inferred to be `List<SomeBean>`.

Code under review:
public SyntheticBean create(SyntheticCreationalContext<SyntheticBean> context) {
            return new SyntheticBean(context.getInjectedReference(new TypeLiteral<>() {
            }, All.Literal.INSTANCE));
        }

Line commented on: return new SyntheticBean(context.getInjectedReference(new TypeLiteral<>() {

Revised code:
public SyntheticBean create(SyntheticCreationalContext<SyntheticBean> context) {
            return new SyntheticBean(context.getInjectedReference(new TypeLiteral<List<SomeBean>>() {
            }, All.Literal.INSTANCE));
        }
Code review comment: ```suggestion         assertThat(m).containsKeys(String.class, Boolean.class); ```

Code under review:
void unwrapOptionalsPreservesOrder() {
        LinkedOptionalMap<Class<?>, String> map = new LinkedOptionalMap<>();

        map.put("a", String.class, "aaa");
        map.put("b", Boolean.class, "bbb");

        LinkedHashMap<Class<?>, String> m = map.unwrapOptionals();

        assertThat(m).containsKey(String.class);
        assertThat(m).containsValues("aaa", "bbb");
    }

Line commented on: assertThat(m).containsKey(String.class);

Revised code:
void unwrapOptionalsPreservesOrder() {
        LinkedOptionalMap<Class<?>, String> map = new LinkedOptionalMap<>();

        map.put("a", String.class, "aaa");
        map.put("b", Boolean.class, "bbb");

        LinkedHashMap<Class<?>, String> m = map.unwrapOptionals();

        assertThat(m).containsKeys(String.class, Boolean.class);
        assertThat(m).containsValues("aaa", "bbb");
    }
Code review comment: Let's at least drop a TODO to add a more native ParDo.withBadRecordHandler() to reduce this boilerplate.

Code under review:
public PCollection<Integer> expand(PCollection<Integer> input) {
    PCollectionTuple pCollectionTuple =
        input.apply(
            "NoOpDoFn",
            ParDo.of(new OddIsBad(badRecordRouter))
                .withOutputTags(RECORDS, TupleTagList.of(BadRecordRouter.BAD_RECORD_TAG)));

    Coder<BadRecord> badRecordCoder;

    try {
      SchemaRegistry schemaRegistry = input.getPipeline().getSchemaRegistry();
      badRecordCoder =
          SchemaCoder.of(
              schemaRegistry.getSchema(BadRecord.class),
              TypeDescriptor.of(BadRecord.class),
              schemaRegistry.getToRowFunction(BadRecord.class),
              schemaRegistry.getFromRowFunction(BadRecord.class));
    } catch (NoSuchSchemaException e) {
      throw new RuntimeException(e);
    }

    errorHandler.addErrorCollection(
        pCollectionTuple.get(BadRecordRouter.BAD_RECORD_TAG).setCoder(badRecordCoder));

    return pCollectionTuple.get(RECORDS).setCoder(BigEndianIntegerCoder.of());
  }

Line commented on: errorHandler.addErrorCollection(

Revised code:
public PCollection<Integer> expand(PCollection<Integer> input) {
    
    PCollectionTuple pCollectionTuple =
        input.apply(
            "NoOpDoFn",
            ParDo.of(new OddIsBad(badRecordRouter))
                .withOutputTags(RECORDS, TupleTagList.of(BadRecordRouter.BAD_RECORD_TAG)));

    errorHandler.addErrorCollection(
        pCollectionTuple
            .get(BadRecordRouter.BAD_RECORD_TAG)
            .setCoder(BadRecord.getCoder(input.getPipeline())));

    return pCollectionTuple.get(RECORDS).setCoder(BigEndianIntegerCoder.of());
  }
Code review comment: Please do not use `Map.Entry`, just use `Entry`

Code under review:
private Collection<EncryptColumnMetaData> getTableEncryptColumnMetaDatas() {
        Collection<EncryptColumnMetaData> result = new LinkedList<>();
        for (Map.Entry<String, ColumnMetaData> entry : schemaMetaData.get(tableName).getColumns().entrySet()) {
            if (!(entry.getValue() instanceof EncryptColumnMetaData)) {
                continue;
            }
            result.add((EncryptColumnMetaData) entry.getValue());
        }
        return result;
    }

Line commented on: for (Map.Entry<String, ColumnMetaData> entry : schemaMetaData.get(tableName).getColumns().entrySet()) {

Revised code:
private Collection<EncryptColumnMetaData> getTableEncryptColumnMetaDatas() {
        Collection<EncryptColumnMetaData> result = new LinkedList<>();
        for (Entry<String, ColumnMetaData> entry : schemaMetaData.get(tableName).getColumns().entrySet()) {
            if (entry.getValue() instanceof EncryptColumnMetaData) {
                result.add((EncryptColumnMetaData) entry.getValue());
            }
        }
        return result;
    }
Code review comment: How do we know when data distribution has to completed if we add new nodes?  Since we know from the event what the target is, wouldn't it be better to identify the nodes that we are scaling to (i.e. the `count` nodes that are `active`, not retired and have the same resources as the `cluster.targetResources()`), then for each of those check with cluster controller if the node is in sync?

Code under review:
private Cluster updateCompletion(Cluster cluster, NodeList clusterNodes) {
        if (cluster.lastScalingEvent().isEmpty()) return cluster;
        var event = cluster.lastScalingEvent().get();
        if (event.completion().isPresent()) return cluster;

        
        
        if (clusterNodes.retired().stream()
                        .anyMatch(node -> node.history().hasEventAt(History.Event.Type.retired, event.at())))
            return cluster;

        
        for (NodeTimeseries nodeTimeseries : metricsDb.getNodeTimeseries(event.at(), clusterNodes)) {
            Optional<MetricSnapshot> firstOnNewGeneration =
                    nodeTimeseries.asList().stream()
                                           .filter(snapshot -> snapshot.generation() >= event.generation()).findFirst();
            if (firstOnNewGeneration.isEmpty()) return cluster; 
        }


        
        Instant completionTime = nodeRepository().clock().instant();
        return cluster.with(event.withCompletion(completionTime));
    }


Revised code:
private Cluster updateCompletion(Cluster cluster, NodeList clusterNodes) {
        if (cluster.lastScalingEvent().isEmpty()) return cluster;
        var event = cluster.lastScalingEvent().get();
        if (event.completion().isPresent()) return cluster;

        
        
        if (clusterNodes.retired().stream()
                        .anyMatch(node -> node.history().hasEventAt(History.Event.Type.retired, event.at())))
            return cluster;

        
        for (NodeTimeseries nodeTimeseries : metricsDb.getNodeTimeseries(event.at(), clusterNodes)) {
            Optional<MetricSnapshot> firstOnNewGeneration =
                    nodeTimeseries.asList().stream()
                                           .filter(snapshot -> snapshot.generation() >= event.generation()).findFirst();
            if (firstOnNewGeneration.isEmpty()) return cluster; 
        }


        
        Instant completionTime = nodeRepository().clock().instant();
        return cluster.with(event.withCompletion(completionTime));
    }
Code review comment: I believe so. But it's a good point that we could be more defensive here...

Code under review:
BeanArchivePredicateBuildItem additionalBeanArchives() {
        return new BeanArchivePredicateBuildItem(new Predicate<ApplicationArchive>() {

            @Override
            public boolean test(ApplicationArchive archive) {
                
                return !archive.getIndex().getKnownDirectImplementors(GrpcDotNames.MUTINY_BEAN).isEmpty();
            }
        });
    }

Line commented on: return !archive.getIndex().getKnownDirectImplementors(GrpcDotNames.MUTINY_BEAN).isEmpty();

Revised code:
BeanArchivePredicateBuildItem additionalBeanArchives() {
        return new BeanArchivePredicateBuildItem(new Predicate<ApplicationArchive>() {

            @Override
            public boolean test(ApplicationArchive archive) {
                
                return !archive.getIndex().getKnownDirectImplementors(GrpcDotNames.MUTINY_BEAN).isEmpty();
            }
        });
    }
Code review comment: This is a good question. Flink will restart the pipeline every time we scale up/down.  `addSplitsback` is called only when we enable [Restart Pipelined Region Failover Strategy](https://nightlies.apache.org/flink/flink-docs-master/docs/ops/state/task_failure_recovery/#restart-pipelined-region-failover-strategy) and some readers failed. We will put these splits back to Enumerator and reassign them after the failed readers started.

Code under review:
public void addSplitsBack(List<PulsarPartitionSplit> splits, int subtaskId) {
        
        splitAssigner.addSplitsBack(splits, subtaskId);

        
        if (context.registeredReaders().containsKey(subtaskId)) {
            LOG.debug(
                    "Reader {} has been restarted after crashing, we will put splits back to it.",
                    subtaskId);
            
            List<Integer> readers = new ArrayList<>(context.registeredReaders().keySet());
            assignPendingPartitionSplits(readers);
        }
    }

Line commented on: assignPendingPartitionSplits(readers);

Revised code:
public void addSplitsBack(List<PulsarPartitionSplit> splits, int subtaskId) {
        
        splitAssigner.addSplitsBack(splits, subtaskId);

        
        if (context.registeredReaders().containsKey(subtaskId)) {
            LOG.debug(
                    "Reader {} has been restarted after crashing, we will put splits back to it.",
                    subtaskId);
            
            List<Integer> readers = new ArrayList<>(context.registeredReaders().keySet());
            assignPendingPartitionSplits(readers);
        }
    }
Code review comment: Addressed in https://github.com/ballerina-platform/ballerina-lang/pull/38093/commits/f6fc6deae6ab6005be97a5ccea07619863dcab63.

Code under review:
public void testSymbolLookupInModuleAlias() {
        Project project = BCompileUtil.loadProject("test-src/symbol_lookup_with_module_alias_test.bal");
        SemanticModel model = getDefaultModulesSemanticModel(project);
        Document srcFile = getDocumentForSingleSource(project);
        List<Symbol> visibleSymbols = model.visibleSymbols(srcFile, LinePosition.from(19, 25));
        List<String> expectedModuleSymbols = List.of("obj", "sys", "regexp", "'array");
        ArrayList<Object> symbols = new ArrayList<>();
        for (Symbol visibleSymbol : visibleSymbols) {
            if (visibleSymbol.kind() == MODULE
                    && visibleSymbol.getName().isPresent()
                    && expectedModuleSymbols.contains(visibleSymbol.getName().get())) {
                symbols.add(visibleSymbol.getName().get());
            }
        }
        assertEquals(symbols.size(), 2);
    }

Line commented on: assertEquals(symbols.size(), 2);

Revised code:
public void testSymbolLookupInModuleAlias() {
        Project project = BCompileUtil.loadProject("test-src/symbol_lookup_with_module_alias_test.bal");
        SemanticModel model = getDefaultModulesSemanticModel(project);
        Document srcFile = getDocumentForSingleSource(project);
        List<Symbol> visibleSymbols = model.visibleSymbols(srcFile, LinePosition.from(23, 23));
        List<String> expectedModuleSymbols = List.of("obj", "tbl", "regexp", "arr", "foo");
        int moduleSymbolsCount = 0;
        for (Symbol visibleSymbol : visibleSymbols) {
            if (visibleSymbol.kind() == MODULE
                    && visibleSymbol.getName().isPresent()
                    && expectedModuleSymbols.contains(visibleSymbol.getName().get())) {
                moduleSymbolsCount++;
            }
        }
        assertEquals(moduleSymbolsCount, expectedModuleSymbols.size());
    }
Code review comment: I'd move this to line 118, you don't need to query the typeToken if the classNamestack is empty.

Code under review:
private void checkMethodNamePrefix(DetailAST methodDefToken) {
        
        if (!TokenUtil.findFirstTokenByPredicate(methodDefToken,
            c -> c.getType() == TokenTypes.PARAMETERS && c.getChildCount() == 1).isPresent()) {
            return;
        }

        
        final DetailAST typeToken = methodDefToken.findFirstToken(TokenTypes.TYPE);
        if (classNameStack.isEmpty()) {
            return;
        }

        if (!TokenUtil.findFirstTokenByPredicate(
            typeToken, c -> c.getType() == TokenTypes.IDENT && c.getText().equals(classNameStack.peekLast())).isPresent()) {
            return;
        }

        final String methodName = methodDefToken.findFirstToken(TokenTypes.IDENT).getText();
        
        avoidStartWords.forEach(avoidStartWord -> {
            if (methodName.length() >= avoidStartWord.length() && methodName.startsWith(avoidStartWord)) {
                log(methodDefToken, String.format(FLUENT_METHOD_ERR, methodName, avoidStartWord));
            }
        });
    }

Line commented on: final DetailAST typeToken = methodDefToken.findFirstToken(TokenTypes.TYPE);

Revised code:
private void checkMethodNamePrefix(DetailAST methodDefToken) {
        
        if (TokenUtil.findFirstTokenByPredicate(methodDefToken, parameters ->
            parameters.getType() == TokenTypes.PARAMETERS && parameters.getChildCount() != 1).isPresent()) {
            log(methodDefToken, "A fluent method should only have one parameter.");
        }

        
        final DetailAST typeToken = methodDefToken.findFirstToken(TokenTypes.TYPE);
        if (TokenUtil.findFirstTokenByPredicate(typeToken, ident -> ident.getType() == TokenTypes.IDENT
            && !ident.getText().equals(classNameStack.peekLast())).isPresent()) {
            log(methodDefToken, "Return type of fluent method should be the class itself");
        }

        final String methodName = methodDefToken.findFirstToken(TokenTypes.IDENT).getText();
        
        avoidStartWords.forEach(avoidStartWord -> {
            if (methodName.length() >= avoidStartWord.length() && methodName.startsWith(avoidStartWord)) {
                log(methodDefToken, String.format("''%s'' fluent method name should not start with keyword ''%s''.",
                    methodName, avoidStartWord));
            }
        });
    }
Code review comment: How about return both `slot id` and `expr`. I'm afraid that `slot id` may be used for some tracing debug.  And please fix all related unit tests after you change here.

Code under review:
public String toSqlImpl() {
        StringBuilder sb = new StringBuilder();

        
        
        
        
        
        
        
        if (tblName != null) {
            return tblName.toSql() + "." + label + sb.toString();
        } else if (label != null) {
            return label + sb.toString();
        } else if (desc.getSourceExprs() != null) {
            for (Expr expr : desc.getSourceExprs()) {
                sb.append(expr.toSql());
                sb.append(" ");
            }
            return sb.toString();
        } else {
            return "<slot " + Integer.toString(desc.getId().asInt()) + ">" + sb.toString();
        }
    }

Line commented on: return sb.toString();

Revised code:
public String toSqlImpl() {
        StringBuilder sb = new StringBuilder();

        
        
        
        
        
        
        
        if (tblName != null) {
            return tblName.toSql() + "." + label + sb.toString();
        } else if (label != null) {
            return label + sb.toString();
        } else if (desc.getSourceExprs() != null) {
            if (desc.getId().asInt() != 1) {
                sb.append("<slot " + Integer.toString(desc.getId().asInt()) + ">");
            }
            for (Expr expr : desc.getSourceExprs()) {
                sb.append(" ");
                sb.append(expr.toSql());
            }
            return sb.toString();
        } else {
            return "<slot " + Integer.toString(desc.getId().asInt()) + ">" + sb.toString();
        }
    }
Code review comment: Don't we need to change the doc comment with respect to the change? As this method only sets compression header now.

Code under review:
public static void setCompressionHeaders(Context context, HTTPCarbonMessage outboundMessage) {
        AnnAttachmentInfo configAnn = context.getServiceInfo().getAnnotationAttachmentInfo(
                HttpConstants.PROTOCOL_PACKAGE_HTTP, HttpConstants.ANN_NAME_CONFIG);
        if (configAnn != null) {
            AnnAttributeValue compressionEnabled = configAnn.getAttributeValue(
                    HttpConstants.ANN_CONFIG_ATTR_COMPRESSION_ENABLED);
            if (compressionEnabled != null && !compressionEnabled.getBooleanValue()) {
                outboundMessage.setHeader(HttpHeaderNames.CONTENT_ENCODING.toString(),
                                          Constants.HTTP_TRANSFER_ENCODING_IDENTITY);
            }
        }
    }

Line commented on: AnnAttributeValue compressionEnabled = configAnn.getAttributeValue(

Revised code:
public static void setCompressionHeaders(Context context, HTTPCarbonMessage outboundMessage) {
        AnnAttachmentInfo configAnn = context.getServiceInfo().getAnnotationAttachmentInfo(
                HttpConstants.PROTOCOL_PACKAGE_HTTP, HttpConstants.ANN_NAME_CONFIG);
        if (configAnn != null) {
            AnnAttributeValue compressionEnabled = configAnn.getAttributeValue(
                    HttpConstants.ANN_CONFIG_ATTR_COMPRESSION_ENABLED);
            if (compressionEnabled != null && !compressionEnabled.getBooleanValue()) {
                outboundMessage.setHeader(HttpHeaderNames.CONTENT_ENCODING.toString(),
                                          Constants.HTTP_TRANSFER_ENCODING_IDENTITY);
            }
        }
    }
Code review comment: In which situation is it NESTED? We made the change originally because it wasn't prefixed, so trying to understand what changed.

Code under review:
private void prepareStateBackend(K key) {
          
          
          
          ByteBuffer encodedKey =
              FlinkKeyUtils.removeNestedContext(key, (Coder<ByteString>) keyCoder);
          keyedStateBackend.setCurrentKey(encodedKey);
        }


Revised code:
private void prepareStateBackend(K key) {
          
          ByteBuffer encodedKey = FlinkKeyUtils.fromEncodedKey(key);
          keyedStateBackend.setCurrentKey(encodedKey);
        }
Code review comment: > I think we should keep this. because when TableEnvironment supports new operation but forgets to update SqlCommandParser  I think we should never do this, either to refactor the code to keep SQL_CLI synced with table environment or keep the logic clean.

Code under review:
private Optional<SqlCommandCall> parseCommand(String line) {
		final Optional<SqlCommandCall> parsedLine;
		try {
			parsedLine = SqlCommandParser.parse(executor.getSqlParser(sessionId), line);
		} catch (SqlExecutionException e) {
			printExecutionException(e);
			return Optional.empty();
		}
		if (!parsedLine.isPresent()) {
			printError(CliStrings.MESSAGE_UNKNOWN_SQL);
		}
		return parsedLine;
	}

Line commented on: printError(CliStrings.MESSAGE_UNKNOWN_SQL);

Revised code:
private Optional<SqlCommandCall> parseCommand(String line) {
		final SqlCommandCall parsedLine;
		try {
			parsedLine = SqlCommandParser.parse(executor.getSqlParser(sessionId), line);
		} catch (SqlExecutionException e) {
			printExecutionException(e);
			return Optional.empty();
		}
		return Optional.of(parsedLine);
	}
Code review comment: Modified it as `'Clazz' is abstract, and cannot be instantiated` per discussion.

Code under review:
private List<Executable> getExecutables(Class<?> clazz, String methodName, JMethodKind kind) {

        if (kind == JMethodKind.CONSTRUCTOR) {
            if (Modifier.isAbstract(clazz.getModifiers())) {
                throw new JInteropException(DiagnosticErrorCode.INSTANTIATION_ERROR,
                        "'" + clazz.getName() + "' is abstract; cannot be instantiated");
            }
            return Arrays.asList(getConstructors(clazz));
        } else {
            List<Executable> list = new ArrayList<>();
            for (Method method : getMethods(clazz)) {
                if (method.getName().equals(methodName)) {
                    list.add(method);
                }
            }
            return list;
        }
    }

Line commented on: throw new JInteropException(DiagnosticErrorCode.INSTANTIATION_ERROR,

Revised code:
private List<Executable> getExecutables(Class<?> clazz, String methodName, JMethodKind kind) {

        if (kind == JMethodKind.CONSTRUCTOR) {
            if (Modifier.isAbstract(clazz.getModifiers())) {
                throw new JInteropException(DiagnosticErrorCode.INSTANTIATION_ERROR,
                        "'" + clazz.getName() + "' is abstract, and cannot be instantiated");
            }
            return Arrays.asList(getConstructors(clazz));
        } else {
            List<Executable> list = new ArrayList<>();
            for (Method method : getMethods(clazz)) {
                if (method.getName().equals(methodName)) {
                    list.add(method);
                }
            }
            return list;
        }
    }
Code review comment: does it mean we no longer set error if all retries has failed?

Code under review:
private Mono<Void> sendInternal(Flux<EventData> events, SendOptions options) {
        final String partitionKey = options.getPartitionKey();
        final String partitionId = options.getPartitionId();

        if (!CoreUtils.isNullOrEmpty(partitionKey)
            && !CoreUtils.isNullOrEmpty(partitionId)) {
            return monoError(logger, new IllegalArgumentException(String.format(Locale.US,
                "SendOptions.getPartitionKey() and SendOptions.getPartitionId() are both set. Only one or the"
                    + " other can be used. partitionKey: '%s'. partitionId: '%s'",
                partitionKey, partitionId)));
        }

        return getSendLink(options.getPartitionId())
            .flatMap(link -> link.getLinkSize()
                .flatMap(size -> {
                    final int batchSize = size > 0 ? size : MAX_MESSAGE_LENGTH_BYTES;
                    final CreateBatchOptions batchOptions = new CreateBatchOptions()
                        .setPartitionKey(options.getPartitionKey())
                        .setPartitionId(options.getPartitionId())
                        .setMaximumSizeInBytes(batchSize);
                    return events.collect(new EventDataCollector(batchOptions, 1, link::getErrorContext,
                        tracerProvider));
                })
                .flatMap(list -> sendInternal(Flux.fromIterable(list))));
    }

Line commented on: final String partitionKey = options.getPartitionKey();

Revised code:
private Mono<Void> sendInternal(Flux<EventData> events, SendOptions options) {
        final String partitionKey = options.getPartitionKey();
        final String partitionId = options.getPartitionId();

        if (!CoreUtils.isNullOrEmpty(partitionKey)
            && !CoreUtils.isNullOrEmpty(partitionId)) {
            return monoError(logger, new IllegalArgumentException(String.format(Locale.US,
                "SendOptions.getPartitionKey() and SendOptions.getPartitionId() are both set. Only one or the"
                    + " other can be used. partitionKey: '%s'. partitionId: '%s'",
                partitionKey, partitionId)));
        }

        return getSendLink(options.getPartitionId())
            .flatMap(link -> link.getLinkSize()
                .flatMap(size -> {
                    final int batchSize = size > 0 ? size : MAX_MESSAGE_LENGTH_BYTES;
                    final CreateBatchOptions batchOptions = new CreateBatchOptions()
                        .setPartitionKey(options.getPartitionKey())
                        .setPartitionId(options.getPartitionId())
                        .setMaximumSizeInBytes(batchSize);
                    return events.collect(new EventDataCollector(batchOptions, 1, link::getErrorContext,
                        tracerProvider));
                })
                .flatMap(list -> sendInternal(Flux.fromIterable(list))));
    }
Code review comment: Is there absolutely no way for cassandra.start()/cluster.connect() to throw an exception?

Code under review:
public static void closeCassandra() {
        session.close();

        Cluster cluster = session.getCluster();
        if (cluster != null) {
            cluster.close();
        }

        if (cassandra != null) {
            cassandra.stop();
        }
    }

Line commented on: session.close();

Revised code:
public static void closeCassandra() {
        if (session != null) {
            session.close();
        }
        if (cluster != null) {
            cluster.close();
        }
        CASSANDRA_CONTAINER.stop();
    }
Code review comment: The setup and execution is identical to that used in testIntHash(), except the data type and field name. Consider refactor into a helper function. As it stands I think it is harder to read as I was looking for differences between the two tests.

Code under review:
public void testLongHash() throws ParseException {
        var expression = Expression.fromString("input myText | hash | attribute 'myLong'");

        SimpleTestAdapter adapter = new SimpleTestAdapter();
        adapter.createField(new Field("myText", DataType.STRING));
        var intField = new Field("myLong", DataType.LONG);
        adapter.createField(intField);
        adapter.setValue("myText", new StringFieldValue("input text"));
        expression.setStatementOutput(new DocumentType("myDocument"), intField);

        
        VerificationContext verificationContext = new VerificationContext(adapter);
        assertEquals(DataType.LONG, expression.verify(verificationContext));

        ExecutionContext context = new ExecutionContext(adapter);
        context.setValue(new StringFieldValue("input text"));
        expression.execute(context);
        assertNotNull(context);
        assertTrue(adapter.values.containsKey("myLong"));
        assertEquals(7678158186624760752L, adapter.values.get("myLong").getWrappedValue());
    }

Line commented on: expression.execute(context);

Revised code:
public void testLongHash() throws ParseException {
        var expression = Expression.fromString("input myText | hash | attribute 'myLong'");

        SimpleTestAdapter adapter = new SimpleTestAdapter();
        adapter.createField(new Field("myText", DataType.STRING));
        var intField = new Field("myLong", DataType.LONG);
        adapter.createField(intField);
        adapter.setValue("myText", new StringFieldValue("input text"));
        expression.setStatementOutput(new DocumentType("myDocument"), intField);

        
        VerificationContext verificationContext = new VerificationContext(adapter);
        assertEquals(DataType.LONG, expression.verify(verificationContext));

        ExecutionContext context = new ExecutionContext(adapter);
        context.setValue(new StringFieldValue("input text"));
        expression.execute(context);
        assertTrue(adapter.values.containsKey("myLong"));
        assertEquals(7678158186624760752L, adapter.values.get("myLong").getWrappedValue());
    }
Code review comment: Yes, good point, fixed and made sure to actually test both forms

Code under review:
private static ApplicationId fromIdString(String idString, String splitCharacter) {
        String[] parts = idString.split(splitCharacter);
        String errorMessage = "Application ids must be on the form tenant" +
                splitCharacter + "application" + splitCharacter + "instance, but was " + idString;
        if (parts.length < 3)
            throw new IllegalArgumentException(errorMessage);
        
        
        if (parts.length > 3)
            log.log(SEVERE, errorMessage);
        return from(parts[0], parts[1], parts[2]);
    }

Line commented on: splitCharacter + "application" + splitCharacter + "instance, but was " + idString;

Revised code:
private static ApplicationId fromIdString(String idString, String splitCharacter) {
        String[] parts = idString.split(Pattern.quote(splitCharacter));
        String errorMessage = "Application ids must be on the form tenant" +
                splitCharacter + "application" + splitCharacter + "instance, but was " + idString;
        if (parts.length < 3)
            throw new IllegalArgumentException(errorMessage);
        
        
        if (parts.length > 3)
            log.log(SEVERE, errorMessage);
        return from(parts[0], parts[1], parts[2]);
    }
Code review comment: The same to [test_split_with_element_allowed_splits](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/worker/bundle_processor_test.py#L81)

Code under review:
public static Iterable<Object[]> data() {
      return ImmutableList.<Object[]>builder()
          
          .add(
              new Object[] {
                channelSplitResult(4L),
                16L,
                ImmutableList.of("A"),
                0.25,
                ImmutableList.of(2L, 3L, 4L, 5L)
              })
          
          .add(
              new Object[] {
                channelSplitResult(5L),
                16L,
                ImmutableList.of("A"),
                0.25,
                ImmutableList.of(2L, 3L, 5L)
              })
          .add(
              new Object[] {
                channelSplitResult(3L),
                16L,
                ImmutableList.of("A"),
                0.25,
                ImmutableList.of(2L, 3L, 6L)
              })
          
          
          .add(
              new Object[] {
                channelSplitResult(5L),
                16L,
                ImmutableList.of("A"),
                0.25,
                ImmutableList.of(5L, 6L, 7L)
              })
          .add(
              new Object[] {
                channelSplitResult(3L),
                16L,
                ImmutableList.of("A"),
                0.25,
                ImmutableList.of(1L, 2L, 3L)
              })
          
          .add(
              new Object[] {
                ProcessBundleSplitResponse.getDefaultInstance(),
                16L,
                ImmutableList.of("A", "B", "C", "D", "F", "G"),
                0.25,
                ImmutableList.of(1L, 2L, 3L)
              })
          .build();
    }

Line commented on: new Object[] {

Revised code:
public static Iterable<Object[]> data() {
      return ImmutableList.<Object[]>builder()
          
          .add(new Object[] {channelSplitResult(1L), 0L, 0, 0, 16L})
          
          .add(new Object[] {channelSplitResult(4L), 0L, 0, 0.24, 16L})
          .add(new Object[] {channelSplitResult(4L), 0L, 0, 0.25, 16L})
          .add(new Object[] {channelSplitResult(4L), 0L, 0, 0.26, 16L})
          
          .add(new Object[] {channelSplitResult(8L), 0L, 0, 0.5, 16L})
          .add(new Object[] {channelSplitResult(9L), 2, 0, 0.5, 16L})
          .add(new Object[] {channelSplitResult(11L), 6L, 0, 0.5, 16L})
          
          .add(new Object[] {channelSplitResult(1L), 0L, 0.5, 0.25, 4L})
          .add(new Object[] {channelSplitResult(2L), 0L, 0.9, 0.25, 4L})
          .add(new Object[] {channelSplitResult(2L), 1L, 0, 0.25, 4L})
          .add(new Object[] {channelSplitResult(2L), 1L, 0.1, 0.25, 4L})
          .build();
    }
Code review comment: After confirming with andrey, we could keep the previous behavior to still ignore `IOException` during close, because we just try best to delete files for releasing resources. If one file was already deleted by other factors before, it is no need to cause unnecessary failover.   If we want to refactor this behavior future, we could do it separately, not to block this PR. I remove `try catch` from `IOManager#close` and add some javadoc for this method.

Code under review:
public void afterTest() throws Exception {
		this.ioManager.close();
		if (!this.ioManager.isProperlyShutDown()) {
			Assert.fail("I/O Manager was not properly shut down.");
		}
		
		if (this.memoryManager != null && testSuccess) {
			Assert.assertTrue("Memory leak: not all segments have been returned to the memory manager.", 
				this.memoryManager.verifyEmpty());
			this.memoryManager.shutdown();
			this.memoryManager = null;
		}
	}

Line commented on: if (!this.ioManager.isProperlyShutDown()) {

Revised code:
public void afterTest() throws Exception {
		this.ioManager.close();
		
		if (this.memoryManager != null && testSuccess) {
			Assert.assertTrue("Memory leak: not all segments have been returned to the memory manager.", 
				this.memoryManager.verifyEmpty());
			this.memoryManager.shutdown();
			this.memoryManager = null;
		}
	}
Code review comment: Do we want to install both if we already know one of them is installed?

Code under review:
public void testInstall() {
        TaskContext taskContext = mock(TaskContext.class);
        TestCommandSupplier commandSupplier = new TestCommandSupplier(taskContext);

        commandSupplier.expectCommand("yum list installed package-1", 0, "");
        commandSupplier.expectCommand("yum list installed package-2", 1, "");
        commandSupplier.expectCommand(
                "yum install --assumeyes --enablerepo=repo-name package-1 package-2",
                0,
                "");

        Yum yum = new Yum(taskContext, commandSupplier);
        yum.install("package-1", "package-2")
                .enableRepo("repo-name")
                .converge();

        commandSupplier.verifyInvocations();
    }

Line commented on: "yum install --assumeyes --enablerepo=repo-name package-1 package-2",

Revised code:
public void testInstall() {
        TaskContext taskContext = mock(TaskContext.class);
        TestCommandSupplier commandSupplier = new TestCommandSupplier(taskContext);

        commandSupplier.expectCommand("yum list installed package-1", 0, "");
        commandSupplier.expectCommand("yum list installed package-2", 1, "");
        commandSupplier.expectCommand(
                "yum install --assumeyes --enablerepo=repo-name package-1 package-2",
                0,
                "");

        Yum yum = new Yum(taskContext, commandSupplier);
        yum.install("package-1", "package-2")
                .enableRepo("repo-name")
                .converge();

        commandSupplier.verifyInvocations();
    }
Code review comment: For completeness, why don't you pass here `stateHandle.getStateSize()`?

Code under review:
public IncrementalLocalKeyedStateHandle createLocalStateHandleForDownloadedState() {
        return new IncrementalLocalKeyedStateHandle(
                stateHandle.getBackendIdentifier(),
                stateHandle.getCheckpointId(),
                
                
                new DirectoryStateHandle(downloadDestination, 0L),
                stateHandle.getKeyGroupRange(),
                stateHandle.getMetaDataStateHandle(),
                stateHandle.getSharedState());
    }

Line commented on: new DirectoryStateHandle(downloadDestination, 0L),

Revised code:
public IncrementalLocalKeyedStateHandle createLocalStateHandleForDownloadedState() {
        return new IncrementalLocalKeyedStateHandle(
                stateHandle.getBackendIdentifier(),
                stateHandle.getCheckpointId(),
                new DirectoryStateHandle(downloadDestination, stateHandle.getStateSize()),
                stateHandle.getKeyGroupRange(),
                stateHandle.getMetaDataStateHandle(),
                stateHandle.getSharedState());
    }
Code review comment: When users need to disable checkpoint during backlog, they can configure checkpoint-interval-during-backlog into -1L.  In checkpoint-interval's case, user can disable checkpoint by not setting this configuration. However, according to FLIP-309, not setting checkpoint-interval-during-backlog stands for using the same value as checkpoint-interval, so we need to find an alternative special value for this function.  According to [1] and [2], checkpoint interval uses -1L to represent intervals of infinite length, or disabling checkpoint interval[3]. Thus I adopted -1L as the special value in accordance with the current semantics for checkpoint interval.  [1] https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/environment/CheckpointConfig.java#L201 [2] https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamingJobGraphGenerator.java#L1918 [3] https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/CheckpointCoordinator.java#L1946

Code under review:
public void setCheckpointIntervalDuringBacklog(long checkpointInterval) {
        if (checkpointInterval != -1L && checkpointInterval < MINIMAL_CHECKPOINT_TIME) {
            throw new IllegalArgumentException(
                    String.format(
                            "Checkpoint interval must be larger than or equal to %s ms",
                            MINIMAL_CHECKPOINT_TIME));
        }
        configuration.set(
                ExecutionCheckpointingOptions.CHECKPOINTING_INTERVAL_DURING_BACKLOG,
                Duration.ofMillis(checkpointInterval));
    }

Line commented on: if (checkpointInterval != -1L && checkpointInterval < MINIMAL_CHECKPOINT_TIME) {

Revised code:
public void setCheckpointIntervalDuringBacklog(long checkpointInterval) {
        if (checkpointInterval != 0 && checkpointInterval < MINIMAL_CHECKPOINT_TIME) {
            throw new IllegalArgumentException(
                    String.format(
                            "Checkpoint interval must be zero or larger than or equal to %s ms",
                            MINIMAL_CHECKPOINT_TIME));
        }
        configuration.set(
                ExecutionCheckpointingOptions.CHECKPOINTING_INTERVAL_DURING_BACKLOG,
                Duration.ofMillis(checkpointInterval));
    }
Code review comment: Our Uid format is internal and if we change the format then this will break (similarly if Quarkus replaces the underlying transaction engine). Testing uid format does not provide any positive benefit?  However the test that verifies that the `@Transactional` annotation correctly starts a transaction is valid and is sufficient.

Code under review:
public void test() {
        RestAssured.when().get("/uid").then().assertThat().body(MatchesPattern.matchesPattern("[:0-9a-f]+"));

        RestAssured.when().get("/status").then().body(is("0"));
    }

Line commented on: RestAssured.when().get("/uid").then().assertThat().body(MatchesPattern.matchesPattern("[:0-9a-f]+"));

Revised code:
public void test() {
        RestAssured.when().get("/status").then().body(is("0"));
    }
Code review comment: `finishBundle` contains a different blocking version of flush that only blocks on the records emitted by this bundle. It would actually be harmful to do that: Using `producer.flushSync()` in `finishBundle` would likely block that dofn instance "forever" because the producer is shared and other instances keep writing to the producer. `flushSync` basically loops infinitely until there's no more pending records in the producer.

Code under review:
private void teardownSharedProducer() {
        synchronized (producerRefCount) {
          if (producerRefCount.decrementAndGet() == 0) {
            
            if (producer == null) {
              return;
            }
            if (producer.getOutstandingRecordsCount() > 0) {
              producer.flushSync();
            }
            producer.destroy();
            producer = null;
          }
        }
      }

Line commented on: producer.flushSync();

Revised code:
private void teardownSharedProducer() {
        IKinesisProducer obsolete = null;
        synchronized (KinesisWriterFn.class) {
          if (--producerRefCount == 0) {
            obsolete = producer;
            producer = null;
          }
        }
        if (obsolete != null) {
          obsolete.flushSync(); 
          obsolete.destroy();
        }
      }
Code review comment: Please rename variable name `utils` because of there is no any relationship with variable name and class type.

Code under review:
public UpdateStatementContext(final UpdateStatement sqlStatement) {
        super(sqlStatement);
        TableExtractor utils = new TableExtractor();
        utils.extractTablesFromUpdate(sqlStatement);
        tablesContext = new TablesContext(utils.getRewriteTables());
    }

Line commented on: TableExtractor utils = new TableExtractor();

Revised code:
public UpdateStatementContext(final UpdateStatement sqlStatement) {
        super(sqlStatement);
        TableExtractor tableExtractor = new TableExtractor();
        tableExtractor.extractTablesFromUpdate(sqlStatement);
        tablesContext = new TablesContext(tableExtractor.getRewriteTables());
    }
Code review comment: We don't really have something, but it would likely make sense to add it

Code under review:
Object aroundInvoke(InvocationContext ctx) throws Exception {
        if (ctx.getMethod().getReturnType().equals(Uni.class)) {
            return invokeUni(ctx);
        }

        return invoke(ctx);
    }

Line commented on: if (ctx.getMethod().getReturnType().equals(Uni.class)) {

Revised code:
Object aroundInvoke(InvocationContext ctx) throws Exception {
        switch (ReactiveType.valueOf(ctx.getMethod())) {
            case UNI:
                return invokeUni(ctx);
            case MULTI:
                return invokeMulti(ctx);
            case STAGE:
                return invokeStage(ctx);
            default:
                return invoke(ctx);
        }
    }
Code review comment: Only these test jobs will be triggered automatically. Undeclared jobs may still be triggered by users, though, but that likely won't happen if they aren't shown here :)

Code under review:
public List<StepStatus> allSteps() {
        List<JobId> firstTestJobs = List.of(firstDeclaredOrElseImplicitTest(systemTest),
                                            firstDeclaredOrElseImplicitTest(stagingTest));
        return allSteps.stream()
                       .filter(step -> step.isDeclared() || firstTestJobs.contains(step.job().orElseThrow()))
                       .collect(toUnmodifiableList());
    }

Line commented on: List<JobId> firstTestJobs = List.of(firstDeclaredOrElseImplicitTest(systemTest),

Revised code:
public List<StepStatus> allSteps() {
        List<JobId> firstTestJobs = List.of(firstDeclaredOrElseImplicitTest(systemTest),
                                            firstDeclaredOrElseImplicitTest(stagingTest));
        return allSteps.stream()
                       .filter(step -> step.isDeclared() || firstTestJobs.contains(step.job().orElseThrow()))
                       .collect(toUnmodifiableList());
    }
Code review comment: IMHO, if the exist check is expensive, we shouldn't do it and rather rely on the implementation but add a comment in the `FlinkKubeClient.deleteConfigMapsBy*` implementations. Doing the exist call would just be more coherent with the actual contract of the interface provided by the fabric8 client.

Code under review:
public CompletableFuture<Void> deleteConfigMapsByLabels(Map<String, String> labels) {
        return CompletableFuture.runAsync(
                () -> {
                    if (!this.internalClient.configMaps().withLabels(labels).delete()) {
                        final List<ConfigMap> notDeletedConfigMaps =
                                internalClient.configMaps().withLabels(labels).list().getItems();
                        if (!notDeletedConfigMaps.isEmpty()) {
                            final String notDeletedConfigMapsNameStr =
                                    notDeletedConfigMaps.stream()
                                            .map(cm -> cm.getMetadata().getName())
                                            .collect(Collectors.joining(", "));
                            throw new CompletionException(
                                    new KubernetesException(
                                            "The following ConfigMaps labeled with "
                                                    + labelsToString(labels)
                                                    + " couldn't be deleted for unknown reasons: "
                                                    + notDeletedConfigMapsNameStr));
                        }
                    }
                },
                kubeClientExecutorService);
    }

Line commented on: if (!this.internalClient.configMaps().withLabels(labels).delete()) {

Revised code:
public CompletableFuture<Void> deleteConfigMapsByLabels(Map<String, String> labels) {
        
        
        return CompletableFuture.runAsync(
                () -> this.internalClient.configMaps().withLabels(labels).delete(),
                kubeClientExecutorService);
    }
Code review comment: our code style unfortunately does not cover the placement of such braces; hence we reject any changes such as this to existing code.

Code under review:
private static List<ReporterSetup> setupReporters(Map<String, MetricReporterFactory> reporterFactories, List<Tuple2<String, Configuration>> reporterConfigurations) {
		List<ReporterSetup> reporterSetups = new ArrayList<>(reporterConfigurations.size());
		for (Tuple2<String, Configuration> reporterConfiguration: reporterConfigurations) {
			String reporterName = reporterConfiguration.f0;
			Configuration reporterConfig = reporterConfiguration.f1;

			try {
				Optional<MetricReporter> metricReporterOptional = loadReporter(reporterName, reporterConfig, reporterFactories);
				metricReporterOptional.ifPresent(reporter -> {
					MetricConfig metricConfig = new MetricConfig();
					reporterConfig.addAllToProperties(metricConfig);
					reporterSetups.add(createReporterSetup(reporterName, metricConfig, reporter));
				});
			} catch (Throwable t) {
				LOG.error("Could not instantiate metrics reporter {}. Metrics might not be exposed/reported.", reporterName, t);
			}
		}
		return reporterSetups;
	}

Line commented on: reporterConfig.addAllToProperties(metricConfig);

Revised code:
private static List<ReporterSetup> setupReporters(Map<String, MetricReporterFactory> reporterFactories, List<Tuple2<String, Configuration>> reporterConfigurations) {
		List<ReporterSetup> reporterSetups = new ArrayList<>(reporterConfigurations.size());
		for (Tuple2<String, Configuration> reporterConfiguration: reporterConfigurations) {
			String reporterName = reporterConfiguration.f0;
			Configuration reporterConfig = reporterConfiguration.f1;

			try {
				Optional<MetricReporter> metricReporterOptional = loadReporter(reporterName, reporterConfig, reporterFactories);
				metricReporterOptional.ifPresent(reporter -> {
					MetricConfig metricConfig = new MetricConfig();
					reporterConfig.addAllToProperties(metricConfig);
					reporterSetups.add(createReporterSetup(reporterName, metricConfig, reporter));
				});
			}
			catch (Throwable t) {
				LOG.error("Could not instantiate metrics reporter {}. Metrics might not be exposed/reported.", reporterName, t);
			}
		}
		return reporterSetups;
	}