muttakin2301/ccr2000 huggingface dataset:
Code review comment: Agree. What I want to do is create a "cloud config" which defines the properties of a cloud and then only test for those properties.

Code under review:
public List<Node> removeRecursively(Node node, boolean force) {
        try (Mutex lock = lockUnallocated()) {
            requireRemovable(node, false, force);

            if (node.type().isHost()) {
                List<Node> children = list().childrenOf(node).asList();
                children.forEach(child -> requireRemovable(child, true, force));
                db.removeNodes(children);
                List<Node> removed = new ArrayList<>(children);
                if (zone.cloud().value().equals("aws"))
                    db.removeNodes(List.of(node));
                else
                    move(node, State.deprovisioned, Agent.system, Optional.empty());
                removed.add(node);
                return removed;
            }
            else {
                db.removeNodes(List.of(node));
                return List.of(node);
            }
        }
    }

Line commented on: if (zone.cloud().value().equals("aws"))

Revised code:
public List<Node> removeRecursively(Node node, boolean force) {
        try (Mutex lock = lockUnallocated()) {
            requireRemovable(node, false, force);

            if (node.type().isDockerHost()) {
                List<Node> children = list().childrenOf(node).asList();
                children.forEach(child -> requireRemovable(child, true, force));
                db.removeNodes(children);
                List<Node> removed = new ArrayList<>(children);
                if (zone.cloud().value().equals("aws"))
                    db.removeNodes(List.of(node));
                else
                    move(node, State.deprovisioned, Agent.system, Optional.empty());
                removed.add(node);
                return removed;
            }
            else {
                db.removeNodes(List.of(node));
                return List.of(node);
            }
        }
    }
Code review comment: only when is there a bug, never encounter the unconsistent case here

Code under review:
public void replayBatchRemoveTransaction(BatchRemoveTransactionsOperationV2 operation) {
        writeLock();
        try {
            int numOfClearedTransaction = 0;
            if (operation.getLatestTxnIdForShort() != -1) {
                while (!finalStatusTransactionStateDequeShort.isEmpty()) {
                    TransactionState transactionState = finalStatusTransactionStateDequeShort.pop();
                    clearTransactionState(transactionState.getTransactionId());
                    numOfClearedTransaction++;
                    if (operation.getLatestTxnIdForShort() == transactionState.getTransactionId()) {
                        break;
                    }
                }
            }

            if (operation.getLatestTxnIdForLong() != -1) {
                while (!finalStatusTransactionStateDequeLong.isEmpty()) {
                    TransactionState transactionState = finalStatusTransactionStateDequeLong.pop();
                    clearTransactionState(transactionState.getTransactionId());
                    numOfClearedTransaction++;
                    if (operation.getLatestTxnIdForLong() == transactionState.getTransactionId()) {
                        break;
                    }
                }
            }
            Preconditions.checkState(numOfClearedTransaction == operation.getNumOfClearedTransaction());
        } finally {
            writeUnlock();
        }
    }

Line commented on: TransactionState transactionState = finalStatusTransactionStateDequeShort.pop();

Revised code:
public void replayBatchRemoveTransaction(BatchRemoveTransactionsOperationV2 operation) {
        writeLock();
        try {
            if (operation.getLatestTxnIdForShort() != -1) {
                while (!finalStatusTransactionStateDequeShort.isEmpty()) {
                    TransactionState transactionState = finalStatusTransactionStateDequeShort.pop();
                    clearTransactionState(transactionState.getTransactionId());
                    if (operation.getLatestTxnIdForShort() == transactionState.getTransactionId()) {
                        break;
                    }
                }
            }

            if (operation.getLatestTxnIdForLong() != -1) {
                while (!finalStatusTransactionStateDequeLong.isEmpty()) {
                    TransactionState transactionState = finalStatusTransactionStateDequeLong.pop();
                    clearTransactionState(transactionState.getTransactionId());
                    if (operation.getLatestTxnIdForLong() == transactionState.getTransactionId()) {
                        break;
                    }
                }
            }
        } finally {
            writeUnlock();
        }
    }
Code review comment: I don't like the spec here. _How much_ can you write without blocking? Generally, this should be a `while` loop, but since you only do a single write here, and then close, I guess this is fine.

Code under review:
public void run() {
            StatusResponse statusResponse = target.requestStatusHtml();
            servletResponse.setStatus(statusResponse.statusCode);
            if (statusResponse.contentType != null) {
                servletResponse.setHeader("Content-Type", statusResponse.contentType);
            }
            output.setWriteListener(new WriteListener() {
                @Override
                public void onWritePossible() throws IOException {
                    if (output.isReady()) {
                        if (statusResponse.content != null) {
                            output.write(statusResponse.content);
                        }
                        asyncContext.complete();
                    }
                }

                @Override
                public void onError(Throwable t) {
                    log.log(Level.FINE, t, () -> "Failed to write status response: " + t.getMessage());
                    asyncContext.complete();
                }
            });
        }

Line commented on: asyncContext.complete();

Revised code:
public void run() {
            StatusResponse statusResponse = target.requestStatusHtml();
            servletResponse.setStatus(statusResponse.statusCode);
            if (statusResponse.contentType != null) {
                servletResponse.setHeader("Content-Type", statusResponse.contentType);
            }
            output.setWriteListener(new WriteListener() {
                @Override
                public void onWritePossible() throws IOException {
                    if (output.isReady()) {
                        if (statusResponse.content != null) {
                            output.write(statusResponse.content);
                        }
                        asyncContext.complete();
                    }
                }

                @Override
                public void onError(Throwable t) {
                    log.log(Level.FINE, t, () -> "Failed to write status response: " + t.getMessage());
                    asyncContext.complete();
                }
            });
        }
Code review comment: Can we add a comment on why these numbers are chosen

Code under review:
public void testBigQueryStorageQueryWithErrorHandling1M() throws Exception {
    setUpTestEnvironment("1M");
    Pipeline p = Pipeline.create(options);
    ErrorHandler<BadRecord, PCollection<Long>> errorHandler =
        p.registerBadRecordErrorHandler(new ErrorSinkTransform());
    PCollection<Long> count =
        p.apply(
                "Read",
                BigQueryIO.read(FailingTableRowParser.INSTANCE)
                    .fromQuery("SELECT * FROM `" + options.getInputTable() + "`")
                    .usingStandardSql()
                    .withMethod(Method.DIRECT_READ)
                    .withErrorHandler(errorHandler))
            .apply("Count", Count.globally());

    errorHandler.close();

    PAssert.thatSingleton(count).isEqualTo(10381L);
    PAssert.thatSingleton(errorHandler.getOutput()).isEqualTo(10592L - 10381L);
    p.run().waitUntilFinish();
  }

Line commented on: PAssert.thatSingleton(errorHandler.getOutput()).isEqualTo(10592L - 10381L);

Revised code:
public void testBigQueryStorageQueryWithErrorHandling1M() throws Exception {
    setUpTestEnvironment("1M");
    Pipeline p = Pipeline.create(options);
    ErrorHandler<BadRecord, PCollection<Long>> errorHandler =
        p.registerBadRecordErrorHandler(new ErrorSinkTransform());
    PCollection<Long> count =
        p.apply(
                "Read",
                BigQueryIO.read(FailingTableRowParser.INSTANCE)
                    .fromQuery("SELECT * FROM `" + options.getInputTable() + "`")
                    .usingStandardSql()
                    .withMethod(Method.DIRECT_READ)
                    .withErrorHandler(errorHandler))
            .apply("Count", Count.globally());

    errorHandler.close();

    
    PAssert.thatSingleton(count).isEqualTo(10381L);
    
    PAssert.thatSingleton(errorHandler.getOutput()).isEqualTo(10592L - 10381L);
    p.run().waitUntilFinish();
  }
Code review comment: I think that we should introduce a new annotation; e.g. something like `@EndpointDisabled`.

Code under review:
public Supplier<Boolean> apply(ClassInfo classInfo) {
                            
                            if (classInfo.name().toString().contains("DummyResource")) { 
                                return recorder.isPropertyTrue("foo.bar");
                            }

                            return null;
                        }


Revised code:
public Supplier<Boolean> apply(ClassInfo classInfo) {
                            AnnotationInstance instance = classInfo.declaredAnnotation(ENDPOINT_DISABLED);
                            if (instance == null) {
                                return null;
                            }
                            String propertyName = instance.value("name").asString();
                            String propertyValue = instance.value("stringValue").asString();
                            AnnotationValue disableIfMissingValue = instance.value("disableIfMissing");
                            boolean disableIfMissing = disableIfMissingValue != null && disableIfMissingValue.asBoolean();
                            return recorder.disableIfPropertyMatches(propertyName, propertyValue, disableIfMissing);
                        }
Code review comment: I think the default value for concurrencyLimit should be 0 or -1, means no limit. shouldn't return null for getConcurrencyLimit.

Code under review:
private boolean isGroupSlotAvailable(ResourceGroup group, int numAllocatedSlotsOfGroup) {
        if (group == null) {
            return true;
        }

        Integer numTotalSlots = group.getConcurrencyLimit();
        return numTotalSlots == null || numTotalSlots <= 0 || numAllocatedSlotsOfGroup < numTotalSlots;
    }

Line commented on: Integer numTotalSlots = group.getConcurrencyLimit();

Revised code:
private boolean isGroupSlotAvailable(ResourceGroup group, int numAllocatedSlotsOfGroup) {
        if (group == null) {
            return true;
        }

        Integer numTotalSlots = group.getConcurrencyLimit();
        return numTotalSlots == null || numTotalSlots <= 0 || numAllocatedSlotsOfGroup < numTotalSlots;
    }
Code review comment: Same comment as above regarding this.

Code under review:
public void visit(BLangMatch matchNode) {
        if (PositionUtil.withinBlock(this.linePosition, matchNode.getPosition())) {
            this.scope = this.symbolEnv;
            matchNode.patternClauses.forEach(patternClause -> acceptNode(patternClause, this.symbolEnv));
        }
        this.acceptNode(matchNode.onFailClause, symbolEnv);
    }

Line commented on: this.acceptNode(matchNode.onFailClause, symbolEnv);

Revised code:
public void visit(BLangMatch matchNode) {
        if (PositionUtil.withinBlock(this.linePosition, matchNode.getPosition())) {
            this.scope = this.symbolEnv;
            matchNode.patternClauses.forEach(patternClause -> acceptNode(patternClause, this.symbolEnv));
            this.acceptNode(matchNode.onFailClause, symbolEnv);
        }
    }
Code review comment: Yup. This now works because of the build items being in the modules (which are directly included and which have no dependency on the actual jackson/jsonb modules). So if Jsob and / Jackson are not present in the application, the SPI build items will just be dropped on the floor. If they are present, the corresponding Build Step will happily use them. This sort of pattern is very flexible and has been used in other parts of Quarkus as well (see the Kubernetes or Undertow extensions for example).

Code under review:
void registerJacksonSerDeser(BuildProducer<JacksonModuleBuildItem> customSerDeser) {
        customSerDeser.produce(
                new JacksonModuleBuildItem.Builder("ObjectIdModule")
                        .add(io.quarkus.mongodb.panache.jackson.ObjectIdSerializer.class.getName(),
                                io.quarkus.mongodb.panache.jackson.ObjectIdDeserializer.class.getName(),
                                ObjectId.class.getName())
                        .build());
    }

Line commented on: new JacksonModuleBuildItem.Builder("ObjectIdModule")

Revised code:
void registerJacksonSerDeser(BuildProducer<JacksonModuleBuildItem> customSerDeser) {
        customSerDeser.produce(
                new JacksonModuleBuildItem.Builder("ObjectIdModule")
                        .add(io.quarkus.mongodb.panache.jackson.ObjectIdSerializer.class.getName(),
                                io.quarkus.mongodb.panache.jackson.ObjectIdDeserializer.class.getName(),
                                ObjectId.class.getName())
                        .build());
    }
Code review comment: According to your comment, fixed within new version.

Code under review:
public byte[] getBody() {
        final AmqpMessageBodyType type = amqpAnnotatedMessage.getBody().getBodyType();
        switch (type) {
            case DATA:
                return amqpAnnotatedMessage.getBody().getFirstData();
            case SEQUENCE:
            case VALUE:
                throw logger.logExceptionAsError(new UnsupportedOperationException("Not supported AmqpBodyType: "
                    + type.toString()));
            default:
                throw logger.logExceptionAsError(new IllegalArgumentException("Unknown AmqpBodyType: "
                    + type.toString()));
        }
    }

Line commented on: case DATA:

Revised code:
public byte[] getBody() {
        return annotatedMessage.getBody().getFirstData();
    }
Code review comment: Ok, I see.  Are we scheduling periodic materialisation at fixed "wall clock" rate? Or once every N checkpoints?

Code under review:
public void start() {
        if (!started) {

            started = true;

            LOG.info("Task {} starts periodic materialization", subtaskName);

            scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));
        }
    }

Line commented on: scheduleNextMaterialization(new Random().nextInt((int) periodicMaterializeDelay));

Revised code:
public void start() {
        if (!started) {

            started = true;

            LOG.info("Task {} starts periodic materialization", subtaskName);

            scheduleNextMaterialization(initialDelay);
        }
    }
Code review comment: @prabushi Can't we get these details through `ProjectPaths.isStandaloneBalFile(path)` and `ProjectPaths.packageRoot(path)` util methods? `ProjectLoader.loadProject(filePath.get())` is a costly operation.

Code under review:
public CompletableFuture<BallerinaProject> project(BallerinaProjectParams params) {
        return CompletableFuture.supplyAsync(() -> {
            BallerinaProject ballerinaProject = new BallerinaProject();
            try {
                Optional<Path> filePath = CommonUtil.getPathFromURI(params.getDocumentIdentifier().getUri());
                if (filePath.isEmpty()) {
                    return ballerinaProject;
                }
                Project project = ProjectLoader.loadProject(filePath.get());
                ballerinaProject.setPath(project.sourceRoot().toString());
                ProjectKind projectKind = project.kind();
                if (projectKind != ProjectKind.SINGLE_FILE_PROJECT) {
                    ballerinaProject.setPackageName(project.currentPackage().packageName().value());
                }
                ballerinaProject.setKind(projectKind.name());
            } catch (Throwable e) {
                String msg = "Operation 'ballerinaDocument/project' failed!";
                this.clientLogger.logError(msg, e, params.getDocumentIdentifier(), (Position) null);
            }
            return ballerinaProject;
        });
    }

Line commented on: ProjectKind projectKind = project.kind();

Revised code:
public CompletableFuture<BallerinaProject> project(BallerinaProjectParams params) {
        return CompletableFuture.supplyAsync(() -> {
            BallerinaProject ballerinaProject = new BallerinaProject();
            try {
                Optional<Path> filePath = CommonUtil.getPathFromURI(params.getDocumentIdentifier().getUri());
                if (filePath.isEmpty()) {
                    return ballerinaProject;
                }
                Project project = ProjectLoader.loadProject(filePath.get());
                ballerinaProject.setPath(project.sourceRoot().toString());
                ProjectKind projectKind = project.kind();
                if (projectKind != ProjectKind.SINGLE_FILE_PROJECT) {
                    ballerinaProject.setPackageName(project.currentPackage().packageName().value());
                }
                ballerinaProject.setKind(projectKind.name());
            } catch (Throwable e) {
                String msg = "Operation 'ballerinaDocument/project' failed!";
                this.clientLogger.logError(msg, e, params.getDocumentIdentifier(), (Position) null);
            }
            return ballerinaProject;
        });
    }
Code review comment: please remove all instances of `InternalObjectNode` this is internal type and is going to be removed.

Code under review:
private List<CosmosItemOperation> mapToCosmosItemOperation(final Map<Key, ObjectNode> records) {
        return records.entrySet()
            .stream()
            .map(record -> {
                final String partitionKey = record.getKey().getPartitioningKey();
                final InternalObjectNode objectNode = new InternalObjectNode(record.getValue());
                return BulkOperations.getCreateItemOperation(objectNode, new PartitionKey(partitionKey));
            })
            .collect(Collectors.toList());
    }

Line commented on: final InternalObjectNode objectNode = new InternalObjectNode(record.getValue());

Revised code:
private List<CosmosItemOperation> mapToCosmosItemOperation(final Map<Key, ObjectNode> records) {
        return records.entrySet()
            .stream()
            .map(record -> {
                final String partitionKey = record.getKey().getPartitioningKey();
                final ObjectNode value = record.getValue();
                return BulkOperations.getCreateItemOperation(value, new PartitionKey(partitionKey));
            })
            .collect(Collectors.toList());
    }
Code review comment: I'm more worried that the test will fail if something else is running on that port.

Code under review:
public void testInterruptsNotCached() throws Exception {
		ConnectionID connectionId = new ConnectionID(new InetSocketAddress(InetAddress.getLocalHost(), 8080), 0);
		AwaitingNettyClient nettyClient = new AwaitingNettyClient();
		PartitionRequestClientFactory factory = new PartitionRequestClientFactory(nettyClient, 0);

		nettyClient.awaitForInterrupts = true;
		connectAndInterrupt(factory, connectionId);

		nettyClient.awaitForInterrupts = false;
		factory.createPartitionRequestClient(connectionId);
	}

Line commented on: ConnectionID connectionId = new ConnectionID(new InetSocketAddress(InetAddress.getLocalHost(), 8080), 0);

Revised code:
public void testInterruptsNotCached() throws Exception {
		ConnectionID connectionId = new ConnectionID(new InetSocketAddress(InetAddress.getLocalHost(), 8080), 0);
		try (AwaitingNettyClient nettyClient = new AwaitingNettyClient()) {
			PartitionRequestClientFactory factory = new PartitionRequestClientFactory(nettyClient, 0);

			nettyClient.awaitForInterrupts = true;
			connectAndInterrupt(factory, connectionId);

			nettyClient.awaitForInterrupts = false;
			factory.createPartitionRequestClient(connectionId);
		}
	}
Code review comment: nit: add an explanation comment: ``` InputGate on CheckpointBarrier can enqueue a mailbox action to execute and StreamTaskNetworkInput must allow this action to execute before processing a following record. ```

Code under review:
public void testNoDataProcessedAfterCheckpointBarrier() throws Exception {
		CheckpointBarrier barrier = new CheckpointBarrier(0, 0, CheckpointOptions.forCheckpointWithDefaultLocation());

		List<BufferOrEvent> buffers = new ArrayList<>(2);
		buffers.add(new BufferOrEvent(barrier, 0));
		buffers.add(createDataBuffer());

		VerifyRecordsDataOutput output = new VerifyRecordsDataOutput<>();
		StreamTaskNetworkInput input = createStreamTaskNetworkInput(buffers, output);

		assertHasNextElement(input, output);
		assertEquals(0, output.getNumberOfEmittedRecords());
	}

Line commented on: List<BufferOrEvent> buffers = new ArrayList<>(2);

Revised code:
public void testNoDataProcessedAfterCheckpointBarrier() throws Exception {
		CheckpointBarrier barrier = new CheckpointBarrier(0, 0, CheckpointOptions.forCheckpointWithDefaultLocation());

		List<BufferOrEvent> buffers = new ArrayList<>(2);
		buffers.add(new BufferOrEvent(barrier, 0));
		buffers.add(createDataBuffer());

		VerifyRecordsDataOutput output = new VerifyRecordsDataOutput<>();
		StreamTaskNetworkInput input = createStreamTaskNetworkInput(buffers, output);

		assertHasNextElement(input, output);
		assertEquals(0, output.getNumberOfEmittedRecords());
	}
Code review comment: should use global var for WS and NL

Code under review:
public ReturnTypeDescriptorNode transform(ReturnTypeDescriptorNode returnTypeDescriptorNode) {
        Token returnsKeyword = formatToken(returnTypeDescriptorNode.returnsKeyword(), 1, 0);
        NodeList<AnnotationNode> annotations = formatNodeList(returnTypeDescriptorNode.annotations(), 0, 0, 1, 0);
        Node type = formatNode(returnTypeDescriptorNode.type(), 1, 0);

        return returnTypeDescriptorNode.modify()
                .withReturnsKeyword(returnsKeyword)
                .withAnnotations(annotations)
                .withType(type)
                .apply();
    }

Line commented on: Node type = formatNode(returnTypeDescriptorNode.type(), 1, 0);

Revised code:
public ReturnTypeDescriptorNode transform(ReturnTypeDescriptorNode returnTypeDescriptorNode) {
        Token returnsKeyword = formatToken(returnTypeDescriptorNode.returnsKeyword(), 1, 0);
        NodeList<AnnotationNode> annotations = formatNodeList(returnTypeDescriptorNode.annotations(), 0, 0, 1, 0);
        Node type = formatNode(returnTypeDescriptorNode.type(), this.trailingWS, this.trailingNL);

        return returnTypeDescriptorNode.modify()
                .withReturnsKeyword(returnsKeyword)
                .withAnnotations(annotations)
                .withType(type)
                .apply();
    }
Code review comment: @mcruzdev I guess you should have the answer in the StaticHandler

Code under review:
private byte[] getClasspathResourceContent(String name) {
        String resourceName = GeneratedStaticResourcesRecorder.META_INF_RESOURCES + name;
        URL resource = getClassLoader().getResource(resourceName);
        if (resource == null) {
            LOG.warnf("The resource '%s' does not exist on classpath", resourceName);
            return null;
        }
        try {
            try (InputStream inputStream = resource.openStream()) {
                return inputStream.readAllBytes();
            }
        } catch (IOException e) {
            LOG.error("Error while reading file from Classpath for path " + resourceName, e);
            return null;
        }
    }

Line commented on: return inputStream.readAllBytes();

Revised code:
private byte[] getClasspathResourceContent(String name) {
        String resourceName = GeneratedStaticResourcesRecorder.META_INF_RESOURCES + name;
        URL resource = getClassLoader().getResource(resourceName);
        if (resource == null) {
            LOG.warnf("The resource '%s' does not exist on classpath", resourceName);
            return null;
        }
        try {
            try (InputStream inputStream = resource.openStream()) {
                return inputStream.readAllBytes();
            }
        } catch (IOException e) {
            LOG.error("Error while reading file from Classpath for path " + resourceName, e);
            return null;
        }
    }
Code review comment: Update `attemptsBeforeResetBackoff` to `Integer.MAX_VALUE` for all tests not related to `maxAttempts`.  `Integer.MAX_VALUE` makes it easier for other developers to know that attemptsBeforeResetBackoff does not take effect.

Code under review:
void testResetBackoff() {
        final long initialBackoffMS = 1L;
        final long resetBackoffThresholdMS = 8L;
        final ManualClock clock = new ManualClock();

        final ExponentialDelayRestartBackoffTimeStrategy restartStrategy =
                new ExponentialDelayRestartBackoffTimeStrategy(
                        clock, initialBackoffMS, 5L, 2.0, resetBackoffThresholdMS, 0.25, 100);

        clock.advanceTime(
                resetBackoffThresholdMS + restartStrategy.getBackoffTime() - 1,
                TimeUnit.MILLISECONDS);
        restartStrategy.notifyFailure(failure);
        assertThat(restartStrategy.getBackoffTime())
                .as("Backoff should be increased")
                .isEqualTo(2L);

        clock.advanceTime(
                resetBackoffThresholdMS + restartStrategy.getBackoffTime(), TimeUnit.MILLISECONDS);
        restartStrategy.notifyFailure(failure);
        assertThat(restartStrategy.getBackoffTime())
                .as("Backoff should be reset")
                .isEqualTo(initialBackoffMS);
    }

Line commented on: clock, initialBackoffMS, 5L, 2.0, resetBackoffThresholdMS, 0.25, 100);

Revised code:
void testResetBackoff() {
        final long initialBackoffMS = 1L;
        final long resetBackoffThresholdMS = 8L;
        final ManualClock clock = new ManualClock();

        final ExponentialDelayRestartBackoffTimeStrategy restartStrategy =
                new ExponentialDelayRestartBackoffTimeStrategy(
                        clock,
                        initialBackoffMS,
                        5L,
                        2.0,
                        resetBackoffThresholdMS,
                        0.25,
                        Integer.MAX_VALUE);

        clock.advanceTime(
                resetBackoffThresholdMS + restartStrategy.getBackoffTime() - 1,
                TimeUnit.MILLISECONDS);
        restartStrategy.notifyFailure(failure);
        assertThat(restartStrategy.getBackoffTime())
                .as("Backoff should be increased")
                .isEqualTo(2L);

        clock.advanceTime(
                resetBackoffThresholdMS + restartStrategy.getBackoffTime(), TimeUnit.MILLISECONDS);
        restartStrategy.notifyFailure(failure);
        assertThat(restartStrategy.getBackoffTime())
                .as("Backoff should be reset")
                .isEqualTo(initialBackoffMS);
    }
Code review comment: This is just to iterate through the union members to see if there are compatible raw template types. No need to call `getEffectiveMemberTypes()` for that right?

Code under review:
private BType getCompatibleRawTemplateType(BType expType, DiagnosticPos pos) {
        if (expType.tag != TypeTags.UNION) {
            return expType;
        }

        BUnionType unionType = (BUnionType) expType;
        List<BType> compatibleTypes = new ArrayList<>();

        for (BType type : unionType.getMemberTypes()) {
            if (types.isAssignable(type, symTable.rawTemplateType)) {
                compatibleTypes.add(type);
            }
        }

        if (compatibleTypes.size() == 0) {
            return expType;
        }

        if (compatibleTypes.size() > 1) {
            dlog.error(pos, DiagnosticCode.MULTIPLE_COMPATIBLE_RAW_TEMPLATE_TYPES, symTable.rawTemplateType, expType);
            return symTable.semanticError;
        }

        return compatibleTypes.get(0);
    }

Line commented on: for (BType type : unionType.getMemberTypes()) {

Revised code:
private BType getCompatibleRawTemplateType(BType expType, DiagnosticPos pos) {
        if (expType.tag != TypeTags.UNION) {
            return expType;
        }

        BUnionType unionType = (BUnionType) expType;
        List<BType> compatibleTypes = new ArrayList<>();

        for (BType type : unionType.getMemberTypes()) {
            if (types.isAssignable(type, symTable.rawTemplateType)) {
                compatibleTypes.add(type);
            }
        }

        if (compatibleTypes.size() == 0) {
            return expType;
        }

        if (compatibleTypes.size() > 1) {
            dlog.error(pos, DiagnosticCode.MULTIPLE_COMPATIBLE_RAW_TEMPLATE_TYPES, symTable.rawTemplateType, expType);
            return symTable.semanticError;
        }

        return compatibleTypes.get(0);
    }
Code review comment: It should be, but again, this is similar to the tag handling of  `StreamElementSerializer`  Would you mind keeping as it is for now?

Code under review:
byte[] serializeRecord(IN record, Long timestamp) {
			if (dos == null) {
				dos = new DataOutputSerializer(16);
			}

			try {
				if (timestamp == null) {
					dos.writeInt(TAG_REC_WITHOUT_TIMESTAMP);
				} else {
					dos.writeInt(TAG_REC_WITH_TIMESTAMP);
					dos.writeLong(timestamp);
				}
				serializer.serialize(record, dos);

			} catch (IOException e) {
				throw new RuntimeException("Unable to serialize record", e);
			}

			byte[] ret = dos.getCopyOfBuffer();
			dos.clear();
			return ret;
		}

Line commented on: byte[] ret = dos.getCopyOfBuffer();

Revised code:
byte[] serializeRecord(IN record, Long timestamp) {
			if (dos == null) {
				dos = new DataOutputSerializer(16);
			}

			try {
				dos.write(KAFKA_SHUFFLE_VERSION);

				if (timestamp == null) {
					dos.write(TAG_REC_WITHOUT_TIMESTAMP);
				} else {
					dos.write(TAG_REC_WITH_TIMESTAMP);
					dos.writeLong(timestamp);
				}
				serializer.serialize(record, dos);

			} catch (IOException e) {
				throw new RuntimeException("Unable to serialize record", e);
			}

			byte[] ret = dos.getCopyOfBuffer();
			dos.clear();
			return ret;
		}
Code review comment: Can this ever be null? From the current code, it doesn't look like so.

Code under review:
public String getUserAgent() {
        String userAgent = null;
        if (this.feedResponseDiagnostics != null) {
            userAgent = this.feedResponseDiagnostics.getUserAgent();
        } else if (this.clientSideRequestStatistics != null) {
            userAgent = this.clientSideRequestStatistics.getUserAgent();
        }

        if (userAgent != null) {
            return userAgent;
        }

        return "";
    }

Line commented on: if (userAgent != null) {

Revised code:
public String getUserAgent() {
        if (this.feedResponseDiagnostics != null) {
            return this.feedResponseDiagnostics.getUserAgent();
        }

        return this.clientSideRequestStatistics.getUserAgent();
    }
Code review comment: We can assert these in Ballerina itself.

Code under review:
public void testIncompatibleStringToFloat() {
        BValue[] args = {new BString("2222.333f")};
        BValue[] returns = BRunUtil.invoke(result, "stringtofloat", args);
        Assert.assertTrue(returns[0] instanceof BError);
        BError error = (BError) returns[0];
        String errorMsg = ((BMap) error.getDetails()).get("message").stringValue();
        Assert.assertEquals(errorMsg, "'string' value '2222.333f' cannot be converted to 'float'");
    }

Line commented on: Assert.assertEquals(errorMsg, "'string' value '2222.333f' cannot be converted to 'float'");

Revised code:
public void testIncompatibleStringToFloat() {
        BValue[] args = {new BString("2222.333f")};
        BValue[] returns = BRunUtil.invoke(result, "stringtofloat", args);
        Assert.assertTrue(returns[0] instanceof BError);
        BError error = (BError) returns[0];
        String errorMsg = ((BMap) error.getDetails()).get("message").stringValue();
        Assert.assertEquals(errorMsg, "'string' value '2222.333f' cannot be converted to 'float'");
    }
Code review comment: I wanted to keep it consistent.

Code under review:
private void head(OperatorID headOperatorID, StreamOperatorFactory<?> headOperatorFactory) {
		headConfig.setStreamOperatorFactory(headOperatorFactory);
		headConfig.setOperatorID(headOperatorID);
		headConfig.setChainStart();
		headConfig.setChainIndex(chainIndex);
		headConfig.setBufferTimeout(bufferTimeout);
	}

Line commented on: headConfig.setBufferTimeout(bufferTimeout);

Revised code:
private void head(OperatorID headOperatorID, StreamOperatorFactory<?> headOperatorFactory) {
		headConfig.setStreamOperatorFactory(headOperatorFactory);
		headConfig.setOperatorID(headOperatorID);
		headConfig.setChainStart();
		headConfig.setChainIndex(chainIndex);
		headConfig.setBufferTimeout(bufferTimeout);
	}
Code review comment: I wonder if the reason why the memory of checkpoint catalog is not reclaimed is that the getValue() method references the members of checkpoint catalog?

Code under review:
public ReportHandler() {
        GaugeMetric<Long> gauge = new GaugeMetric<Long>(
                "report_queue_size", MetricUnit.NOUNIT, "report queue size") {
            @Override
            public Long getValue() {
                return (long) reportQueue.size();
            }
        };
        MetricRepo.PALO_METRIC_REGISTER.addPaloMetrics(gauge);
    }

Line commented on: return (long) reportQueue.size();

Revised code:
public ReportHandler() {
        GaugeMetric<Long> gauge = new GaugeMetric<Long>(
                "report_queue_size", MetricUnit.NOUNIT, "report queue size") {
            @Override
            public Long getValue() {
                return (long) reportQueue.size();
            }
        };
        MetricRepo.PALO_METRIC_REGISTER.addPaloMetrics(gauge);
    }
Code review comment: If there is no empty string, just checking for null is enough, why introduce strings?

Code under review:
private void checkDatabaseName(final String databaseName) {
        if (Strings.isNullOrEmpty(databaseName)) {
            throw new NoDatabaseSelectedException();
        }
        if (!ProxyContext.getInstance().databaseExists(databaseName)) {
            throw new UnknownDatabaseException(databaseName);
        }
    }

Line commented on: if (Strings.isNullOrEmpty(databaseName)) {

Revised code:
private void checkDatabaseName(final String databaseName) {
        if (null == databaseName) {
            throw new NoDatabaseSelectedException();
        }
        if (!ProxyContext.getInstance().databaseExists(databaseName)) {
            throw new UnknownDatabaseException(databaseName);
        }
    }
Code review comment: What's the purpose of this check?

Code under review:
public void put(Object key, BValue value) {
            super.put(key, value);
            if (constructed) {
                setAttribute((String) key, value.stringValue());
            }
        }

Line commented on: if (constructed) {

Revised code:
public void put(Object key, BValue value) {
            super.put(key, value);
            if (constructed) {
                setAttribute((String) key, value.stringValue());
            }
        }
Code review comment: Isn't ChatThreadClient created by the builder now? Please be consistent: eight getChatThreadClient from chatClient or get it from the builder NOT both

Code under review:
public void getChatThread() {
        ChatClient chatClient = createChatClient();

        ChatThreadClient chatThreadClient = chatClient.getChatThreadClient("Id");
        ChatThreadProperties chatThreadProperties = chatThreadClient.getProperties();
    }

Line commented on: ChatThreadClient chatThreadClient = chatClient.getChatThreadClient("Id");

Revised code:
public void getChatThread() {
        ChatClient chatClient = createChatClient();

        ChatThreadClient chatThreadClient = chatClient.getChatThreadClient("Id");
        ChatThreadProperties chatThreadProperties = chatThreadClient.getProperties();
    }
Code review comment: Possible Assert4j shorthand: ```java         assertThat(TestShuffleMaster.partitionEvents)                 .containsExactly(                         PARTITION_REGISTRATION_EVENT,                         PARTITION_REGISTRATION_EVENT,                         EXTERNAL_PARTITION_RELEASE_EVENT,                         EXTERNAL_PARTITION_RELEASE_EVENT); ```

Code under review:
void testShuffleMasterLifeCycle() throws Exception {
        try (MiniCluster cluster = new MiniCluster(createClusterConfiguration(false))) {
            cluster.start();
            cluster.executeJobBlocking(createJobGraph());
        }
        assertThat(TestShuffleMaster.currentInstance.get().closed).isTrue();

        String[] expectedPartitionEvents =
                new String[] {
                    PARTITION_REGISTRATION_EVENT,
                    PARTITION_REGISTRATION_EVENT,
                    EXTERNAL_PARTITION_RELEASE_EVENT,
                    EXTERNAL_PARTITION_RELEASE_EVENT,
                };
        assertThat(TestShuffleMaster.partitionEvents.toArray()).isEqualTo(expectedPartitionEvents);
    }

Line commented on: assertThat(TestShuffleMaster.partitionEvents.toArray()).isEqualTo(expectedPartitionEvents);

Revised code:
void testShuffleMasterLifeCycle() throws Exception {
        try (MiniCluster cluster = new MiniCluster(createClusterConfiguration(false))) {
            cluster.start();
            cluster.executeJobBlocking(createJobGraph());
        }
        assertThat(TestShuffleMaster.currentInstance.get().closed).isTrue();
        assertThat(TestShuffleMaster.partitionEvents)
                .containsExactly(
                        PARTITION_REGISTRATION_EVENT,
                        PARTITION_REGISTRATION_EVENT,
                        EXTERNAL_PARTITION_RELEASE_EVENT,
                        EXTERNAL_PARTITION_RELEASE_EVENT);
    }
Code review comment: Yeah, the channel name detection is a bit of a hack; maybe the service registry team could have a though on how to do this more elegantly?

Code under review:
private List<String> extractChannels(Config configIn) {

        var list = new ArrayList<String>();

        for (String propertyName : configIn.getPropertyNames()) {
            if (propertyName.startsWith(INCOMING_PREFIX)) {
                var channelName = propertyName.replace(INCOMING_PREFIX, "").split("\\.")[0];
                list.add(INCOMING_PREFIX + channelName + ".");
            } else if (propertyName.startsWith(OUTGOING_PREFIX)) {
                var channelName = propertyName.replace(OUTGOING_PREFIX, "").split("\\.")[0];
                list.add(OUTGOING_PREFIX + channelName + ".");
            }
        }
        return list;
    }

Line commented on: for (String propertyName : configIn.getPropertyNames()) {

Revised code:
private List<String> extractChannels(Config configIn) {

        var list = new ArrayList<String>();

        for (String propertyName : configIn.getPropertyNames()) {
            if (propertyName.startsWith(INCOMING_PREFIX)) {
                var channelAndProp = StringUtils.substringAfter(propertyName, INCOMING_PREFIX);
                
                var channelName = StringUtils.substringBefore(channelAndProp, ".");
                if (!StringUtils.isBlank(channelName))
                    list.add(INCOMING_PREFIX + channelName + ".");
            } else if (propertyName.startsWith(OUTGOING_PREFIX)) {
                var channelAndProp = StringUtils.substringAfter(propertyName, OUTGOING_PREFIX);
                
                var channelName = StringUtils.substringBefore(channelAndProp, ".");
                if (!StringUtils.isBlank(channelName))
                    list.add(OUTGOING_PREFIX + channelName + ".");
            }
        }
        return list;
    }
Code review comment: This logic has been removed from BE, right? Is it still necessary to persist the txn id now?

Code under review:
protected void runPendingJob() throws AlterCancelException {
        
        List<Partition> partitions = Lists.newArrayList();
        OlapTable olapTable;
        Database db = GlobalStateMgr.getCurrentState().getDb(dbId);

        if (db == null) {
            throw new AlterCancelException("database does not exist, dbId:" + dbId);
        }
        db.readLock();
        try {
            olapTable = (OlapTable) db.getTable(tableName);
            if (olapTable == null) {
                throw new AlterCancelException("table does not exist, tableName:" + tableName);
            }
            partitions.addAll(olapTable.getPartitions());
        } finally {
            db.readUnlock();
        }

        
        
        
        
        if (this.watershedTxnId == -1) {
            this.watershedTxnId =
                    GlobalStateMgr.getCurrentGlobalTransactionMgr().getTransactionIDGenerator().getNextTransactionId();
            GlobalStateMgr.getCurrentState().getEditLog().logAlterJob(this);
        }

        try {
            for (Partition partition : partitions) {
                updatePartitionTabletMeta(db, olapTable.getName(), partition.getName(), metaValue, metaType);
            }
        } catch (DdlException e) {
            throw new AlterCancelException(e.getMessage());
        }

        this.jobState = JobState.RUNNING;
    }


Revised code:
protected void runPendingJob() throws AlterCancelException {
        
        List<Partition> partitions = Lists.newArrayList();
        OlapTable olapTable;
        Database db = GlobalStateMgr.getCurrentState().getDb(dbId);

        if (db == null) {
            throw new AlterCancelException("database does not exist, dbId:" + dbId);
        }
        db.readLock();
        try {
            olapTable = (OlapTable) db.getTable(tableName);
            if (olapTable == null) {
                throw new AlterCancelException("table does not exist, tableName:" + tableName);
            }
            partitions.addAll(olapTable.getPartitions());
        } finally {
            db.readUnlock();
        }

        if (this.watershedTxnId == -1) {
            this.watershedTxnId =
                    GlobalStateMgr.getCurrentGlobalTransactionMgr().getTransactionIDGenerator().getNextTransactionId();
            GlobalStateMgr.getCurrentState().getEditLog().logAlterJob(this);
        }

        try {
            for (Partition partition : partitions) {
                updatePartitionTabletMeta(db, olapTable.getName(), partition.getName(), metaValue, metaType);
            }
        } catch (DdlException e) {
            throw new AlterCancelException(e.getMessage());
        }

        this.jobState = JobState.RUNNING;
    }
Code review comment: I'm not sure we want to have the user provide a coder, to keep the interface simple.  We could change registration to Pipeline::registerBadRecordHandler, and just keep that logic in the pipeline. That has the side benefit of letting us know more about specific error handlers for the future case where we support other kinds of error handling

Code under review:
public void close() {
      closed = true;
      PCollection<ErrorT> flattened;
      if (errorCollections.isEmpty()) {
        LOG.warn("Empty list of error pcollections passed to ErrorHandler.");
        
        flattened = pipeline.apply(Create.of(new ArrayList<ErrorT>()));
      } else {
        flattened = PCollectionList.of(errorCollections).apply(Flatten.pCollections());
      }
      LOG.debug(
          "{} error collections are being sent to {}",
          errorCollections.size(),
          sinkTransform.getName());
      String sinkTransformName = sinkTransform.getName();
      sinkOutput =
          flattened
              .apply(
                  "Record Error Metrics to " + sinkTransformName,
                  new WriteErrorMetrics<ErrorT>(sinkTransformName))
              .apply(
                  "Write to error Sink",
                  sinkTransform.addAnnotation(
                      "FeatureMetric", "ErrorHandler".getBytes(StandardCharsets.UTF_8)));
    }

Line commented on: closed = true;

Revised code:
public void close() {
      if (closed) {
        throw new IllegalStateException(
            "Error handler is already closed, and may not be closed twice");
      }
      closed = true;
      PCollection<ErrorT> flattened;
      if (errorCollections.isEmpty()) {
        LOG.info("Empty list of error pcollections passed to ErrorHandler.");
        flattened = pipeline.apply(Create.empty(coder));
      } else {
        flattened = PCollectionList.of(errorCollections).apply(Flatten.pCollections());
      }
      LOG.debug(
          "{} error collections are being sent to {}",
          errorCollections.size(),
          sinkTransform.getName());
      String sinkTransformName = sinkTransform.getName();
      sinkOutput =
          flattened
              .apply(
                  "Record Error Metrics to " + sinkTransformName,
                  new WriteErrorMetrics<ErrorT>(sinkTransformName))
              .apply(
                  "Write to error Sink",
                  sinkTransform.addAnnotation(
                      "FeatureMetric", "ErrorHandler".getBytes(StandardCharsets.UTF_8)));
    }
Code review comment: is possible we only call convertIfNeeded() once in constructor?

Code under review:
public String toString() {
        convertIfNeeded();

        return azureCoreHeaders.toString();
    }

Line commented on: convertIfNeeded();

Revised code:
public String toString() {
        convertIfNeeded();

        return azureCoreHeaders.toString();
    }
Code review comment: Yeah, if doable, I think we'd better list the problematic ones rather than be too broad. That won't be perfect but less dangerous.

Code under review:
public boolean canConvert(Class type) {
        return List.class.isAssignableFrom(type);
    }

Line commented on: return List.class.isAssignableFrom(type);

Revised code:
public boolean canConvert(Class type) {
        return (type != null) && SUPPORTED_CLASS_NAMES.contains(type.getName());
    }
Code review comment: This means that my comment is not clear enought ;) `ctx.expression(1).accept(this);` will calls `visitParameterExpression()` that will escape the parameter with single quote. But if the parameter is a JavaScript patern, we need to un-escape it as the character in position.  So the parameter in position zero is `'`.  I didn't want to add regex specific stuff inside `visitParameterExpression()` ... mainly because there is no easy way to know inside this method that we are in a like expression (maybe possible via ctx.getParent() ...)

Code under review:
public String visitLikePredicate(HqlParser.LikePredicateContext ctx) {
        String parameter = ctx.expression(1).accept(this);
        if (parameter.indexOf('/') == 1 && parameter.lastIndexOf('/') > 1) {
            
            
            parameter = parameter.substring(1, parameter.length() - 1);
        }
        return ctx.expression(0).accept(this) + ":{'$regex':" + parameter + "}";
    }

Line commented on: if (parameter.indexOf('/') == 1 && parameter.lastIndexOf('/') > 1) {

Revised code:
public String visitLikePredicate(HqlParser.LikePredicateContext ctx) {
        String parameter = ctx.expression(1).accept(this);
        if (parameter.indexOf('/') == 1 && parameter.lastIndexOf('/') > 1) {
            
            
            parameter = parameter.substring(1, parameter.length() - 1);
        }
        return ctx.expression(0).accept(this) + ":{'$regex':" + parameter + "}";
    }