muttakin2301/ccr2000 huggingface dataset:
Code review comment: Yeah, this can be ignored since we have set the binder name as "kafka"

Code under review:
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
        if (bean instanceof BindingServiceProperties) {
            BindingServiceProperties bindingServiceProperties = (BindingServiceProperties) bean;
            if (bindingServiceProperties.getBinders().isEmpty()) {
                BinderProperties kafkaBinderSourceProperty = new BinderProperties();
                kafkaBinderSourceProperty.setType(KAKFA_BINDER_TYPE);
                configureBinderSources(kafkaBinderSourceProperty, AzureKafkaSpringCloudStreamConfiguration.AZURE_KAFKA_SPRING_CLOUD_STREAM_CONFIGURATION_CLASS);

                Map<String, BinderProperties> kafkaBinderPropertyMap = new HashMap<>();
                kafkaBinderPropertyMap.put(KAKFA_BINDER_DEFAULT_NAME, kafkaBinderSourceProperty);

                bindingServiceProperties.setBinders(kafkaBinderPropertyMap);
            } else {
                for (Map.Entry<String, BinderProperties> entry : bindingServiceProperties.getBinders().entrySet()) {
                    if (entry.getKey() != null && entry.getValue() != null
                            && (KAKFA_BINDER_TYPE.equalsIgnoreCase(entry.getValue().getType())
                            || KAKFA_BINDER_DEFAULT_NAME.equalsIgnoreCase(entry.getKey()))) {
                        configureBinderSources(entry.getValue(), buildKafkaBinderSources(entry.getValue()));
                    }
                }
            }
        }
        return bean;
    }

Line commented on: kafkaBinderSourceProperty.setType(KAKFA_BINDER_TYPE);

Revised code:
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
        if (bean instanceof BindingServiceProperties) {
            BindingServiceProperties bindingServiceProperties = (BindingServiceProperties) bean;
            if (bindingServiceProperties.getBinders().isEmpty()) {
                BinderProperties kafkaBinderSourceProperty = new BinderProperties();
                configureBinderSources(kafkaBinderSourceProperty, AzureKafkaSpringCloudStreamConfiguration.AZURE_KAFKA_SPRING_CLOUD_STREAM_CONFIGURATION_CLASS);

                Map<String, BinderProperties> kafkaBinderPropertyMap = new HashMap<>();
                kafkaBinderPropertyMap.put(KAKFA_BINDER_DEFAULT_NAME, kafkaBinderSourceProperty);

                bindingServiceProperties.setBinders(kafkaBinderPropertyMap);
            } else {
                for (Map.Entry<String, BinderProperties> entry : bindingServiceProperties.getBinders().entrySet()) {
                    if (entry.getKey() != null && entry.getValue() != null
                            && (KAKFA_BINDER_TYPE.equalsIgnoreCase(entry.getValue().getType())
                            || KAKFA_BINDER_DEFAULT_NAME.equalsIgnoreCase(entry.getKey()))) {
                        configureBinderSources(entry.getValue(), buildKafkaBinderSources(entry.getValue()));
                    }
                }
            }
        }
        return bean;
    }
Code review comment: why compare row number first then sort key ?

Code under review:
public int compare(CandidateContext context1, CandidateContext context2) {
            
            int ret = Integer.compare(context1.getGroupbyColumnNum(), context2.getGroupbyColumnNum());
            if (ret != 0) {
                return ret;
            }
            
            ret = Double.compare(context1.getMvStatistics().getOutputRowCount(),
                    context2.getMvStatistics().getOutputRowCount());
            if (ret != 0) {
                return ret;
            }

            
            ret = Integer.compare(context2.sortScore, context1.sortScore);
            if (ret != 0) {
                return ret;
            }

            
            ret = Integer.compare(context1.getSchemaColumnNum(), context2.getSchemaColumnNum());
            if (ret != 0) {
                return ret;
            }

            ret = Double.compare(context1.getMvStatistics().getComputeSize(), context2.getMvStatistics().getComputeSize());
            return ret != 0 ? ret : Integer.compare(context1.getIndex(), context2.getIndex());
        }


Revised code:
public int compare(CandidateContext context1, CandidateContext context2) {
            
            int ret = Integer.compare(context1.getGroupbyColumnNum(), context2.getGroupbyColumnNum());
            if (ret != 0) {
                return ret;
            }

            
            ret = Integer.compare(context2.sortScore, context1.sortScore);
            if (ret != 0) {
                return ret;
            }

            
            ret = Double.compare(context1.getMvStatistics().getOutputRowCount(),
                    context2.getMvStatistics().getOutputRowCount());
            if (ret != 0) {
                return ret;
            }

            
            ret = Integer.compare(context1.getSchemaColumnNum(), context2.getSchemaColumnNum());
            if (ret != 0) {
                return ret;
            }

            ret = Double.compare(context1.getMvStatistics().getComputeSize(), context2.getMvStatistics().getComputeSize());
            return ret != 0 ? ret : Integer.compare(context1.getIndex(), context2.getIndex());
        }
Code review comment: One already created bean, which generally won't happen until runtime when you actually have vert.x.   Currently if you fail in static init then this method is called, but vert.x is not ready yet (as it is a runtime bean), so you get an extra unrelated exception in your logs (and possible unclean shutdown).

Code under review:
void undeployVerticles(@Observes @BeforeDestroyed(ApplicationScoped.class) Object event, BeanManager beanManager) {
        
        Set<Bean<?>> beans = beanManager.getBeans(AbstractVerticle.class, Any.Literal.INSTANCE);
        Context applicationContext = beanManager.getContext(ApplicationScoped.class);
        for (Bean<?> bean : beans) {
            if (ApplicationScoped.class.equals(bean.getScope())) {
                
                Object instance = applicationContext.get(bean);
                if (instance != null) {
                    
                    try {
                        AbstractVerticle verticle = (AbstractVerticle) instance;
                        io.vertx.mutiny.core.Vertx mutiny = beanManager.createInstance()
                                .select(io.vertx.mutiny.core.Vertx.class).get();
                        mutiny.undeploy(verticle.deploymentID()).await().indefinitely();
                        LOGGER.debugf("Undeployed verticle: %s", instance.getClass());
                    } catch (Exception e) {
                        
                        LOGGER.debugf("Unable to undeploy verticle %s: %s", instance.getClass(), e.toString());
                    }
                }
            }
        }
    }

Line commented on: io.vertx.mutiny.core.Vertx mutiny = beanManager.createInstance()

Revised code:
void undeployVerticles(@Observes @BeforeDestroyed(ApplicationScoped.class) Object event, BeanManager beanManager) {
        
        Set<Bean<?>> beans = beanManager.getBeans(AbstractVerticle.class, Any.Literal.INSTANCE);
        Context applicationContext = beanManager.getContext(ApplicationScoped.class);
        for (Bean<?> bean : beans) {
            if (ApplicationScoped.class.equals(bean.getScope())) {
                
                Object instance = applicationContext.get(bean);
                if (instance != null) {
                    
                    try {
                        AbstractVerticle verticle = (AbstractVerticle) instance;
                        io.vertx.mutiny.core.Vertx mutiny = beanManager.createInstance()
                                .select(io.vertx.mutiny.core.Vertx.class).get();
                        mutiny.undeploy(verticle.deploymentID()).await().indefinitely();
                        LOGGER.debugf("Undeployed verticle: %s", instance.getClass());
                    } catch (Exception e) {
                        
                        LOGGER.debugf("Unable to undeploy verticle %s: %s", instance.getClass(), e.toString());
                    }
                }
            }
        }
    }
Code review comment: In here the effectiveTypeDescriptor method gives the correct result. We are using that type symbol to get the langlibs. So this is not an issue right?  ``` type Type1 object {  };  type Type2 readonly & Type1;  type Type3 Type2;  type Type4 Type3;  function name() {   Type4 t =  object {};   t.<cursor> } ``` I tried with the above example.

Code under review:
public List<FunctionSymbol> langLibMethods() {
        if (this.langLibFunctions == null) {
            if (this.effectiveTypeDescriptor().typeKind() == TypeDescKind.OBJECT) {
                this.langLibFunctions = this.effectiveTypeDescriptor().langLibMethods();
            } else if (this.effectiveTypeDescriptor().typeKind() == TypeDescKind.TYPE_REFERENCE) {
                TypeReferenceTypeSymbol typeRef = (TypeReferenceTypeSymbol) this.effectiveTypeDescriptor();
                this.langLibFunctions = typeRef.typeDescriptor().langLibMethods();
            } else {
                LangLibrary langLibrary = LangLibrary.getInstance(this.context);
                List<FunctionSymbol> functions = langLibrary.getMethods(
                        ((AbstractTypeSymbol) this.effectiveTypeDescriptor()).getBType());
                this.langLibFunctions = filterLangLibMethods(functions, this.getBType());
            }
        }

        return this.langLibFunctions;
    }

Line commented on: TypeReferenceTypeSymbol typeRef = (TypeReferenceTypeSymbol) this.effectiveTypeDescriptor();

Revised code:
public List<FunctionSymbol> langLibMethods() {
        if (this.langLibFunctions == null) {
            if (this.effectiveTypeDescriptor().typeKind() == TypeDescKind.OBJECT) {
                this.langLibFunctions = this.effectiveTypeDescriptor().langLibMethods();
            } else if (this.effectiveTypeDescriptor().typeKind() == TypeDescKind.TYPE_REFERENCE) {
                TypeReferenceTypeSymbol typeRef = (TypeReferenceTypeSymbol) this.effectiveTypeDescriptor();
                this.langLibFunctions = typeRef.typeDescriptor().langLibMethods();
            } else {
                LangLibrary langLibrary = LangLibrary.getInstance(this.context);
                List<FunctionSymbol> functions = langLibrary.getMethods(
                        ((AbstractTypeSymbol) this.effectiveTypeDescriptor()).getBType());
                this.langLibFunctions = filterLangLibMethods(functions, this.getBType());
            }
        }

        return this.langLibFunctions;
    }
Code review comment: AFAIK the generated subclass has a "forward method" for every intercepted method and this method is invoked if you do `IC.proceed()` in the last interceptor from the chain.

Code under review:
void forEachMethod(ClassInfo clazz, Consumer<MethodInfo> action) {
        for (MethodInfo method : clazz.methods()) {
            if (method.name().startsWith("<")) {
                
                continue;
            }
            if (Modifier.isPrivate(method.flags())) {
                
                continue;
            }
            if (Modifier.isFinal(method.flags())) {
                
                continue;
            }
            if (method.isSynthetic()) {
                
                continue;
            }

            action.accept(method);
        }

        DotName parentClassName = clazz.superName();
        if (parentClassName == null || parentClassName.equals(DotNames.OBJECT)) {
            return;
        }
        ClassInfo parentClass = index.getClassByName(parentClassName);
        if (parentClass == null) {
            return;
        }
        forEachMethod(parentClass, action);
    }


Revised code:
void forEachMethod(ClassInfo clazz, Consumer<MethodInfo> action) {
        for (MethodInfo method : clazz.methods()) {
            if (method.name().startsWith("<")) {
                
                continue;
            }
            if (method.isSynthetic()) {
                
                continue;
            }

            action.accept(method);
        }

        DotName parentClassName = clazz.superName();
        if (parentClassName == null || parentClassName.equals(DotNames.OBJECT)) {
            return;
        }
        ClassInfo parentClass = index.getClassByName(parentClassName);
        if (parentClass == null) {
            return;
        }
        forEachMethod(parentClass, action);
    }
Code review comment: Not sure why those weren't added before (they also exist prior to 3.0) and I was seeing a (non-blocking?) NPE w.r.t. to `mariadb.properties`, so I added [both](https://github.com/mariadb-corporation/mariadb-connector-j/tree/3.0.3/src/main/resources).

Code under review:
void addNativeImageResources(BuildProducer<NativeImageResourceBuildItem> resources) {
        resources.produce(new NativeImageResourceBuildItem("mariadb.properties", "driver.properties"));
    }

Line commented on: resources.produce(new NativeImageResourceBuildItem("mariadb.properties", "driver.properties"));

Revised code:
void addNativeImageResources(BuildProducer<NativeImageResourceBuildItem> resources) {
        
        
        
        resources.produce(new NativeImageResourceBuildItem("mariadb.properties"));
    }
Code review comment: JDBC maybe does not have these time attributes

Code under review:
public List<String> supportedProperties() {
		List<String> properties = new ArrayList<>();

		
		properties.add(CONNECTOR_DRIVER_NAME);
		properties.add(CONNECTOR_URL);
		properties.add(CONNECTOR_TABLE_NAME);
		properties.add(CONNECTOR_USERNAME);
		properties.add(CONNECTOR_PASSWORD);

		
		properties.add(CONNECTOR_SCAN_PARTITION_COLUMN_NAME);
		properties.add(CONNECTOR_SCAN_PARTITION_SIZE);
		properties.add(CONNECTOR_SCAN_PARTITION_LOWER);
		properties.add(CONNECTOR_SCAN_PARTITION_UPPER);

		
		properties.add(CONNECTOR_LOOKUP_CACHE_MAX_SIZE);
		properties.add(CONNECTOR_LOOKUP_CACHE_EXPIRE_MS);
		properties.add(CONNECTOR_LOOKUP_MAX_RETRY_TIMES);

		
		properties.add(CONNECTOR_SINK_UPDATE_MODE);
		properties.add(CONNECTOR_SINK_FLUSH_MAX_SIZE);
		properties.add(CONNECTOR_SINK_FLUSH_INTERVAL_MS);
		properties.add(CONNECTOR_SINK_MAX_RETRY_TIMES);

		
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".

		
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".

		return properties;
	}

Line commented on: properties.add(SCHEMA + ".

Revised code:
public List<String> supportedProperties() {
		List<String> properties = new ArrayList<>();

		
		properties.add(CONNECTOR_DRIVER);
		properties.add(CONNECTOR_URL);
		properties.add(CONNECTOR_TABLE);
		properties.add(CONNECTOR_USERNAME);
		properties.add(CONNECTOR_PASSWORD);

		
		properties.add(CONNECTOR_READ_PARTITION_COLUMN);
		properties.add(CONNECTOR_READ_PARTITION_NUM);
		properties.add(CONNECTOR_READ_PARTITION_LOWER_BOUND);
		properties.add(CONNECTOR_READ_PARTITION_UPPER_BOUND);
		properties.add(CONNECTOR_READ_FETCH_SIZE);

		
		properties.add(CONNECTOR_LOOKUP_CACHE_MAX_ROWS);
		properties.add(CONNECTOR_LOOKUP_CACHE_TTL);
		properties.add(CONNECTOR_LOOKUP_MAX_RETRIES);

		
		properties.add(CONNECTOR_WRITE_FLUSH_MAX_ROWS);
		properties.add(CONNECTOR_WRITE_FLUSH_INTERVAL);
		properties.add(CONNECTOR_WRITE_MAX_RETRIES);

		
		properties.add(SCHEMA + ".
		properties.add(SCHEMA + ".

		return properties;
	}
Code review comment: > Ah! Not sure if inferring makes the code more readable here though.  I can see why it is confusing - I had the type here explicitly but IDEA told me it isn't needed. Let me add it back :)

Code under review:
public SyntheticBean create(SyntheticCreationalContext<SyntheticBean> context) {
            return new SyntheticBean(context.getInjectedReference(new TypeLiteral<>() {
            }, All.Literal.INSTANCE));
        }

Line commented on: return new SyntheticBean(context.getInjectedReference(new TypeLiteral<>() {

Revised code:
public SyntheticBean create(SyntheticCreationalContext<SyntheticBean> context) {
            return new SyntheticBean(context.getInjectedReference(new TypeLiteral<List<SomeBean>>() {
            }, All.Literal.INSTANCE));
        }
Code review comment: Let's invert this case and just have if -> throw and everything else outside of an if/else block

Code under review:
private void unpackAndValidateId(String keyId) {
        if (keyId != null && keyId.length() > 0) {
            try {
                URL url = new URL(keyId);
                String[] tokens = url.getPath().split("/");
                String endpoint = url.getProtocol() + ":
                String keyName = (tokens.length >= 3 ? tokens[2] : null);
                version = (tokens.length >= 4 ? tokens[3] : null);
                if(Strings.isNullOrEmpty(endpoint)) {
                    throw new IllegalArgumentException("Key endpoint in key id is invalid");
                } else if (Strings.isNullOrEmpty(keyName)) {
                    throw new IllegalArgumentException("Key name in key id is invalid");
                } else if(Strings.isNullOrEmpty(version)) {
                    throw new IllegalArgumentException("Key version in key id is invalid");
                }
            } catch (MalformedURLException e) {
                e.printStackTrace();
            }
        } else {
            throw new IllegalArgumentException("Key Id is invalid");
        }
    }

Line commented on: if (keyId != null && keyId.length() > 0) {

Revised code:
private void unpackAndValidateId(String keyId) {
        if (ImplUtils.isNullOrEmpty(keyId)) {
            throw new IllegalArgumentException("Key Id is invalid");
        }
        try {
            URL url = new URL(keyId);
            String[] tokens = url.getPath().split("/");
            String endpoint = url.getProtocol() + ":
            String keyName = (tokens.length >= 3 ? tokens[2] : null);
            String version = (tokens.length >= 4 ? tokens[3] : null);
            if (Strings.isNullOrEmpty(endpoint)) {
                throw new IllegalArgumentException("Key endpoint in key id is invalid");
            } else if (Strings.isNullOrEmpty(keyName)) {
                throw new IllegalArgumentException("Key name in key id is invalid");
            } else if (Strings.isNullOrEmpty(version)) {
                throw new IllegalArgumentException("Key version in key id is invalid");
            }
        } catch (MalformedURLException e) {
            throw new IllegalArgumentException("The key identifier is malformed", e);
        }
    }
Code review comment: I'm curious why there isn't any sort of TokenCredential or AzureSasCredential policy here. Does this service not have any sort of authentication? I can't find it in the .NET repo either

Code under review:
public ModelsRepositoryAsyncClient buildAsyncClient() {
        Objects.requireNonNull(endpoint, "'endpoint' cannot be null");

        Configuration buildConfiguration = this.configuration;
        if (buildConfiguration == null)
        {
            buildConfiguration = Configuration.getGlobalConfiguration().clone();
        }

        
        ModelsRepositoryServiceVersion serviceVersion = this.serviceVersion;
        if (serviceVersion == null)
        {
            serviceVersion = ModelsRepositoryServiceVersion.getLatest();
        }

        
        RetryPolicy retryPolicy = this.retryPolicy;
        if (retryPolicy == null)
        {
            retryPolicy = DEFAULT_RETRY_POLICY;
        }

        if (this.httpPipeline == null) {
            this.httpPipeline = buildPipeline(
                this.endpoint,
                this.httpLogOptions,
                this.clientOptions,
                this.httpClient,
                this.additionalPolicies,
                retryPolicy,
                buildConfiguration,
                this.properties);
        }

        return new ModelsRepositoryAsyncClient(this.endpoint, this.httpPipeline, serviceVersion, this.jsonSerializer);
    }

Line commented on: Objects.requireNonNull(endpoint, "'endpoint' cannot be null");

Revised code:
public ModelsRepositoryAsyncClient buildAsyncClient() {
        Objects.requireNonNull(endpoint, "'endpoint' cannot be null");

        Configuration buildConfiguration = this.configuration;
        if (buildConfiguration == null)
        {
            buildConfiguration = Configuration.getGlobalConfiguration().clone();
        }

        
        ModelsRepositoryServiceVersion serviceVersion = this.serviceVersion;
        if (serviceVersion == null)
        {
            serviceVersion = ModelsRepositoryServiceVersion.getLatest();
        }

        
        RetryPolicy retryPolicy = this.retryPolicy;
        if (retryPolicy == null)
        {
            retryPolicy = DEFAULT_RETRY_POLICY;
        }

        if (this.httpPipeline == null) {
            this.httpPipeline = buildPipeline(
                this.endpoint,
                this.httpLogOptions,
                this.clientOptions,
                this.httpClient,
                this.additionalPolicies,
                retryPolicy,
                buildConfiguration,
                this.properties);
        }

        return new ModelsRepositoryAsyncClient(this.endpoint, this.httpPipeline, serviceVersion, this.jsonSerializer);
    }
Code review comment: I think this will be best left for the user, as latency can change and the limit can be changed by GCP project owners.

Code under review:
public void processElement(ProcessContext context) {
      Iterable<AnnotateImageRequest> value = context.element().getValue();
      ArrayList<AnnotateImageRequest> annotateImageRequests = new ArrayList<>();
      value.forEach(annotateImageRequests::add);
      context.output(annotateImageRequests);
    }

Line commented on: ArrayList<AnnotateImageRequest> annotateImageRequests = new ArrayList<>();

Revised code:
public void processElement(ProcessContext context) {
      context.output(getResponse(Objects.requireNonNull(context.element().getValue())));
    }
Code review comment: Since the default limit is 30QPS but the latency will depend a lot on the size of the input let's be cautious, assume smaller sized input data, and use something small like 5 keys. We should make this choice clear in the documentation of the class, constructor with the default, and recommend that a user updates this to better reflect their desired QPS.

Code under review:
public void processElement(ProcessContext context) {
      Iterable<AnnotateImageRequest> value = context.element().getValue();
      ArrayList<AnnotateImageRequest> annotateImageRequests = new ArrayList<>();
      value.forEach(annotateImageRequests::add);
      context.output(annotateImageRequests);
    }

Line commented on: ArrayList<AnnotateImageRequest> annotateImageRequests = new ArrayList<>();

Revised code:
public void processElement(ProcessContext context) {
      context.output(getResponse(Objects.requireNonNull(context.element().getValue())));
    }
Code review comment: ok, done (and there goes the 2-line patch :( )

Code under review:
public void requestPartitions() throws IOException, InterruptedException {
		if (!requestedPartitionsFlag) {
			synchronized (requestLock) {
				if (isReleased) {
					throw new IllegalStateException("Already released.");
				}

				
				if (numberOfInputChannels != inputChannels.size()) {
					throw new IllegalStateException("Bug in input gate setup logic: mismatch between" +
							"number of total input channels and the currently set number of input " +
							"channels.");
				}

				for (InputChannel inputChannel : inputChannels.values()) {
					inputChannel.requestSubpartition(consumedSubpartitionIndex);
				}
			}

			requestedPartitionsFlag = true;
		}
	}

Line commented on: if (!requestedPartitionsFlag) {

Revised code:
public void requestPartitions() throws IOException, InterruptedException {
		if (requestedPartitionsFlag) {
			return;
		}

		synchronized (requestLock) {
			if (isReleased) {
				throw new IllegalStateException("Already released.");
			}

			
			if (numberOfInputChannels != inputChannels.size()) {
				throw new IllegalStateException("Bug in input gate setup logic: mismatch between" +
						"number of total input channels and the currently set number of input " +
						"channels.");
			}

			for (InputChannel inputChannel : inputChannels.values()) {
				inputChannel.requestSubpartition(consumedSubpartitionIndex);
			}
		}

		requestedPartitionsFlag = true;
	}
Code review comment: Hmm ... so should we try to set a compile version for non-java projects as well, then? Assign the controller version at the time of submission, or use the one from deployment.xml if set?

Code under review:
private void validateDeprecatedElements(ApplicationPackage applicationPackage) {
        for (var deprecatedElement : applicationPackage.deploymentSpec().deprecatedElements()) {
            if (applicationPackage.compileVersion().isEmpty()) continue;
            if (deprecatedElement.majorVersion() >= applicationPackage.compileVersion().get().getMajor()) continue;
            throw new IllegalArgumentException(deprecatedElement.humanReadableString());
        }
    }

Line commented on: if (applicationPackage.compileVersion().isEmpty()) continue;

Revised code:
private void validateDeprecatedElements(ApplicationPackage applicationPackage) {
        int wantedMajor = applicationPackage.compileVersion().map(Version::getMajor)
                                            .or(() -> applicationPackage.deploymentSpec().majorVersion())
                                            .or(() -> controller.readVersionStatus().controllerVersion()
                                                                .map(VespaVersion::versionNumber)
                                                                .map(Version::getMajor))
                                            .orElseThrow(() -> new IllegalArgumentException("Could not determine wanted major version"));
        for (var deprecatedElement : applicationPackage.deploymentSpec().deprecatedElements()) {
            if (applicationPackage.compileVersion().isEmpty()) continue;
            if (deprecatedElement.majorVersion() >= wantedMajor) continue;
            throw new IllegalArgumentException(deprecatedElement.humanReadableString());
        }
    }
Code review comment: Looks to me there is not need to have the process to allocate slot1, offer slot and release it. The slot offering only would be enough to add a free slot. I think we can simplify it, maybe in a separate commit.

Code under review:
public void testAllocateWithFreeSlot() throws Exception {
		final CompletableFuture<SlotRequest> slotRequestFuture = new CompletableFuture<>();
		resourceManagerGateway.setRequestSlotConsumer(slotRequestFuture::complete);

		try (SlotPoolImpl slotPool = createAndSetUpSlotPool()) {
			slotPool.registerTaskManager(taskManagerLocation.getResourceID());

			SlotRequestId requestId1 = new SlotRequestId();
			CompletableFuture<PhysicalSlot> future1 = requestNewAllocatedSlot(
				slotPool,
				requestId1
			);
			assertFalse(future1.isDone());

			final SlotRequest slotRequest = slotRequestFuture.get(timeout.toMilliseconds(), TimeUnit.MILLISECONDS);

			final SlotOffer slotOffer = new SlotOffer(
				slotRequest.getAllocationId(),
				0,
				DEFAULT_TESTING_PROFILE);

			assertTrue(slotPool.offerSlot(taskManagerLocation, taskManagerGateway, slotOffer));

			PhysicalSlot slot1 = future1.get(1, TimeUnit.SECONDS);
			assertTrue(future1.isDone());

			
			slotPool.releaseSlot(requestId1, null);

			assertEquals(1, slotPool.getAvailableSlots().size());
			assertEquals(0, slotPool.getAllocatedSlots().size());

			Optional<PhysicalSlot> optional = slotPool.allocateAvailableSlot(
				new SlotRequestId(),
				slotRequest.getAllocationId()
			);

			
			assertTrue(optional.isPresent());
			PhysicalSlot slot2 = optional.get();

			assertEquals(slot1, slot2);
		}
	}

Line commented on: slotPool.releaseSlot(requestId1, null);

Revised code:
public void testAllocateWithFreeSlot() throws Exception {
		final CompletableFuture<SlotRequest> slotRequestFuture = new CompletableFuture<>();
		resourceManagerGateway.setRequestSlotConsumer(slotRequestFuture::complete);

		try (SlotPoolImpl slotPool = createAndSetUpSlotPool(resourceManagerGateway)) {

			final AllocationID allocationId = new AllocationID();
			assertTrue(registerAndOfferSlot(taskManagerLocation, slotPool, allocationId));

			assertEquals(1, slotPool.getAvailableSlots().size());
			assertEquals(0, slotPool.getAllocatedSlots().size());

			Optional<PhysicalSlot> physicalSlot = slotPool.allocateAvailableSlot(
				new SlotRequestId(),
				allocationId);

			assertTrue(physicalSlot.isPresent());
			assertEquals(0, slotPool.getAvailableSlots().size());
			assertEquals(1, slotPool.getAllocatedSlots().size());
		}
	}
Code review comment: It's better to just let the `SecretNotFoundException` be thrown as it's more descriptive than `orElseThrow()`.

Code under review:
private static TlsCredentials latestValidCredentials(SecretStore secretStore, TlsConfig tlsConfig) {
        int version = latestVersionInSecretStore(secretStore, tlsConfig).orElseThrow();
        return new TlsCredentials(certificates(secretStore, tlsConfig, version), privateKey(secretStore, tlsConfig, version));
    }

Line commented on: int version = latestVersionInSecretStore(secretStore, tlsConfig).orElseThrow();

Revised code:
private static TlsCredentials latestValidCredentials(SecretStore secretStore, TlsConfig tlsConfig) {
        int version = latestVersionInSecretStore(secretStore, tlsConfig);
        return new TlsCredentials(certificates(secretStore, tlsConfig, version), privateKey(secretStore, tlsConfig, version));
    }
Code review comment: Possibly double that as we do prepare, then prepareWithLocks.

Code under review:
public boolean canAllocateTenantNodeTo(Node host, boolean dynamicProvisioning) {
        if ( ! host.type().canRun(NodeType.tenant)) return false;
        if (host.status().wantToRetire()) return false;
        if (host.allocation().map(alloc -> alloc.membership().retired()).orElse(false)) return false;
        if (suspended(host)) return false;

        if (dynamicProvisioning)
            return EnumSet.of(Node.State.active, Node.State.ready, Node.State.provisioned).contains(host.state());
        else
            return host.state() == Node.State.active;
    }

Line commented on: if (suspended(host)) return false;

Revised code:
public boolean canAllocateTenantNodeTo(Node host, boolean dynamicProvisioning) {
        if ( ! host.type().canRun(NodeType.tenant)) return false;
        if (host.status().wantToRetire()) return false;
        if (host.allocation().map(alloc -> alloc.membership().retired()).orElse(false)) return false;
        if (suspended(host)) return false;

        if (dynamicProvisioning)
            return EnumSet.of(Node.State.active, Node.State.ready, Node.State.provisioned).contains(host.state());
        else
            return host.state() == Node.State.active;
    }
Code review comment: instead of `break`, can we return `value` here? I think it will make things more clear.  We can add a `else` block at the end, and just return `value` from else condition.

Code under review:
static Object getValue(JsonNode value) {
        if (value.isValueNode()) {
            switch (value.getNodeType()) {
                case BOOLEAN:
                    return value.asBoolean();
                case NUMBER:
                    if (value.isInt()) {
                        return value.asInt();
                    } else if (value.isLong()) {
                        return value.asLong();
                    } else if (value.isDouble()) {
                        return value.asDouble();
                    }
                    break;
                case STRING:
                    return value.asText();
                default:
                    throw new IllegalStateException("Unexpected value: " + value.getNodeType());
            }
        }
        return value;
    }

Line commented on: break;

Revised code:
static Object getValue(JsonNode value) {
        if (value.isValueNode()) {
            switch (value.getNodeType()) {
                case BOOLEAN:
                    return value.asBoolean();
                case NUMBER:
                    if (value.isInt()) {
                        return value.asInt();
                    } else if (value.isLong()) {
                        return value.asLong();
                    } else if (value.isDouble()) {
                        return value.asDouble();
                    } else{
                        return value;
                    }
                case STRING:
                    return value.asText();
                default:
                    throw new IllegalStateException("Unexpected value: " + value.getNodeType());
            }
        }
        return value;
    }
Code review comment: Unrelated but useful: always move away "unlikely" paths ie cold ones out of the "main" method logic: it would increase chances to inline it (see http://normanmaurer.me/blog_in_progress/2013/11/07/Inline-all-the-Things/ that's still very actual and these things hasn't changed across JDK versions, really). Most inlining decisions on OpenJDK happens based on the bytecode size (although not always; sometime the native method size is important too), meaning that using any bytecode plugin viewer (I use https://plugins.jetbrains.com/plugin/9248-jclasslib-bytecode-viewer) can help checking how much the bytecode size change while grouping differently code into methods: eg  ```java if (LOG.isTraceEnabled()) {     traceActivate(...); } ``` this is going to reduce a bit the bytecode size of the original method, increasing its chance to be inlined in the common use case

Code under review:
public void activate(ContextState initialState) {
        if (LOG.isTraceEnabled()) {
            String stack = Arrays.stream(Thread.currentThread().getStackTrace())
                    .skip(2)
                    .limit(7)
                    .map(se -> "\n\t" + se.toString())
                    .collect(Collectors.joining());
            LOG.tracef("Activate %s %s\n\t...",
                    initialState != null ? Integer.toHexString(initialState.hashCode()) : "new", stack);
        }
        if (initialState == null) {
            currentContext.set(new RequestContextState(new ConcurrentHashMap<>()));
            
            fireIfNotEmpty(initializedNotifier);
        } else {
            if (initialState instanceof RequestContextState) {
                currentContext.set((RequestContextState) initialState);
            } else {
                throw new IllegalArgumentException("Invalid initial state: " + initialState.getClass().getName());
            }
        }
    }

Line commented on: String stack = Arrays.stream(Thread.currentThread().getStackTrace())

Revised code:
public void activate(ContextState initialState) {
        if (LOG.isTraceEnabled()) {
            String stack = Arrays.stream(Thread.currentThread().getStackTrace())
                    .skip(2)
                    .limit(7)
                    .map(se -> "\n\t" + se.toString())
                    .collect(Collectors.joining());
            LOG.tracef("Activate %s %s\n\t...",
                    initialState != null ? Integer.toHexString(initialState.hashCode()) : "new", stack);
        }
        if (initialState == null) {
            currentContext.set(new RequestContextState(new ConcurrentHashMap<>()));
            
            fireIfNotEmpty(initializedNotifier);
        } else {
            if (initialState instanceof RequestContextState) {
                currentContext.set((RequestContextState) initialState);
            } else {
                throw new IllegalArgumentException("Invalid initial state: " + initialState.getClass().getName());
            }
        }
    }
Code review comment: Isn't it better if we move this logic to the function definition node's overridden method itself?  As the return type  of  a function represents  the particular  function, I feel that this logic does not belong here.

Code under review:
public void visit(ReturnStatementNode returnStatementNode) {
        this.semanticModel.typeOf(returnStatementNode).ifPresent(this::checkAndSetTypeResult);
        if (resultFound) {
            return;
        }

        
        returnStatementNode.parent().accept(this);
        if (resultFound && returnTypeSymbol.typeKind() == TypeDescKind.FUNCTION) {
            FunctionTypeSymbol functionTypeSymbol = (FunctionTypeSymbol) returnTypeSymbol;
            functionTypeSymbol.returnTypeDescriptor().ifPresentOrElse(this::checkAndSetTypeResult, this::resetResult);
        } else {
            resetResult();
        }
    }

Line commented on: functionTypeSymbol.returnTypeDescriptor().ifPresentOrElse(this::checkAndSetTypeResult, this::resetResult);

Revised code:
public void visit(ReturnStatementNode returnStatementNode) {
        this.semanticModel.typeOf(returnStatementNode).ifPresent(this::checkAndSetTypeResult);
        if (resultFound) {
            return;
        }

        
        returnStatementNode.parent().accept(this);
        if (resultFound && returnTypeSymbol.typeKind() == TypeDescKind.FUNCTION) {
            FunctionTypeSymbol functionTypeSymbol = (FunctionTypeSymbol) returnTypeSymbol;
            functionTypeSymbol.returnTypeDescriptor().ifPresentOrElse(this::checkAndSetTypeResult, this::resetResult);
        } else {
            resetResult();
        }
    }
Code review comment: I think this block should be the `finally` block of the `try { test.run() } finally { ... }`

Code under review:
protected void runTest(RunnableWithException test) {
		Throwable testFailure = null;
		try {
			test.run();
		} catch (Throwable t) {
			testFailure = t;
		}

		try {
			Deadline deadline = Deadline.now().plus(Duration.ofSeconds(10));

			boolean isAnyJobRunning = yarnClient.getApplications().stream()
				.anyMatch(YarnTestBase::isApplicationRunning);

			while (deadline.hasTimeLeft() && isAnyJobRunning) {
				try {
					Thread.sleep(500);
				} catch (InterruptedException e) {
					Assert.fail("Should not happen");
				}
				isAnyJobRunning = yarnClient.getApplications().stream()
					.anyMatch(YarnTestBase::isApplicationRunning);
			}

			if (isAnyJobRunning) {
				final List<String> runningApps = yarnClient.getApplications().stream()
					.filter(YarnTestBase::isApplicationRunning)
					.map(app -> "App " + app.getApplicationId() + " is in state " + app.getYarnApplicationState() + '.')
					.collect(Collectors.toList());
				if (!runningApps.isEmpty()) {
					Assert.fail("There is at least one application on the cluster that is not finished." + runningApps);
				}
			}
		} catch (Throwable t) {
			throw new AssertionError(ExceptionUtils.firstOrSuppressed(t, testFailure));
		}
	}

Line commented on: try {

Revised code:
protected void runTest(RunnableWithException test) throws Exception {
		
		try (final CleanupYarnApplication ignored = new CleanupYarnApplication()) {
			test.run();
		}
	}
Code review comment: Replace `System.out` with log statement

Code under review:
InstanceIdentity sendInstanceRegisterRequest(InstanceRegisterInformation instanceRegisterInformation, String athenzUrl) {
        try(CloseableHttpClient client = HttpClientBuilder.create().build()) {
            ObjectMapper objectMapper = new ObjectMapper();
            System.out.println(objectMapper.writeValueAsString(instanceRegisterInformation));
            HttpUriRequest postRequest = RequestBuilder.post()
                    .setUri(athenzUrl + "/instance")
                    .setEntity(new StringEntity(objectMapper.writeValueAsString(instanceRegisterInformation), ContentType.APPLICATION_JSON))
                    .build();
            CloseableHttpResponse response = client.execute(postRequest);
            if(HttpStatus.isSuccess(response.getStatusLine().getStatusCode())) {
                return objectMapper.readValue(response.getEntity().getContent(), InstanceIdentity.class);
            } else {
                String s = EntityUtils.toString(response.getEntity());
                System.out.println("s = " + s);
                throw new RuntimeException(response.toString());
            }
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

Line commented on: System.out.println(objectMapper.writeValueAsString(instanceRegisterInformation));

Revised code:
InstanceIdentity sendInstanceRegisterRequest(InstanceRegisterInformation instanceRegisterInformation, String athenzUrl) {
        try(CloseableHttpClient client = HttpClientBuilder.create().build()) {
            ObjectMapper objectMapper = new ObjectMapper();
            System.out.println(objectMapper.writeValueAsString(instanceRegisterInformation));
            HttpUriRequest postRequest = RequestBuilder.post()
                    .setUri(athenzUrl + "/instance")
                    .setEntity(new StringEntity(objectMapper.writeValueAsString(instanceRegisterInformation), ContentType.APPLICATION_JSON))
                    .build();
            CloseableHttpResponse response = client.execute(postRequest);
            if(HttpStatus.isSuccess(response.getStatusLine().getStatusCode())) {
                return objectMapper.readValue(response.getEntity().getContent(), InstanceIdentity.class);
            } else {
                String s = EntityUtils.toString(response.getEntity());
                System.out.println("s = " + s);
                throw new RuntimeException(response.toString());
            }
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }
Code review comment: Inverted. I left it out initially since 2 vs 3 lines of code doesn't matter that much and I was just moving this code and too many modifications while moving the code can also be confusing :)

Code under review:
private Optional<BufferOrEvent> handleEmptyBuffer() throws Exception {
		if (!inputGate.isFinished()) {
			return Optional.empty();
		}

		if (!endOfStream) {
			
			endOfStream = true;
			releaseBlocksAndResetBarriers();
			return pollNext();
		}
		else {
			isFinished = true;
			return Optional.empty();
		}
	}

Line commented on: if (!endOfStream) {

Revised code:
private Optional<BufferOrEvent> handleEmptyBuffer() throws Exception {
		if (!inputGate.isFinished()) {
			return Optional.empty();
		}

		if (endOfStream) {
			isFinished = true;
			return Optional.empty();
		} else {
			
			endOfStream = true;
			releaseBlocksAndResetBarriers();
			return pollNext();
		}
	}
Code review comment: But you're sure that all the response handlers we added earlier are always executed on the right thread at the right time? That's what gets me a bit worried.  Note that I have no idea how this all works so I'm asking naive questions.

Code under review:
public void handle(final RoutingContext ctx) {

        ctx.response()
                .endHandler(currentManagedContextTerminationHandler)
                .exceptionHandler(currentManagedContextTerminationHandler)
                .closeHandler(currentManagedContextTerminationHandler);
        if (!currentManagedContext.isActive()) {
            currentManagedContext.activate();
        }
        try {
            handleWithIdentity(ctx);
            currentManagedContext.deactivate();
        } catch (Throwable t) {
            currentManagedContext.terminate();
            throw t;
        }
    }

Line commented on: currentManagedContext.deactivate();

Revised code:
public void handle(final RoutingContext ctx) {

        ctx.response()
                .endHandler(currentManagedContextTerminationHandler)
                .exceptionHandler(currentManagedContextTerminationHandler)
                .closeHandler(currentManagedContextTerminationHandler);
        if (!currentManagedContext.isActive()) {
            currentManagedContext.activate();
        }
        try {
            handleWithIdentity(ctx);
            currentManagedContext.deactivate();
        } catch (Throwable t) {
            currentManagedContext.terminate();
            throw t;
        }
    }
Code review comment: I agree that current implementation might be safer but seems like it significantly limits the usability of RestrictionTrackers as well. I think source authors are advanced users. So it might be OK to stay in the side of usability in some cases over safety. But this is debatable.  Another option might be to add a method to the RestrictionTracker to mark progress without claiming. We have such a method in the old RangeTracker interface: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/iobase.py#L393

Code under review:
public void makeProgress() throws Exception {
        progress += approximateRecordSize;
        if (progress > totalWork) {
          throw new IOException("Making progress out of range");
        }
      }

Line commented on: progress += approximateRecordSize;

Revised code:
public void makeProgress() throws Exception {
        progress += approximateRecordSize;
        if (progress > totalWork) {
          throw new IOException("Making progress out of range");
        }
      }
Code review comment: ```suggestion             throw ErrorCreator.createError(StringUtils.fromString("ballerina: error occurred while waiting for " + ``` This might be better

Code under review:
public void stop() {
        if (!moduleInitialized) {
            throw ErrorHelper.getRuntimeException(ErrorCodes.INVALID_FUNCTION_INVOCATION_BEFORE_MODULE_INIT, "stop");
        }
        if (moduleStopped) {
            throw ErrorHelper.getRuntimeException(ErrorCodes.FUNCTION_ALREADY_CALLED, "stop");
        }
        scheduler.poison();
        try {
            schedulerThread.join();
        } catch (InterruptedException e) {
            throw ErrorCreator.createError(StringUtils.fromString("ballerina: error occurred in while waiting for " +
                    "scheduler thread to finish"), e);
        }
        invokeModuleStop();
        moduleStopped = true;
    }

Line commented on: throw ErrorCreator.createError(StringUtils.fromString("ballerina: error occurred in while waiting for " +

Revised code:
public void stop() {
        if (!moduleInitialized) {
            throw ErrorHelper.getRuntimeException(ErrorCodes.INVALID_FUNCTION_INVOCATION_BEFORE_MODULE_INIT, "stop");
        }
        if (moduleStopped) {
            throw ErrorHelper.getRuntimeException(ErrorCodes.FUNCTION_ALREADY_CALLED, "stop");
        }
        scheduler.poison();
        try {
            schedulerThread.join();
        } catch (InterruptedException e) {
            throw ErrorCreator.createError(StringUtils.fromString("error occurred while waiting for the scheduler " +
                    "thread to finish"), e);
        }
        invokeModuleStop();
        moduleStopped = true;
    }