Paper: FARS: FSM-Augmentation to Make LLMs Hallucinate the Right APIs (ICLR-2024)

Reviewer comment (Evaluation):
The approach is more like eliminating "syntax" error but not "semantic" error. Will FARS eliminate "syntax" error but increase "semantic" error? For example, the function is originally semantically correct but with a wrong format, whereas FARS corrects the syntax but brings semantic errors. It will be good to see how much performance is obtained by "syntax" correction and if FARS introduces more "semantic" errors.

Author response:
In our implementation, we simply constrain the allowed token set without changing any of the probability scores. The highest probability token will remain the highest after constraining, provided it doesn’t interfere with the allowed arguments, API names, or format. Now, when we constrain and force the model to choose a token from the valid subset of tokens, which is different from the original global max, the question of whether that affects the rest of the generation is a hard one to answer in theory. We believe a good model should be able to generate a good continuation from any prefix, so we would argue that no, it doesn’t introduce any more semantic errors. To support this, we ran a quick empirical study on the MultiWOZ dataset on how many individual samples the unconstrained more performs better than the constrained model (not just the average, on which we know the constrained model is clearly better). We found that over 1000 dialogs and 7372 API turns, the unconstrained model is better than constrained just 6 times (0.08%).

Reviewer comment (Reproducibility):
What temperature is used in the experiments? The comparison may be less convincing if the temperature is high for baseline models.

Author response:
We don’t use temperature at all in our experiments. We use greedy decoding for API Generation since we don’t care about sampling diversity and just want the API call to be predicted correctly. This is for both constrained and unconstrained models. In the public baseline numbers for MultiWOZ from other previous works, we verified that it was also greedy, without any temperature sampling (those are all smaller models SFTed for this task anyway, not text generation LLMs). We included a section in the appendix about these inference details.

Reviewer comment (Presentation):
The paper lacks a thorough discussion of the limitations and potential future directions of the proposed approach.

Author response:
Thank you for bringing this to our notice and apologies for lack of more information on this. Due to space constraints, we limited this discussion to just the last paragraph of our paper. We however mention two concrete limitations and future directions to address them - 1) our model’s inability to handle APIs from unknown domains due to always constraining it to an API bank and 2) lack of analysis on the model’s ability to conditionally choose to engage the FSM. We will add a section on this in the appendix after further discussion among ourselves.

Reviewer comment (Reproducibility):
The effectiveness of FARS is contingent on the FSM's knowledge of the available API catalog. It is not clear how easy to update FSM if there is a new API function added.

Author response:
Updating the FSM with the new API is very easy. In fact, it is one of the advantages of our approach since we don’t have to touch the model or exemplars at all. 
We currently use a dynamic trie structure to represent the FSM. This trie is constructed from a schema dictionary that contains all the APIs, arguments, and allowed values. To add a new API, we can simply add it to the dictionary and re-run the trie initialization. This adds the sequence corresponding to generating that API name to the trie and includes the possible arguments in a tracker. Since the trie is dynamically expanded to cover arguments as the sequence keeps progressing during inference, we already have logic to go through all available APIs and extend the trie so nothing needs to be done for the new API there. We added these details in the paper to make this clear.

Reviewer comment (Experiments):
The scope of the paper's evaluation is limited to the Vicuna-33B model's performance on specific datasets. A broader assessment across various models would provide a more comprehensive understanding of FARS's effectiveness and generalizability.

Author response:
We performed evaluation with internal models of different sizes and found similar trends as well but reported only on Vicuna for reproducibility. Our approach required us to change the underlying generation strategy so we couldn’t go with closed models such as ChatGPT, where we don’t have access to the weights to run generation ourselves. We required an instruct-tuned LM and we believed that Vicuna-33B was a good strong candidate to go with. We also briefly experimented with LLAMA-2-Chat but found that that model was very resistant to generating APIs, no matter how we prompted it with exemplars.