# Speech Act Theory as an Evaluation Tool for Human-Agent Communication
## 8 Conclusions and Limitations  
To study the requirements of natural, enriched, and effective task-oriented communication with agents in the collaborative virtual world, we designed a communication model, and a scenario was developed where both the human and the virtual agent have to collaborate. SAT was used as a tool to evaluate the effectiveness of the verbal communication in conveying the agent's intentions. This study aimed to present SAT concepts as applicable tools to evaluate an agent's verbal communication. Different studies and scenarios that include human-agent communication will incorporate unique exchanged messages to achieve a unique objective; thus, there is no one method that can fit all cases and scenarios. The objective of the current study is to propose a general-purpose framework that is built on SAT concepts that can be adjusted to match each study. We discovered the following major findings from our study:  
* Besides being used as a reference in designing a communication model, SAT could be used as an evaluation tool to provide understanding of verbal communication from different perspectives, including the structure of utterances, the ratios of use of different intentions, the clearness of the intention behind the utterances, and the effect of the clearness of verbal messages on the decisions taken.
* Results show that the verbal component is very effective in conveying requests and intentions in a collaborative virtual environment.
* There is a positive relationship between participants' understanding of the agent's intention conveyed in his request and their acceptance of the request, and hence, to the understanding of the incidents in the collaboration situation.
* There is a positive relationship between the mutual understanding between the human and the agent in the collaborative environment and the human participants' perceptions of the flow of actions and overall performance.
* There is a positive relationship between the agent's consideration of the human, as represented in its utterances and actions, and the level of understanding the human has of these utterances and actions.  
This study had a number of limitations and challenges. The study showed that SAT concepts could be used to evaluate an agent's verbal communication; however, this is a general framework that should be adapted to fit each unique scenario and different messages. Another challenge could be the way to access the performance of a team including a human and an agent. As each scenario should have its own objective, evaluating the performance of human-agent teamwork will depend on the nature of each scenario. Additionally, potential improvement in human-agent teamwork may involve other factors that could contribute to performance improvement.  
**Author Contributions:** Conceptualization, N.H. and D.R.; methodology, N.H. and D.R.; software, N.H.; validation, N.H. and D.R.; formal analysis, N.H.; investigation, N.H. and D.R.; resources, D.R.; data curation, N.H.; writing--original draft preparation, N.H.; writing--review and editing, N.H. and D.R.; visualization, N.H.; supervision, D.R.; project administration, D.R.; funding acquisition, D.R.  
**Funding:** This research was funded by an Australian Research Council (ARC) Discovery Project number DP1093170.  
**Acknowledgments:** The authors would like to thank [PERSON] and [PERSON] for their technical assistance in designing the virtual reality system and in running the experiments. We would also like to thank [PERSON] for allowing us to conduct one of the experiments in her class.  
**Conflicts of Interest:** The authors declare no conflict of interest.