Source: http://sisl.disi.unitn.it/publications/?tgid=46
Timestamp: 2019-04-22 19:54:49+00:00

Document:
All years20172016201520142013201220112010200920082007200620052004200320022001200019991998199719961995199419931992 All types ArticlesBooksConferencesPresentationsProceedings All authors Abella A.Alam F.Allauzen C.Alonso T.Alshawi H.Anantharaman J.Andre’ E.Arai K.Atallah M. J.Baggia P.Balamurali A. R.Ballestrero ABangalore S.Bayer A. O.Bechet F.Berra E.Bisazza A.Bocchieri E.Bonin F.Bordel G.Buntschuh B.Callejas Z.Calvo M.Carenini G.Castagnino NCelli F.Cervone A.Chetouani M.Chiarcos C.Chowdhury AChowdhury A. S.Chowdhury S.Chowdhury S. A.Chu-Carroll J.Cimiano P.Cohn F. J.Conkie A.Coppola B.Cristani M.Cutugno F.Danieli M.Danieli M..De Mori R.Di Fabbrizio G.Dinarelli M.Dipper S.DonDouglas S.Dutton D.Eckert W.Esposito A.Fabbrizio G.Falavigna D.Fan J.Favre B.Fogarolli A.Franc J.Francesconi F.Fratti M.Fuhrmann F.Fulcheri C.Garcia F.Ghosh AGhosh A.Ghosh S.Gilmartin E.Goffin V.Gokhan T.Gonzalez M.Gorin A.Gorin A. L.Gretter R.Griol D.Gupta G.Gupta N.Götze M.Haecker M.Hakkani-Tur D.Hammal Z.Han S.Heylen D.Hollister B.Hurtado L. F.Ivanov A. V.Jakub F.Johansson R.Joshi A.Kaiser R.Kamm C.Karahan M.Kashkarev I.Kim H. K.Klasinas I.Koutsombogera M.Lefevre F.Lehnen P.Levin E.Lin C.Liscombe J.Ljolje A.Lopez-Cozar R.Ludwig B.Maffei MMayor J. M.Mayor Torres J.M.Mayor Torres Juan M.McTear M.Menardi P.Mian G. A.Mogessie A. M.Mogessie MMogessie M.Mosca N.Moschitti A.Mumolo E.Murdock V.Narayanan S.Nencioni ANey H.Nguyen T. T.Parodi SParthasarathy S.Patrone F.Patwardhan S.Petrovska-Delacretaz D.Pieraccini R.Poesio M.Poesio P.Potamianos A.Prasad R.Quarteroni S.Rabbia F.Rahim M.Raymond C.Rebelli A.Renals S.Riccardi GRiccardi G.Riley M.Roberti P.Rochery M.Rodríguez K.Rodríguez K. J.Ronchetti M.Rose R. C.Rönnqvist S.Salah A. G.Sanchis E.Sara Tonelli S.Saraclar M.Sarin P.Saul L.Schaeller R.Schapire R.Schenk N.Schroeter J.Sebastian V.Segarra E.Singla K.Sporka A. J.Stepanov A. E.Stepanov E.Stepanov E. A.Testa E.Tonelli S.Topkara M.Tortolina LTortoreto G.Trione J.Tur D.Tur G.Tur H.Unger C.Vagin A.Varges S.Veglio F.Vinciarelli A.Wisniewska J.Wright J.Wright J. H.Yao H.Zenker B.Zoppoli Gt K.
Alam F., Celli F., Stepanov A. E., Ghosh A. and Riccardi G.
Proc. PEOPLES Workshop at COLING, Osaka 2016, 2016.
In this paper, we address the issue of automatic prediction of readers’ mood from newspaper articles and comments. As online newspapers are becoming more and more similar to social media platforms, users can provide affective feedback, such as mood and emotion. We exploited the self-reported annotation of mood categories obtained from the metadata of the Italian online newspaper corriere.it to design and evaluate a system for predicting five different mood categories from news articles and comments: indignation, disappointment, worry, satisfaction and amusement. The outcome of our experiments shows that overall, bag-of-word-ngrams performs better compared to all other feature sets, however, stylometric features perform better for the mood score prediction of articles. Our study shows that such self-reported annotations can be used to design automatic systems.
Alam F. , Chowdhury S. , Danieli M. and Riccardi G.
Proc. COLING, Osaka, 2016., 2016.
2) correlation between the interlocutors using psycholinguistic features, such as linguistic styles, psychological process, personal concerns among others, and 3) relation of interlocutors turn-taking behaviors such as competitiveness. To study the degree of coordination in different emotional segments, we conducted our experiments using real dyadic conversations collected from call centers in which agent’s emotional state include empathy and customer’s emotional states include anger and frustration. Our findings suggest that the most coordination occurs between the interlocutors inside anger segments, where as, a little coordination was observed when the agent was empathic, even though an increase in the amount of non-competitive overlaps was observed. We found no significant difference between anger and frustration segment in terms of turn-taking behaviors. However, the length of pause significantly decreases in the preceding segment of anger where as it increases in the preceding segment of frustration.
Chowdhury S. , Stepanov A. E. and Riccardi G.
Proc. INTERSPEECH, San Francisco, 2016., 2016.
User satisfaction is an important aspect of the user experience while interacting with objects, systems or people. Traditionally user satisfaction is evaluated a-posteriori via spoken or written questionnaires or interviews. In automatic behavioral analysis we aim at measuring the user emotional states and its descriptions as they unfold during the interaction. In our approach, user satisfaction is modeled as the final state of a sequence of emotional states and given ternary values positive, negative, neutral. In this paper, we investigate the discriminating power of turn-taking in predicting user satisfaction in spoken conversations. Turn-taking is used for discourse organization of a conversation by means of explicit phrasing, intonation, and pausing. In this paper, we train different characterization of turn-taking, such as competitiveness of the speech overlaps. To extract turn-taking features we design a turn segmentation and labeling system that incorporates lexical and acoustic information. Given a human-human spoken dialog, our system automatically infers any of the three values of the state of the user satisfaction. We evaluate the classification system on real-life call-center human-human dialogs. The comparative performance analysis shows that the contribution of the turn-taking features outperforms both prosodic and lexical features.
Chowdhury A,, Danieli M. and Riccardi G.
psycholinguistic features. Our findings also suggest that the lexical selections of the overlapper are good indicators of speaker’s competitive or non-competitive intentions.
Favre B., Stepanov A. E., Trione J. , Bechet F. and Riccardi G.
datasets released for the task will allow more research on abstractive dialog summarization.
Vinciarelli A., Esposito A., Andre’ E., Bonin F., Chetouani M., Cohn F. J., Cristani M., Fuhrmann F., Gilmartin E., Hammal Z., Heylen D., Kaiser R., Koutsombogera M., Potamianos A., Renals S., Riccardi G., Salah A. G.
Cognitive Computation, pp. 1-17, April 2015, 2015.
especially when it comes to technologies that make sense of human–human and human–machine interactions. This article outlines some of the most important issues that still need to be addressed to ensure substantial progress in the field, namely (1) development and adoption of virtuous data collection and sharing practices, (2) shift in the focus of interest from individuals to dyads and groups, (3) endowment of artificial agents with internal representations of users and context, (4) modelling of cognitive and semantic processes underlying social behaviour and (5) identification of application domains and strategies for moving from laboratory to the real-world products.
For a long time, the research on human-machine conversation and interaction has inspired futuristic visions created by film directors and science fiction writers. Nowadays, there has been great progress towards this end by the extended community of artificial intelligence scientists spanning from computer scientists to neuroscientists. In this paper we first review the tension between the latest advances in the technology of virtual agents and the limitations in the modality, complexity and sociability of conversational agent interaction. Then we identify a research challenge and target for the research and technology community. We need to create a vision and research path to create personal agents that are perceived as devoted assistants and counselors in helping end-users managing their own healthcare and well-being throughout their life. Such target is a high-payoff research agenda with high-impact on the society. In this position paper, following a review of the state-of-the-art in conversational agent technology, we discuss the challenges in spoken/multimodal/multi-sensorial interaction needed to support the development of Healthcare Personal Agents.
Griol D., Callejas Z., Lopez-Cozar R. and Riccardi G.
Computer Speech and Language, to be published in 2014, 2014.
This paper proposes a domain-independent statistical methodology to develop dialog managers for spoken dialog systems. Our methodology employs a data-driven classiﬁcation procedure to generate abstract representations of system turns taking into account the previous history of the dialog. A statistical framework is also introduced for the development and evaluation of dialog systems created using the methodology, which is based on a dialog simulation technique. The beneﬁts and ﬂexibility of the proposed methodology have been validated by developing statistical dialog managers for four spoken dialog systems of different complexity, designed for different languages (English, Italian, and Spanish) and application domains (from transactional to problem-solving tasks). The evaluation results show that the proposed methodology allows rapid development of new dialog managers as well as to explore new dialog strategies, which permit developing new enhanced versions of already existing systems. © 2013 Elsevier Ltd. All rights reserved.
Riccardi G., Ghosh A., Chowdhury S. A. and Bayer A. O.
A widely used strategy in human and machine performance enhancement is achieved through feedback. In this paper we investigate the effect of live motivational feedback on motivating crowds and improving the performance of the crowdsourcing computational model. The provided feedback allows workers to react in real-time and review past actions (e.g. word deletions); thus, to improve their performance on the current and future (sub) tasks. The feedback signal can be controlled via clean (e.g. expert) supervision or noisy supervision in order to trade-off between cost and target performance of the crowd-sourced task. The feedback signal is designed to enable crowd workers to improve their performance at the (sub) task level. The type and performance of feedback signal is evaluated in the context of a speech transcription task. Amazon Mechanical Turk (AMT) platform is used to transcribe speech utterances from different corpora. We show that in both clean (expert) and noisy (worker/turker) real-time feedback conditions the crowd workers are able to provide significantly more accurate transcriptions in a shorter time.
Riccardi G., Cimiano P., Potamianos A., and Unger C.
In the last two decades, information-seeking spoken dialog systems (SDS) have moved from research prototypes to real-life commercial applications. Still, dialog systems are limited by the scale, complexity of the task and coverage of knowledge required by problemsolving machines or mobile personal assistants. Future spoken interaction are required to be multilingual, understand and act on large scale knowledge bases in all its forms (from structured to unstructured). The Web research community have striven to build large scale and open multilingual resources (e.g. Wikipedia) and knowledge bases (e.g. Yago). We argue that a) it is crucial to leverage this massive amount of Web lightly structured knowledge and b) the scale issue can be addressed collaboratively and design open standards to make tools and resources available to the whole speech and language community.
Ludwig B., Haecker M., Schaeller R., Zenker B., Ivanov A. V. and Riccardi G.
Providing navigation assistance to users is a complex task generally consisting of two phases: planning a tour (phase one) and supporting the user during the tour (phase two). In the first phase, users interface to databases via constrained or natural language interaction to acquire prior knowledge such as bus schedules etc. In the second phase, often unexpected external events, such as delays or accidents, happen, user preferences change, or new needs arise. This requires machine intelligence to support users in the navigation realtime task, update information and trip replanning. To provide assistance in phase two, a navigation system must monitor external events, detect anomalies of the current situation compared to the plan built in the first phase, and provide assistance when the plan has become unfeasible. In this paper we present a prototypical mobile speech-controlled navigation system that provides assistance in both phases. The system was designed based on implications from an analysis of real user assistance needs investigated in a diary study that underlines the vital importance of assistance in phase two.
Francesconi F., Ghosh A., Riccardi G., Ronchetti M. and Vagin A.
In this paper we propose an approach to lightweight acquisition, sharing and annotation of experience-based corpora via mobile devices. Corpora acquisition is the crucial and often costly process in speech and language science and engineering. To address this problem, we have built a system for creating a location based corpora annotated with multimedia tags (e.g. text, speech, image) generated by end-users. We describe a relevant case study for the collection of mobile user life logs. We plan to make publicly available such tools and platforms to the research community for collaborative development and distributed experiential corpora collection.
Varges S., Riccardi G., Quarteroni S. , and Ivanov A. V.
Stepanov E. and Riccardi G.
Questionnaire-based surveys and on-line product reviews resemble each other in that they both have user comments and satisfaction ratings. Since a comment might be a general opinion about a product or only one or a set of its attributes, in which case the text might not reflect the rating; surveys and reviews share the problem of pairing freetext comments with these ratings. To train accurate models for automatic evaluation of products from free-text, it is important to distinguish these two kinds of opinions. In this paper we present experiments on detecting general opinions that target a product as a whole; thus, reflect the user sentiments better. The task is different from subjectivity detection, since the goal is to detect generality of an opinion regardless of the rest of the documents being opinionated or not. The task complements feature-based opinion analysis and opinion polarity classification, since it can be applied as a preceding step to both tasks. We show that when used as a classification feature user ratings are not useful in the general opinion detection task. However, they are effective in predicting the polarity of a comment once it is identified as a general opinion.
Moschitti A., Chu-Carroll J., Patwardhan S., Fan J. and Riccardi G.
The last decade has seen many interesting applications of Question Answering (QA) technology. The Jeopardy! quiz show is certainly one of the most fascinating, from the viewpoints of both its broad domain and the complexity of its language. In this paper, we study kernel methods applied to syntactic/semantic structures for accurate classification of Jeopardy! definition questions. Our extensive empirical analysis shows that our classification models largely improve on classifiers based on word-language models. Such classifiers are also used in the state-of-the-art QA pipeline constituting Watson, the IBM Jeopardy! system. Our experiments measuring their impact on Watson show enhancements in QA accuracy and a consequent increase in the amount of money earned in game-based evaluation.
Quarteroni S., Gonzalez M., Riccardi G. and Varges S.
Statistical user simulation is an efficient and effective way to train and evaluate the performance of a (spoken) dialog system. In this paper, we design and evaluate a modular data-driven dialog simulator where we decouple the “intentional” component of the User Simulator from the Error Simulator representing different types of ASR/SLU noisy channel distortion. While the former is composed by a Dialog Act Model, a Concept Model and a User Model, the latter is centered around an Error Model. We test different Dialog Act Models and Error Models against a baseline dialog manager and compare results with real dialogs obtained using the same dialog manager. On the grounds of dialog act, task and concept accuracy, our results show that 1) datadriven Dialog Act Models achieve good accuracy with respect to real user behavior and 2) data-driven Error Models make task completion times and rates closer to real data.
Quarteroni S. and Riccardi G.
Dialog acts represent the illocutionary aspect of the communication; depending on the nature of the dialog and its participants, different types of dialog act occur and an accurate classification of these is essential to support the understanding of human conversations. We learn effective discriminative dialog act classifiers by studying the most predictive classification features on Human-Human and Human-Machine corpora such as LUNA and SWITCHBOARD; additionally, we assess classifier robustness to speech errors. Our results exceed the state of the art on dialog act classification from reference transcriptions on SWITCHBOARD and allow us to reach a very satisfying performance on ASR transcriptions.
Varges S., Quarteroni S., Riccardi G. and Ivanov A. V.
We investigate the clarification strategies exhibited by a hybrid POMDP dialog manager based on data obtained from a phone-based user study. The dialog manager combines task structures with a number of POMDP policies each optimized for obtaining an individual concept. We investigate the relationship between dialog length and task completion. In order to measure the effectiveness of the clarification strategies, we compute concept precisions for two different mentions of the concept in the dialog: first mentions and final values after clarifications and similar strategies, and compare this to a rulebased system on the same task. We observe an improvement in concept precision of 12.1% for the hybrid POMDP compared to 5.2% for the rule-based system.
Gonzalez M., Quarteroni S., Riccardi G. and Varges S.
Statistical user simulation is a promising methodology to train and evaluate the performance of (spoken) dialog systems. We work with a modular architecture for data-driven simulation where the “intentional” component of user simulation includes a User Model representing userspecific features. We train a dialog simulator that combines traits of human behavior such as cooperativeness and context with domain-related aspects via the Expectation-Maximization algorithm. We show that cooperativeness provides a finer representation of the dialog context which directly affects task completion rate.
Dinarelli M.,Stepanov E.,Varges S. and Riccardi G.
We present a call routing application for complex problem solving tasks. Up to date work on call routing has been mainly dealing with call-type classification. In this paper we take call routing further: Initial call classification is done in parallel with a robust statistical Spoken Language Understanding module. This is followed by a dialogue to elicit further taskrelevant details from the user before passing on the call. The dialogue capability also allows us to obtain clarifications of the initial classifier guess. Based on an evaluation, we show that conducting a dialogue significantly improves upon call routing based on call classification alone. We present both subjective and objective evaluation results of the system according to standard metrics on real users.
Varges S., Quarteroni S., Riccardi G., Ivanov A. V. and Roberti P.
Spoken Language Understanding performs automatic concept labeling and segmentation of speech utterances. For this task, many approaches have been proposed based on both generative and discriminative models. While all these methods have shown remarkable accuracy on manual transcription of spoken utterances, robustness to noisy automatic transcription is still an open issue. In this paper we study algorithms for Spoken Language Understanding combining complementary learning models: Stochastic Finite State Transducers produce a list of hypotheses, which are re-ranked using a discriminative algorithm based on kernel methods. Our experiments on two different spoken dialog corpora, MEDIA and LUNA, show that the combined generative-discriminative model reaches the state-ofthe-art such as Conditional Random Fields (CRF) on manual transcriptions, and it is robust to noisy automatic transcriptions, outperforming, in some cases, the state-of-the-art.
Griol D., Riccardi G. and Sanchis E.
In this paper, we present an approach for the development of a statistical dialog manager, in which the system response is selected by means of a classification process which considers all the previous history of the dialog to select the next system response. In particular, we use decision trees for its implementation. The statistical model is automatically learned from training data which are labeled in terms of different SLU features. This methodology has been applied to develop a dialog manager within the framework of the European LUNA project, whose main goal is the creation of a robust natural spoken language understanding system. We present an evaluation of this approach for both human machine and human-human conversations acquired in this project. We demonstrate that a statistical dialog manager developed with the proposed technique and learned from a corpus of human-machine dialogs can successfully infer the task-related topics present in spontaneous humanhuman dialogs.
Varges S., Riccardi G., Quarteroni S., Ivanov A. V.
Conversational systems use deterministic rules that trigger actions such as requests for confirmation or clarification. More recently, Reinforcement Learning and (Partially Observable) Markov Decision Processes have been proposed for this task. In this paper, we investigate action selection strategies for dialogue management, in particular the exploration/exploitation trade-off and its impact on final reward (i.e. the session reward after optimization has ended) and lifetime reward (i.e. the overall reward accumulated over the learner’s lifetime). We propose to use interleaved exploitation sessions as a learning methodology to assess the reward obtained from the current policy. The experiments show a statistically significant difference in final reward of exploitation-only sessions between a system that optimizes lifetime reward and one that maximizes the reward of the final policy.
Varges S., Riccardi G., Quarteroni S., Ivanov A. V. and Roberti P.
We have developed a complete spoken dialogue framework that includes rule-based and trainable dialogue managers, speech recognition, spoken language understanding and generation modules, and a comprehensive web visualization interface. We present a spoken dialogue system based on Reinforcement Learning that goes beyond standard rule based models and computes on-line decisions of the best dialogue moves. Bridging the gap between handcrafted (e.g. rule-based) and adaptive (e.g. based on Partially Observable Markov Decision Processes - POMDP) dialogue models, this prototype is able to learn high rewarding policies in a number of dialogue situations.
We are interested in the problem of understanding human conversation structure in the context of human-machine and human-human interaction. We present a statistical methodology for detecting the structure of spoken dialogs based on a generative model learned using decision trees. To evaluate our approach we have used the LUNA corpora, collected from real users engaged in problem solving tasks. The results of the evaluation show that automatic segmentation of spoken dialogs is very effective not only with models built using separately human-machine dialogs or human-human dialogs, but it is also possible to infer the task-related structure of human-human dialogs with a model learned using only human-machine dialogs.
Sporka A. J., Jakub F. and Riccardi G.
Riccardi G., Mosca N., Roberti P. and Baggia P.
Riccardi G., Baggia P. and Roberti P.
Sebastian V., Riccardi G. and Quarteroni S.
Fogarolli A., Riccardi G. and Ronchetti M.
Varges S. and Riccardi G.
Riccardi G. and Baggia P.
Edizione della Normale di Pisa, 2006, 2006.
Potamianos A., Narayanan S. and Riccardi G.
Potamianos A., Narayanan S and Riccardi, G., 2005.
Hakkani-Tur D., Tur G., Riccardi G. and Kim H. K.
Bechet F., Riccardi G. and Hakkani-Tur D.
Riccardi G. and Hakkani-Tur D.
Lecture Notes in Computer Science, Springer-Verlag, , pp. 144 – 154, 2005, 2005.
Abstract. In this paper we investigate the role of user emotions in human-machine goal-oriented conversations. There has been a growing interest in predicting emotions from acted and non-acted spontaneous speech. Much of the research work has gone in determining what are the correct labels and improving emotion prediction accuracy. In this paper we evaluate the value of user emotional state towards a computational model of emotion processing. We consider a binary representation of emotions (positive vs. negative) in the context of a goal-driven conversational system. For each human-machine interaction we acquire the temporal emotion sequence going from the initial to the final conversational state. These traces are used as features to characterize the user state dynamics. We ground the emotion traces by associating its patterns to dialog strategies and their effectiveness. In order to quantify the value of emotion indicators, we evaluate their predictions in terms of speech recognition and spoken language understanding errors as well as task success or failure. We report results on the 11.5K dialog corpus samples from the How may I Help You? corpus.
Fabbrizio G., Dutton D., Gupta N., Hollister B., Rahim M., Riccardi G., Schapire R. and Schroeter J.
Gorin A., Abella A., Alonso T., Riccardi G. and Wright J.
IEEE Computer, vol. 35, n.4, pp. 51-56, April, 2002 (invited paper), 2002.
Engineers have long sought to design systems that understand and act upon spoken language. Extracting meaning from natural, unconstrained speech over the telephone is technically challenging, and quantifying semantic content is crucial for engineering and evaluating such systems.
Rahim M., Pieraccini R., Eckert W., Levin E., Di Fabbrizio G., Riccardi G., Kamm C., Narayanan S.
Riccardi G. and Gorin A. L.
IEEE Trans. on Speech and Audio, vol. 8, pp. 3-10, 2000, 2000.
We are interested in adaptive spoken dialog systems for automated services. Peoples’ spoken language usage varies over time for a given task, and furthermore varies depending on the state of the dialog. Thus, it is crucial to adapt automatic speech recognition (ASR) language models to these varying conditions. We characterize and quantify these variations based on a database of 30K user-transactions with AT&T’s experimental How May I Help You? spoken dialog system. We describe a novel adaptation algorithm for language models with time and dialog-state varying parameters. Our language adaptation framework allows for recognizing and understanding unconstrained speech at each stage of the dialog, enabling context-switching and error recovery. These models have been used to train state-dependent ASR language models. We have evaluated their performance with respect to word accuracy and perplexity over time and dialog states. We have achieved a reduction of 40% in perplexity and of 8.4% in word error rate over the baseline system, averaged across all dialog states.
Rahim M., Pieraccini R., Eckert W., Levin E., Di Fabbrizio G., Riccardi G., Lin C., Kamm C.
Riccardi G., Bangalore S. and Sarin P.
Gorin A. L. and Riccardi G.
Rose R. C. and Riccardi G.
Arai K., Wright J. H., Riccardi G. and Gorin A. L.
Speech Communication, vol. 27, no. 1, Jan. 1999, 1999.
Riccardi G., Potamianos A. and Narayanan S.
Rose R. C., Yao H., Riccardi G. and Wright J.
Riccardi G. and Bangalore S.
Arai K., Wright J., Riccardi G. and Gorin A.
Gorin A. L., Riccardi G. and Wright J. H.
Speech Communication, vol. 23, Oct. 1997, pp. 113-127., 1997.
We are interested in providing automated services via natural spoken dialog systems. By natural, we mean that the machine understands and acts upon what people actually say, in contrast to what one would like them to say. There are many issues that arise when such systems are targeted for large populations of non-expert users. In this paper, we focus on the task of automatically routing telephone calls based on a user’s fluently spoken response to the open-ended prompt of ‘‘How may I help you?’’. We first describe a database generated from 10,000 spoken transactions between customers and human agents. We then describe methods for automatically acquiring language models for both recognition and understanding from such data. Experimental results evaluating call-classification from speech are reported for that database. These methods have been embedded within a spoken dialog system, with subsequent processing for information retrieval and formfilling. q 1997 Elsevier Science B.V.
Riccardi G., Gorin A. L., Ljolje A. and Riley M.
Wright J. H., Gorin A. L. and Riccardi G.
Riccardi G., Pieraccini R. and Bocchieri E.
Computer Speech and Language, vol. 10(4), 1996, pp. 265-293, 1996.
Riccardi G., Bocchieri E. and Pieraccini R.

References: V. 
 V.

 V.

 V. 
 V.

 V.