Package: cltl.combot
CLTL Leolani CombotCLTL Leolani Combot provides the framework for applications that implement human-robot interaction with conversation.About the ProjectThis is the successor of theLeolani platformwith an improved modular architecture.ApplicationsClone one of the application parents from this project space and follow the instructions there to run them:Eliza appGet to know you appLeolani-mmaiComponentsCurrently, the following components are implemented for the framework:Application RepositoryRepository to share artifacts and external dependencies between components.EMISSORRepresentation of interaction data.BackendHardware integration and signal generation.Automatic Speech Recognition (ASR)Transcription of audio signals to text.Voice Activity Detection (VAD)Detection of speech in audio signals.Face RecognitionCurrently includes face detection, age-gender detection, face recognitionChat UISimple Chat client to display and interact with the conversationEliza chatEliza based chat.Get To Know You (G2KY) chatEstablish name based on face recognition.About-agentAnswer questions about the agent itself.Knowledge extractionExtracts factoid triples and perspectives from statements and gives back responses or it extracts SPARQL queries from questions and generates answers.Knowledge linkingResolves IRIs for mentions and perceptions of things and people such that triples are augmented with IRIs.Knowledge representationThe models and functions to support the episodic Knowledge Graph.Language generationGenerates texts from triples.Mention detectionDetects mentions of entities and visual objects in text.Object recognitionDetects people and objects in images.Question processorAnswers questions about the world through the Internet.Visual responderAnswers questions to the visual context.To create a new component follow the instructions in thetemplate component.Getting StartedRunning an applicationClone one of the aboveapplicationsand follow the instructions there to run them.PrerequisitesPythonMost of the repositories require at least Python 3.8.One option to manage Python versions is to usepyenv. Note, however, thatpyenvdoesn't
work well together with anaconda. To detect if you are using anaconda, runwhich python. If you use anaconda, you can use that to manage your Python
version.makeTo build the application,makeis used.On OS X it is recommeded to upgrademake. Since OS X doesn't use standard GNU
utils due their restrictive licence, defaultmakeon OS X is way outdated.One option is to use homebrew:brew install makeand add the installedgmakecommand by addingPATH="$(brew --prefix)/opt/make/libexec/gnubin:$PATH"to your~/.zshrcDockerTo be added.DevelopmentTo work on the development of a specificapplication, start from the parent repository
and follow the steps described below. The description uses theEliza appas example.Check-outTo check out all code needed for the Eliza App, follow the instructions in theEliza app.Build and run the applicationThe application is structured into separate components which have their owngitrepositories
and can be run as separate Python applications. The parent repository of the application contains
all those component repositories asgitsubmodules.There is a central application (cltl-eliza-app) that configures and runs
all the necessary components it needs, either inside a Python application or as containerized services in aKubernetescluster or usingdocker compose. To run the application, first all components need to be packaged and made available
to the application. For this purpose there aremakefilesavailable in the components and the application parent that
automate this process. To build the application runmake buildfrom theparent repository. This command will download external dependencies tocltl-requirements, setup virtual environments for all components, package
them and publish the packages tocltl-requirementsto make them available
to the application and other components.To run the application follow the instructions in theEliza parent.Make changes to the codeIndividual components in the parent repository are edited and committed separately, and, after a stable version is
reached, the state of the components is commited in the parent repository, for the workflow seeWorking with git submodules.Modularization allows developing components in isolation. The application and other components depend on a packaged
version of a component only, therefore changes will become available outside of the component only after rebuilding
the application, see above.To use PyCharm for development see the instructions inWorkflow using PyCharm.To commit changes made to the application see the instructions inWorking with git submodules.Adding a new componentTo add a new component to an application follow the instruction in thetemplate component.Create a new applicationHOWTOsWorkflow using PyCharmWorking with git submodulesSetup a new componentAdd a component to a Python appContent of this repositoryThis repo provides infrastructre and general code for the platform:InfrastructureThecltl.combot.inframodule contains library code for infrastructre used in
the application.Event busComponents of the application can communicate via an event bus. Thecltl.combot.infra.eventmodule provides the interface and different implementations
of the event bus.Thecltl.combot.infra.topic_workermodule provides a convenience class to
implement the subscription to one or multiple topics in the event bus.Configuration managerConfiguration is made available in the application via a configuration manager.
Thecltl.combot.infra.configmodule provides the interface and different
implementations of the configuration manager.Resource managerAccess to resources in the application is made available via a resource manager.
This includes providing resources and waiting for resources to become available
as well as managing access to shared resources. Thecltl.combot.infra.resourcemodule provides the interface and different implementations of the resource
manager.Time utilThecltl.combot.infra.time_utilmodule provides time related utilities to
ease the usage of a consistent time format throughout the application.Dependency injectionThecltl.combot.infra.di_containermodule provides a simple utility to use
dependency inject in the application.Common librariesTo be added.Events based on EMISSORThecltl.combot.eventmodule contains common event payloads.ContributingContributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make aregreatly appreciated.Fork the ProjectCreate your Feature Branch (git checkout -b feature/AmazingFeature)Commit your Changes (git commit -m 'Add some AmazingFeature')Push to the Branch (git push origin feature/AmazingFeature)Open a Pull RequestLicenseDistributed under the MIT License. SeeLICENSEfor more information.AuthorsTaewoon KimThomas BaierSelene Báez SantamaríaPiek Vossen