TBM2 - Welcoming Visitors

## Introduction

Recognise visitors at the door and take appropriate action depending on who they are.

## Approach

The robot waits for the doorbell to be rung, then navigates to the front door where it instructs the visitor to stand in front of the camera. The robot detects faces in the video stream, which it matches against training images of Dr. Kimble using a similarity threshold. If Dr. Kimble is not recognised then the robot attempts to recognise the postman or deliman using statistical properties (e.g. median red, green and blue) of the chest region (e.g. beneath the detected face).

If the robot recognises Dr. Kimble, it leads him to the bedroom, then waits for him to leave. The robot detects that Dr. Kimble has left the bedroom by looking for change in the laser scan data. The robot then navigates back to the front door and wishes Dr. Kimble goodbye.

If the robot recognises the postman, it instructs him to open the door and leave the parcel on the hallway floor. It then wishes him goodbye.

If the robot recognises the deliman, it leads him to the kitchen then instructs him to leave the breakfast box on the table. The robot then navigates back to the front door and wishes the deliman goodbye.

Otherwise the robot indicates to the visitor that it doesn't recognise them and cannot open the door.

The robot ends the interaction by returning to its home position.

## Installation Instructions

* axis\_camera:
	1. git clone https://github.com/ros-drivers/axis\_camera.git
	2. recompile
	3. source ~/tb\_ws/devel/setup.bash
* opencv (???)

## Run Instructions

1. create a map:
	1. roslaunch hearts\_navigation hearts\_navigation\_map\_begin.launch
	2. edit map\_name parameter in hearts\_navigation\_map\_end.launch (optional)
	3. roslaunch hearts\_navigation hearts\_navigation\_map\_end.launch
	4. edit map\_name parameter in hearts\_navigation\_navigate.launch
	5. to match hearts\_navigation\_map\_end.launch (TODO: remove parameter from tbm2\_welcoming\_visitors.launch)
2. input coordinates of points of interest to locations.json:
	1. rosrun hearts\_navigation qeconverter.py
	2. roslaunch hearts\_navigation hearts\_navigation\_navigate.launch
	3. set 2D pose estimate
	4. ensure joystick has control of TIAGo, then set 2D nav goal at point of interest
	5. copy coordinates into locations.json
	6. repeat steps 4 and 5 for all points of interest (home, hallway, entrance, outside bedroom)
3. set benchmarking box parameters:
	1. edit rsbb\_key and rsbb\_host parameters of roah\_rsbb\_comm node in tbm2\_welcoming\_visitors.launch
4. configure Axis camera:
	1. edit hostname, username and password parameters of axis\_driver nodes in hearts\_uniform.launch and hearts\_face\_uniform\_reg.launch
5. train face recogniser:
	1. roslaunch hearts\_face\_uniform\_reg hearts\_face\_uniform\_reg.launch
	2. have Dr. Kimble stand in front of the camera - making small circular head movements - until the training data have been captured and the launch file has terminated
	3. optionally change the sensitivity threshold in macros.hpp and recompile
6. train uniform recogniser:
	1. roslaunch hearts\_face\_uniform hearts\_face\_uniform.launch
	2. have the deliman and postman stand in front of the camera, then get the min/max ranges of the statistical parameters over several image frames
	3. edit the macros.hpp file with the min/max ranges and recompile
7. run launch file:
	1. roslaunch tbm2\_welcoming\_visitors tbm2\_welcoming\_visitors.launch

## Dependencies

* hearts\_face\_uniform: face and uniform recognition
* hearts\_face\_uniform\_reg: face registration (training)
* hearts\_tts: text-to-speech
* hearts\_navigation: navigation