Discovery Center - Gesture controlled robot

The goal of the project is to equip the Discovery Center of Idaho with their first fully functioning, motion actuated robotics exhibit. This exhibit will feature a robot arm that can be controlled using hand gestures as well as a variety of activities that users can complete based on different skill tiers (beginner, intermediate, and advanced). 
==Problem Definition==
The Discovery Center of Idaho is looking to incorporate interactive exhibits that expose K-12 students to modern robotics technology. This project features a four degree of freedom robot arm that can perform a number of sorting activities based on hand motion in free space, without direct physical contact with an actuator.
==Background==
Currently, the Discovery Center of Idaho does not have a robotics exhibit to show. This project will allow children and patrons of all ages to learn about robotics as well as interact with a live robot that they can control using only their hand gestures.
==Deliverables==
Fully assembled robot arm with leap motion sensor
Software program to control robot
Documentation on how to operate/ maintain/ troubleshoot the system 
==Specifications==
Robot arm must be compact and transportable
2-3 activities that the robot arm can perform based on user skill
 Beginner
 Intermediate
 Advanced 
User must be able to control robot using a Leap Motion controller
Activities must be able to be reset easily for the next user
Parts used (actuators, microcontrollers, etc.) must be easily replaceable and maintainable
Exhibit must be safe for all ages to operate
==Project Learning==
===Leap Motion===
The Leap Motion controller is a motion tracking device that can be used to track hand gestures. The controller is capable of tracking an individual's palm, forearm, and each individual finger joint. In addition, the sensor can track two hands at a time. This controller tracks hand gestures using two different cameras, and is found to perform better when overhead light is at a minimum. 
 300px | Leap motion controller tracking hand gestures
===Servo Diagram===
This diagram shows the configuration of the servos used to actuate the robot arm.
*Servo 1 controls the base rotation
*Servos 2 and 3 control shoulder movement
*Servos 4 and 5 control elbow movement
*Servo 6 controls the vertical movement of the wrist
*Servo 7 controls the wrist rotation
*Servo 8 controls the end effector
 300px
===System Diagram 1===
This diagram shows the system using a Arbotix micro-controller to control the servos and provide the robot with power. 
===System Diagram 2===
This diagram shows the system using a U2D2 USB converter, which would subvert the micro-controller to control the servos. 
===System Diagram 3===
This diagram shows the system using an XBee receiver for a wireless hand-held controller, replacing the Leap Motion.
===XBee wireless controller=== 
In order to learn how the robot responds to certain inputs, we opted to try and control the robot via a handheld remote. 
We bought the remote from the robot's manufacturer and used the code provided to gain insights into how we could eventually use 
the Leap Motion controller. 
==Final Design== 
For our final system design, we decided it would be best if we used the Arbotix micro-controller since it reduced
complexity and freed the U2D2 device to be used to ID other servos. 
===Final design system diagram===
File: UIDiscovery sd1.png 
===Software Diagram===
 700px 
==Project Progress==
===XBee===
We were able to successfully control the robot using the XBee controller and demonstrated this at snapshot #2. 
===Gripper Modification===
Since the size of our initial gripper was inadequate, we designed and 3D-printed a new, wider gripper so that we 
could grasp larger objects. 
	
 
===Inverse Kinematics functionality===
We were able to implement the use of inverse kinematics to move the robot arm using the Leap Motion controller by retrofitting the code
from the XBee controller to use the input coordinates from Leap Motion. 
===Gripper functionality===
We were able to control the gripper by tracking the thumb and other digits and then mapping the distance between the digits to a range of servo values.
===Full Functionality Testing===
The following photo shows the robot being operated and tested prior to Senior Design Expo.
 600px 
==Design Validation==
===Leap Motion===
Call Out 	Test 	Target Date	Completed 	Performance 	Result	Actions
Base Servo-Motor Actuation (Servo-Motor ID #1) 	Free Rotation, with the 300 Degrees of Motion, Using leap motion to rotate the arm 90 deg. w/ hand motion 	11/1/2018 	10/24/2018	Completed, with refinements on the way. Actuation occurs, though it could be improved. 	Good 	Purchase the U2D2 Converter, in hopes of reducing the amount of conversions needed. Look into reducing latency.
 	Rotation using Servo-Motor ID #1 on the base via Arduino Software. 	11/1/2018 	11/1/2018 	Completed, with refinements on the way. The Arm actuates by itself to the upright position, through Arduino Coding 	Good 	Continue progress, and look into actuating the arm through Inverse Kinematics through Leap Motion. Look into reducing latency.
 	Rotation using Servo-Motor ID #1 on the base via Arduino/C++ Code Modification 	2/13/2019 	1/30/2019 	Completed, base servo actuates, and moves side-to-side, mirroring user input. Latency isn't detrimental 	Good 	See if reaction speed can be improved, but more than anything, functionality is of highest priority
Shoulder Servo-Motor Actuation (Servo-Motor ID #2 & #3) 	Use Arduino Coding to activate the robotic arm into the upright position 	11/1/2018 	10/24/18 	Completed, with refinements on the way. The Arm actuates by itself to the upright position, through Arduino Coding 	Good 	Continue progress, and look into actuating the arm through Inverse Kinematics through Leap Motion
 	Lateral Motion upwards and downwards, following user input via C++ Coding to Leap Motion Controller 	2/13/2019 	1/30/19 	Completed, shoulder mirrors what the individual inputs to the Leap Motion Controller 	Good 	See if reaction speed can be improved, but more than anything, functionality is of highest priority
Elbow Servo-Motor Actuation (Servo-Motor ID #4 & #5) 	Using Arduino Coding to activate the robotic arm to the upright position. 	11/1/18 	11/1/18 	Completed, with refinements on the way. The Arm actuates itself to horizontal/vertical position, through Arduino Coding 	Good 	Continue progress, begin looking into how to make the arm actuate smoother, transition wise under Leap Motion Control
 	Using C++ Coding w/ Leap Motion Sensory device, we use upward and downward hand motions to test functionality 	2/13/19 	1/30/19 	Complete, the elbow mirrors user input, just like the base and shoulder. 	Good 	Refinements in the overall movement, via restrictions on certain movements and over-corrections, so that we can reduce risks of broken parts, and any unneeded wear and tear on the robotic arm
Wrist (Lateral Motion) Actuation (Servo-Motor ID #6) 	Using Arduino Coding to activate the robotic arm to and upright position. 	11/1/2018 	11/3/18 	Actuation occurs and it is seamless 	Good 	Continue progress, now see if we can attempt to do the same thing through inverse-kinematics (IK)
Wrist Rotation Actuation (Servo-Motor #-NULL) 	Removed w/ DCI's Permission, due to possible issues with miscommunication between Leap Motion Device and Hand Position	2/13/2019 	N/A 	Removed and replaced with 'Faux' Servo-Motor that maintains similar weight, and arm extension, without compromising gripper performance	Good 	N/A
Gripper Actuation (Servo-Motor ID #7) 	Use U2D2 converter to run Dynamixel Software, to see maximum and minimum gripper opening 	2/27/2019 	2/27/19 	Minimum and Maximum Gripper Operating width, found and specified within the code. 	Good 	None
 	Set limits in code, just below maximum and minimum for Servo #7, then test the code via Leap Motion Controller 	3/1/2019 	3/3/19 	Smoother Operation from Minimum to Maximum opening of the gripper 	Good 	Smooth function, though it differs for different block size. Make alterations to minimum gripper opening vaule, so that the servo isn't overworked
Implementation of U2D2 Converter Directly into Base Servo Motor 	Using Leap Motion and Inverse Kinematics Equations (w/ help of Dr. Perry) to actuate robotic arm 	N/A 	N/A 	N/A 	N/A 	No ability to actually implement such software due to the fact that it wouldn't necessarily provide better results than that of the Arduino/C++ Configuration already being run.
Implementation of the ArbotiX Commander v2.0 Controller 	Movement/Resolution and overall testing of the Servo-Motor's Actuation, before continual progress with Leap Motion 	11/29/2018 	11/28/18 	Completed, motion is somewhat lagging, might be able to increase reaction time 	Good 	Continue progress, and see what can be done before Snapshot Day #2, and introduction to Ralph. Gain insight from Arduino code and see how it can be implemented to make Leap Motion work
Unit Test for checking servo-motor actuation through automated program	Submit the code through either a C++ or Arduino format to test/debug any actuation errors within servo-motors 	2/22/2019 	2/27/19 	Servo-Motors work, and will provide any feedback by individually moving to specified test location. 	Good 	Keep this software for any troubleshooting tat may be needed in the future, for the group and for DCI.
===Micro-controller===
Call Out 	Test 	Target Date	Completed 	Performance 	Result	Actions
Base Servo-Motor Actuation (Servo-Motor ID #1) 	Actuates the Arm into the Upright Position, before rotation ensues 	11/1/2018 	10/24/2018	Completed, with refinements on the way. Actuation occurs, though it could be improved. 	Good 	Look into implementing the Leap Motion Device through the U2D2 Converter or ArbotiX Micro-Controller
Shoulder Servo-Motor Actuation (Servo-Motor ID #2 & #3)	Actuates the Arm into the Upright Position, before rotation ensues 	11/1/2018 	10/24/18 	Completed, with refinements on the way. The Arm actuates by itself to the upright position, through Arduino Coding 	Good 	Look into implementing the Leap Motion Device through the U2D2 Converter or ArbotiX Micro-Controller
Elbow Servo-Motor Actuation (Servo-Motor ID #4 & #5) 	Actuates the Arm into the Upright Position, before rotation ensues 	11/1/2018 	10/24/18 	Actuation occurs and is seamless with the hard coding 	Good 	Look into implementing the Leap Motion Device through the U2D2 Converter or ArbotiX Micro-Controller
Wrist (Lateral Motion) Actuation (Servo-Motor ID #6) 	Actuates the Arm into the Upright Position, before rotation ensues 	11/1/2018 	10/24/18 	Actuation occurs and is seamless with the hard coding 	Good 	Look into implementing the Leap Motion Device through the U2D2 Converter or ArbotiX Micro-Controller
Wrist Rotation Actuation (Servo-Motor #-NULL) 	See Explanation Above within Leap Motion Section 	N/A 	N/A 	N/A 	N/A 	N/A
Gripper Actuation (Servo-Motor ID #7) 	Actuates the Gripper laterally with Servo-Motor rotational motion 	2/27/2019 	3/3/19 	Actuation Occurs and is seamless with the hard coding 	Good 	Look into implementing specific movement of the Gripper, by limiting the closing of the Gripper, thus reducing Servo-Motor freezing
System Operation 	Implement IIR (Infinite Impulse Response) Filter Code, to help facilitate more seamless and smooth response w/o adding pauses within the system	4/2/2019 	4/5/19 	Actuation Occurs, is smoother, and is definitely refined. Works much better than the pauses that were within the code previously.	Good 	Make actuation even more seamless by tweaking the IIR Filter's equation. Change it slightly in one aspect of the robot arm, and see how the robot are actuates.
==Budget==
 1000px 
==Design Expo==
File: DCI expo booth.png 
File: DCI expo 2.png 
==Team Members==
File: DCI expo group.png 
thumb|left	Evangelos Stratigakes 
Major: Computer Science
Hometown: Boise, ID
Graduation Date: May, 2019 
Future Goals: After graduating, Evangelos wants to work as a software engineer, eventually moving into the field of cloud computing. 
Email: stra8803@vandals.uidaho.edu
thumb|left	Zhihui Wang 
Major: Mechanical Engineering
Hometown: Beijing, China 
Graduation Date: May, 2019 
Future Goals: After graduation, Zhihui is attending graduate school at the University of Idaho.
Email: wang6162@vandals.uidaho.edu
thumb|left	Chaeun Kim 
Major: Computer Science 
Hometown: Ulsan, South Korea 
Graduation Date: May, 2019 
Future Goals: After graduation, Chaeun wants to continue working as a software engineer, with hopes to gain experience in AI in automobiles. Chaeun also plans on furthering his academic career in the near future. 
Email: kim7885@vandals.uidaho.edu
thumb|left	Austyn Sullivan-Watson 
Major: Mechanical Engineering
Hometown: Star, ID 
Graduation Date: May, 2019 
Future Goals: After graduating, Austyn wants to work as a Mechanical Engineer within industry, with the hope of creating and/or improving automobile functionality.
Email: sull1271@vandals.uidaho.edu
==Additional Documentation==
===GitHub===
Code Repository
===Project Schedule===
 Fall Schedule
 Spring Schedule
===Presentations===
 Design EXPO Presentation
===Design EXPO Poster===
File: DCI Capstone Design Poster Final Design.pdf 
===Final Report===
 Final report 
===User Guide===
 User Guide