Package: cognifly
cognifly-pythonControl the CogniFly open-source drone remotely from your python script.Quick linksDrone setup instructionsInstallationUsageManual controlPython APIPro APISchool APIStreamingTroubleshootingAdvancedCustom estimator and PIDsInstallationPlease followdrone setup instructionsto properly set up the flight controller and raspberry pi.cognifly-pythoncan be installed from PyPI.
This is done by simply executing the following on both the Raspberry Pi and the remote-controlling computer:pip3install--upgradepip
pip3installcogniflyUsageIn order to use the installed library, thecognifly_controller.pyscript must first be running on Cognifly.
This can be done by connecting to the drone through SSH, and executing the following command:cognifly-controllerWe advise setting up a service on the rapsberry pi to launch this script automatically on CogniFly at startup, so that the user doesn't need to SSH the drone.Note: if using our Raspbian image on the Raspberry Pi, this service is set up already.Note: On the Raspberry Pi, thecognifly-controllercommand may not become available immediately after installation. If not, try closing the terminal and opening a new one.
Worst case scenario, this command is an alias forpython3 -m cognifly controller.Manual controlPS4 controller:We recommend using a PS4 bluetooth gamepad for manual control of the drone, as this is pretty fun and allows you to use the drone everywhere.First, pair your PS4 controller with the Raspberry Pi (you need to do this only once):ssh the Raspberry Pi:ssh pi@my_drone_hostname.localinstall the prerequisite:sudo apt install bluetooth pi-bluetooth bluezreboot the pi:sudo rebootssh again:ssh pi@my_drone_hostname.localexecutesudo bluetoothctlconfigure the bluetooth agent by executing the following commands:agent on
discoverable on
pairable on
default-agentlaunch the scan for available bluetooth devices:scan onstart bluetooth pairing on the PS4 controller by pressingshare, then thePSbuttons simultaneously until a white light flashesin the scan output, look for something like[NEW] Device AC:FD:93:14:25:D3 Wireless Controllercopy the MAC address (AC:FD:93:14:25:D3in this example but yours will differ)turn off scanning:scan offpair:pair XX:XX:XX:XX:XX:XXby replacingXX:XX:XX:XX:XX:XXwith your copied MAC addressconnect to the controller:connect XX:XX:XX:XX:XX:XX(replacingXX:XX:XX:XX:XX:XXwith your copied MAC address)the light stops flashing and turns bluetrust the controller:trust XX:XX:XX:XX:XX:XX(replacingXX:XX:XX:XX:XX:XXwith your copied MAC address)Once this is done, you will not need to ssh the drone anymore; just turn your PS4 controller on and it will connect to the Raspberry Pi automatically (it may sometimes take several trials).Controls:Note: when taking control over the API, the flight controller of the API is disabled but theDISARMemergency button and the video stream will still work.
When releasing control to the API, theDISARMbuttons of the PS4 controller will also keep working.
By default, when a PS4 controller is connected, it takes control over the API.
TheGamepadindicator in the GUI monitors this.Keyboard:It is possible to manually control the drone with the keyboard via SSH, by focusing the session that executescognifly_controller.pyon the raspberry pi:A: armD: disarmT: take offL: land8: forward5: backward7: left9: right4: left yaw6: right yawpageup: uppagedown: downR: reset the board and exit the scriptControl though the python APIThe remote control API is defined incognifly_remote.py(please read the docstrings for thorough documentation).Connecting to the drone is as simple as creating aCogniflyobject.
By default, this will also pop a simple Graphic User Interface:fromcogniflyimportCognifly# connect to the drone and pop the GUI:cf=Cognifly(drone_hostname="my_drone_name.local")time.sleep(10.0)The API is divided into a "pro" and a "school" API.Pro APIThe "pro" API is fairly simple and is what you should use for serious applications.
It enables the user to control Cognifly either by velocity or by position, in two possible coordinate systems:world frame: X and Y relative to the starting point of the drone, Yaw relative to the starting orientation of the drone, and Z relative to the ground,drone frame: X, Y and Yaw relative to the current position and orientation of the drone, and Z relative to the ground.Example-script using the "pro" API for control:importtimefromcogniflyimportCognifly# create a Cognifly object (resets the controller):cf=Cognifly(drone_hostname="my_drone_name.local")# arm the drone:cf.arm()time.sleep(1.0)# take off to default altitude :cf.takeoff_nonblocking()time.sleep(10.0)# go 0.2 m/s frontward for 1.0 second:cf.set_velocity_nonblocking(v_x=0.2,v_y=0.0,v_z=0.0,w=0.0,duration=1.0,drone_frame=True)time.sleep(2.0)# go rightward and upward while rotating for 1.0 second:cf.set_velocity_nonblocking(v_x=0.0,v_y=0.2,v_z=0.1,w=0.5,duration=1.0,drone_frame=True)time.sleep(2.0)# retrieve battery, pose, speed and health flags:telemetry=cf.get_telemetry()print(f"telemetry:\n{telemetry}")# go to (-0.5, -0.5, 0.5) and back to the initial yaw (0.0) at a max speed of 0.25 m/s, 0.5 rad/s:cf.set_position_nonblocking(x=-0.5,y=-0.5,z=0.5,yaw=0.0,max_velocity=0.25,max_yaw_rate=0.5,max_duration=10.0,relative=False)time.sleep(5.0)# go frontward for 0.5 m at 0.25 m/s, staying at an altitude of 0.5 m:cf.set_position_nonblocking(x=0.5,y=0.0,z=0.5,yaw=0.0,max_velocity=0.25,max_yaw_rate=0.5,max_duration=10.0,relative=True)time.sleep(5.0)# go back home:cf.set_position_nonblocking(x=0.0,y=0.0,z=0.5,yaw=0.0,max_velocity=0.25,max_yaw_rate=0.5,max_duration=10.0,relative=False)time.sleep(5.0)# land:cf.land_nonblocking()time.sleep(2.0)# disarm the dronecf.disarm()# reset the drone (reinitializes coordinate system):cf.reset()Note that this API is non-blocking, a new call will override the previous call
(position and velocity are controlled by PIDs, so violent changes WILL make the drone unstable).School APIThe "school" API is an overlay of the "pro" API, built for students who need an easy and relatively safe API for class purpose.
It is vastly inspired from theeasytellolibrary, of which it reproduces most of the interface, adapted to Cognifly.
Contrary to the "pro" API, calls to the "school" API are blocking and return only when the command has been fully performed (or when it times out).
It mostly consists of hidden calls to the position control "pro" API with an additional hidden callback through thesleep_until_donemethod (see the code to reproduce similar behavior with the "pro" API).Whereas the units of the "pro" API are meters and radians, the units of the "school" API are centimeters and degrees.Example using the "school" API for control:fromcogniflyimportCognifly# create a Cognifly object (resets the controller):cf=Cognifly(drone_hostname="my_drone_name.local")# take off (resets the controller):cf.takeoff()# go forward for 50 cm:cf.forward(50)# turn clockwise by 90 degrees:cf.cw(90)# turn counter-clockwise by 45 degrees:cf.ccw(45)# other movements:cf.backward(20)cf.up(30)cf.down(20)cf.right(20)cf.left(10)# go to (0, 0, 0.5) (cm) with a yaw of 90Â° counter-clockwise compared to the initial orientationcf.go(0,0,0.5,-90)# sequence of position targets (when 4 items, the last is yaw):cf.position_sequence([[0.2,0.2,0.5,0.0],[0.0,0.1,0.5],[0.0,0.0,0.5]])# get telemetry:battery=cf.get_battery()height=cf.get_height()speed=cf.get_speed()x,y,z=cf.get_position()vx,vy,vz=cf.get_velocity()# land (disarms the drone):cf.land()StreamingCognifly can stream from the raspberry pi camera (note: frames are transferred directly through the local network).
First make sure that the camera is enabled in the raspberry pi, and that it works correctly.It is possible to display the video or to retrieve frames for processing:fromcogniflyimportCognifly# create a Cognifly object (resets the controller):cf=Cognifly(drone_hostname="my_drone_name.local")# take off (resets the controller):cf.takeoff()# display the stream at 24 fps:cf.stream(fps=24)time.sleep(10.0)# stop the stream:cf.streamoff()time.sleep(5.0)# turn the stream on at 5 fps, with no display:cf.streamon(fps=5)# retrieve a frame for processing:cv2_image=cf.get_frame()# turn the stream off:cf.streamoff()# land:cf.land()TroubleshootingDrift: A slight horizontal drift of less than 1cm/s is to be expected.
However, if the drone drifts badly, disarm it, move it around and check that the position and velocity estimates make sense.If some estimates remain fixed: the drone is probably not in EST_POS debug mode. Carefully setup the flight controller again, according to thedrone setup instructions.If some estimates behave crazily: the floor is probably not textured enough. The current iteration of CogniFly uses a cheap optical flow sensor to estimate its location, and this sensor needs a lot of texture on the ground to work properly. See thecustom estimatorsection to circumvent this issue.Advanced usageCustom estimator and PIDscognifly-pythonsupports custom estimators.
A custom estimator overrides the position and velocity estimates that come from the flight controller, as these can be very poor when the ground is textureless or badly lit, due to the optical flow sensor performing poorly.In order to design and use your own custom estimator, you must not use thecognifly-controllerbash command, but instead write a python script in which you instantiate aCogniflyControllerobject, passing a customPoseEstimatorto thepose_estimatorargument.
Doing this also enables customizing the PID values without resorting to the API.This can be achieved as follows:# Script to be executed on the drone, instead of the cognifly-controller bash commandfromcognifly.cognifly_controller.cognifly_controllerimportCogniflyController,PoseEstimator# Definition of a custom estimator:classMyCustomEstimator(PoseEstimator):defget(self):"""Must return a tuple of 8 values:(pos_x_wf, pos_y_wf, pos_z_wf, yaw, vel_x_wf, vel_y_wf, vel_z_wf, yaw_rate)If any is None, this is considered failure and the onboard estimator will be used instead.These values represent the drone attitude in the world frame:pos_x_wf = x position (m)pos_y_wf = y position (m)pos_z_wf = z position (m)yaw: yaw (rad)vel_x_wf = x velocity (m/s)vel_y_wf = y velocity (m/s)vel_z_wf = z velocity (m/s)yaw_rate: yaw rate (rad/s)CAUTION: your estimator needs to respect the cognifly coordinate system, which is not standard:x: FORWARDy: RIGHTz: UP"""# compute pos_x_wf, pos_y_wf, pos_z_wf, yaw, vel_x_wf, vel_y_wf, vel_z_wf, yaw_rate herereturnpos_x_wf,pos_y_wf,pos_z_wf,yaw,vel_x_wf,vel_y_wf,vel_z_wf,yaw_rateif__name__=='__main__':# instantiate of the custom estimator:ce=MyCustomEstimator()# instantiate of the CogniflyController object:cc=CogniflyController(print_screen=True,# set to false if you want to run headlesspose_estimator=ce,# your custom estimatortrace_logs=False,# set to True to save PID logsvel_x_kp=750.0,# proportional gain for X velocityvel_x_ki=200.0,# integral gain for X velocityvel_x_kd=10.0,# derivative gain for X velocityvel_y_kp=750.0,# proportional gain for Y velocityvel_y_ki=200.0,# integral gain for Y velocityvel_y_kd=10.0,# derivative gain for Y velocityvel_z_kp=5.0,# proportional gain for Z velocityvel_z_ki=2.0,# integral gain for Z velocityvel_z_kd=0.05,# derivative gain for Z velocityvel_w_kp=75.0,# proportional gain for yaw ratevel_w_ki=50.0,# integral gain for yaw ratevel_w_kd=0.0,# derivative gain for yaw ratepid_limit=400,# PID clipping value, should not exceed 500x_vel_gain=0.5,# gain from X vector to X velocity (position control)y_vel_gain=0.5,# gain from Y vector to Y velocity (position control)z_vel_gain=0.2)# gain from Z vector to Z velocity (position control)# run the controller:cc.run_curses()In case you need the estimates of the flight controller in your custom estimator (e.g., to merge them with your own estimates in a Kalman filter), thePoseEstimatorclass has aget_fc_estimate()method for that purpose:defget(self):pos_x_wf,pos_y_wf,pos_z_wf,yaw,vel_x_wf,vel_y_wf,vel_z_wf,yaw_rate=self.get_fc_estimate()# do your stuff here# (note that the fc estimates will be None before run_curses())returnpos_x_wf,pos_y_wf,pos_z_wf,yaw,vel_x_wf,vel_y_wf,vel_z_wf,yaw_rateNote 1:cognifly-pythondoes not entirely override the flight controller estimates internally.
Instead, it consists of an external control loop.
The internal control loop performed within the flight controller is not altered, and still uses the flight controller estimates.
Thus, the drone behavior may still differ depending on the texture of the ground.Note 2:mode 2of the gamepad uses your custom estimator, whereasmode 1shuts down the external control loop.
You can usemode 2to debug your estimator.
Thetrace_logsargument can also be set toTruefor tuning your PIDs.
Logs will then be saved undercognifly/cognifly_controller/trace_logs.:warning:Caution: taking off with the gamepad inmode 2momentarily shuts down the external control loop.
However, you can take off using your estimator thanks to the API, usingtakeoff_nonblocking(track_xy=True).