Package: panoramacli
PanoramaDevPythonCLIThe purpose of this package is to provide a CLI tool to facilitate Panorama developer creating a local application as well as being able to upload the application to the cloud using the CLI.DependenciesYou will need Docker and AWS CLI installed on your machine.
Docker is required for building a package and AWS CLI is needed for downloading a model from S3 and packaging the application to Panorama Cloud.Docker Setuphttps://docs.docker.com/get-docker/Since Panorama CLI builds ARM Docker images, it needs these extra steps on Linux to build cross platform images. Installing Docker Desktop on Mac should automatically handle cross platform builds.On Debian based distros$sudoapt-getinstallqemubinfmt-supportqemu-user-static
$sudodockerrun--rm--privilegedmultiarch/qemu-user-static--reset-pyesOn CentOS/RHEL based distros$sudoyuminstallqemubinfmt-supportqemu-user-static
$sudodockerrun--rm--privilegedmultiarch/qemu-user-static--reset-pyesAWS CLI Setuphttps://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.htmlAWS CLI version should be >=2.3.0 for v2 and >=1.21.0 for v1InstallPanorama CLI is only supported on Linux and macOS right now.$pip3installpanoramacliUpgradeTo upgrade to the latest release$pip3install--upgradepanoramacliCommandsBasic structure of the commands is as follows$panorama-cli<command>[optionsandparameters]To view help documentation, use one of the following$panorama-cli-h
$panorama-cli<command>-hDeploying a sample applicationInstructions for downloading and deploying a sample application can be found athttps://docs.aws.amazon.com/panorama/latest/dev/gettingstarted-deploy.htmlRelated Github RepositoriesDeveloper Guide -https://github.com/awsdocs/aws-panorama-developer-guideSample Applications -https://github.com/aws-samples/aws-panorama-samplesPanorama Docshttps://docs.aws.amazon.com/panorama/Application creation flow exampleThis is an example of a sample app which has two node packages. people_counter package has core logic for counting the number of people, call_node has the model which people_counter package uses. We will also add an abstract camera to the application which can be linked to a real camera from the console while deploying the application.$panorama-cliinit-project--nameexample_project
Successfullycreatedtheprojectskeletonat<path>

$cdexample_project

$panorama-clicreate-package--namepeople_counter

$panorama-clicreate-package--namecall_node--typeModelApplication StructureAt this point, the application structure looks as follows.graph.jsonundergraphsdirectory lists down all the packages, nodes and edges in this application. Nodes and Edges are the way to define an application graph in Panorama.package.jsonin each package has details about the package and the assets it uses. Interface definitions for the package need to be defined in this as well.
Model packagecall-nodehas adescriptor.jsonwhich needs to have the metadata required for compiling the model. More about this in the models section.
Inpeople_counterpackage which is the default i.e container type, all the implementation related files go into thesrcdirectory anddescriptor.jsonhas details about which command and file to use when the container is launched. More about container package management later.assetsdirectory is where all the assets reside. Developer is not expected to make any changes in this directory.├──assets
├──graphs
│└──example_project
│└──graph.json
└──packages├──accountXYZ-call_node-1.0│├──descriptor.json│└──package.json└──accountXYZ-people_counter-1.0├──Dockerfile├──descriptor.json├──package.json└──srcSetting up Cameras for PanoramaPanorama has a concept of Abstract Camera Package which the developers can use while developing their apps. This abstract camera package can be overriden and linked to an actual camera in the developer's Panorama account while deploying.Let's add an abstract camera to this application by running the following command.$ panorama-cli add-panorama-package --type camera --name front_door_cameraThis command defines an abstract camera package in thepackagessection and adds the following snippet in thenodessection of thegraph.json. You can modify the title and description to be more relevant to the use case.{"name":"front_door_camera","interface":"panorama::abstract_rtsp_media_source.rtsp_v1_interface","overridable":true,"launch":"onAppStart","decorator":{"title":"Camera front_door_camera","description":"Default description for camera front_door_camera"}}rtsp_v1_interfaceis the defined interface for an abstract camera and it has an output port calledvideo_outwhich can be used to forward the camera output to another node. More details about connecting this camera are discussed in theapp graph section belowPreparing a model for PanoramaRaw models are compiled using Sagemaker Neo on Panorama Cloud before being deployed onto the device. All models for this reason are paired with adescriptor.jsonwhich has the required meta deta for compiling the raw model on the cloud.Details about using Sagemaker Neo to compile models can be found athttps://docs.aws.amazon.com/sagemaker/latest/dg/neo-job-compilation.htmlAll the model info that is used to compile models on Sagemaker Neo are part of thedescriptor.json. Values used in this example are specific to the squeezenet model that is being used in this example.Since call_node has the model in this example, editpackages/accountXYZ-call-node-1.0/descriptor.jsonand add the following snippet into it.{"mlModelDescriptor":{"envelopeVersion":"2021-01-01","framework":"PYTORCH","inputs":[{"name":"data","shape":[1,3,224,224]}]}}Now we can add the model by passing in the path to the descriptor file which we just updated.If you want to download the model from S3 and then add it pass--model-s3-urias shown below. Otherwise just use--model-local-pathto pass the local model path instead.--packages-pathcan be used to pass all the packges where this model is being used and after downloading the model, DX CLI automatically adds the downloaded model into assets section of all the specified packages.$panorama-cliadd-raw-model--model-asset-namecallable_squeezenet--model-s3-uris3://<s3_bucket_path>/raw_models/squeezenet1_0.tar.gz--descriptor-pathpackages/accountXYZ-call_node-1.0/descriptor.json--packages-pathpackages/accountXYZ-call_node-1.0
download:s3://<s3_bucket_path>/squeezenet1_0.tar.gztoassets/callable_squeezenet.tar.gz
Successfullydownloadedcompiledartifacts(s3://<s3_bucket_path>/squeezenet1_0.tar.gz)to./assets/callable_squeezenet.tar.gz{"name":"callable_squeezenet","implementations":[{"type":"model","assetUri":"fd1aef48acc3350a5c2673adacffab06af54c3f14da6fe4a8be24cac687a386e.tar.gz","descriptorUri":"df53d7bc3666089c2f93d23f5b4d168d2f36950de42bd48da5fdcafd9dbac41a.json"}]}If you make any updates to your model ordesriptor.jsonfile after running this command, just re-run the command with the same--model-asset-nameand the old asset will be updated with the new assets.Writing code and building a containerpeople_counter package has the core logic to count the number of people, so let's create a file calledpeople_counter_main.pyatpackages/accountXYZ-people_counter-1.0/srcand add the relevant code to that.
Editpackages/accountXYZ-people_counter-1.0/descriptor.jsonto have the following content{"runtimeDescriptor":{"envelopeVersion":"2021-01-01","entry":{"path":"python3","name":"/panorama/people_counter_main.py"}}}descriptor.json basically provides the path for the command that needs to run and the path to the file that needs to be executed once the container starts.We can now build the package using the following command to create a container asset.$panorama-clibuild-container--container-asset-namepeople_counter_container_binary--package-pathpackages/accountXYZ-people-counter-package-1.0If you make any updates to your code ordesriptor.jsonorDockerfilefile after building a container, just re-run the command with the same--container-asset-nameand the old assets will be updated with the new assets.Defining interfaces and app graphLet's take a look at how thepackage.jsonlooks for people_counter package after running build-container{"nodePackage":{"envelopeVersion":"2021-01-01","name":"people_counter","version":"1.0","description":"Default description for package people_counter","assets":[{"name":"people_counter_container_binary","implementations":[{"type":"container","assetUri":"4e3a68e5fc0be9b7f3d8540a0a7d9855d6baae0b6dfc280b68431fd90b1e2c90.tar.gz","descriptorUri":"15545511b51d390a0a252537a41719498efd04f707deae17c6618d544e40e996.json"}]}],"interfaces":[{"name":"people_counter_container_binary_interface","category":"business_logic","asset":"people_counter_container_binary","inputs":[{"name":"video_in","type":"media"}],"outputs":[{"name":"video_out","type":"media"}]}]}}A new asset namedpeople_counter_container_binaryhas been added under assets and a new interface namedpeople_counter_container_binary_interfacehas been defined. In Panorama, interfaces are a way to programtically interact with a package and each interface is linked to an asset. For example,people_counter_container_binary_interfacehas an asset field which points topeople_counter_container_binary. That means that we are defining an interface to that asset. In this case, since our asset is a container with your code in it, all the inputs your code expects can be part of the inputs under interfaces. In this example, the code just expects one input which is a video stream. If output of your code needs to be consumed by another asset, that can be part of the ouputs. Similarly, a new interface was added to the call-node package when we canadd-raw-modelcommand. In that case, interface was linked to the model asset which we added using that command.At this point,graph.jsonunder the graphs directory looks like this{"nodeGraph":{"envelopeVersion":"2021-01-01","packages":[{"name":"accountXYZ::people_counter","version":"1.0"},{"name":"accountXYZ::call_node","version":"1.0"},{"name":"panorama::abstract_rtsp_media_source","version":"1.0"}],"nodes":[{"name":"front_door_camera","interface":"panorama::abstract_rtsp_media_source.rtsp_v1_interface","overridable":true,"launch":"onAppStart","decorator":{"title":"Camera front_door_camera","description":"Default description for camera front_door_camera"}},{"name":"callable_squeezenet","interface":"accountXYZ::call_node.callable_squeezenet_interface"},{"name":"people_counter_container_binary_node","interface":"accountXYZ::people_counter.people_counter_container_binary_interface","overridable":false,"launch":"onAppStart"}],"edges":[]}}packagessection here has all the packages that are part of this application and we can see thatnodessection has some nodes defined already. To be able to use any package, we need to define a corresponding nodes in thegraph.jsonfor all the interfaces that are part of the package.people_counter_container_binary_nodenode is linked topeople_counter_container_binary_interfaceinterface from people_counter package which we just looked at and similarlycallable_squeezenetnode is linked tocallable_squeezenet_interfaceinterface from the call_node package. We already discussed thefront_door_cameranode insetting up cameras sectionNext thing we will do is set up the edges for the application graph.people_counter_container_binary_interfacehad one inputvideo_inas part of the interface definition and that was the video input to the code in that package. We can connect that input to the camera node's output by adding the following edge under theedgessection."edges":[{"producer":"front_door_camera_node.video_out","consumer":"people_counter_container_binary_node.video_in"},]Registering and Uploading all local packages in the CloudWhen the application is ready, use the following command to upload all the packages to the cloud$panorama-clipackage-applicationDeploying the applicationAfter packaging the application, you can now use the graph.json from the package to start a deployment from the cloud!Panorama Application ConceptsContainer Package BasicsHandling implementation related filesThis is a directory tree of how an example container package. All the implementation related files for this package go into thesrcdirectory. In this package,people_counter_main.pyhas the logic for processing the frames from the camera andpeople_counter_main.pydepends onblueprint.csvandrequirements.jsonfor some of its functionality and therefore those are under thesrcdirectory as well. If the application requires multiple.pyfiles then all those will be under thesrcdirectory as well.accountXYZ-people_counter-1.0
├──Dockerfile
├──descriptor.json
├──package.json
└──src├──blueprint.csv├──people_counter_main.py└──requirements.jsonLet's now take a look at the Dockerfile provided as part of the packageFROMpublic.ecr.aws/panorama/panorama-applicationCOPYsrc/panoramaIn the second line, we are basically copying all the contents of thesrcdirectory into the/panoramadirectory of the Docker image. Therefore, its important to note that whenpeople_counter_main.pyaccesses other files which were originally part of thesrcdirectory, they are actually under/panoramawhen the application is running on the Panorama Appliance.Handling dynamic dataSince all the containers run in read-only mode on the Panorama Appliance, its not possible to create new files at all paths. To facilitate this, Panorama base image(i.e first line in Dockerfile) has two directories/opt/aws/panorama/logsand/opt/aws/panorama/storagewhich are empty. During runtime, these directories are mounted to the device file system and allow the developer to store new files and data dynamically./opt/aws/panorama/logsis the location to store all the logs and all files created in the directory are uploaded to CloudWatch for the account which was used to provision the device./opt/aws/panorama/storageis a good location to store all the dynamic info that the application might need.
When the device re-starts, all the memory locations are deleted but the data under these two directories is persistent and therefore should contain all the context for the application to function from where it left off on a reboot.Installing dependenciesThe image(public.ecr.aws/panorama/panorama-application) provided by Panorama is a ARM64v8 Ubuntu image with just Panorama base software installed so all the additional dependencies for the application must be installed separately. For example, add the following line to the Dockerfile to install OpenCV and boto3. This Dockerfile fetches the latestpanorama-applicationimage by default, to use a specific version of the image, refer tohttps://gallery.ecr.aws/panorama/panorama-applicationand tag the image in the Dockerfile with the version you're planning to use.FROMpublic.ecr.aws/panorama/panorama-applicationCOPYsrc/panoramaRUNpip3installopencv-pythonboto3Overriding Abstract CamerasWe added an Abstract Camera insetting up cameras section. If you're deploying an app through Panorama console, you will be automatically promted to replace the abstract camera node with a data source in your account. This section mentions how to create anoverride.jsonwhich can be used to replace abstract camera with a real camera while deploying applications from command line.Create a Camera node using the following command.--output-package-nameis the name of the camera package and--node-namespecifies the node name under this package. We are keeping the names for both of these asmy_rtsp_camerato keep it simple. Update--template-parameterswith the correct Username, Password and StreamUrl.$awspanoramacreate-node-from-template-job--template-typeRTSP_CAMERA_STREAM--output-package-namemy_rtsp_camera--output-package-version"1.0"--node-namemy_rtsp_camera--template-parameters'{"Username":"admin","Password":"123456","StreamUrl": "rtsp://<url>"}'This command will return an output like{"JobId":"d1d81752-d8ab-4131-8e48-8cf638685e71"}Let's make sure camera was created successfully by running the following command using the job id from above.$awspanoramadescribe-node-from-template-job--job-idd1d81752-d8ab-4131-8e48-8cf638685e71{"JobId":"d1d81752-d8ab-4131-8e48-8cf638685e71","Status":"SUCCEEDED","CreatedTime":"2021-10-11T13:22:51.284000-07:00","LastUpdatedTime":"2021-10-11T13:22:51.284000-07:00","OutputPackageName":"my_rtsp_camera","OutputPackageVersion":"1.0","NodeName":"my_rtsp_camera_node","NodeDescription":"my_rtsp_camera_node","TemplateType":"RTSP_CAMERA_STREAM","TemplateParameters":{"Password":"SAVED_AS_SECRET","StreamUrl":"rtsp://<url>","Username":"SAVED_AS_SECRET"}}Since the status says succeeded, we are now ready to use this camera. Now, let's create anoverride.jsonfor this camera. Here we are replacingfront_door_cameranode which we created insetting up cameras sectionwith the newly createdmy_rtsp_camera{"nodeGraphOverrides":{"envelopeVersion":"2021-01-01","packages":[{"name":"accountXYZ::my_rtsp_camera","version":"1.0"}],"nodes":[{"name":"my_rtsp_camera_node","interface":"accountXYZ::my_rtsp_camera.my_rtsp_camera","overridable":true,"overrideMandatory":false,"launch":"onAppStart"}],"nodeOverrides":[{"replace":"front_door_camera","with":[{"name":"my_rtsp_camera_node"}]}]}}Processing streams from multiple cameras togetherIf you want to process streams from two or more cameras together, you can also replace thefront_door_cameranode with multiple cameras. For processing two streams together for example, create a second camera using the steps mentioned above and use the following override file.{"nodeGraphOverrides":{"envelopeVersion":"2021-01-01","packages":[{"name":"accountXYZ::my_rtsp_camera","version":"1.0"},{"name":"accountXYZ::my_other_camera","version":"1.0"}],"nodes":[{"name":"my_rtsp_camera_node","interface":"accountXYZ::my_rtsp_camera.my_rtsp_camera","overridable":true,"overrideMandatory":false,"launch":"onAppStart"},{"name":"my_other_camera_node","interface":"accountXYZ::my_other_camera.my_other_camera","overridable":true,"overrideMandatory":false,"launch":"onAppStart"}],"nodeOverrides":[{"replace":"front_door_camera","with":[{"name":"my_rtsp_camera_node"},{"name":"my_other_camera_node"}]}]}}Using multiple camerasAt the end of last section, we looked at how to modify the override file we want to process streams from  multiple cameras together. This section speaks about using multiple camera streams in the same application and processing them separately. This is useful when multiple cameras are being used for different purposes.Let's take thepackage.jsonof people_counter package from thedefining interfaces sectionand modify it to have two video inputs.{"nodePackage":{"envelopeVersion":"2021-01-01","name":"people_counter","version":"1.0","description":"Default description for package people_counter","assets":[{"name":"people_counter_container_binary","implementations":[{"type":"container","assetUri":"4e3a68e5fc0be9b7f3d8540a0a7d9855d6baae0b6dfc280b68431fd90b1e2c90.tar.gz","descriptorUri":"15545511b51d390a0a252537a41719498efd04f707deae17c6618d544e40e996.json"}]}],"interfaces":[{"name":"people_counter_container_binary_interface","category":"business_logic","asset":"people_counter_container_binary","inputs":[{"name":"video_in_1","type":"media"},{"name":"video_in_2","type":"media"}],"outputs":[{"name":"video_out","type":"media"}]}]}}Let's add another camera to the application by using the following command.$ panorama-cli add-panorama-package --type camera --name back_door_cameraA node namedback_door_camerawill be added into thenodessection ofgraph.jsonand let us connect both the cameras to the video inputs defined above in theedgessection as shown below.{"nodeGraph":{"envelopeVersion":"2021-01-01","packages":[{"name":"accountXYZ::people_counter","version":"1.0"},{"name":"accountXYZ::call_node","version":"1.0"},{"name":"panorama::abstract_rtsp_media_source","version":"1.0"},],"nodes":[{"name":"front_door_camera","interface":"panorama::abstract_rtsp_media_source.rtsp_v1_interface","overridable":true,"launch":"onAppStart","decorator":{"title":"Camera front_door_camera","description":"Default description for camera front_door_camera"}},{"name":"callable_squeezenet","interface":"accountXYZ::call_node.callable_squeezenet_interface"},{"name":"people_counter_container_binary_node","interface":"accountXYZ::people_counter.people_counter_container_binary_interface","overridable":false,"launch":"onAppStart"},{"name":"back_door_camera","interface":"panorama::abstract_rtsp_media_source.rtsp_v1_interface","overridable":true,"launch":"onAppStart","decorator":{"title":"Camera back_door_camera","description":"Default description for camera back_door_camera"}}],"edges":[{"producer":"front_door_camera_node.video_out","consumer":"people_counter_container_binary_node.video_in_1"},{"producer":"back_door_camera_node.video_out","consumer":"people_counter_container_binary_node.video_in_2"},]}}Viewing output on a HDMIIn this people counter example application, if we also want to draw bounding boxes around people and view those processed frames on a screen, we can do that as well by adding a Data Sink node. Data Sink node forwards the input it receives to the HDMI port.Like the abstract camera package, Panorama also provides a data sink package and we can create a data_sink using the following command.$ panorama-cli add-panorama-package --type data_sink --name data_sink_nodeThis command adds the following node in thenodessection ofgraph.json{
    "name": "data_sink_node",
    "interface": "panorama::hdmi_data_sink.hdmi0",
    "overridable": false,
    "launch": "onAppStart"
}We can now connect thisdata_sink_nodeto the output ofpeople_counter_container_binary_nodein theedgessection. At this point,graph.jsonlooks as follows.{"nodeGraph":{"envelopeVersion":"2021-01-01","packages":[{"name":"accountXYZ::people_counter","version":"1.0"},{"name":"accountXYZ::call_node","version":"1.0"},{"name":"panorama::abstract_rtsp_media_source","version":"1.0"},{"name":"panorama::hdmi_data_sink","version":"1.0"}],"nodes":[{"name":"front_door_camera","interface":"panorama::abstract_rtsp_media_source.rtsp_v1_interface","overridable":true,"launch":"onAppStart","decorator":{"title":"Camera front_door_camera","description":"Default description for camera front_door_camera"}},{"name":"callable_squeezenet","interface":"accountXYZ::call_node.callable_squeezenet_interface"},{"name":"people_counter_container_binary_node","interface":"accountXYZ::people_counter.people_counter_container_binary_interface","overridable":false,"launch":"onAppStart"},{"name":"data_sink_node","interface":"panorama::hdmi_data_sink.hdmi0","overridable":false,"launch":"onAppStart"}],"edges":[{"producer":"front_door_camera_node.video_out","consumer":"people_counter_container_binary_node.video_in"},{"producer":"people_counter_container_binary_node.video_out","consumer":"data_sink_node.video_in"}]}}