Installation of RedHawk™ 8.4.7 Gold on the
NVIDIA® Jetson AGX Orin
Release Notes
September 18th, 2023
0898003-8.4.7-Orin

This page intentionally left blank1. Introduction
This document describes the process of installing the ARM64 version of RedHawk™ 8.4.7 on the
Jetson AGX Orin. The directions in this document supersede all others – they are specific to installingthe software on Concurrent Real-Time’s RedHawk systems.

2. Requirements•••

Jetson AGX Orin with Internet accessAn x86_64 host system running Ubuntu 20.04 with Internet access; note thatthe host system should be up to date with the latest available Ubuntu package updates
The RedHawk 8.4.7 for Jetson AGX Orin optical media disk

3. Installation
3.1.

Download SDK Manager onto the host system

On an x86_64 host system running Ubuntu 20.04, download version 1.9.3-10904 of SDK
Manager from the NVIDIA website via this URL:
https://developer.nvidia.com/jetpack
Click on the NVIDIA SDK Manager link to download the SDK Manager package (login may berequired). You should also click on the Jetson AGX Orin Developer Kit User Guide to
download the User’s Guide PDF.
Place these downloaded files into an orin directory in the home directory of the current user.

3.2.

Install the downloaded SDK Manager package

Open a terminal window and issue the following commands to install the SDK Manager:$ cd ~/orin$ sudo apt install ./sdkmanager_1.9.3-10904_amd64.deb

3.3.

Connect the Orin system to the host system

The Orin must be connected to the host system for SDK Manager to be able to detect it andsuccessfully flash it. Two different USB cables are required to connect the Orin to the hostsystem. Refer to the cables and documentation included with your Orin.

3.4.

Put the Orin into recovery mode

Newer versions of SDK Manager work best when the Orin is initially placed into recovery mode. If
the Orin is powered on and booted, issue the following command to enter recovery mode:
reboot --force forced-recovery
Alternatively, connect the power cable with the middle button pressed, then press the left button,
and then release both buttons at the same time. Refer to NVIDIA documentation for more details.

3.5.

Use SDK Manager to flash the Orin

Invoke the following command as the current user to begin using the SDK Manager:$ sdkmanager

The first action required by the SDK Manager is to log into the nvidia.com website using yourdeveloper or partner account email address and password. Enter the appropriate information andpress LOGIN to continue.
After logging in you may be presented with the following Update Notification dialog:

If this dialog appears, do not click Update Now and instead click Later so that flashing can be
performed with the currently installed SDK Manager, which should be at version 1.9.3-10904.
NOTE
RedHawk 8.4.7 has only been validated with JetPack version 5.1.2.
Other versions are incompatible with RedHawk 8.4.7 and must not be used.
At this point, you can generally follow the instructions in the Jetson AGX Orin Developer Kit User
Guide along with the following RedHawk-specific instructions.
SDK Manager screenshots are provided below.
1. On the STEP 01 - DEVELOPMENT ENVIRONMENT screen, click on the . . . icon in the Target
Hardware box to reveal the list of supported hardware configurations, as shown in thefollowing screenshot:Click on the Jetson AGX Orin modules target hardware option to make it the selected target,
as shown in the following screenshot:

2. Click on the . . . in the Linux box to reveal the list of supported JetPack versions, as
shown in the following screenshot:3. Click on the JetPack 5 . 1 . 2 version to make it the selected version, as shown in thefollowing screenshot:

4. Press Continue to advance to the STEP 02 – DETAILS AND LICENSE screen and click
I accept the terms and conditions of the license agreements at the bottom of
the window to accept the licenses, as shown in the following screenshot:5. Press Continue to advance to the STEP 03 – SETUP PROCESS screen and SDK Manager
will begin downloading all the software components from NVIDIA, as shown below:

Depending on your network’s speed, the download can take 30 minutes or more. During
the download and setup, a detailed output log can be viewed under the TERMINAL tab.

6. Once the download and setup has completed, you will be presented with a Flash dialog.
Notice the Automatic setup pulldown menu as shown in the following dialog:

Click on Automatic setup and change the setting to Manual Setup as shown in thefollowing dialog:

Review section 3.4, “Put the Orin into recovery mode” above, if necessary.
Next, fill in the New Username and New Password fields for the default user that willbe created on the Orin, as shown in the following dialog:

Follow the steps listed on the Flash dialog to place the Orin into recovery mode and thenclick Flash to begin flashing.
NOTE
Issue the lsusb command on the host to verify that an NVidia Corp.
entry with device ID 0955:7023 is displayed; this indicates the Orinhas entered recovery mode and is ready for flashing.

7. During flashing the SDK Manager will prompt you to fill in Username and Password
fields as shown in the following dialog:

These fields default to the values specified in the previous step, so simply click Install tocontinue the software installation.
NOTE
To complete software installation the Orin must be connected to the Internet.

8. Once the flashing and software installation have completed you will automaticallyproceed to the STEP 04 – SUMMARY FINALIZATION screen.

Click Finish and Exit to exit the SDK Manager

3.6.

Set NVIDIA Power Model to Maximum Performance

After SDK Manager has completed the Ubuntu installation on the Orin, and while the Orin is stillbooted with the NVIDIA stock kernel, log into the Orin using the username and password chosen
during flashing and run the following command:$ sudo nvpmodel -m 0

The following output should be displayed:
NVPM ERROR: Error writing 0 to /sys/devices/gpu.0/tpc_pg_mask: 16
NVPM WARN: Reboot required for changing to this power mode: 0
NVPM WARN: DO YOU WANT TO REBOOT NOW? enter YES/yes to confirm:

Type yes and press ENTER to reboot the Orin and then log in again and issue the following
command:$ sudo nvpmodel -q

The following output should be displayed:
NV Power Mode: MAXN
0This confirms that the Orin has been correctly configured for maximum performance.
Note that once the Orin’s power mode has been set it will persist across reboots until it isexplicitly changed again using the nvpmodel command.

3.7.

Copy RedHawk packages to the Orin
NOTE
This section assumes you are using the RedHawk 8.4.7 for Jetson AGX Orin
optical media disc. Alternatively, if you wish to install RedHawk over the network, refer
to Appendix A, “Install RedHawk from Concurrent Real-Time Repositories” at the end of
this document. Once installed, continue with section 3.9 below to complete installation.

After SDK Manager has completed the Ubuntu installation on the Orin, insert the RedHawk 8.4.7
for Jetson AGX Orin optical disc into the Ubuntu host’s optical drive and it should automaticallymount under the /media/$USER directory (e.g., /media/jane/RedHawk 8.4.7 aarch64).
Change to the mounted directory on the host and use networking to copy the RedHawk packages
into the newly installed Orin, as illustrated by the following commands:$ cd /media/jane/RedHawk-8.4.7-aarch64$ scp *.gz user@Orin-IP-Address:/tmp

NOTE
Use the username, hostname and password chosen during flashing.

3.8.

Install RedHawk packages on the Orin

To install the RedHawk packages, log into the Orin (either via ssh or via the graphical console)and issue the following commands:$ cd /tmp$ sudo tar -xf core-*.tar.gz$ cd redhawk$ sudo ./installEnter the password for the current user if the sudo command prompts for it.
NOTE
Warnings may be displayed during package installation, including:• Warning: couldn’t identify type of root file system for fsck hook
• Failed to stop turbo.service: Unit turbo.service not loaded.
• Failed to disable unit: Unit file turbo.service does not exist.
You can safely ignore these messages as they are not relevant
to RedHawk 8.4.7 on Jetson platform embedded installations.
At this point, all RedHawk software should be successfully installed on the Orin, however
additional steps in the next section are required before you can boot RedHawk kernels.

Add RedHawk entries to the extlinux.conf file

3.9.

Perform the following steps on the Orin to create boot entries for the RedHawk kernels:
1.

Create a backup of the /boot/extlinux/extlinux.conf file.
$ cd /boot/extlinux$ sudo cp extlinux.conf extlinux.bak

2.

Open the extlinux.conf file in your preferred editor. For example, if using vi:$ sudo vi extlinux.conf

3.

Duplicate all the primary kernel entry text lines to create a second identical entry
below the initial entry. For example:
TIMEOUT 30
DEFAULT primary
MENU TITLE L4T boot options
LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
FDT /boot/dtb/kernel_tegra234-p3701-0000-p3737-0000.dtb
INITRD /boot/initrd
APPEND ${cbootargs} quiet ...many other boot options...
LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
FDT /boot/dtb/kernel_tegra234-p3701-0000-p3737-0000.dtb
INITRD /boot/initrd
APPEND ${cbootargs} quiet ...many other boot options...

4.

Customize the first kernel entry to point to the desired RedHawk kernel. For example, tocreate an entry for the RedHawk trace kernel, modify the LABEL, MENU LABEL and
LINUX lines as follows (leaving the other lines unchanged):
LABEL redhawk-trace
MENU LABEL redhawk-trace kernel
LINUX /boot/Image-5.10.120-rt70-r35.4.1-tegra-RedHawk-8.4.7-trace
FDT /boot/dtb/kernel_tegra234-p3701-0000-p3737-0000.dtb
INITRD /boot/initrd
APPEND ${cbootargs} quiet ...many other boot options...

NOTE
Modifying the first kernel entry will allow the NVIDIA kernel to be booted asa fallback if any typos were entered during the customization of the first entry.
Failure to follow this guideline can render the Orin unbootable andrequire re-flashing if a typo is accidentally introduced during editing.
5. Change the default kernel to the newly added RedHawk kernel by changing the DEFAULT
entry at the top of the file:
TIMEOUT 30
DEFAULT redhawk-trace

NOTE
If the Orin has been set up with a serial console, you may choose
to skip this step and instead interactively decide which kernel to bootwhen you are presented with the boot menu on the serial console.

3.10.

Verify RedHawk kernel installation

Upon reboot, the Orin should now be running the selected RedHawk kernel. To verify this, issue
the following command:$ uname -r

You should see output displayed like the following:
5.10.120-rt70-r35.4.1-tegra-RedHawk-8.4.7-trace

Please contact Concurrent Real-Time technical support if you had any problems during thisinstallation (support@concurrent-rt.com or 1-800-245-6453).

4. Known Issues
Special consideration should be given to the following areas.

4.1.

Orin sibling cores can interfere with shielding

Orin cores utilize a 4-tier cache as shown in the following diagram:

Each core has its own L1 and L2 caches, but L3 caches are always shared between four sibling
cores. In the above diagram, cores 0, 1, 2, and 3 are siblings, cores 4, 5, 6, and 7 are siblings,
and cores 8, 9, 10, and 11 are siblings.
When shielding a core on the Orin for a real-time application, activity on the shielded core’s
sibling cores can modify the shared L3 cache and impact the determinism and performance of thereal-time application. To achieve maximum real-time performance, all sibling cores must be
shielded, with the unused sibling cores kept idle.

4.2.

Maximizing clock speeds

NVIDIA provides the jetson_clocks utility to control the speeds of various hardware clocks on
the Orin. This utility provides several options and must be run as the root user. Invoke the utility
with the --show option to view all current clock speeds. For example:
root@ubuntu:~# jetson_clocks --show 2>/dev/null
SOC family:tegra234 Machine:Jetson AGX Orin
Online CPUs: 0-11
cpu0: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu1: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu10: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu11: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu2: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu3: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=1420800 IdleStates: WFI=1 c7=1
cpu4: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=1728000 IdleStates: WFI=1 c7=1
cpu5: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=1728000 IdleStates: WFI=1 c7=1
cpu6: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu7: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu8: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
cpu9: Online= Governor=schedutil MinFreq=729600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=1 c7=1
GPU MinFreq=306000000 MaxFreq=1300500000 CurrentFreq=306000000
EMC MinFreq=204000000 MaxFreq=3199000000 CurrentFreq=2133000000 FreqOverride=0
DLA0_CORE:
Online=1 MinFreq=0 MaxFreq=1600000000 CurrentFreq=1600000000
DLA0_FALCON: Online=1 MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
DLA1_CORE:
Online=1 MinFreq=0 MaxFreq=1600000000 CurrentFreq=1600000000
DLA1_FALCON: Online=1 MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
PVA0_VPS0: Online=1 MinFreq=0 MaxFreq=1152000000 CurrentFreq=1152000000
PVA0_AXI: Online=1 MinFreq=0 MaxFreq=832000000 CurrentFreq=832000000
FAN Dynamic Speed control=active hwmon2_pwm=56
NV Power Mode: MAXN
root@ubuntu:~#

Notice that, by default, some of the current frequency values above are significantly lower thanthe maximum frequencies supported by the hardware.
To achieve the best performance, set all clock speeds to their maximum values by invoking the
jetson_clocks utility without any options. Then, to verify that all clock speeds have beenmaximized, run the jetson_clocks utility again with the --show option. For example:

root@ubuntu:~# jetson_clocks
root@ubuntu:~# jetson_clocks --show 2>/dev/null
SOC family:tegra234 Machine:Jetson AGX Orin
Online CPUs: 0-11
cpu0: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu1: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu10: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu11: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu2: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu3: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu4: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu5: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu6: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu7: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu8: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
cpu9: Online= Governor=schedutil MinFreq=2201600 MaxFreq=2201600 CurrentFreq=2201600 IdleStates: WFI=0 c7=0
GPU MinFreq=1300500000 MaxFreq=1300500000 CurrentFreq=1300500000
EMC MinFreq=204000000 MaxFreq=3199000000 CurrentFreq=3199000000 FreqOverride=1
DLA0_CORE:
Online=1 MinFreq=0 MaxFreq=1600000000 CurrentFreq=1600000000
DLA0_FALCON: Online=1 MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
DLA1_CORE:
Online=1 MinFreq=0 MaxFreq=1600000000 CurrentFreq=1600000000
DLA1_FALCON: Online=1 MinFreq=0 MaxFreq=844800000 CurrentFreq=844800000
PVA0_VPS0: Online=1 MinFreq=0 MaxFreq=1152000000 CurrentFreq=1152000000
PVA0_AXI: Online=1 MinFreq=0 MaxFreq=832000000 CurrentFreq=832000000
FAN Dynamic Speed control=active hwmon2_pwm=56
NV Power Mode: MAXN
root@ubuntu:~#

Notice that all the current frequency values above have been configured to their maximum
speeds.
NOTE
Unlike nvpmodel, clock speed changes made by jetson_clocks do
not persist across a reboot. Thus, you must always run jetson_clocks
again after a reboot to ensure that all clock speeds are maximized.

4.3.

Maximizing fan speed during heavy loads

By default, the speed of Orin’s CPU cooling fan will be dynamically adjusted, as needed, toalways ensure proper cooling. This default dynamic fan behavior should be more than adequate
for almost all use cases.
However, for periods of prolonged benchmarking and/or stress testing, Concurrent recommends
maximizing the CPU cooling fan speed before starting tests to prevent fast temperature rises.
Temperatures that rise too quicky may result in slowing down CPU speeds, which can negatively
impact benchmarking.
Invoke jetson_clocks with the --fan option to maximize the speed of the CPU cooling fan.

4.4.

Entropy gathering daemon interferes with real-time

Ubuntu 20.04 enables the Hardware Volatile Entropy Gathering daemon (haveged) that
continuously samples various hardware resources (e.g., cache, branch predictions, memorytranslation tables) to collect entropy and improve random number generation.
This daemon’s constant hardware probing activity can negatively impact real-time performanceeven on shielded CPU cores, and the daemon should be stopped before starting real-timeworkloads by invoking the following command as the root user:# systemctl stop haveged.service
Do not disable the entropy gathering service because various Ubuntu startup services require aminimum level of entropy to be gathered before they will start; this includes services required forthe X Server to properly initialize. However, after the Orin has started with graphics displaying
you can safely stop the service to improve real-time performance.

4.5.

Debug kernel not supplied with RedHawk 8.4.7

The RedHawk debug kernel is not provided with RedHawk 8.4.7 due to NVIDIA incompatibilities
with various debug-related kernel configuration options. Concurrent is working with NVIDIA toaddress this issue and hopefully it will be corrected in a future RedHawk release.

4.6.

RedHawk kernels and OEM verification failures

Booting any of the RedHawk kernels on the Orin will result in the following warnings being
displayed almost immediately after the boot process begins:[0014.855] I> Validate kernel ...
[0014.855] I> T19x: Authenticate kernel (bin_type: 37), max size 0x5000000[0014.856] E> Stage2Signature validation failed with SHA2!!
[0014.856] C> OEM authentication of kernel header failed!
[0014.857] W> Failed to validate kernel binary (err=1077936152, fail=1)[0014.857] W> Security fuse not burned, continue to bootThese warnings can be safely ignored because OEM security fuses are not burned by default on
Jetson hardware platforms.
For more information, refer to the NVIDIA Jetson AGX Orin Fuse Specification document that isavailable on the NVIDIA Developer website.

4.7.

Change to PAM Capability module directory

The PAM Capability module directory that is documented in some versions of the RedHawk
User’s Guide is incorrect for the Orin. The correct PAM Capability module location should beshown as:/lib/aarch64-linux-gnu/security/pam_capability.so
If you wish to use the PAM Capability feature with services in the /etc/pam.d directory, thenyou should refer to the PAM Capability module as follows:
session required /lib/aarch64-linux-gnu/security/pam_capability.so

4.8.

Custom kernels will not install kernel Image files

When custom kernels are built and installed on the Orin, the make install step will install acustom vmlinuz-… file into /boot, but it will not install a matching custom Image-… file.
The newly installed custom vmlinuz-… file path can simply be entered for the custom kernel
entry’s LINUX line in extlinux.conf to reference the newly installed custom kernel.
Alternatively, the newly built uncompressed Image-… file can be manually copied to /boot withthe following command:sudo cp ./arch/arm64/boot/Image /boot/Image-$(make ccur-kernel-release)
The custom kernel entry’s LINUX line can then reference the copied custom Image-… file.

A. Install RedHawk from Concurrent Real-Time Repositories
RedHawk can be installed over the network utilizing the Concurrent Real-Time software repositories.
This has the advantage that it will install the latest updates that have been released, however some
up-front setup is required to perform the installation; follow the steps below to complete this setup.

A.1.

Install Concurrent Real-Time Public Keys

Concurrent Real-Time signs its APT repositories with a GPG-generated DSA and ElGamal key
pair. You need to install Concurrent Real-Time's public key so that the APT system can authorize
usage of the repositories.
You can download the ccur-public-keys file and import the key file as needed into APT's key ring.
You should import the key file before attempting to access software from Concurrent Real-Time's
repositories.
The following commands will download the public key and install it.
wget -q http://redhawk.concurrent-rt.com/network/ccur-public-keys
apt-key add ccur-public-keys
If you wish to check the authenticity of the public keys file you downloaded from Concurrent RealTime's web site, request the key fingerprints from Concurrent Software Support and comparethem to the fingerprints of the keys you downloaded.
You can obtain the fingerprints of the keys you downloaded with the following command:
gpg --with-fingerprint ccur-public-keys

A.2.

Install Repository Definition Files

Repository definition files tell the APT installation and update system how to access specificrepositories. The definition files must reside in the /etc/apt/sources.list.d/ directory andthe file name must end .list; e.g. /etc/apt/sources.list.d/ccur.list.
For example, use the following format for the repository definition of the RedHawk 8.4.7 for
Jetson AGX Orin product:deb [arch=arm64] http://redhawk.concurrent-rt.com/ubuntu/login/passwd/redhawk/agx-orin 8.4.n core

For example, the following entry substitutes a fictional login and password:deb [arch=arm64] http://redhawk.concurrent-rt.com/ubuntu/L12345/xyzzy/redhawk/agx-orin 8.4.n core

Note that if the NightStar tools have been purchased for this machine, then you should also add arepository definition for the NightStar tools below the previously added line. For example:
deb [arch=arm64] http://redhawk.concurrent-rt.com/ubuntu/L12345/xyzzy/nightstar 5.1 rt ubu20

Once the repository definition file is in place the repository setup is complete. Run the following
command as root to ensure that the software repositories have been added correctly:
sudo apt update
If you see errors or warnings produced that reference the newly added ccur.list file, thenverify that all the above steps have all been completed as documented.

A.3.

Install RedHawk Packages

Now that the setup is done, simply issue the following command to install all the latest RedHawk
8.4.7 packages (and NightStar 4.6 packages if purchased) onto the current system:
apt install 'ccur-*'Once this command completes the RedHawk packages are installed. Continue with Section 3.8,
“Add RedHawk entries to the extlinux.conf file," to finalize the RedHawk installation.

A.4.

SupportIf you need assistance, please contact the Concurrent Real-Time Software Support Center at our
toll-free number 1-800-245-6453. For calls outside the continental United States, the number is 1954-283-1822. The Software Support Center operates Monday through Friday from 8 a.m. to 5
p.m., Eastern Standard Time.
You may also submit a request for assistance at any time by using the Concurrent Real-Time
website at http://concurrent-rt.com/support or by sending an email to support@concurrent-rt.com.