AWS CLI Commands – Bulk Operations

Graphical User Interface (GUI) was introduced to reduce the use of command inputs. Why we still need to use Command Line Interface (CLI)? There are many reasons. One of them is that CLI can achieve certain outcomes that GUI may not be able to (easily) achieve, e.g. bulk operations. We will take a look at how to use AWS CLI to do bulk operations in this article.


A typical format of an AWS CLI command is like below:

aws <service name> <action> <parameter>

  • The aws prefix is to distinguish AWS commands from the native OS commands, e.g. PowerShell commands
  • The <service name> part is to tell which service the command should act upon
  • The <action> part is to tell what the command should be doing
  • The <parameter> part is about how the command should be conducted

For example, an S3 bucket listing command is like aws s3 ls s3://mybucket --recursive.

Command Helper

We will see the actual bulk operation commands examples in the later section. But when there are no example to follow, questions like “what parameter should I use” will arise. There are a few ways to address it:

  • Use the auto prompt option, e.g.
    • aws --cli-auto-prompt
    • aws <service name> --cli-auto-prompt
    • aws <service name> <action> --cli-auto-prompt
  • Use help, e.g.
    • aws help
    • aws <service name> help
    • aws <service name> <action> help
  • Use AWS CLI Command Reference


AWS CLI supports various output formats: json, yaml, yaml-stream, text and table.

To specify the output of a particular command, you can add --output at the end of your command, e.g. aws ec2 describe-regions --output text. If you want to set a format as the default format, you can put output=<format name> in your config file. For details of output configuration, see AWS CLI Commands – Output Format.

Bulk Operation Commands

Note that in the command examples below, the parts highlighted in red need to be replaced with the actual value.

ServiceActionCommand Example
S3Bulk and parallel copy files from one bucket to anotheraws s3 cp s3://bucket_name_1/ s3://bucket_name_2 --recursive --exclude "*" --include "keyword1*" --include "keyword2*"
and execute another command as
aws s3 cp s3://bucket_name_1/ s3://bucket_name_2 --recursive --exclude "*" --include "keyword3*" --include "keyword4*"
S3Bulk upload local files (in the current folder) to an S3 bucketaws s3 sync . s3://bucket_name
S3Bulk download S3 bucket files to the current local folderaws s3 sync s3://bucket_name .
S3Bulk rename files within a bucketaws s3api list-objects --bucket bucket_name --prefix "identifying_keyword" --delimiter "|" --output text| ForEach-Object { $_.split("`t")[2] } | Select-String -Pattern tobe_replaced_keyword | ForEach-Object -Process {$outputFile = $_ -replace 'tobe_replaced_keyword', 'replacing_keyword'; aws s3 mv s3://bucket_name/$_ s3://bucket_name/$outputFile}

if you are using SSO, --profile profile_name need to be added after each aws command
S3List the size of a bucket and number of items stored in itaws s3api list-objects --bucket bucket_name --output json --query "[sum(Contents[].Size), length(Contents[])]"
aws s3 ls --summarize --human-readable --recursive s3://bucket_name
EC2Bulk add multiple tags to multiple ec2 instancesaws ec2 create-tags --resources instance_id_1 instance_id_2 --tags 'Key=\"key1_name\", Value=\"key1_value\"' 'Key=\"key2_name\", Value=\"key2_value\"'
EC2Bulk start ec2 instancesaws ec2 start-instances --instance-ids instance_id_1 instance_id_2
EC2Bulk stop ec2 instancesaws ec2 stop-instances --instance-ids instance_id_1 instance_id_2
EC2Display all ec2 images’ name filtered by owner and platformaws ec2 describe-images --owners amazon --filters "Name=platform, Values=windows" --query "Images[*].[Name]" --output text
EC2Display all ec2 instances that have been stopped and reason for the stopaws ec2 describe-instances --filters Name=instance-state-name,Values=stopped --region ap-southeast-2 --query "Reservations[].Instances[].[StateReason.Message]"
Specify the resource(s) to retain when stack is in the DELETE_FAILED stateaws cloudformation delete-stack --stack-name stack_name --retain-resources "resource_logic_id_1" "resource_logic_id_2"
IAMList users by ARNaws iam list-users --query "Users[].[Arn]"


AWS CLI Command Line Interface

AWS CLI Command Reference

You May Also Like

About the Author: Richard Zhao

My name is Richard Zhao. I'm a solution architect and owner of Having built knowledge bases for many companies, I'd like to use this cloud studio to share knowledge and ideas with wider people on the internet.

Leave a Reply

Your email address will not be published. Required fields are marked *