Exploring Batch Operations in S3 | Scribe

Exploring Batch Operations in S3

  • Hafeez Baig |
  • 7 steps |
  • 51 seconds
  • Amazon Web ServicesAmazon Web Services
information ordinal icon
Refer to the following guide to learn how to use IAM in the cloud\ <https://scribehow.com/shared/Creating_an_IAM_User_and_User_Group_in_AWS_Management_Console__hWNiiXlcRJKFqYZZEJD-cg>
1
Sign in to **AWS Management Console** as an **IAM User** **Pre-requisite** - Ensure you have selected the AWS region closest to your location\ For this guide, we will be using us-east-2 (Ohio) as a preferred choice
2
Type **S3** in the search bar and click on **S3** to view the S3 Dashboard
information ordinal icon
**What is S3 in AWS?** Amazon S3 (Simple Storage Service) is a scalable object storage service offered by Amazon Web Services (AWS). It allows users to store and retrieve any amount of data from anywhere on the web. S3 is designed to provide durability for objects and offers high availability and security features such as encryption, access control, and data management policies.
3
On Amazon S3 dashboard, on the left bar click on the **Batch Operation**
information ordinal icon
**What is Batch operations in AWS S3 bucket?** Batch operations in AWS S3 buckets refer to a set of capabilities that allow users to perform large-scale operations on objects stored within Amazon S3. These operations are typically executed concurrently and efficiently, enabling users to manage large volumes of data more effectively.
4
Batch Operations wizard will open, on the right side click on the **Create job** option
5
**Choose Region and manifest** wizard will open, Choose the AWS Region from the dropdown. We will be using us-east-2 (Ohio) as a preferred choice
6
Scroll to the **Manifest** section and configure the settings
information ordinal icon
**What is Manifest Format?** A manifest file in AWS S3 replication is typically in JSON format, containing object details like keys, versions, metadata, and replication configurations. 1. **Types of Formats**: - **JSON Format**: Preferred for its flexibility and ease of serialization, allowing complex data representation. - **CSV Format**: Less common but viable, especially for simpler use cases or compatibility needs. 2. **Creating Manifest Using S3 Replication**: - Define replication rules in the source bucket's replication configuration. - S3 automatically generates and manages manifest files. - These files ensure objects are replicated according to defined rules and configurations.
7
Scroll the page and click on the **Next** button to create a new job
information ordinal icon
Congratulations on successfully creating the job in Batch Operations! And completing this lab!