Transferring data to cloud - Online, Transfer Service and Transfer App | Scribe

Transferring data to cloud - Online, Transfer Service and Transfer App

  • Hafeez Baig |
  • 7 steps |
  • 21 seconds
  • Google CloudGoogle Cloud
1
Sign in to the **Google Cloud Console**
2
Type "**Storage Transfer**" in the search bar and click on the **Storage Transfer** option
information ordinal icon
**What is Storage Transfer?**\ \ **Storage Transfer** in Google Cloud refers to a service that allows you to automate the transfer of data between different storage locations. It can be used to move data from on-premises storage, other cloud providers, or between Google Cloud Storage buckets. Storage Transfer helps in migrating large datasets, synchronizing data across locations, or backing up data by providing scheduled and managed data transfers with options for filtering, scheduling, and error handling.
3
**Transfer wizard** will open, click on the **CREATE TRANSFER JOB** button
information ordinal icon
**What is TRANSFER JOB?**\ \ A **Transfer Job** in Google Cloud refers to a task created within the Storage Transfer Service to move or copy data from one storage location to another. This can involve transferring data between different Google Cloud Storage buckets, from external cloud providers, or from on-premises systems to Google Cloud. A Transfer Job allows you to specify the source, destination, schedule, and any filters or options for the data transfer. It's a way to automate and manage large-scale data migrations or backups.
4
**Create a transfer job** wizard will open, on the right side click on the **Source type** dropdown section and here you can select the Source type for the Transfer job
information ordinal icon
**What is the Source type?**\ \ The **Source Type** under a Transfer Job in Google Cloud Storage Transfer Service refers to the origin of the data you want to transfer. It specifies where the data is currently stored before being moved to the destination. Common source types include: - **Google Cloud Storage**: Transferring data between different Cloud Storage buckets. - **Amazon S3**: Transferring data from an Amazon S3 bucket. - **HTTP/HTTPS**: Transferring data from publicly accessible web servers. - **On-Premises**: Transferring data from your on-premises storage to Google Cloud. Selecting the appropriate source type is the first step in setting up a transfer job.
5
Scroll to the **Destination type** dropdown section and here you can select the Destination type which is Google Cloud Storage and POSIX filesystem **Note:** For POSIX filesystem option it requires agent pool setup or selection
information ordinal icon
**What is the Destination type?**\ \ The **Destination Type** under a Transfer Job in Google Cloud Storage Transfer Service specifies where the data will be moved or copied to during the transfer. It defines the target location for the data being transferred. The common destination type is: - **Google Cloud Storage**: The data is transferred into a specified Google Cloud Storage bucket. This setting allows you to direct the data from your chosen source type to the appropriate storage location within Google Cloud.
6
Scroll to the **Scheduling mode** section and here you can select the mode as Batch or Event-driven **Note:** Event-driven option has been newly added by GCP in the Transfer job.
information ordinal icon
**What is Batch Scheduling Mode?**\ \ **Batch Scheduling Mode** in the context of Google Cloud Storage Transfer Service refers to a scheduling option that allows you to group multiple transfer jobs into a single batch and execute them at a specific time. This mode is useful for managing large-scale data transfers by running them during off-peak hours or at a time that best fits your operational needs. It helps optimize resource usage and can be configured to minimize the impact on network and system performance during the transfer.
7
To view the Agents pools, click on the **Agent pools** option from the left bar
information ordinal icon
**What are Agent pools?**\ \ **Agent Pools** in Google Cloud refer to a collection of agents that manage and execute data transfer tasks for the Storage Transfer Service. Agents are software components that handle the actual transfer of data between source and destination storage locations. In the context of **Transfer Appliance** or **Transfer Service**: - **Agent Pools** allow you to manage and scale the agents used for data transfer tasks, ensuring efficient handling of large or complex data transfers. - They can be configured to handle multiple transfer jobs simultaneously and are used to balance the workload, improving performance and reliability. Agent Pools are particularly useful for managing transfers involving on-premises data, large-scale migrations, or complex transfer scenarios.
information ordinal icon
Congratulations! on completing this lab and Understanding transferring data to cloud - Online, Transfer Service and Transfer App