Google Cloud Platform (GCP) Service Account for Automated Data Transfers

Setting up your service account is very simple.

In this brief guide, we’ll walk you through creating and setting up a Google Cloud Platform (GCP) service account with the necessary permissions to enable automated data transfers between SFTP, Google Cloud Storage (GCS), and Google BigQuery.

What is a Service Account and why do I need it?

A service account is a special Google account used by applications to interact with Google APIs. It allows Netice and your automated data transfer tasks to access Google Cloud services securely and such access can be provoked at any time.

For scheduled automated data transfers, a service account provides a secure way to authenticate and authorize your application to read from and write to GCS and BigQuery.

1. Creating a Service Account

Step-by-Step Instructions

1. Go to the GCP Console:

   • Visit Google Cloud Console.

2. Navigate to the IAM & Admin Section:

   • Click the Navigation Menu (three horizontal lines) in the upper-left corner.
   • Select IAM & Admin > Service Accounts.

Create a New Service Account:

Click the + CREATE SERVICE ACCOUNT button at the top.

Enter a Service Account Name (this can be anything) and Description (e.g. “For Netice Data Transfer Plaform)”.

Click CREATE AND CONTINUE.

2. Granting Permissions to the Service Account

Once you have created your service accounts, you will see it listed in the Service Accounts page. To enable your service account to perform the required operations, you need to grant it specific roles.

Step-by-Step Instructions

Now, go to IAM and locate the service account you created.

Click the pen icon:

We can now give the needed permissions for the service account. Under “Assign roles” click “Add another role”.

Add the roles of “Secret Manager Admin”, “Storage Admin” and “BigQuery Admin”, and click “Save”:

Secret Manager Admin allows full access to administer Secret Manager resources meaning your SFTP private key, passphrase, password, GCP service account key which are required to successfully complete your data transfers.

Storage Admin allows full control over buckets and objects, including creating, deleting, modifying, and accessing them.

BigQuery Admin allows full control over BigQuery resources, including creating, deleting, modifying datasets and tables, and accessing them.

3. Creating a Service Account Key

Finally, once you have created your service account and granted it the necessary permissions, you need to generate a key for authentication.

Step-by-Step Instructions

Let’s return to the Service Accounts view under IAM.

Click on the service account you have created and granted the permissions for.

Click the KEYS tab.

Click ADD KEY -> Create new key.

Choose the JSON key type and click Create:

A JSON file containing your key will be downloaded automatically. Store this file securely as it contains the credentials Netice will use to authenticate with Google Cloud Platform. Do not share the key with anyone else.

The content of this file is the Google Cloud Service Account Key that you will use when setting up your automated data transfer tasks. You can open it with e.g. Notepad and copy and paste the contents of the key to the Service Account Key text area when setting up a data transfer task:

Security Best Practices

By granting only the necessary permissions, we ensure that your service account has the least privilege required to perform its tasks, minimizing security risks. Always monitor and review permissions regularly to ensure they are up-to-date and appropriate.

And that’s it.

Setting up a GCP service account with the correct permissions is a critical step in enabling secure and automated data transfers between SFTP, GCS, and BigQuery. Follow this guide to ensure your service account is correctly configured and secure.

For more detailed information, you can refer to the official Google documentation:

• IAM Roles for GCS
• IAM Roles for BigQuery

If you have any questions or need further assistance, please contact our support team.