Common errors
With data transfers between Google BigQuery, Cloud Storage, SFTP...

Subscription level

Error

Table size ****.** MB exceeds the allowed limit of ****.00 MB for subscription level: *****. Please consider upgrading your subscription level.

Solution

Either transfer a smaller file or upgrade your subscription as your current subscription level is not for such large data files. Premium subscriptions enable the transferring of files up to 1.5 GB and paid amounts are prorated by the time a subscription has been in use for each billing period, meaning that the client gets fair pricing even for immediate, middle-of-the-billing-period upgrades and for the current month, does not need to pay the upgraded subscription’s price for the full month but only for the proportion of the billing period that each subscription type has been in use.

Broken file

Error

Error uploading file *****.csv to Cloud Storage: Error tokenizing data. C error: EOF inside string starting at row *****

Solution

A row is incomplete and cuts abruptly. This is common in a file that has been arbitrarily split and in such case the last row could be the one causing the issue. Pandas library in Python does not accept to process the file as such and the broken row as specified by the row number should either be fixed or removed.

Google Cloud location mismatch

Error

Error during BigQuery to GCS transfer: 400 Cannot read and write in different locations: source: EU, destination: US; reason: invalid, message: Cannot read and write in different locations: source: EU, destination: US

Solution

When creating a new bucket for the client, the tool tries to create it in the same location where the BigQuery dataset is located, and for some reason there has been a failure to do so. If this error persists, please create the GCS bucket manually in the correct location and set it as the input for the GCS bucket in your transfer task, and run your task again.