Troubleshooting: Timeout: we could not import your data within 9 minutes
Data import fails with the error:
Timeout: we could not import your data within 9 minutes. Please try to reduce amount of data to import.
Error Meaning
This error typically means that the dataset you're pulling takes more than 9 minutes to be returned by your source API. In other words, your source application's API does not return your data to Coupler.io and eventually fails with a timeout.
Usually, reducing the data for import helps to fix the issue.
Data needs to be reduced on step 1 - Data Source using basic/ advanced settings of the source.
If you apply filters during step 2 - Transformations - it does not reduce the original dataset pulled from the source API = does not impact the time it takes for your source API to return the data before Coupler.io shows it on Transformations.
How to reduce the dataset
1. Using the advanced settings on Step 1 - Data Source
Most of the sources have advanced settings that can be used for filtering the dataset for import.
E.g. Shopify Orders have the following wide range of filters that can be applied to the data set:
2. Splitting data between several sources
If you need all the data and using filters does not apply, you may split your data by setting up multiple sources for your one importer.
There is a separate timeout for every source within one importer. If you split your data into several sources within one importer, data will be returned in chunks by your source API for each source thus reducing the time for import.
You can do so either by clicking "+" to connect a new source, or by hitting "duplicate":
- "connect a new source" adds a "fresh" source to your importer w/o any applied settings
- "duplicate" adds a copy of your source with already applied settings
The easiest way is to click "duplicate" and just change the date filters you applied. For example: if you want to pull Shopify Orders for 2023, you may use the following settings:
- Source 1 fetches data from Jan 2023 to Mar 2023
- Source 2 fetches data from Apr 2023 to Jun 2023
- and so on, depending on your data needs
Tip: Coupler.io supports macros you can use for dynamic date filters.
During Step 2 - Transformations - combine all your sources using the "Append" mode:
Timeout for Google Sheets source
The reasons for this issue might be:
- big dataset
- a spreadsheet is too heavy
- a spreadsheet uses many formulas and it hangs
Big dataset
Follow the steps from How to reduce the dataset?
A spreadsheet is too heavy/ uses many formulas and hangs
Check if there is a way for you to clean up your source file. E.g.check if there is unneeded data that can be removed or if there are many formulas that could be cleaned up.
Otherwise, you might want to divide your data. Please try the following:
1. Create several (we can start from 2) importers
2. Setup 1st importer:
2.1. Specify the first half of the data range you need to import (For example, the data we wanted to import is on the range A1:Z20, then we'll first specify "A1:Z10" on this first importer). The goal is to reduce the data being fetched to prevent the timeout error.
2.2. Set the Import Mode: Replace (default mode)
3. Setup 2nd importer:
3.1. Specify the second half of your data range (Ex.: A11:Z20)
3.2. Set the Import mode: Append
4. Setup sequence of importer runs: 1st importer goes first, and after it, the 2nd importer runs.
4.1. You could set the sequence via schedule ( How to set up automatic data refresh?) or via webhook (Webhooks support)
In case 2 importers are not enough (you are still getting timeout errors), please try to split the data into 3 or more importers.
Alternatively, you may split the data within one importer by using multiple sources. See the Shopify example above: Splitting data between several sources