JSON troubleshooting: how to use API with pagination?

In the established integrations (Shopify, Xero, etc.) we use APIs of the source apps that support pagination hence all dataset you’re pulling from these apps with Coupler.io is returned smoothly per one run.

However, if you are pulling data with JSON importer - please note that unfortunately, Coupler.io does not support pagination in JSON importer automatically. 
We understand that our users need this feature and already included it in our product backlog. While we are working on the implementation - we suggest the next workaround:
  1. Setup different importers in replace mode: each of them will load data from one page with the maximum number of results supported by API (i.e. page1, page2 with 100 results to a separate sheet in Google Spreadsheet i.e. sheet1, sheet2.)
  2. You can schedule all importers from step 1 at the same time - i.e. at 8 AM.
  3. Setup one importer in replace mode that will merge data from all sheets to one with the few sheets selected at the source settings and schedule it at 9 AM.

You can also run all these importers manually: importers from step 1 - first and an importer from step 3 next. Using this approach you will get all your products in 1 sheet.

Notes:

1. Once your data source size is increased and new data is available on a new page - you need to create +1 importer manually (as described in step 1).

2. To speed up configuration - you can use the approach described here: How to merge several sheets into one?.
In a few words:
- name sheets in step 1 importers using similar namings - i.e. woocommerce_products_page1, woocommerce_products_page2
-  set the "Sheets" setting in the step 3 importer using the regular expression woocommerce_products.+

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.