How to Incrementally Import / Prevent Duplicates from syncing.

Created by Nicolas Wussler, Modified on Tue, 16 Aug, 2022 at 8:33 AM by Igor Galic

TABLE OF CONTENTS

 


 

1) Description

Some integrations do not support Incremental Imports while it can be necessary to achieve the desired workflow.

 

Therefore a custom workaround can be created through Google Sheets to make Incremental Import possible.

 


 

2) Setting up the Tasks

 

2.1 Task 1: Source to Google Sheets

 

2.1.1 Create task 1

 

Source

The Source can be any Integration that does not support incremental imports in the configuration. 

 

Target

This workaround would require the use of the Google Sheets integration

If you do not have this set up already please proceed with the setup according to How to integrate Google Sheets

 

 

2.1.2 Target Configuration

 

To make the Incremental Import possible the following Configurations have to be applied to the Google Sheet.

 

Export behavior 

  • Update or Create using UID 

     

 

 

  • Action: Only Create

 

 

  • Create Required data Fields

 

 

Here the Data fields imported from the Source should be added so that they can be passed on to the Target integration in Task 2

 

  • Set UID field

 

 

Setting a column as UID (Unique Identifier) will serve as reference to system on checking for updated records.

 

You need to specify which columns should be treated as unique. The system will use this when checking for updated records. 

At least one column must be selected. 

 

  • Overview of Config example

 

 

2.1.3 Task Setting: Trigger

The Trigger can be set up as desired.
For all the trigger options please consult:
Task Settings Section 4) Task Automation

 


 

2.2 Task 2: Google Sheets to Target

 

2.2.1 Create Task 2

 

Source

The Source for this task will have to be Google Sheets, specifically the sheet that was created in Task 1 needs to be used.

 

Target

The Target for this task should be the Integration you desire to transfer the date to fetched from the Source in Task 1

 

 

2.2.2 Source Configuration

 

 

  • Record Processing: Update Records

 

During task execution the system will compare the current file with file from previous execution and only added and/or changed records will be sent to the destination. 

 

 

 

 

  • Unique Column

A Unique column has to be selected, this should be the same as the column marked as UID (Unique Identifier) in Task 1

 

 

 

2.2.3 Task Setting: Trigger

Triger: On event
When: Task Succeeded event 

Task: Task 1

 

This has to be selected in order to execute the task as soon as Task 1 has been completed. 

This is selected so that the incremental import does not need any manual intervention.  

 


 

We hope that this covers everything. If you have any further questions or need additional clarification please reach out to us via [email protected] or our Chat Widget!

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article