Historical Sync

Introduction

Byteline Data Sync efficiently synchronizes existing data in both apps so that you can keep valuable data. This process is called historical sync. The process involves getting existing data from both apps and then updating, creating and deleting records to make the data in sync.

For 2-Way sync, records can be updated on both apps. However, for 1-Way sync, records are updated only on the target app.

Understanding Unique Keys in Data Sync

When you're syncing data between two different apps using Byteline's Data Sync, you want to make sure that the information is matched up correctly. This is where unique keys come into play. Don't worry if you're not a tech expert—this guide will break it down for you in simple terms.

What Are Unique Keys?

Think of unique keys as special identifiers for your data. Imagine you have two lists—one from App A and one from App B—and each list contains names and email addresses. A unique key could be something like an email address because it’s a piece of information that is unique to each person on your list. No two people should have the same email address, right? So, this unique piece of information helps Byteline figure out who’s who when syncing the data between your two apps.

Unique keys might sound technical, but they’re really just a simple way to make sure your data stays accurate when syncing between apps. By using unique keys, Byteline ensures that your information is always matched up correctly, saving you time and hassle.

Why Are Unique Keys Important?

When you’re syncing data, you might have records in both App A and App B that need to be matched up. If you don’t use unique keys, Byteline won’t know how to correctly pair the records from one app to another. Unique keys ensure that the information for "John Doe" in App A matches up with the information for "John Doe" in App B, even if the two apps store the data slightly differently.

How Do Unique Keys Work?

1. Unique Key Selection:
   1. Automatic (default): Byteline automatically determines the unique keys based on your data. This default approach works for most of the sync configuration. However, for fine control over the historical sync process, you can manually select the unique keys.
   2. Manual: For greater control over the historical sync process, you can manually select unique keys. The first step in the manual selection is to decide what your unique key will be. In many cases, it’s something straightforward like an email address or a customer ID. You’ll pick this key when setting up your data sync for the first time. See the Manually Configuring Unique Key(s) section below for the details.
2. Historical Sync: When you run your sync for the first time, Byteline looks at the unique keys you’ve chosen to match up existing records between your two apps. This process is called a historical sync, meaning it’s looking at data that’s already there, not just new data.
   Unique keys are only considered for the historical sync, i.e., to sync existing data. Once a sync is active, unique keys are not checked.

Manually Configuring Unique Key(s)

To manually select unique keys for real-time sync:

1. During configuring real-time sync, you need to toggle the option to manually choose the unique key(s). Below is the screenshot for enabling this setting.

   

2. Click the Continue button, which takes you to the unique key configuration page. On this page, define your unique keys by clicking on the key icons (illustrated in the screenshot)

How to trigger historical sync?

The historical sync is automatically performed when a sync is started. The only exception is if you choose the "Future data only" option, in which case historical sync is skipped. Once a sync is live, our ongoing sync process will keep your data in sync.

If you're planning to change the structure of your tables and want to perform historical sync afterward. You should follow the below steps:

1. Stop the sync.
2. Make modifications to your table(s) structure.
3. Edit the sync for the new table structure.
4. Make the sync live, which will perform the historical sync.

Ignoring or Deleting Unmatched Records

Historical sync is a one-way process, allowing you to sync data either from left to right or right to left. If the target app contains more records than the source, these extra records are considered unmatched. You can choose to either leave these records unchanged or delete them.

If your sync includes multiple tables, you can configure this setting individually for each table.

Deleting Unmatched Records

If at least one unmatched record is detected, an option to manage these records appears. To delete the unmatched records:

1. Select the "Delete items" radio button.
2. Click the "Apply" button.

   

When the historical sync runs, it will automatically remove the unmatched records from the target app.

Troubleshooting

The record count doesn't match

You have completed the historical sync but subsequently discovered a discrepancy in the record counts between the tables of the two apps. This situation often arises when incorrect unique key(s) are selected. Occasionally, a field designated as a unique key may not be truly unique. To ensure accuracy, you can verify the validity of the unique key(s) by counting the distinct values across all rows. The resulting count should match the total number of rows.

Please note that you have the option to select multiple fields as unique. In such cases, Byteline considers the values of both fields as criteria for matching records (using an AND condition).