Delta sync approach focuses on transmitting only the changes or differences (delta) between the platform and Klevu backend. Instead of reading/transferring the entire dataset, delta sync identifies the modified portions and transmits only those portions to the destination, minimizing the amount of data that needs to be sent.
Klevu provides delta sync support for various platforms. In this discussion, we will explore each platform, examining its sync mechanism and the delta sync approach employed by Klevu.
Shopify
In terms of syncing with Shopify, Klevu initially creates a product feed of the store's catalog using Shopify APIs. This feed serves as the basis for data synchronization with the backend.
However, Klevu also offers delta sync functionality for the Shopify platform. With delta sync, Klevu attempts to identify products that have been modified or added in the last 15 minutes and syncs them to the backend every 10 minutes. However, there are a few limitations to consider:
1. Dependence on the "updated_at" field: The Delta Feed Updates feature relies on the "updated_at" field in Shopify. Consequently, if any products are not added or updated, and their "updated_at" field remains unaffected, they will not be included in the delta feed. Certain scenarios where this may occur include:
- Currency changes
- Creation of new currencies
- Creation of metafields assigned to the product
- Changing metafield values already assigned to a product.
- Product inventory changes when an order is placed
- When collections are added, updated, or deleted.
2. Product limit threshold: When fetching data, Klevu needs to set a threshold limit for the number of products to be included in the delta updates. For example, this limit could be set at 250 or 500, depending on factors such as the store's catalog size, metafields, and collections. This threshold is in place to prevent the overall data sync process from being slowed down due to Shopify API rate limits. If the number of delta updates exceeds this threshold, Klevu will rely on the main product sync process, which continues to function as usual.
3. Product delete: The delta sync updates provided by Klevu do not account for product deletions. In the case of product deletions, Klevu relies solely on the main product sync process.
To enable the delta sync feature for the Shopify store, it is recommended to reach out to the Klevu support team. They will assess your store's data and configure the setup accordingly.
Bigcommerce
In terms of syncing with BigCommerce, Klevu initially creates a product feed of the store's catalog using BigCommerce APIs. This feed serves as the basis for data synchronization with the backend.
However, Klevu also offers delta sync functionality for the BigCommerce platform. With delta sync, Klevu attempts to identify products that have been modified or added in the last 15 minutes and syncs them to the backend every 10 minutes. However, there are a few limitations to consider:
1. Dependence on the "date_modified" field: The Delta Feed Updates feature relies on the "date_modified" field in BigCommerce. Consequently, if any products are not added or updated, and their "date_modified" field remains unaffected, they will not be included in the delta feed. Certain scenarios where this may occur include:
- Currency is changed.
- A new Currency is created.
- Metafields are created and assigned to the product.
- Changing metafield values already assigned to a product.
- Pricelists are created and assigned to products.
- Customer group discount rules are created and assigned to products.
- Product Inventory changes when an Order is placed.
2. Product limit threshold: When fetching data, Klevu needs to set a threshold limit for the number of products to be included in the delta updates. For example, this limit could be set at 250 or 500, depending on factors such as the store's catalog size, metafields, and pricelists. This threshold is in place to prevent the overall data sync process from being slowed down due to BigCommerce API rate limits. If the number of delta updates exceeds this threshold, Klevu will rely on the main product sync process, which continues to function as usual.
3. Product delete: The delta sync updates provided by Klevu do not account for product deletions. However, we offer an alternative method for managing product deletions by leveraging Bigcommerce webhooks. With this approach, we can detect when products are deleted or disabled in your Bigcommerce store and subsequently remove them from the Klevu backend.
To enable the delta sync feature and delete products via webhook for the BigCommerce store, it is recommended to reach out to the Klevu support team. They will assess your store's data and configure the setup accordingly.
Magento
By default, Klevu supports delta updates for data synchronization with Magento data sync. During each sync, Klevu attempts to identify newly added, updated, or deleted data from the Magento system and syncs it to the Klevu backend.
However, there is a limitation to consider. Klevu relies on the updated_at field of the catalog_product_entity table in Magento to determine updated data. If your website utilizes an ERP (Enterprise Resource Planning) system, it is recommended to enable a specific setting or ensure that the "updated_at" field of the "catalog_product_entity" table is appropriately modified whenever a product is updated. This ensures that Klevu can accurately identify and sync the updated data.
Salesforce Commerce Cloud
By default, Klevu supports delta updates for data synchronization with SFCC data sync. To get more details on this, we recommend following this article.
Commercetools
By default, once the Klevu Commercetools connector is set up, delta sync is also a part of this setup. Again With delta sync, Klevu attempts to identify products that have been modified or added in the last 15 minutes and syncs them to the backend every 10 minutes. However, there are a few limitations to consider:
1. Dependence on the "lastModifiedAt" field: The Delta Feed Updates feature relies on the "lastModifiedAt" field in commercetools. Consequently, if any products are not added or updated, and their "lastModifiedAt" field remains unaffected, they will not be included in the delta feed.
2. Product delete: The delta sync updates provided by Klevu do not account for product deletions. In the case of product deletions, Klevu relies solely on the main product sync process.
Other Platforms
For other platform, please visit the link to set up delta updates.