Step 2: Set Up the Sync

If your feed file contains more than 20,000 items, contact your Technical Account Manager to sync your feed. If your feed is smaller, go to Assets › Data Feeds and create a new product feed.

After naming the feed, you can select one of the following sync options:

Sync a URL or upload a file
Sync securely via Amazon S3
Sync securely via SFTP

Sync a URL or upload a file

Upload a feed file (up to 25 MB), or refer to a URL location where a feed file is stored. If you are using HTTP authentication, add the username and password to the feed URL as follows http://user:password@url.

🚧
Make sure that the response header content type is one of the following:

application/json

application/xml

text/csv

Any other content will be rejected and resolved as a non-supported file format. Set this at the server level, where the file is hosted.

Click Add Another Source to combine up to 10 sources:

You can then use a parser function to control how the two feed files are merged, as seen in the following example:

function parse(feed1, feed2) {  
  var combinedFeeds =  feed1.concat(feed2);  
  return combinedFeeds;  
}

See the full parser function example.

Sync securely via Amazon S3

Use a password-protected S3 bucket managed by Dynamic Yield that contains your product feed file.

Click Create a Bucket to create a new S3 bucket. The credentials are only displayed once, so make sure you copy them to a secure location. You can generate new credentials using the Generate New Credentials option, but this is also limited to 2 times. Experience OS checks the bucket for a new file six times a day. If you just uploaded a file to the bucket, you can also click Sync Now to force the initial sync.

To perform the initial upload to the bucket, use the AWS CLI tool, as shown in the following example for a US account (EU endpoints are different and are provided in the Experience OS console upon feed creation). Note that the filename must be productfeed.csv.

How:

Push the file to S3:
aws s3 cp productfeed.csv s3://com.dynamicyield.feeds/[SECTION_ID]/productfeed.csv
Verify that the file is in S3:
aws s3 ls s3://com.dynamicyield.feeds/[SECTION_ID]/

Make sure to add “/” at the end of the endpoint to avoid an “access denied error”.

Sync securely via SFTP

For more detailed information on this method, see Upload a Product Feed via SFTP. Use an S3 bucket, secured using SFTP protocol, managed and provided by Dynamic Yield, which contains your product feed file.

Enter your SSH key to register it with Experience OS (only one can be registered per OS section at any time).
Use the Endpoint, Server ID, and username to upload your feed (see screenshot). The filename must be productfeed.csv. You can change the SSH key at any time using the Change Public SSH Key link.
Experience OS checks the bucket for a new file six times a day. If you just uploaded a file to the bucket, you can also click Sync Now to force the initial sync.

Sync frequency

By default, the product feed syncs with the source file every 24 hours. However, you can change this setting if your product catalog updates more frequently. Make the adjustment in Advanced Settings. If you sync the file via S3 or SFTP, Experience OS checks the bucket for a new file six times a day. If you just uploaded a file to the bucket, you can also click Sync Now to force the initial sync.

Alternatively, you can use an API to update the feed on demand. This option enables you to add, delete, or modify entire rows or values in your data feed and sync it more quickly.