Data Warehouse Syncing

How data warehouse syncing works in Calixa

When you set up the Bigquery or Snowflake integration, Calixa will automatically start syncing data directly from your warehouse.

In order to do so efficiently, Calixa requires an "updated_at" field that shows the last time that row was updated in the database. Calixa can then sync only the rows that have been added or changed since the previous sync. The integration syncs every 15 minutes by default.

It is a database best practice to always have an "updated_at" column, but we realize that not all tables will follow this convention. If we detect that your database's "updated_at" field doesn't have millisecond granularity, we will assume it's a different date timestamp and should not be used as an update cursor.

In this case, since we can't use your "updated_at" field to limit the rows to sync, Calixa will switch to doing a full sync hourly (instead of every 15 mins). You will see an alert while creating your warehouse sync to let you know if your field isn't properly formatted.

Depending on how much data you have, these full syncs may increase your data warehouse bill. If you have questions or concerns about this, please reach out to us at team (at) calixa (dot) io.

Missing Fields

Calixa will skip any records that don't have required fields (fields like a User ID or email). We show a warning with the number of rows being skipped, so that you can correct the data in your before syncing.