Step 1: Select salesforce ingest connector
- From the main menu, click on Add connector
- Select Salesforce Ingest from dropdown list
- Input connector name
- Choose the destination warehouse
- Enable Salesforce Sandbox connector in the checkbox
Step 2: Provide connector credentials
- Input the username, password and the login URL for your sandbox account
- Click on Check Connection for confirmation of successful connection and proceed to the Next page
The password here needs to be
For example, if password is
pass123 and your security token in
xqObSRDsWgZ6fGt then enter in the password field
Step 3: Configure source information
- Filtering options:
- Table whitelist : Add the list of tables to include here
- Table blacklist : Add the list of tables to exclude here
Use regular expressions or exact names
- Interval : Set the frequency of data extraction for the connector
- Sync Historical Data : Check to load the entire past database as a one time activity
- Click on Fetch Source Metadata to see the confirmation message and click Next
You can go back and uptate the black/whitelist and re-fetch source metadata or manually add loadunits in the next screen.
Step 4: Loadunits extraction settings
The list of loadunits with extraction mode and schedule is displayed.
Extraction mode is auto detected based on the table size and availability of primary key and timestamp column at the source table.
- Extraction mode: Can be snapshot, incremental update or incremental append. For incremental extractions timestamp column must be added
- Filtering Options:
- ColumnList : Add the entire column name or regex here
- Column Blacklist : As the name suggests, blacklist columns by adding column name or regex rule here
- Interval: The frequency of the extraction mode ranges in discrete intervals starting from 5 minutes.
- Source table name: Is auto selected for the loadunit during edit, should be added when adding a loadunit
Step 5: Edit Data Layouts
Datacoral allows you to change the datatype for different fields
- Update DataTypes as needed per loadunit (Please note that reducing the string size is not supported)
- Click on Next to add the connector
Step 6: Warehouse Configuration
For each of the load units on the left edit the load configuration as follows
- Load Mode: Datacoral supports three load modes * **Replace** : This is a wipe and load operation, replaces all the rows of the destination table with the results of the transformation query* **Append**: Insert operation where, the result of the transformation query are inserted into the destination table, rows already in the destination table are not updated* **Merge**: Upsert operation where the result of the transformation query can have rows that indicate that the destination table rows have to be inserted, updated, or even deleted. This mode allows for efficient incremental updates to destination tables
For Merge load mode, it is mandatory to provide the primary key, however we recommend that the timestamp column be shared as well.
- Copy options: Add the copy options (For more information visit Redshift documentation and Snowflake documentation)
Step 7: Confirmation of the configuration
You have successfully added the connector once you landed on the below page.