Amazon S3 Connector

The Datacoral S3 Connector enables data flow from an Amazon S3 bucket into your Redshift data warehouse. You can periodically scan S3 for new files and load files into your warehouse. No programming, no plumbing, no additional logic, and no scripting required.

Features & Capabilities

  • Backfill: Full historical sync of your entire data
  • Data Extraction Modes: snapshot, incremental with pagination
  • Data Load Modes: replace, append and merge
  • Tables and Columns selection: The ability to select individual schemas, tables and columns for replication in the Datacoral's UI.
  • Data-layout: Automatically detect the data layout and columns from the S3 files (except for CSV format files)
  • Customizations: Update the configurations easily using the UI
  • Scheduling: Periodically scan a S3 bucket and load data in files into a warehouse (or other S3 files)
  • Select data:
    • Filter files by: Suffix and Prefix (such as a portion of the file name)
    • Specify nested folders within an S3 bucket through explicit names or partitioned by date-time through use of dynamic variable substitution. Supported variables are: {YYYY} - Year,{MM} - Month, {DD} - Day, {HH} - Hour, {mm} - Minute and {ss} - Second
  • File Formats:
    • Use gzip-compressed files, if needed.
    • Use these file formats: Delimited, such as CSV, JSON, AVRO and PARQUET

Read more about our Features and Capabilities in the next tab

Next Steps

Additional Information

Got a question?

Please contact Datacoral's Support Team, we'd be more than happy to answer any of your questions.