Datacoral Version 1.41.0

Connectors:

  • New API connectors launched
  • Hubspot connector now fully supports contacts and deals loadunits
  • PostgreSQL connector
    • Page size is automatically calculated for snapshot paginate based loadunits
    • Supports multiple schemas
  • GitHub connector: Loadunit pulls supports all status
  • PG-CDC connector:
    • Supports automatic backfill of newly added tables and columns along with column rename
    • Supports multiple schemas
  • Connector interface :
    • Connector add interface will disallow creating the connector with datacoral keywords (ie manifest, splits)

Transformations:

  • Organize commands now support a new option --warehouse-name for specifying the given warehouse.
  • Organize commands --slice-name is deprecated, will be end-of-life by 2020-12-31
  • Introducing generate command which allows creating system-generated parameters for different options
  • Introducing automatic DQ pipeline generation from CLI with the dq-pipeline command allowing the generation of input parameters for setting up the different types of DQ checks
  • Matview-reprocess now supports --force flag to reprocess sequential matviews

Onboarding:

  • Onboarding New Feature: Datacoral now supports the Ireland region (eu-west-1)

System improvements:

  • Security
    • Enhanced security in compliance with SOC2 for user password hash
    • Additional security extended for password changes
    • Enhanced security for user-name and team-name from DOS attack