Datacoral Version 1.29.0

  • Cloudformation Templates

    • Cross Account Role
      • List dynamodb backup tables and drop backup tables
      • Add ability to create lambdas with layers - used in jdbc connector
    • Lambda exec role
      • Lambda role to be able to read from SQS
  • Connectors

    • Postgres CDC - new way of doing change data capture from Postgres' Logical replication slot using batch compute transformations to minimize load on destination warehouse.
    • NetSuite JDBC new - Ability to connect to netsuite/openaccess using JDBC
    • Events - geo enriching of user specified IPs
  • Transformations

    • Improvements to batchcompute to speed up UDF creation
    • The UDF transform function now has an additional system_context parameter with information like the name of the Datacoral installation, timelabel of the specific transformation invocation, AWS region, and upstream dependencies of the transformation.
    • Allow addition of custom base images for UDF containers. Custom base images can be used to include other languages like Java (for JDBC connections) within the UDFs.
  • System improvements

    • New app shell with top-nav
    • Onboarding flow cleanup
    • Metadata events are now available in an SQS Queue in the case of table and column changes at the source. These events can be used to trigger workflows to handle the changes in metadata.