Datacoral Version 1.8.0

  • Datacoral policy updates
    • Datacoral cross account role now uses installation as a parameter, i.e, cross account roles are installation specific
    • Datacoral roles now include Batch Role policies
    • Datacoral lambda execution rule has two additional policies:
      • To chain lambda invocations
      • To allow athena queries
    • Datacoral installation role has changes in subnet group naming convention. (now prefixed with datacoral)
  • Webapp
    • Materialized view status and details are now available in the webapp
  • CLI
    • No changes
  • Collect Slices
    • Postgres Logical Decoding Slice - New Slice to extract changes from postgres by fetching write ahead logs
    • Launchdarkly Collect Slice - New Slice to collect exposure logs from Launchdarkly
    • Fountain Collect Slice - New Slice to collect data from Fountain (hiring platform)
    • Zendesk Collect Slice - added calls and legs loadunits
  • Organize Slices
    • GDPR Slice - New Slice to support the "Right to be forgotten" rule of GDPR.
    • Managed Redshift Slice
      • Added support to have loadunits as timeseries tables with retention
      • Retention of Timeseries tables is now timelabel-aware
  • Harness Slices
    • MySQL publisher - Enhanced to support replace mode
    • Postgres publisher - Enhanced to support replace mode
    • S3 publisher - Now compatible with Amazon Athena
    • Salesforce publisher - New Slice to insert/update objects in Salesforce
    • DynamoDB publisher - New Slice to insert/update entries in DynamoDB tables
  • Security Improvements
    • More refined restrictions around datacoral roles, cross account role and console user roles.
    • Restricting access to resources via roles ARN
    • Metabase specific resources are tagged with datacoral & installation name.
    • Installation role is now tagged with datacoral & installation name.
  • System Improvements
    • Monitoring and recovery improvements to catch all missing timelabels for collect loadunits as well as materialized views
    • Collect slices
      • Streamline fanout hierarchy in loadunits and pagination in collect slices
        • All parent pages are done before child pages are fetched
        • Handle fanout hierarchy in case parent loadunit has zero data
        • Load all pages for incremental load units at once through manifest files
    • Publisher slices
      • Handle pagination
      • Create timelabel summaries