Datacoral Version 1.9.0

  • Datacoral policy updates
    • Datacoral Role has additional firehose:PutRecordsBatch permission in Batch Role policies
    • Installation Role is scoped to specific resources for aws-service-role
    • Cross Account Role changed the scope of ec2:RunInstances permission in order to be able to create ec2 instance successfully for jupyterhub slice.
    • Limited the scope of cloudwatch event permissions to the resources that are part of datacoral installation.
  • Webapp
    • New UI for Organize slices that is consistent with Collect slice hierarchy
    • Organize / sliceName / schemaName / mvName
    • UI spreadsheets ( collect/slices, organize/schemas ) now support filtering by each column
  • CLI
    • Added ability to download MV definitions and update a MV via a file through the CLI
  • Collect Slices
    • Zendesk slice changes
      • Bug fixes to handle the case where the number of chats fetched in the chats loadunit exceeded a threshold
    • Fountain slice changes
      • New loadunit applicant_transitions
      • Added filterBy deploy parameter to set field used for backfilling last_transitioned_at or created_at
    • LaunchDarkly slice changes
      • Handling higher volumes of users data via users-search api and batching 50 api calls in single lambda
      • Users loadunits accept the user defined columnList in the deploy params
    • Google adwords data relocation into schema google, and bugfix including required libraries
  • Organize Slices
    • S3 and Glue Data Catalog Retention: automated deletion of partitions that reach specified retention age
    • Added lifecycle support for Redshift Materialized View updates
      • Store MV definition history
      • Support where clause changes in MV definitions
        • Allow users to specify new view definition and the timelabel (update-timelabel) at which the new view definition is to be applied
        • Notify users of downstream dependencies that might be affected
      • GDPR slice improvements
        • Replacement id needs to use the column value for deduplication
  • Harness Slices
    • PG Publisher - fixed an issue where the last page of query results was handled incorrectly