Datacoral Version 1.7.0

  • Datacoral stack policy updates
    • Added GlueReadOnlyAccess permissions to secure console user
    • Added CloudWatch permission "cloudwatch:PutMetricData" for lambda exec role to enable publishing cloudwatch metrics from internal alarms raised by alarm slice
  • Webapp
    • You can now look at timelabel history of a loadunit by clicking on the loadunit
    • Added sync mode for every load unit on the slice detail page
  • CLI
    • Removed dependency on S3 when checking for latest versions for upgrade
  • Collect Slice CLI
    • Changed default format for collect list command to json from csv
    • Added slice version option to the collect list command
    • Fixed collect slice update issue that was failing randomly
    • Bug fixes in JIRA slice to handle certain types of data which were causing data load failures
    • Added support for new load units in JIRA (project and component)
    • Bug fixes in pg slice bootstrap when the metadata queries return no results for certain tables
    • Bug fixes in pg slice to handle empty table black lists
    • Bug fixes in mysql slice to accommodate for replica lags while sync'ing loadunits in incremental mode
    • Support added for snapshot paginate mode in S3 slice. We can now load a series of files from a given S3 path in snapshot mode
    • Support added for pulling Facebook insights from version 2.12 in Facebook slice
    • Mysql password parameter name has been moved from secretKey to password
  • Organize CLI
    • Deprecated and changed frequency parameter in matview-create command to interval-in-minutes
    • Changed the refresh rules for materialized views to be based on timelabels
    • matview-stats command now returns stats based on timelabels
    • matview-reprocess command is added to the CLI to enabled reprocess of mvs for given timelabels
    • matview-describe command now takes a new option 'dependents' to list all the dependents of a given mv
    • Added validation checks to matview-create command to disable mv creation when the name has uppercase letters
    • Added Validation checks to matview-delete command to fail mv deletion when other mvs depend on it
  • System improvements
    • Re-architected the way materialized views refreshes are handled to make the mv refresh logic more predictable
    • Added better error handling in managed glue while creating partitions