Hightouch integrates with Datadog to help you gain greater observability into your syncs and monitor their health. Once you've authorized access to Datadog in Hightouch, we send real-time sync metrics and events to your Datadog account. You can then aggregate these statistics to gain insight into the health of your syncs.
You need your Datadog site and API key to set up the extension. You may want to create a new API key for Hightouch. Make sure that the API key creator in Datadog has the necessary write permissions for custom metrics (Metric Tags Write, Host Tags Write, Metrics Metadata Write) and events (Event Config Write).
Make sure to use the API Key itself and not the Key ID. Also ensure you use an API Key and not an Application Key to avoid the Datadog configuration is invalid error. You can find more information about the difference between these keys in Datadog's documentation.
Go to the Monitoring section of the Extensions page and select Configure extension.
Hightouch emits both metrics and events. Metrics are best suited for dashboards and statistical aggregations, while events are best for filtering. Don't hesitate to if there's another metric or event you're interested in using.
With just a few metrics, you can build complex dashboards and monitors. For example, you can create dashboards that target specific syncs that are critical to your operations and report the total number of operations or top syncs that have warnings or failures.
One common use case is to build a basic dashboard that tracks completed syncs by status over time.
The preceding chart uses the hightouch.sync.complete metric and sums the counts by status. You can filter further by including only statuses that are status:success or NOT status:success.
Anomaly detection can help flag significant changes, like many users suddenly being deleted or created. You can use the Datadog extension for anomaly detection in added, changed, or removed rows.
In Datadog, create a metric monitor with the Anomaly Detection method.
Define the metric by selecting the hightouch.sync.row_processed metric. Specify the op_type, for example, removed, and sync name. Select the metric aggregation as sum by and enter sync.
In Alert Conditions, set the Alert threshold and Warning threshold to values that make sense for your use case.
(Optional) You can inject the sync name into the monitor's subject or message body using {{sync.name}}.