Data Monitor

What is Data Monitoring

Data Monitoring is the concept of assessing historical data and generating alerts based on expected criteria. This is one of the unique monitoring capabilities of Serverless360, which adds so much value to our customers. Data Moniotring can facilitate monitoring the Azure resources like Service Bus Queues , Topics, Logic Apps, Function Apps , Event Grid Topics and Event Subscriptions in various perspectives like,

• Evaluate availability of Service Bus Queue / Topic by monitoring on combination of metrics 'Server Errors', 'User Errors' and 'Size'
• Check on efficiency of Service Bus Queue / Topic by verifying if you are processing the right number of messages in the specified time window hence ensuring that you meet your business requirements
• Check on reliability of the Logic App by monitoring the number failed runs 'Runs Failed' every hour
• Get alerted on consumption of the Logic App, by monitoring if the 'Total Billable Executions' exceeds a certain volume etc.
  This feature can be of help to Azure portal administrators to ease tasks that an administrator is forced to do on daily basis, and it is a monotonous/repetitive task.
  This feature can also be used by Azure Developers to understand if the applications that interact with Azure Service Bus Queue / Topic handle them the right way, by monitoring the User Errors generated.

Configuring a Data Monitor

Pre-requisite to Data Monitor an Azure resource in Serverless360 is, the resource should be associated to your Serverless360 account using Service Principal. The Azure resource to be Data Monitored can be a Service Bus Queue , Topic, Logic App, Function App, Event Grid Topic or EventGrid Event Subscription. It is now possible to map multiple resources to a single Data Monitor.

To create a Data Monitor:

1. Log in to your 'Serverless360 account'
2. Click the 'Manage' button in the Composite Application in which you intend to create a Data Monitor
3. Click the 'Monitor' icon in the left navigation menu to get the list of default monitors. You can either choose to associate resources to the defualt Data Monitor which would be scheduled to run at a Frequency of 15 minutes and to end never, with 'Notify On Success' enabled
4. To create a new Data Monitor, click the 'Add Monitor' icon and select Data Monitor to open the 'Add Data Monitor' blade
5. Enter a Friendly Monitor name and Summary to the Data Monitor
6. Click 'Next' to set the schedule intervals, you can choose from the available frequency list.

The Data Monitor will calculate the next run time considering the monitor create time and selected frequency. For example, if your next run time is set at 10:15 and the schedule is to run every 15 minutes, the Data Monitor will fetch the consolidated data for specified metric from Azure API for duration between 10:00 and 10:15. This value will be compared against the threshold values to compute the results with one of 3 states: Error, Warning or Success.

9. Click 'Next' to choose the notification channels, toggle on/off the notification channels to deliver the alerts
10. 'Notify on Success' option sends a success alert when all the configured metrics are back to healthy state
11. Click 'Save' to configure the Data Monitor

12. Click on the Data Monitor to land on the dashboard. Click 'Manage Resources' to associate resources to this Data Monitor and define the warning and error threshold values for the resource metrics to be monitored. Choose the metrics based on the purpose of the monitoring.

Data Monitoring Dashboard

The data monitor status information will be plotted against the corresponding time on the calendar chart control. The status information is color-coded based on the actual status of the data monitor. To understand data monitoring dashboard better, let’s assume the scenario – “The administrator wants to get notified with a warning when there are more than 10 Runs Failed and receive an error when there are more than 25 Runs Failed in the associated Logic App".

In this case, when the number of Runs Failed is, say, 10 during a particular time period, the status will be plotted in Green color on the calendar chart control. Similarly, when the number of Runs Failed is, say, 20 during a particular time period, the status will be plotted in Yellow color on the calendar chart control indicating a warning. When the number of Runs Failed is, say, 30 during a particular time period, the status will be plotted in Red color on the calendar chart control indicating a breach in the error threshold condition. When the status message is clicked, the system will display a pop up with the actual status information of the data monitor.

1. Calendar Chart Area - The status information of the data monitor will be plotted in this section. The information will be plotted against the time interval on the Y-axis. Say, for instance, a data monitor is set up to monitor the Logic Apps once every 30 minutes during the business hours from 10 AM to 7 PM. In this case, the points will be plotted at every 30-minute slots starting at 10:00 AM through 7:00 PM. Depending on the status of the data monitor, the points will be plotted with either Green/Red/Yellow color code tick/ cross mark against the data point.

2. Calendar Control - The calendar control can be used to pick a date of interest and view the data monitor information. Every Data Monitor will have an exclusive calendar control with dashboard to list its executions. For easy identification, the dates that have data monitor configured are color-coded as Red, Green, and Grey.

   1. Red - At least one of the configured resources on the specific date has failed to meet the configured criteria
   2. Green - All the configured resources on the specific date has successfully met the configured criteria
   3. Grey - Indicates the current date

3. By clicking on the specific execution, you will open a blade with the actual results and the configuration of that specific data monitor.