Telemetry cost control in Business Central

In the dynamic world of Business Central, telemetry data plays a crucial role in providing actionable insights into system performance and usage patterns. However, as valuable as telemetry is, managing its associated costs can become challenging if not properly controlled. In this blog post, we will explore strategies to effectively manage, and control costs related to telemetry insights in Business Central, ensuring that you get the maximum value without overspending.

Telemetry pricing

5 GB is free.

–Do you want to stay in that range?

▪Set a daily cap at 160 MB per day (to avoid unexpected charges).

One event typically takes 2-10 KB.

This totals in 5,000 – 80,000 events.

32,000 events per day are possible with an average event size of 5 KB.

This approach helps in effectively monitoring system performance while keeping costs under control.

What to Filter out?

High-Volume Data Sources:

• Web Service Calls: These can generate significant amounts of data.
• Background Sessions: Often produce large volumes of telemetry data.
• Other Noisy Data: Identify and exclude less critical data to stay within free limits.

Why Filter?

• Cost Efficiency: Reducing unnecessary data helps in staying within the free 5 GB limit.
• Relevant Insights: Focus on essential data for better insights and decision-making.

Filtering ensures cost efficiency by reducing unnecessary data and allows you to focus on essential information, leading to better insights and more informed decision-making.

What if you would like to pay?

Pricing Details:

• Cost: $2.76 per GB beyond the free 5 GB.
• Monthly Volume: Allows for 100,000 events.
• Daily Volume: Equivalent to about 3,300 events per day.

Considerations:

• Budget Planning: Understanding costs helps in budgeting for additional data needs.
• Data Prioritization: Pay for the most valuable data to ensure meaningful insights.

Remember, it’s important to plan your budget accordingly and focus on the most valuable data to make the most out of your investment.

Too much signals?

Let’s talk about the noise in our telemetry data. You see, sometimes we are bombarded with too many events flooding our system, but not all of them are really what we need. It’s like trying to find a needle in a haystack, but the haystack is constantly growing! So, now let’s discuss how to handle that noise, focus on the signals that matter, and finally, get only meaningful insights for our business.

Here we have a visual representation of how our telemetry investment areas have evolved across different versions of Business Central. It offers insights into where our focus and resources have been directed over time, guiding us in understanding the shifting priorities and strategic decisions made in telemetry development.

Number of events trough BC versions?

As Business Central evolves, so does the volume of telemetry events generated. Comparing versions, we’ve seen a steady increase: BC19 had 1671 events, BC21 rose to 1977, and in BC24, we’re now handling 2161 events. This trend underscores the expanding complexity of our system and the need for efficient data management strategies moving forward.

Limiting signals

As it turns out, every telemetry signal generated by Business Central is logged into Application Insights.

For example when you have external webservice consumers, every “Authorization Succeeded is logged… It gets big quite fast.

The easiest way to avoid unexpected cost, is by setting a ‘Daily cap’ inside of the related Log Analytics workspace. However, you might miss out on some crucial telemetry data after the daily cap has been reached.

Daily cap setting

To manage daily caps, head to Azure Insight Resource, from there

Go to the “Usage and estimated cost” section.

Use action data sampling to set your desired daily limit.  In this example I set 160MB per day as it is in our free plan.

If this limit is reached, you’ll receive notifications as displayed above, to avoid receiving this notification, maybe we should set some Data Collection Rules.

Filter events with Data Collection Rules

While we’re working with Azure settings, let’s talk about tweaking our Data Collection Rules.

These rules are like filters for our telemetry data—they let us pick and choose which events to include or exclude based on certain criteria. To set those click on the Workspace link in the right of the Application Insight Resource.

Then in the settings section, click on Tables.
In the list of the Tables we need to find AppTraces.

Once there, locate AppTraces, right-click, and select ‘Create Transformation’.

In the transformation editor, it’s time to craft our query to specify what we want to filter out.

This step is crucial as it allows us to define the criteria for excluding specific events or data points from our telemetry.

Next, we will see what are the signals that should be filtered out.

Determine which signals to limit

Let’s decide which signals to limit.

This process involves prioritizing critical events and considering their business impact.

We should focus on the key signals, and ensure that our telemetry data remains concise and valuable

• •RT0003 and RT0004  – Authorization Succeeded, log for webclient only, otherwise too many records (tostring(Properties.eventId) <> “RT0003” or tostring(Properties.clientType) == “WebClient”)
• •RT0006 – Report rendered: 1320 – Notification Email
• •RT0008 – Incoming WS calls for specific publishers that generates large amount of API calls
• •RT0019 – Outcoming WS calls for specific publishers that generates large amount of API calls

As we review our telemetry, signals such as RT0003 and RT0004, which capture “Authorization Succeeded” logs for web clients exclusively, are candidates for exclusion to prevent record overload.

Additionally, events like RT0008 and RT0019, associated with incoming and outgoing web service calls for specific publishers, may contribute to large API activity and could be considered for removal.

• • RT0030 – Error dialogs with empty text (not really errors) –(tostring(Properties.eventId) <> ‘RT0030’ or tostring(Message) !startswith “Error dialog shown:” or Properties has “alErrorMessage”)
• •AL0000JBO – Posting Preview called
• • AL0000E24 – Job Queue Lifecycle  Job queue entry enqueued
• • AL0000E25 – Job Queue Lifecycle  Job queue entry started
• • AL0000E26 – Job Queue Lifecycle  Job queue entry finished
• • AL0000CTV – Email sent
• •LC0040 – Task XYZ created: TIME scheduled to run after DATETIME. Ready to run: True
• •LC0043 – Task XYZ main codeunit ID completed.

Expanding our review, signals like RT0030, capturing error dialogs with empty text, may not align with actual errors and could be excluded from logs. How to filter those properly is shown on the list above.

Similarly, additional events shown on the list above, that are representing various system activities like posting previews, job queue lifecycle events, email sends, and task creations, can be excluded if they don’t meet our telemetry goals. By filtering out these signals, we ensure our telemetry data remains focused for decision-making.

Advanced filtering example rule

Source
| where not (tostring (Properties.eventId) in ("RT0003", "RT0004", "RT0006", "RT0008", "RT0010", "RT0014", "RT0015", "RT0016", "RT0019", "RT0020", "LC0024", "LC0025", "LC0040", "LC0042", "LC0043", "LC0045"))
| where tostring(Properties.environmentType) == "Production"

This query serves as an example to exclude specific events based on their EventId, ensuring that only relevant signals are present in our telemetry data.

AppPageView rules

For the AppPageView rules, there is not much to filter out.

But as we focus on only signals from Production environments, and while environmentType does not exist in properties of this Table, we need to use similar information from Measurements.

•0 = Production, 1 = Sandbox

source | where tostring(Measurements.environmentType) == "0"

It has same name, environmentType, with options 0 for production and 1 for sandbox.

All Event IDs

We covered roughly what would be a must to filter out. But all of the telemetry signals are well documented on Microsoft Learn and can be further investigated and determined if you would filter something more out.

Telemetry Event IDs in Application Insights – Business Central | Microsoft Learn

Stay tuned for more insights for Telemetry in Business Central! 🚀