How to filter Azure FrontDoor health check requests from application insights

Azure FrontDoor is a great product. However, there's some hidden costs that are not immediately obvious when implementing Azure FrontDoor.

  • keep an eye on the number of routing rules that you have
  • 'outbound data from Azure FrontDoor' costs. Note the costs for the different zones.
  • application insights/log analytics costs. This is what we are discussing in this blog post. If your app users application insights/log analytics, data ingestion into could cost - a lot. For more details, have a look here- https://docs.microsoft.com/en-us/azure/frontdoor/front-door-health-probes. There's a number of global edge probes that make requests to a host. See here for details
  • the cost of having developers trying to search through application insights for issues where the majority of requests are coming from FrontDoor.

Re costs, this is the cost profile for a typical app that used FrontDoor, where filtering hasn't been implemented for application insights.

profile

Application insights costs for this app have been captured in light blue.

Notice how 20-30% of the cost of hosting an app is data ingestion caused by health check requests from Azure FrontDoor.

I found this interesting post which documented a great approach to preventing certain types of requests from flooding into application insights.

https://blog.hildenco.com/2020/03/how-to-suppress-application-insights.html

An added benefit is that by reducing FD requests from being captured by application insights - it should vastly reduce application insights data ingestion cost.

This is an app insights graph showing before and after the application of the change being discussed was implemented. The requests being made before 12:45PM were are being made by FrontDoor.

Server requests

Add a class that implements ITelemetryProcessor and then add some rules.

``` public class SuppressHealthStaticAndBotsResourcesFilter : ITelemetryProcessor { private ITelemetryProcessor Next { get; set; }

    // some of the static resources that I'd like to exclude from my telemetry
    static readonly List<string> Names = new List<string> { ".ico", "bootstrap", "jquery", ".css", ".js" };

    // next will point to the next TelemetryProcessor in the chain.
    public SuppressHealthStaticAndBotsResourcesFilter(ITelemetryProcessor next)
    {
        Next = next;
    }

    public void Process(ITelemetry item)
    {
        // To exclude requests from our telemetry we should use RequestTelemetry
        // For dependencies, use DependencyTelemetry
        if (item is RequestTelemetry req )
        {
            // static files
            if(Names.Any(n => req.Name.Contains(n)))
                return;

            // bot files
            if (req.Name.Contains("robots.txt", StringComparison.InvariantCultureIgnoreCase))
                return;

            // health checks
            if (req.Name.Contains("/health", StringComparison.InvariantCultureIgnoreCase))
                return;
        }

        // Send everything else
        Next.Process(item);
    }
}

```

Notice the 3 rules that have been implemented by this class. They filter:

  • health requests
  • search bots
  • various static files (i.e. images, js, cs files that as web site developers we are not particularly interested in

Make sure you debug your webapplication and put some break points on the if statements in the Process method to ensure the exclusions work as expected.

For example, browse to this location in your debugging web app:

When you press enter, make sure this return value in the filter is hit:

This confirms that the filter is working as expected. Of course before you can run this in your app, make sure

There's a sample with a code example here:

https://github.com/ossentoo/appinsightedfilteredapp

I've added some unit tests to make it clear how the request telemetry classes should work:

https://github.com/ossentoo/appinsightedfilteredapp/blob/master/unit.tests/AppInsightsFilterShould.cs