Diagnostics and Logging In ASP.NET

Before our application reaches production we need a plan to deal with errors, performance problems, and other unexpected behaviors. The ability to record and retrieve diagnostic information is an important characteristic of robust software. Diagnostic information could include the amount of time needed to execute a critical method, the number of transactions committed per second, or the number of users currently with active sessions.

Make A List, Check It Twice

Before deciding on what information to capture and publish we need to think about what our information needs will be. The needs will vary depending on the type of application, the size of the application, and even the application’s environment (an application hosted by an ISP will typically have fewer options available). One good way to start is to make a prioritized list of diagnostic needs with details on how the system will send and store the information. An example list follows:

Diagnostic	Details
Runtime exception	The system needs to deliver exception messages directly to operators who can troubleshoot problems. The messages should also be persisted in a known location with a date and time stamp.
Incoming bad links	The system should record a 404 response with the incoming referrer to review weekly and notify partners of erroneous links.
Widgets Sold	It should be possible to monitor the system to see the number of widgets sold each second.

The above sample is just an example of what an application may need. Different applications will have different needs and priorities. For instance, a secure application facing the Internet may want to record invalid login attempts along with the date, time, and client IP address. The recorded information may provide clues to track down someone launching a dictionary attack against the application.

Meeting the Requirements

Part of meeting your requirements will be to review the options presented in the rest of this article and matching them against your requirements. In some cases you’ll find an obvious mismatch. For instance, we could widget sales into the machine’s application log, but doing so is not a great idea. The event logs on a machine are not well suited for high frequency events. The administrator will not want to browse through a slew of 404 entries in the Event Viewer application.

On the other hand, sometimes you’ll find the perfect fit. For example, it is a great idea to send email whenever there is a critical exception or error condition in the application. If the application cannot reach the database server, an operator needs to know as soon as possible. Since email can reach not only inboxes, but cell phones and pagers, it seems like a good fit.

We also need to look at our requirements holistically, and determine how much flexibility to put into the system. For instance, we might notify an administrator about an error by calling the following code snippet whenever an exception occurs.

private void LogException(Exception e)
{
    MailMessage mail = new MailMessage();
    mail.To = "admin @ odetocode.com";
    mail.From = "webserver @ odetocode.com”;
    mail.Subject = "Exception!!";
    mail.Body = ex.ToString();
    SmtpMail.Send( mail );
}

The code above carries out very specific instructions. Now suppose LogException looked like the following.

private void LogException(Exception e)
{
    LogUtility.Log(LogPriority.High, ex.ToString());
}

Now we’ve added a layer of abstraction – a class named LogUtility. You can imagine LogUtility might make the decision on who to email (and if it should email) based on the priority parameters. These decision might even be based on settings in a configuration file so we can change settings on the fly without recompiling. Obviously, you might need to write a little more code to pull this off, but the payoffs are increased flexibility and configurability. There are third party libraries available to help as we will cover later in the article.

With all these thoughts in mind, let’s jump into a discussion of the major players in acquiring and distribute diagnostic information.

ASP.NET Tracing

ASP.NET has built in tracing capabilities at the Page and Application level. With tracing enabled, you can view trace information from inside the web browser as ASP.NET can append the trace information to a page to the end of a request. Trace output contains a wealth of information, including execution times, view state size, server variables, user cookies, HTTP headers, query string values, and form values.

By using methods on the ASP.NET Trace object (of type System.Web.TraceContext), you can write custom messages into the Trace output. Tracing can be enabled and disabled with a simple change in the web.config file, and the performance cost of Trace.Write are negligible when tracing is disabled. The following excerpt from a web.config file would enable both page and application level tracing.

<configuration>
   <system.web>
      <trace enabled="true"
         requestLimit="20"
         pageOutput="true"
         localOnly="true" />
   <system.web>
</configuration>

For a good look at the output of a Trace, and how to hook into the trace output, take a look at Dino Esposito’s article “MyTracer Monitors and Traces ASP.NET Apps”.

Tracing is easy to enable and disable, and presents a plethora of information. You can also restrict trace information to only display in response to a request coming from the same machine, to be secure.

Trace can contain an overwhelming amount of information, and only the application developers can make sense of most of the data. Administrators may not be able make much use of trace. Tracing on high volume sites can be overwhelming, and it can be hard to pinpoint a specific problem area. A good bit of the information in a trace (like the page and view state sizes) should not come as a surprise by the time an application reaches production. Trace information is not persisted by default, and enabling a trace can slow down the application.

Performance Counters

ASP.NET exposes a large number of performance counters to view or record with the perfmon application. Performance counters aggregate more than just performance related metrics like requests per second and request wait time. There are also counters for exceptions per second, the cache hit ratio, active sessions, and aborted transactions. For a complete list, see Performance Counters for ASP.NET.

Performance counters are a great place for your application to expose high frequency events. If you have an event happening several times a second, like sales, you might want to write this information to a new, custom, performance counter for monitoring. Administrators can check in on sales using perfmon either on the local computer or from a remote computer. To create and use your own custom performance counters, see “Use Custom Performance Counters from ASP.NET”. Another good article describing the existing performance counters and how to use them is Marquardt’s “ASP.NET Performance Monitoring, and When to Alert Administrators”.

Performance counters are good to use when looking at totals and averages for the entire application over time. Many performance and stress testing tools will automatically record important performance counters to help pinpoint bottlenecks in CPU, memory, disk, or thread resources. When alerts go over a specified threshold, you can instruct the operating system to take action. 

While performance counters can be a great tool to measure the overall health and status of an application, they may not be helpful in diagnosing a specific problem, like using the wrong database name in a database connection string. These types of errors are better suited for the event log.

Event Log

The event log can be both a source and a destination for diagnostic information. As mentioned earlier, we do not want to record high frequency information into the event log, and fill up the log. The best event log messages will give an administrator a clear idea of what the problem is in an application. Recording the stack trace for an exception might not be the best idea, but a message such as “failed to connect to database server FOO” might give an admin a clue. Rob Howard’s article “An Exception To The Rule, Part 2”, shows you how to catch and write unhandled application exceptions into the event log.

The event log, being a local resource, will be around even when there is no network connection to reach a database server or email server. this is important to remember for those errors you absolutely, positively, need to record. Third party software exists to generate reports from the event logs. There are commercial tools available to monitor event logs and alert administrators about messages, as well as generate reports from event logs.

Web Logs

The second requirement we listed at the beginning of this article included retrieving information about file not found errors. You can find this inside the server’s web log, along with client IP addresses, the HTTP referrer header, the HTTP response code, the size of the response, and the amount of time taken to complete the response.

The data points in the web logs make the logs a hearty source of information – it is important to take advantage of the logs. The support article “Configure Web Site Logging in Windows Server 2003” contains a description of the configuration process with links to information for other platforms. There are many tools available, both free and commercial, to analyze and mine information from web logs. The article “How To Use SQL Server to Analyze Web Logs” describes how to build one of your own.

Email, Files, and Databases

There are more places to write diagnostic information than just the event log and performance counters. Emails are a great way to deliver information directly to those who need to know.

The system could write messages with a lower priority to a table in the database. There are several advantages to this approach over both email and file system. First, writing a record into a database table has a more permanent feel than just firing an email message off into the ether of the network. Also, the database can handle the concurrency requirements of a web application servicing multiple requests, and serves as a single location for diagnostic information.

Using the file system means possibly spreading information across as many servers as your application executes on. Whereas the database may not be online and available however, the file system should always be ready to accept information. You’ll need to craft your code carefully, though, to avoid threads colliding when trying to write to the same file. You might consider using a third party component with the ability to write to a file.

Logging & Diagnostic Software

There a few tools available which facilitate the recording of diagnostic information. One such tool is log4net at http://logging.apache.org/log4net/. The log4net software provides the abstraction layer we talked about earlier in the article. All you need to do is write code like the following:

log.Debug(“This is a debug message”);
log.Info(“This is an informational message”);

You can configure log4net to send your messages to a variety of destinations, even multiple destinations through the use of configuration entries. Destinations include an ADO.NET data provider, the ASP.NET Trace context, email, the event log, a UDP packet, and more. The following configuration entry would send message to a .NET remoting sink over TCP.

<appender name="RemotingAppender" type="log4net.Appender.RemotingAppender" >
	<sink value="tcp://localhost:8085/LoggingSink" />
	<lossy value="false" />
	<bufferSize value="95" />
	<onlyFixPartialEventData value="true" />
</appender>

Another piece of software along these lines is the open source Logging Application Block from Microsoft. The latest version of the block supports the encryption of diagnostic information.

Summary

We have covered quite a bit of information in this article to examine some of the popular techniques for acquiring and distributing diagnostic information in ASP.NET (and we have not covered every one!). The three keys to successful diagnostics include: 1) Making a list of requirements. 2) Matching those requirements to the available diagnostic and logging features. 3) Implementing a layer inside your software to make logging flexible, configurable, and effective.

By Scott Allen.