RPC Client Backoff/sec or RPCS Failed exceeded threshold

Topic Last Modified: 2009-05-26

The Microsoft® Exchange Analyzer Tool includes a performance data collection engine that is used to query performance counter objects on computers that are running Exchange 2000 Server, Exchange Server 2003, or Exchange Server 2007. The performance data collection engine collects data from the RPC Client Backoff/sec and Client: RPCs Failed: Server Too Busy/sec performance counters of the MSExchangeIS performance object to analyze performance data.

Exchange Analyzer generates a warning if it determines that either of the following conditions occurred during the sample time slice:

The MSExchangeIS\RPC Client Backoff/sec value was greater than 5
The MSExchangeIS\Client: RPCs Failed:Server Too Busy/sec value was greater than 5
This warning indicates that the server may be experiencing a heavier than normal load and that an increase in RPC latencies or RPC thread exhaustion may be occurring. This issue can manifest itself as mail clients experiencing slowdowns or by the appearance of the Cancel Request dialog box.

To address this warning, follow these steps and this guidance to determine the root cause, and then address the issue as appropriate:

If Client backoffs are occurring, check whether the RPC latencies are high. If the RPC latences are high, begin general troubleshooting to determine whether there are performance bottlenecks, such as high-traffic users, poorly performing disk subsystems, depleted memory, excessive load, and so on.
- Is the RPC rate high?
- Are the RPC latencies high?
  
  Examine the Exchange Performance Troubleshooting Assistant output to determine which client types have the highest latencies.
  
  If backoffs are occurring, examine the MSExchangeIS Client (*)\RPC Average Latency value at each instance to determine which client type has the most latency. Then, look at the MSExchangeIS Client(*)\RPC Operations/sec per protocol value to see which instance (client type) corresponds to the highest number of operations. If the number of RPC Operations/sec is high, this may indicate that the high latencies are occurring because the clients are causing more load than the server can handle.
  
  If the RPC rate is high, try to identify the cause of the high rate. Use the Exchange Server User Monitor (ExMon) tool to collect user information to determine whether individual clients are responsible for the high load.
  
  Even if the rate is high, you will want to identify the resource bottleneck on the server.
  
  Performance Troubleshooting Assistant reports MSExchangeIS\RPC Averaged Latency and MSExchangeIS Client\RPC Average Latency values. MSExchangeIS Client\RPC Average Latency is the latency of RPC operations as reported by the client applications. MSExchangeIS\RPC Averaged Latency is the average time that it takes on the server to process RPC operations. If the MSExchangeIS Client\RPC Average Latency value is significantly higher than the MSExchangeIS\RPC Averaged Latency value, the latency is occurring between the client and the server. In other words, this indicates a problem with network latency.
Look at the Number of Slow RPC Packets performance counter (MSExchangeIS\ RPC Num. of Slow Packets) view to see how many RPCs took longer than two seconds to process on the server.

If the average RPC latency is short and if there are few long RPCs, the problem is external to the Exchange server, such as a high client load.

If the average RPC latency is long or if there are many long RPCs, a resource bottleneck may exist on the server. Investigate the typical sources, such as disk latencies, queues, or CPU use.
- Look for resource bottlenecks on the server
  
  Is there a disk bottleneck?
  
  Is there a CPU bottleneck on the server?

For more information about the MSExchangeIS\RPC Client Backoff/sec and MSExchangeIS\Client: RPCs Failed:Server Too Busy/sec performance counters, see "Monitoring Mailbox Servers" (http://go.microsoft.com/fwlink/?LinkId=145200).

For more information about how to troubleshoot slow RPC request processing issues, see "Troubleshooting Slow RPC Request Processing Issues" (http://go.microsoft.com/fwlink/?LinkId=145201).

For more information about Exchange 2007 performance issues, see "Exchange 2007 Performance Issues" (http://go.microsoft.com/fwlink/?LinkId=116829).

For more information about how to monitor Exchange server performance, see "Monitoring Server Performance" (http://go.microsoft.com/fwlink/?LinkId=145203).