Manage Database Latencies by Using the Troubleshoot-DatabaseLatency.ps1 Script in the Shell

Applies to: Exchange Server 2010 SP3, Exchange Server 2010 SP2

Topic Last Modified: 2012-07-23

The Troubleshoot-DatabaseLatency.ps1 script is used by Microsoft System Center Operations Manager 2007 to detect and correct high latencies on a database. You can create a scheduled task by using Task Scheduler to run this script. Database latencies can be caused by a number of issues, including the following problems:

Disk latencies due to a bad disk Disks with very high read or write latencies can be caused by a bad disk if the latencies persist over long periods of time.
Domain controller latencies A domain controller may exhibit long latencies in responding to LDAP search queries.
High user workload A single user (or set of users) conducting heavy operations can result in database latencies.

The Troubleshoot-DatabaseLatency.ps1 script performs the following actions:

Checks whether database latencies are above the latency threshold (as specified by the LatencyThreshold parameter). The default is 70 milliseconds (ms).
Checks whether the disk's transfers-per-second rate is less than the DiskReadRateThreshold performance counter and whether the disk's seconds-per-transfer rate is greater than the DiskReadLatencyThreshold performance counter. If this is the case, the script determines that the disk must be replaced because it's under low-load conditions but is exhibiting high latencies.
Checks whether a single user is using more than one thread over the last 10 minute period for longer than the value specified by the TimeInServerThreshold parameter. If this is the case, the user is likely contributing to the high latencies, and, as a result, the user's mailbox is quarantined. The user's mailbox is quarantined for six hours, during which the user won't have access to e-mail.

The default settings used in the Troubleshoot-DatabaseLatency.ps1 script are defined in the StoreTSConstants.ps1 script.

Note:
A script must be run from the folder in which it resides. By default, scripts installed with Exchange 2010 are installed at C:\Program Files\Microsoft\Exchange Server\V14\Scripts. The Shell doesn't load scripts automatically. To run a script from the local file, you must precede all scripts with ".\" For example, to run the SampleScript.ps1 script, type `.\SampleScript.ps1`. To run a script and specify the default installation path, type `"C:\Program Files\Microsoft\Exchange Server\V14\Scripts\SampleScript.ps1"`. For more information, see Scripting with the Exchange Management Shell.

Note:

A script must be run from the folder in which it resides. By default, scripts installed with Exchange 2010 are installed at C:\Program Files\Microsoft\Exchange Server\V14\Scripts. The Shell doesn't load scripts automatically. To run a script from the local file, you must precede all scripts with ".\" For example, to run the SampleScript.ps1 script, type .\SampleScript.ps1. To run a script and specify the default installation path, type

"C:\Program
Files\Microsoft\Exchange
Server\V14\Scripts\SampleScript.ps1"

. For more information, see Scripting with the Exchange Management Shell.

Use the Troubleshoot-DatabaseLatency.ps1 script

You need to be assigned permissions before you can perform this procedure. To see what permissions you need, see the "Mailbox databases" entry in the Mailbox Permissions topic.

The following parameter syntax set and table lists the parameters that you can use to detect and troubleshoot database latency issues.

	Copy Code
Troubleshoot-DatabaseLatency.ps1 -MailboxDatabaseName <DatabaseID> [-LatencyThreshold <1-200>] [-TimeInServerThreshold <1-600000>] [-Quarantine <switch>] [-MonitoringContext <switch>]

Parameter	Required	Description
MailboxDatabaseName	Required	The MailboxDatabase parameter specifies the mailbox database on which you're monitoring the database latency. This parameter accepts the following values: GUID Distinguished name (DN) Database name
LatencyThreshold	Optional	The LatencyThreshold parameter specifies the amount of time in ms that the database can be idle before it's considered to be latent. The default value is 70 ms.
MonitoringContext	Optional	The MonitoringContext parameter specifies whether the results of the command are written in the Application event log. If you don't specify this value, the event logs are written to the following location in Event Viewer: Event Viewer > Application and Services Logs > Microsoft > Microsoft-Exchange-Troubleshooters/Operational. You don't have to specify a value with this parameter.
Quarantine	Optional	The Quarantine parameter specifies that heavy users who are contributing to the high latencies will be quarantined. If you don't specify this parameter, users won't be quarantined. You don't have to specify a value with this parameter.
TimeInServerThreshold	Optional	The TimeInServerThreshold parameter specifies how much time (in seconds) that any user activity for which the Exchange store uses a thread can be spent per minute for a single mailbox before the mailbox is considered hazardous to the health of the database. The number of seconds is measured by aggregating the time that all threads working on behalf of the mailbox (during the period reported by the Get-StoreUsageStatistics cmdlet) spend inside the Exchange store. The number of seconds of work per minute is calculated by dividing this aggregate number by the period reported by the Get-StoreUsageStatistics cmdlet (by default 10 minutes). For example, if you set this parameter to 80 seconds, and a single user uses more than one thread for longer than 80 seconds in a 10 minute period, an event error is returned. If you specify the Quarantine parameter, the event error is returned, and the user's mailbox is also quarantined for six hours. The default value is 200 seconds.

Example

This example shows how to run the Troubleshoot-DatabaseLatency.ps1 script with the following settings:

The LatencyThreshold parameter is set to 100 ms. If the database is latent for more than 100 ms, an error is returned.
The TimeInServerThreshold parameter is set to 80 seconds. If a single user uses more than one thread for longer than 80 seconds in a 10 minute period, that user is quarantined.

	Copy Code
.\Troubleshoot-DatabaseLatency.ps1 -MailboxDatabaseName MBD01 -LatencyThreshold 100 -TimeInServerThreshold 80 -Quarantine

Note:
This example shows how to run the command manually once. To produce the data that the troubleshooter needs to effectively monitor your databases, you must run this command at regular intervals. We recommend that you use the Task Scheduler in the Microsoft Windows operating system to set up this task. For more information, see Task Scheduler Overview.

View the database latency troubleshooter output

In Event Viewer, the results of the Troubleshoot-DatabaseLatency.ps1 script are available in the following location: Event Viewer > Application and Services Logs > Microsoft > Microsoft-Exchange-Troubleshooters/Operational.

For example, the following represents output from event ID 5111. This output would be returned if the script ran successfully without errors.

The database latency troubleshooter has detected that the current latency of 1 ms for database MBD01 is within the threshold of 100 ms.

The following table displays the event ID, the description of the event, and, if necessary, the action to take.

Note:
The descriptions in this table are examples of the information that may be included in these events.

Event ID	Description	Action
5110	The database latency troubleshooter started on database MBD01.	Informational only. No action is required.
5111	The database latency troubleshooter has detected that the current latency of 30 ms for database MBD01 is within the threshold of 70 ms.	Informational only. No action is required.
5411	The database latency troubleshooter quarantined user f3bb8007-b6d1-45f5-b748-211d66fa43f6 on database MBD01 due to unusual activity in the mailbox. If the problem persists, manual intervention will be required.	Warning event: Continue monitoring.
5412	The database latency troubleshooter identified a problem with user f3bb8007-b6d1-45f5-b748-211d66fa43f6 on database MBD01 due to unusual activity in the mailbox. The user wasn't quarantined because the Quarantine parameter wasn't specified. If the problem persists, manual intervention will be required.	Warning event: Continue monitoring.
5710	The database latency troubleshooter detected that disk latencies are abnormal for database MBD01. You need to replace the disk.	Error event: You need to replace the disk. Contact your hardware manufacturer for replacement instructions.
5712	The database latency troubleshooter detected high RPC average latencies for database MBD01 but was unable to determine a cause. Manual intervention is required.	Error event: The cause of the latency couldn't be determined. You should create a dump file and analyze it to determine the cause of the issue. For Windows Vista, Windows 7, or Windows Server 2008, see How to Create a user-mode process dump file. For Windows Server 2003 or earlier, see How to use the Userdump.exe tool to create a dump file.