Applies to: Exchange Server 2010 SP3, Exchange Server 2010 SP2
Topic Last Modified: 2011-09-14
Several trends in server hardware apply to the Microsoft Exchange Server 2010 timeframe. One trend is a significant increase in processor performance and an increasing number of processor cores supported on a physical processor. This means that deploying a single Exchange server role on a standard commodity server with multi-core processors might leave a large portion of available CPU underutilized. Some customers expect server virtualization to more effectively utilize server CPU resources. Other customers want to combine Exchange server roles on the same physical server. Both are valid solutions.
Another trend is the availability of server models with multi-core processors and 10 to 16 internal disks. If you consider the number of mailboxes that can be supported by the input/output transactions per second (IOPS) provided by 10 to 16 disks, the Mailbox server role by itself generally won't utilize more than half of the available CPU resources. Adding the Client Access server role and the Hub Transport server role to this server will more effectively utilize the capacity of the server.
You can use the information in this topic as guidance for when to deploy multiple-role server configurations and how to correctly plan for multiple-role server configurations. An example illustrates the server sizing process for multiple-role servers.
Contents
Why Multiple-Role Configurations are Recommended
When Multiple-Role Configurations Are Recommended
When Multiple-Role Configurations Aren't Recommended
Hardware Recommendations for Multiple-Role Servers
Deploying a Multi-Role Server in a DAG
Example of Sizing for an Exchange 2010 Multiple-Role Scenario
Why Multiple-Role Configurations are Recommended
First, and foremost, the hardware you can procure today has processors that are extremely fast, yielding 5,000-6,000 megacycles when compared to our baseline processor configuration. That configuration consists of 2 x 4 core Intel Xeon x5470 3.33-GHz processors. (You can read more about our baseline processor configuration in the section "Example of Capacity Planning for a Mailbox Server" in Mailbox Server Processor Capacity Planning.) If you were to replace the processor architecture in your environment with processors on the market today, while keeping all other environmental factors the same, you would see a significant decrease in processor utilization.
To effectively lower the total cost of ownership on the servers you purchase, you should ensure efficient utilization of the system, which means the system must achieve and sustain near-80 percent CPU utilization during the worst failure mode at peak load. Here are four ways you can efficiently utilize the processors available today:
- Increase the workload, by deploying more active mailboxes per
server.
- Introduce a virtualization layer, and deploy the mailbox role
as a guest machine, along with additional guest machines.
- Deploy additional Exchange server roles onto the system.
- Use a combination of the above methodologies to find the
optimal configuration that utilizes the hardware as efficiently as
possible.
Deploying Exchange 2010 with a multiple-role architecture provides several benefits:
- The multiple-role architecture becomes a building block-based
architecture. With the multiple-role architecture, all servers in
the Exchange environment (excluding Unified Messaging and Edge
Transport) are exactly the same—the same hardware, the same
configuration, and so forth. This uniformity simplifies ordering
the hardware, as well as performing maintenance and management of
the servers.
- From a cost perspective, the overall goal is to ensure that the
architecture is balanced from both a CPU perspective and from a
disk perspective. Deploying server roles on separate machines can
result in long-term cost disadvantages as you may purchase more
CPU, disk, and memory resources than you will actually use. For
example, consider a server that hosts only the Client Access server
role. Many servers enable you to add a given number of disks in a
very economical fashion—when you are deploying that number of disks
and, more importantly utilizing them, the cost is essentially zero.
But if you deploy a server role that uses far less than the given
number of disks, you’re paying for a disk controller that is either
under-utilized or not utilized at all.
- From a cost perspective, the overall goal is to ensure that the
architecture is balanced from both a CPU perspective and from a
disk perspective. Deploying server roles on separate machines can
result in long-term cost disadvantages as you may purchase more
CPU, disk, and memory resources than you will actually use. For
example, consider a server that hosts only the Client Access server
role. Many servers enable you to add a given number of disks in a
very economical fashion—when you are deploying that number of disks
and, more importantly utilizing them, the cost is essentially zero.
But if you deploy a server role that uses far less than the given
number of disks, you’re paying for a disk controller that is either
under-utilized or not utilized at all.
- In many cases, using a multiple-role architecture enables you
to have fewer physical Exchange servers in the environment. Fewer
physical servers mean lower costs for a variety of reasons:
- Operational expenditures are almost always higher than the
capital expenditures. It costs more to manage a server over its
lifetime than it does to purchase it.
- You purchase fewer Exchange server licenses. A multiple-role
server only requires a license for one Exchange server and one
operating system, while breaking out the roles requires multiple
Exchange server licenses and possibly multiple operating system
licenses. For more information, see About Licensing: Licensing for Virtual
Environments.
- Deploying fewer servers has a trickle-down effect across the
rest of the infrastructure. For example, deploying fewer physical
servers may reduce the total rack and floor space required for the
Exchange infrastructure, which in turn reduces power and cooling
costs.
- Operational expenditures are almost always higher than the
capital expenditures. It costs more to manage a server over its
lifetime than it does to purchase it.
- A multi-role architecture ultimately distributes the load
across a greater number of servers than deploying single-role
servers because all Mailbox servers also become Hub Transport
servers and Client Access servers. This architecture provides
two benefits:
- From a scalability perspective, you’re distributing the load
across a greater number of physical machines. During a failure
event, the increased load on the remaining servers only increases
incrementally, which ensures the other functions the server is
performing aren’t adversely affected.
- From a resiliency perspective, the solution can survive a
greater number of Hub Transport and Client Access role (or service)
failures and still provide service.
- From a scalability perspective, you’re distributing the load
across a greater number of physical machines. During a failure
event, the increased load on the remaining servers only increases
incrementally, which ensures the other functions the server is
performing aren’t adversely affected.
For these reasons, the deployment strategy we recommend for Exchange 2010 is a multiple-role server configuration for most scenarios.
When Multiple-Role Configurations are Recommended
Multiple-role configurations are recommended for most scenarios for the following reasons:
- Simple unit of scale Organizations that
anticipate regular growth in the number of mailboxes should
consider deploying multiple-role servers. Because each
multiple-role server represents a building block, this model allows
the easy addition of building blocks to support the need for
increased capacity.
- Large-scale deployments that want to leverage modern
processors Based on scalability testing
performed prior to the release-to-manufacturing (RTM) version of
Exchange 2010, multiple-role servers can effectively utilize hex
core (or more) processors in a single server. This capability
allows large organizations to reduce the number of servers by
combining the Mailbox, Hub Transport, and Client Access server
roles instead of deploying these roles separately on servers with
fewer processor cores. This approach leverages the building block
model described earlier to provide a platform for large-scale
deployments while reducing the overall number of servers required.
Scalability of the multiple-role configuration on larger core count
systems should be validated with lab testing prior to production
deployment.
- Server deployments with internal
storage Many servers available today have two
physical multi-core processors and 10 to 16 internal disks. Several
improvements in Exchange 2010 reduce I/O requirements, making these
servers a cost-effective solution. Depending on user profile and
disk type, these servers generally support up to 4,000 mailboxes.
We recommend adding the Client Access and Hub Transport server
roles to these servers to utilize the additional CPU and make these
servers self-contained building blocks.
- Risk mitigation scenarios where the number of mailboxes
hosted on a Mailbox server is
limited Multiple-role servers are a solution
for deployments where risk management policies limit the number of
mailboxes that can be deployed on a Mailbox server. For example,
say an organization with 10,000 mailboxes has a policy that a
single server outage can't affect more than 25 percent of the
mailboxes in the environment. This requirement limits the number of
mailboxes per Mailbox server to 2,500. The additional capacity on
that server could be utilized by adding the Client Access and Hub
Transport server roles to the server.
- Small organizations and branch office
deployments Except as noted below when Windows
Network Load Balancing is used, a multiple-role deployment is a
recommended solution for deployments where the primary goals are to
minimize the number of physical servers, operating system
instances, and Exchange servers to manage. Running the Client
Access, Hub Transport, and Mailbox server roles on the same
physical server provides the necessary role redundancy with a
minimum requirement of two or three physical servers.
When Multiple-Role Configurations Aren't Recommended
Multiple-role configurations aren't recommended for the following scenarios:
- Small organizations, or branch office deployments, that want
to use Windows Network Load Balancing
(NLB) Multiple-role servers may not work well
for small deployments where two or three multiple-role servers are
being deployed as members of a database availability group (DAG).
For more information about DAGs, see Managing Database
Availability Groups. The clustering component added to Mailbox
servers that are members of a DAG prevents NLB from being installed
on the server. For more information about load balancing
recommendations, see Understanding Load
Balancing in Exchange 2010. However, there's still a
requirement to load balance inbound traffic to the Client Access
servers. In this case, there are two main options:
- Purchase a hardware load balancing appliance. Although there
are some entry-level NLB appliances, this option can be costly,
especially for smaller environments.
- Virtualize the Exchange server roles. In some environments, a
limited number of servers results in having to deploy domain
controllers, file and print servers, and other applications on the
same physical hardware as the Exchange 2010 servers. We recommend
that you implement the physical servers as host servers and isolate
applications inside a virtual environment. With this isolation, you
can run NLB for Client Access servers running on virtual
machines.
- Purchase a hardware load balancing appliance. Although there
are some entry-level NLB appliances, this option can be costly,
especially for smaller environments.
- Virtualization The maximum number of
active mailboxes that can be hosted by a virtual machine may be
reduced based on the combination of message profile and running in
a multi-role configuration. If you have light messaging users,
co-locating server roles in a virtual machine may make sense.
However, if you have heavy messaging users, you may be limited for
resources in a virtual machine, and thus you may need to either
reduce the number of mailboxes per Mailbox virtual machine or split
out the roles into separate virtual machines. In these cases, it
may be more efficient to deploy a single Exchange server role in
each virtual machine, or to deploy one combined Client Access and
Hub Transport virtual machine for every Mailbox server virtual
machine.
Note: You can’t install an Exchange server role on the hypervisor host server. Only management software (for example, antivirus software, backup software, or virtual machine management software) can be deployed on host servers. No other server-based applications should be installed on the host server (for example, Exchange, Microsoft SQL Server, or Active Directory). The host servers should be dedicated to running guest virtual machines.
For more information, see Understanding Client Access and Hub Transport Combined Role Configurations in Capacity Planning.
Hardware Recommendations for Multiple-Role Servers
As a general guideline, a multiple-role server should be sized to use half of the available processor cores for the Mailbox server role and the remaining half for the Client Access and Hub Transport server roles. Microsoft doesn't specify a maximum number of recommended processor cores for multiple-role servers. Instead, a maximum number of populated processor sockets are provided. This refers to the number of processor sockets on the motherboard where multi-core processors are connected. For more information, see Understanding Processor Configurations and Exchange Performance.
In addition to sizing the processor architecture, the memory must also be sized correctly for deploying a multiple-role configuration. For more information, see Understanding Memory Configurations and Exchange Performance.
Deploying a Multiple-Role Server in a DAG
When you're deploying single-role Mailbox servers in a DAG, consider capacity planning for single and multiple server failures in relationship to Mailbox server load. If you have four Mailbox servers in a DAG, size the Mailbox servers at 50 percent capacity so that they can accommodate double the number of active users, in the event of simultaneous failure of two Mailbox servers. Because the Hub Transport and Client Access servers are on different physical servers, the load on those servers isn't impacted much by the loss of one or two Mailbox servers.
When you're deploying multiple-role servers in a DAG, think about capacity planning for Client Access, Hub Transport, and Mailbox server load. If you have four multiple-role servers in a DAG, make sure there is sufficient capacity to accommodate a potential doubling of Hub Transport and Client Access server load. Because the multiple-role configuration aligns with the recommended processor core ratios for server roles, if you correctly size the maximum active databases for the Mailbox server role, Hub Transport and Client Access servers should meet the performance objectives for this scenario.
Example of Sizing for an Exchange 2010 Multiple-Role Scenario
The following example illustrates the server sizing process for multiple-role servers. The example has the following design assumptions:
- Total mailbox count 24,000
- Mailbox profile 100 messages per day
(for example, 20 sent and 80 received)
- Database cache per mailbox 6 MB
(based on a 100 message per day profile)
- Availability requirements Mailbox
resiliency within a single site; protection against simultaneous
failure of three database copies and two servers
- Database requirements 120 databases in
the DAG, 200 mailboxes per database
- Server platform 2 x 6 core
2.26 gigahertz (GHz) processor-based (X5650) server (12
cores)
The following process applies:
- Calculate server count A four-node DAG
is required to protect against the simultaneous failure of two
servers. However, the customer has decided to deploy six servers to
control the maximum number of active mailboxes during a double
server failure event. Therefore, the design begins with six Mailbox
servers within the DAG.
- Calculate maximum active mailboxes per server based on the
activation model Assuming the active databases
are equally distributed across the nodes, each server ideally hosts
4,000 active mailboxes (24,000 ÷ 6). To calculate the active
mailbox count after a double-node failure (based on this example),
the mailbox count is divided by the remaining four nodes, which
equals 6,000 active mailboxes per node (24,000 ÷ 4).
In this example, the MaximumActiveDatabases parameter on the Set-MailboxServer cmdlet is configured for 30 to ensure that no more than 40 percent of the databases become active on a single server.
- Calculate active mailbox CPU
requirements Multiply the maximum number of
active mailboxes on a server by the megacycles per active mailbox
(6,000 × 2 megacycles = 12,000 megacycles), based on the
Estimated IOPS per mailbox based on message activity and mailbox
database cache table in Understanding the
Mailbox Database Cache. Multiply this value by 10 percent
for each additional database copy.
In this example, there's one active copy and three passive copies for every database, so the 12,000 megacycles is increased by 30 percent (12,000 × 1.3 = 15,600 megacycles). For more information, see "Database Cache Metrics" in Understanding the Mailbox Database Cache.
- Calculate passive mailbox CPU
requirements Multiply the number of passive
mailboxes (when a server is hosting the maximum number of active
mailboxes) by the megacycles per passive mailbox (10,000 × 0.3
megacycles = 3,000 megacycles), based on the Estimated IOPS per
mailbox based on message activity and mailbox database cache
table in Understanding the
Mailbox Database Cache. For more information, see "Database
Cache Metrics" in Understanding the
Mailbox Database Cache.
- Add active and passive CPU requirements to get total CPU
requirement In this example, 15,600 active
mailbox megacycles + 3,000 passive mailbox megacycles = 18,600
megacycles total CPU requirement.
- Apply Mailbox CPU requirement to hardware
platform This example uses a 2 x 6 core
2.26-GHz processor-based (x5650) server. Based on the guidance in
Mailbox Server
Processor Capacity Planning, this equates to 60,083 megacycles.
Divide the required megacycles by the available megacycles based on
the server platform to estimate the CPU utilization at peak period
after a double-node failure (18,600 ÷ 60,083 = 31 percent
predicted CPU utilization).
We recommend that the Mailbox server role portion of multiple-role configurations be designed to not exceed 40 percent utilization during peak periods (for example, simultaneous failure of two nodes). This design allows sufficient space to accommodate CPU utilization of Client Access and Hub Transport server roles while maintaining total server CPU utilization at less than 80 percent during peak periods (for example, simultaneous failure of two nodes).
- Calculate active mailbox memory
requirements Multiply the number of active
mailboxes by the required database cache per mailbox. In this
example, with a double server failure, the remaining servers will
host 6,000 active mailboxes (6,000 × 6 MB) ÷ 1,024 = 35.1 GB.
The database cache requirements are based on the mailbox profile.
For more information, see "Database Cache Metrics" in Understanding the
Mailbox Database Cache.
- Apply total memory requirements to hardware
platform The total memory required is based on
the database cache requirements and the server design (dedicated or
multi-role). For more information, see the Default mailbox
database cache sizes table in Understanding the
Mailbox Database Cache. The total memory requirement for the
multi-role server in this example is 52.2 GB ((4 GB +
35.1 GB) ÷ 0.75). Because 52.2 GB isn't a standard memory
configuration, round up to 64 GB or the closest memory
configuration that your server supports.
Return to top