Microsoft has changed the way Offline Address Book (OAB) Distribution works over previous versions of the product to remove a single point of failure in the Exchange 2007/2010 OAB Generation design. While this new method of generating and distributing the Offline Address Book has its advantages, there is also a disadvantage which can result in a breach of privacy especially in multi-tenant environments. In this article we will be looking over how OAB Generation worked in the past as opposed to how it works now highlighting both the good and the bad.
Back in May 2009, I published an article entitled “How OAB Distribution Works” which has received a large number of visits and can be found on my personal blog under the following URL link. This article explains in detail the process behind OAB Generation in Exchange 2007 and 2010 and I highly recommend this read to anyone who is not familiar OAB Generation in previous releases of the product.
If you have not read the above article, let’s quickly summarise. In Exchange 2007/2010 every OAB has a mailbox server responsible for OAB Generation. The mailbox server responsible for OAB generation would generate the OAB according to a schedule and place it on an SMB share under \mailboxservernameExchangeOAB. The Exchange 2007/2010 CAS servers responsible for distributing this Offline Address Book would then download the OAB from this share to a folder advertised through Internet Information Services (IIS). Outlook clients then discover the path of the IIS website through autodiscover and download the files located under the OAB IIS folder through HTTP or HTTPS. If you need to gain a more in-depth understanding of this process again I encourage you to read the blog post above.
Now the problem with the above design is every OAB has one Mailbox server hard coded to be the server responsible for performing OAB Generation. The whole point of Exchange Database Availability Groups is to allow mailbox servers to fail and have databases failover to other mailbox servers which is a member of the same Database Availability Group. This presents a single point of failure. In the event the server responsible for generating the OAB was to fail, this OAB generation process would not failover to another server as the OAB is hardcoded to use that specific mailbox server as the OAB generation server. This means until an administrator brings back the mailbox server which failed or moves the OAB generation process for the specific OAB to another mailbox server, the OAB in question will never get updated.
To fix this in development of Exchange 2013, Microsoft needed a method to allow any mailbox server to fail without disrupting the OAB generation process, after all this was the whole idea behind Database Availability Groups – the ability to allow mailbox servers to fail. Instead of spending development time on putting together a failover technology around OAB Generation, Microsoft decided to incorporate the OAB Generation process into Database Availability Groups. This means instead of having one mailbox server generate the OAB and share it out via SMB, the Exchange 2013 server hosting the active mailbox database containing the Organization Mailbox is now the server responsible for generating the OAB. In fact in Exchange 2013, the OAB is now stored in an Organisation Mailbox so in the event a mailbox server fails or a database failover occurs, the OAB will move along with it. This architecture change has removed the OAB generation single point of failure which caused problems for organisations in previous releases of the product.
Whilst Microsoft removed the single point of failure from the generation process of the OAB, they introduced a problem with the distribution process. In previous releases there was a service running on CAS servers known as the Exchange File Distribution Service, a process which downloaded a copy of the OABs from various mailbox servers performing the OAB Generation task and placed the OABs in a web folder available for clients to download. This allowed companies running multiple OABs to provide NTFS permissions on the OAB folders to restrict who is allowed to download the OAB. This is especially useful in Exchange multi-tenant environments to ensure each tenant is allowed to only download the address book applicable to their organisation.
In Exchange 2013 Client Access Servers the Exchange File Distribution Service has been removed and the Exchange 2013 CAS now proxies any OAB download requests to the Exchange 2013 mailbox server holding the active organisation mailbox containing the requested OAB. The Exchange 2013 CAS finds which mailbox server this is by sending a query to Active Manager. As the Exchange 2013 CAS no longer stores each OAB in a folder under the IIS OAB directory, companies can no longer set NTFS permissions on the folders to restrict who has permissions to download each respective OAB. It is also important to note that inside each organisation mailbox there is no means provided for organisations to lock down who can download each OAB through access control lists. This introduces privacy issues for companies who offer hosted Exchange services as it presents a privacy breach. Someone who knew what they were doing and has a mailbox within the Exchange environment could download OABs from other organisations and in result gather full list of employee contacts for data mining purposes. Microsoft’s response to this threat documented in the multi-tenant guidance for Exchange 2013 is for hosting companies to “monitor the OAB download traffic” – in other words there is no real solution to prevent this from happening.
For more information about the Exchange 2013 OAB distribution process I strongly recommend the following article published by the Exchange Product Team.
Clint Boessen is a Microsoft Exchange MVP located in Perth, Western Australia. Boessen has over 10 years of experience designing, implementing and maintaining Microsoft Exchange Server for a wide range of customers including small- to medium-sized businesses, government, and also enterprise and carrier-grade environments. Boessen works for Avantgarde Technologies Pty Ltd, an IT consulting company specializing in Microsoft technologies. He also maintains a personal blog which can be found at clintboessen.blogspot.com.
Fourth in our series of Guest posts by Exchange MVP’s is Clint Boessen, in the last post he explored the complexities of sharing an email domain- known as a SMTP namespace- essential when migrating between servers or running multiple mail servers. In this post he walks you through the technical steps needed to achieve this when migrating from 2003 to 2010.
Clint Boessen is an Exchange MVP from Western Australia focusing on Microsoft Business Productivity solutions. Clint works for 4Logic Pty Ltd, an Information Technology and Consulting Company who is a Microsoft Partner predominately focusing on Microsoft. Clint has a diploma in Network Engineering, an MCSE, MCITP and is currently working towards his MCM in Exchange Server. Clint maintains his own blog containing lots of hints, tips and tricks for IT Professionals which can be viewed at http://clintboessen.blogspot.com. Part 2: Configuring SMTP Namespace Sharing between Exchange 2003 and Exchange 2010 organizations Welcome to part two of this three part series on SMTP Namespace sharing. In part one of this series we had a look at what SMTP Namespace sharing is and how it can be configured across multiple email systems. If you missed part one of this series you may view it here Part 1: An overview on SMTP Namespace Sharing. In part two we are going to look at how to configure SMTP Namespace Sharing between two Active Directory forests. One forest will run Exchange 2003; the other will run Exchange 2010. I have created a lab environment for our two forests which I have used to document the configuration steps. SMTP Namespace Sharing will be setup for @contoso.com. SMTP Namespace Sharing on Exchange 2010 The first thing we need to configure in Exchange 2010 for SMTP Namespace sharing is the accepted domain. Accepted domains are any SMTP namespace for which an Exchange organization sends and receives e-mail for a given SMTP Namespace. Accepted Domains can be configured in one of 3 ways:
- Authoritative Domain To specify that e-mail messages are delivered to a recipient that has a domain account in your Exchange organization, select this option.
- Internal Relay Domain To specify that e-mail messages are either delivered to recipients in your organization or relayed to a server outside your Exchange organization but still under the authority of your company or IT department, select this option.
- External Relay Domain To relay e-mail messages to an e-mail server outside the Exchange organization, select this option.
To configure SMTP Namespace Sharing for @contoso.com we must setup @contoso.com as an Internal Relay Domain.
Now that Exchange 2010 is able to accept email for @contoso.com provided we have our Receive Connectors configured correctly. Now we must create a Send Connector to route the email to the Exchange 2003 forest for any unresolved recipients. Follow the below steps for setting up the Send Connector: Note: To achieve the correct routing behavior, you must specify a Hub Transport server as the source server for the Send connector. If the Edge Transport server is specified as the source server for the Send connector, a routing loop will occur.
- In Exchange Management Console under Organization Configuration à Hub Transport click New Send Connector on the action pane.
- Provide the Send Connector a descriptive name. The usage type determines the default permissions sets that are assigned on the connector and grants those permissions to the trusted security principals.
- Internal Select this usage type if the e-mail system with which Exchange 2010 shares an address space is another Exchange 2010 organization
- Internet Select this usage type if the e-mail system with which Exchange 2010 shares an address space is a third-party e-mail system.
Select Internet as the foreign email system is Exchange 2003.
On the Address space page, click Add. In the SMTP Address Space dialog box, enter the domain name to which this connector will send mail, for example, contoso.com or *.contoso.com. You may select the Include all subdomains check box to use this connector to send e-mail to all subdomains of the address space. If necessary, you can also provide a specific cost for this connector. When you're finished, click OK. Leave the Scoped send connector check box cleared, and then click Next.
- On the Network settings page, select Route mail through the following smart hosts. Click Add.
In the Add Smart Host dialog box, select IP Address or Fully qualified domain name (FQDN) to specify how to locate the smart host. If you select IP Address, enter the IP address of the smart host. If you select Fully qualified domain name (FQDN), enter the FQDN of the smart host. The sending server must be able to resolve the FQDN. When you're finished, click OK. To add more smart hosts, click Add, and repeat this step. If you want to use a specific list of external DNS servers instead of the DNS servers specified in the adapter settings, select the Use the External DNS Lookup settings on the transport server check box. When you're finished, click Next.
- On the Configure smart host authentication settings page, select the method that's used to authenticate to the smart host. The following smart host authentication methods are available:
- Basic Authentication
- Basic Authentication over TLS
- Exchange Server Authentication
- Externally Secured (for example, with IPsec)
Exchange 2003 by default automatically allows Anonymous email from all IP addresses on its internal subnet. As a result choose None under Authentication type.
Note: It is recommended to configure Basic Authentication over TLS in production environments.
- Click Next.
On the Source Server page, click Add to add a source server. By default, the Hub Transport server that you're currently working on is listed as a source server. In the Select Hub Transport or Subscribed Edge Transport dialog box, select the Hub Transport servers that will be used as the source server for sending messages to the shared address space. When you finish adding source servers, click OK. Click Next.
- On the New Connector page, review the configuration summary for the connector. If you want to modify the settings, click Back. To create the Send connector by using the settings in the configuration summary, click New.
On the Completion page, click Finish.
SMTP Namespace Sharing on Exchange 2003 Configuring Exchange 2003 for SMTP Namespace sharing is very different to Exchange 2010. Exchange 2003 must be authoritative for the primary SMTP address space that is specified in the default recipient policy. Exchange 2003 does not have to be authoritative for any other SMTP address space configured on the default recipient policy. To configure SMTP Namespace sharing we must ensure Exchange 2003 is not authoritative for @contoso.com as this will not allow Exchange 2003 to forward unresolved recipients for @contoso.com to Exchange 2010. As Exchange 2003 must be authoritative for the primary SMTP address specified in the default recipient policy we must create a second recipient policy to set @contoso.com as primary and then click to clear the This Exchange Organization is responsible for all mail delivery to this address check box in the SMTP Address Properties dialog box. This creates the same effect as configuring "Internal Relay Domain" on an Exchange 2010 Accepted Domain shown above. Currently my Exchange 2003 server has @contoso.com setup as my primary SMTP Address on my default recipient policy. As you see I am unable to unselect This Exchange Organization is responsible for all mail delivery to this address as it is the default policy. We need to share the @contoso.com SMTP address space which is specified as the primary SMTP address space in the default policy. As a result we must create a new SMTP address space in the default recipient policy. The new primary SMTP address space that you create does not need to be valid in the Internet DNS. You can use a private SMTP address space such as @localhost or @example.local. In this lab environment we will be using the address space @source.local which Exchange 2003 will use to route internal e-mail messages. Perform the following configuration on your default recipient policy.
- Start Exchange System Manager, click Recipient Policies, right-click Default Policy, and then click Properties.
- In the Default Policy Properties dialog box, click the E-Mail Address (Policy) tab, and then click New.
- In the New E-mail Address dialog box, click SMTP Address, and then click OK.
- In the SMTP Address Properties dialog box, type the SMTP address space for which you want Exchange to be authoritative.
Click to select the This Exchange Organization is responsible for all mail delivery to this address check box, and then click OK.
- Click the new SMTP address that you created, and then click Set as Primary.
- Remove the SMTP address that you want to share from the default recipient policy. To do this, click the SMTP address that you want to share, and then click Remove.
Now create a new recipient policy to be used for SMTP Namespace sharing.
- Create a new recipient policy for the shared SMTP address space. To do this, right-click Recipient Policies, point to New, and then click Recipient Policy.
In the Properties dialog box, type a name for the new recipient policy, click Modify, and then click OK.
Click the E-Mail Addresses (Policy) tab, and then click New.
- Click SMTP Address, and then click OK.
- In the Address box, type the SMTP address space that you want to share. For example, type @example.com, or type @microsoft.com. In this lab we used @contoso.com.
Click to clear the This Exchange Organization is responsible for all mail delivery to this address check box, and then click OK.
Click the new SMTP address that you created, and then click Set as Primary.
Click OK, and then click Yes when you receive the following message:Note: If you use the above procedure and update user SMTP addresses using a recipient update policy, ensure a system state backup of Active Directory is performed on a domain controller before commencing the changes. ADModify.NET is an alternative way for updating user SMTP email addresses if the recipient update service is disabled. This is often the preferred way of performing the task as ADModify.NET allows you to revert changes.
After you configure the recipient policy for SMTP Namespace sharing, you must specify the means for Exchange to determine where to route messages that do not resolve to an object in Active Directory. To do this, create an SMTP connector that has the shared SMTP address space in the Add Address Space dialog box of the connector object. If you do not add the SMTP connector with the shared address space, any incoming e-mail that is destined to the shared SMTP address space is interpreted as an attempt to relay. In this situation, Exchange does not accept the incoming e-mail. Additionally, you must specify a server to which Exchange will forward unresolved e-mail. You can specify this destination server by using its host name or by using its IP address. Perform the following steps to create the SMTP connector.
- In Exchange System Manager, right-click Connectors, point to New, and then click SMTP Connector.
- In the Properties dialog box, type a name for the new connector in the Name box.
Click Forward all mail through this connector to the following smart hosts, and then type the host name of the destination computer or the IP address of the destination computer. You must type square brackets ([ ]) around the IP address. For example, if the IP address of the destination computer is 192.168.1.10, type [192.168.1.10]. In this lab the destination smart host is Ex2010.destination.local. Hostnames do not require brackets around the name.This computer will receive all e-mail that is not resolved to objects in Active Directory.
- Click Add, click an Exchange server in the Add Bridgehead dialog box, and then click OK.
- Click the Address Space tab, click Add, click SMTP in the Add Address Space dialog box, and then click OK
- In the Internet Address Space Properties dialog box, type the shared SMTP address space in the E-mail domain box. When you type the shared SMTP address space, do not include the at (@) symbol. For example, type example.com in the E-mail domain box. Then, click OK. In the lab environment we added contoso.com.
Click to select the Allow messages to be relayed to these domains check box. Because Exchange must also receive messages for the shared e-mail address space, you must let Exchange relay messages to this domain. This setting let's all the SMTP virtual servers that are listed under Local bridgeheads on the General tab accept messages for the shared e-mail address space.
- Click OK.
This email will be coming into Exchange 2010 from anonymous. A receive connector needs to be setup on the Exchange 2010 server to allow anonymous emails to come from the IP addresses of the Exchange 2003 servers in the source forests. For information on how to do this please see:http://clintboessen.blogspot.com/2009/07/allow-network-applications-to-relay.html
SMTP Namespace sharing is now configured between both Exchange 2003 Active Directory and Exchange 2010 Active Directory forests. There is another method for configuring SMTP Namespace sharing on Exchange 2003. This method involves modifying the SMTP virtual server under "Forward all mail with unresolved recipients to host". I do not recommend this method for two reasons.
- It does not stamp the message header which can result in mail loops.
- If you have any users accounts in the Exchange 2003 forest with the Exchange attributes associated to the account but no mailbox, whenever a user in the Exchange 2003 forest sends an Exchange enabled user account an NDR will be generated stating "A configuration error in the e-mail system caused the message to bounce between two servers or to be forwarded between two recipients. Contact your administrator". It is not smart enough to realise the user does not have a mailbox and to forward it to Exchange 2010. In Exchange users that have Exchange attributes enabled but no mailbox are known as "Mail users" whereas users that have mailboxes are known as "Mailbox users". There are times you want Mail users and not Mailbox users. When we perform cross-forest mailbox migration the source user account will revert to a "Mail User" and will still contain the Exchange 2003 attributes. These attributes and SMTP email address are important for keeping the GAL (Global Address List) populated in the source domain.
This concludes part two of this three part series. In part three we will be setting up SMTP Namespace Sharing between Exchange 2010 on-premises and Office 365 in the cloud.
Third in our series of Guest posts by Exchange MVP’s is Clint Boessen, exploring the complexities of sharing an email domain- known as a SMTP namespace- essential when migrating between servers or running multiple mail servers.
Clint Boessen is an Exchange MVP from Western Australia focusing on Microsoft Business Productivity solutions. Clint works for 4Logic Pty Ltd, an Information Technology and Consulting Company who is a Microsoft Partner predominately focusing on Microsoft. Clint has a diploma in Network Engineering, an MCSE, MCITP and is currently working towards his MCM in Exchange Server. Clint maintains his own blog containing lots of hints, tips and tricks for IT Professionals which can be viewed at http://clintboessen.blogspot.com. Part 1: An overview on SMTP Namespace Sharing SMTP Namespace Sharing is the concept of sharing an SMTP Namespace amongst multiple email systems. SMTP Namespace sharing is not limited to vendors or platforms; it can be configured across a wide variety of mail systems including Exchange 5.5 – 2010, Lotus Notes, Novell GroupWise, MDaemon and even many of your open source products such as QMail, Postfix, Sendmail and Zimbra. So what is an SMTP Namespace? An SMTP Namespace is simply the domain name at the end of an email address. Take my work email address firstname.lastname@example.org. The SMTP Namespace here is @4logic.com.au. So how does SMTP Namespace sharing work? Regardless which SMTP mail server you're using, all SMTP mail servers follow the same principles when it comes to SMTP Namespace sharing. To understand how SMTP Namespace sharing works first we must understand the most common options for configuring an SMTP mail server:
- SMTP servers authoritative for specific SMTP Namespaces.
- SMTP servers authoritative for all SMTP Namespaces.
SMTP servers authoritative for specific SMTP Namespaces are generally SMTP servers connected directly to mail storage- typically mailboxes, although mail storage can be anything from a simple file base folder structure to a database system. By authoritative I mean "accept email for". For example my work email server is authoritative for @4logic.com.au however I may also choose to configure my work email server to be authoritative for other SMTP Namespaces such as @contoso.com and @fabrikam.com should I wish to receive email for these suffixes. SMTP servers authoritative for all SMTP Namespaces generally accept the email then relay it on to another SMTP server. These servers are generally referred to as "smart hosts". There are multiple reasons why people do this. For example an SMTP server may be setup to accept email from all SMTP Namespaces for spam/virus filtering purposes in which the mail server cleans the email then forwards the filtered email onto its destination like Mimecast. Other people such as ISP's may setup SMTP servers open to all SMTP Namespaces relay for their customers who would like to send e-mail but don't have their own SMTP server. SMTP servers authoritative for all SMTP Namespaces are generally locked down by using SMTP authentication or locked down to a specific IP range otherwise they would be an "open relays" and open to abuse. Open relays are a plague to internet email; this is where majority of the internet's spam comes from. Spammers have created scripts which troll through internet IP ranges scanning for open relay servers. If you setup an open relay server exposed to the internet there is a good chance within a couple of days you will already be relaying thousands of spam email. Note: Depending on the how the Administrator configures the SMTP server "smart hosts" may also be locked down to only accept email for a predefined range of SMTP Namespaces. Now we understand the how SMTP servers can be setup let's run through an example to explain SMTP Namespace sharing. Say you're migrating from one system to another so we have two different email systems running; you're migrating to Exchange 2010 from Zimbra. Both systems are authoritative for the SMTP Namespace @4logic.com.au and both have user mailboxes assigned to the @4logic.com.au email addresses. When configuring the MX records on the 4logic.com.au DNS zone file without an external smart host with routing, we only have two options. One email system can be the primary destination for @4logic.com.au or we can configure email's to be load balanced between the Exchange 2010 and Zimbra. Regardless which method we choose we run into a problem – there is no way for the sending email server to know which email system contains the recipient's mailbox. If you have followed up until this point it should be easy to understand how it works now. The SMTP Namespace sharing works like this:
- Configure the SMTP servers to receive all email addressed to @4logic.com.au even if they do not have a valid mailbox for the recipient address.
- Let's say the Exchange 2010 server receives an email addressed to email@example.com, for email addresses it does not have a mailbox for it will forward it to the IP address of the Zimbra SMTP server.
- If the Zimbra SMTP server contains a mailbox for the user it will accept the email.
If you're thinking there is a problem with what I have just told you, please give yourself a pat on the back. What happens if the Zimbra email system does not contain a recipient for firstname.lastname@example.org? Zimbra would forward the email back to Exchange 2010. This would result in an infinite email loop between mail servers for any invalid SMTP recipient addresses. So how do email servers address this issue? SMTP Servers can add a flag to the MIME Header inside the email. The flag added to the message header is usually called X-Loop but can vary based on the type of SMTP server. If an SMTP server receives an email back that it has already stamped, it can generate an NDR and return it to the sender or simply delete the email. Not all SMTP Servers add a flag to the MIME Header automatically, configuration may differ. Please revert to vendor specific documentation. This concludes part one of this three part series. In part two we will be setting up SMTP Namespace Sharing between two forests, one running Exchange 2003 and one running Exchange 2010.