Exchange 2010 Outlook Anywhere users receiving prompts when proxied through Exchange 2013


Background

I was working with a customer who had Exchange 2010 & were in the process of migrating to Exchange 2013. As part of their migration process they pointed their Exchange 2010 Outlook Anywhere namespace (let’s call it mail.contoso.com) to Exchange 2013 in DNS. At this point all of their Outlook Anywhere clients should have been connecting to Exchange 2013 & then been proxied to Exchange 2010. While this was somewhat working, they also immediately noticed users were randomly being prompted for credentials, resulting in a negative user experience.

Sometimes the prompts would be when connecting to Public Folders while other times mail or directory connections from Outlook to Exchange.

Resolution

When I was approached with this issue/symptom it sounded familiar. After a search through my OneNote I realized I previously had a discussion with some people I know from Microsoft Support regarding this issue. Turns out this issue was recently addressed via http://support2.microsoft.com/kb/2990117 “Outlook Anywhere users prompted for credentials when they try to connect to Exchange Server 2013”.

This is actually an IIS issue with Server 2008 R2 (the operating system Exchange 2010 was installed on) that’s resolved by a hotfix. After installing the hotfix & rebooting the issue was resolved & their users no longer received the prompts.

 

 

Exchange 2010 SP3 installation fails on SBS 2011


I had an interesting issue with Exchange 2010 SP3 installation on a SBS 2011 server last night. Installation fails on the Hub Transport Server Role with following errors.

sbs 2011 upgrade sp3 error

 

This made me scratching my head. Why is it trying to remove existing certificate that is used by Exchange? It’s also the default SMTP certificate, that’s why setup was not able to remove it.

After investing further, I see this line in the PowerShell script,

Write-ExchangeSetupLog -Info “Removing default Exchange Certificate”;
Get-ExchangeCertificate | where {$_.FriendlyName.ToString() -eq “Microsoft Exchange”} | Remove-ExchangeCertificate

So it’s trying to remove default Exchange certificate that was created during the initial installation, that has friendly name “Microsoft Exchange”.

I’m thinking, there is no way the Godaddy certificate has Friendly Name “Microsoft Exchange”. After looking at the certificate properties, it is indeed the problem. The Friendly Name is showing “Microsoft Exchange”, instead of mail.domain.com.

In order for us to install SP3, we have to use SBS console to import a temporary certificate, so it updates “LeafCertThumbPrint” property in this registry key,

“HKEY_LOCAL_MACHINE\Software\Microsoft\SmallBusinessServer\Networking”

 Note: you can also update the registry manually with one of thumbprint from existing certificate that is already imported.

Exchange 2010 SP3 installs fine after the cert change.  Since we didn’t export the existing GoDaddy certificate before running SP3 setup, it was removed by the setup. In order for Exchange OA and Activesync clients  to continue function,  we have issue a new certificate request with proper Friendly Name, then import the new certificate. You can also reuse the existing certificate on GoDaddy’s website by using “Re-Key” option, but you might end up with a certificate without private key. To repair the missing private key, you can run following command
   certutil –repairstore my <serial number>

 

 

ActiveSync Synching Folders but not Mail


Issue

One of our smaller customers running Exchange 2010 SP3 UR2 was having an issue with one particular mailbox being unable to download mail items via ActiveSync on any device. The odd thing was that the folder structure would come down but no mail items would be synched. The customer said it was working fine until about a week previously.

Troubleshooting

Looking through Event Viewer in the Application logs led me to the following events from “MSExchangeIS Mailbox Store”:
10030

A mismatch was detected between a view of a folder and the actual contents of the folder. The mismatched item was ignored.

Attempts may be made to rebuild the view, but if this message continues to persist for this mailbox, moving the mailbox to a different database may resolve the issue.

Database: Mailbox Database

Folder: [MBX:John Smith][AllItems]

MsgHeader ID: 1110-1E6B08

Folder ID: 1110-3DA14B

View ID: 1110-3DA582

View Name: 1110-3DA14B +A-D-T301c

Document ID: 294529

Function: EcPopulateInitialMsgViewTable(Search)

Followed by:

10031

A folder view which previously experienced consistency issues has been deleted and will be rebuilt the next time it is needed.

Database: Mailbox Database

Folder: [MBX:John Smith][AllItems]

MsgHeader ID: 1110-1E6B08

Folder ID: 1110-3DA14B

View ID: 1110-3DA582

View Name: 1110-3DA14B +A-D-T301c

Function: EcAgeOutOneView

After seeing these events I came to the conclusion that there was logical corruption in this user’s Mailbox preventing ActiveSync from pulling the mail items down. So I immediately went to the handy replacement for ISINTEG, “New-MailboxRepairRequest”. (Reference1 Reference2)

So in this case I ran the following command:

New-MailboxRepairRequest -Mailbox John.Smith -CorruptionType FolderView,ProvisionedFolder,AggregateCounts,SearchFolder

The command lets you know the request was created but not much more than that. To view the logs on Mailbox Repair Requests you need to head back to the Application Log in Event Viewer (Reference )

We can see the below entries in the log:

10047

Mailbox level online integrity check for request ec853fb3-1999-4911-9782-5170a31a37cb started:

Database=Mailbox Database

Mailbox=4F1B824D-5C81-477E-B40B-418C888109F3

Flags=Detect, Fix

Tasks=SearchFolder, View, AggregateCount, ProvisionedFid

10062

Corruptions detected during online integrity check for request ec853fb3-1999-4911-9782-5170a31a37cb

Mailbox:4F1B824D-5C81-477E-B40B-418C888109F3 (John Smith)

Database:Mailbox Database

Corruption          Is Fixed FID         Property              Resolution

“Folder View”, Yes, “1110-1E6B0C (Inbox)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B0C (Inbox)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B0E (Sent Items)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B0E (Sent Items)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B0E (Sent Items)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B0E (Sent Items)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B0E (Sent Items)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B0F (Deleted Items)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B0F (Deleted Items)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B0F (Deleted Items)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B17 (Drafts)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6B1A (Tasks)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-1E6D67 (Junk E-Mail)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-3DA14B (AllItems)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-3DA14B (AllItems)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-3DA14B (AllItems)”, 0x00000001, “Delete the corrupted view”

“Folder View”, Yes, “1110-3DA14B (AllItems)”, 0x00000001, “Delete the corrupted view”

10048

Online integrity check for request ec853fb3-1999-4911-9782-5170a31a37cb completed successfully.

We definitely found corruption so I had the user try the sync again and it worked!…….partly…..   😦

We were able to download mail items but whenever we tried replying to a message we were met with an error message. At this point I was pretty lost in terms of the logging available to me so I used an old trick, manually deleting the Device from underneath the User object in AD using ADSIEDIT (obligatory warning about using ADSIEDIT with care)

I opened ADSIEDIT (Start>Run>adsiedit.msc) & navigated to the Domain Partition (Default Naming Context). I then drilled down to the user object in question. Underneath the object you’ll find a container called CN=ExchangeActiveSyncDevices & underneath that you’ll find the various devices associated with that user.

ADSIEDIT1

ADSIEDIT2

Even if you use TestExchangeConnectivity.com’s ActiveSync test you’ll see an entry for that “device” listed here.

In my case I deleted each of these devices & had the user delete their profiles & re-create them. Unfortunately, we were getting the same errors regarding the inability to reply to messages.

Final Solution

So at this point I had but one reasonable option left, and it had been staring me in the face since I first saw the 10030 Event ID.

“Attempts may be made to rebuild the view, but if this message continues to persist for this mailbox, moving the mailbox to a different database may resolve the issue.”

So I created a new mailbox database as well as a move request for this mailbox & set the Bad Item Limit to 50 (since I expected further corruption that I didn’t catch before with the Repair Request). I checked the status of the move with the below command:

Get-MoveRequest | Get-MoveRequestStatistics -IncludeReport | fl

Once the command completed I was able to see that the move skipped 14 items because they were corrupted. It’s my assumption that there were other issues that were resolved from the move because a move is essentially Exchange copying all the mailbox data into an entirely new mailbox.

After this, the user was finally able to get full functionality out of their ActiveSync devices. Issue resolved!

The Full Story

After resolving the issue I was contacted for I began to ask additional questions about the environment to try & get a better idea of what could cause this type of corruption (admittedly, something I should have done from the very beginning as you’ll soon find out). I found that the customer had gone through a series of hardware issues which resulted in them ultimately running an ESEUTIL /P on their Exchange database. Upon hearing this, not only did things make a bit more sense but I realized it was time for a bit of lecturing.

Now I could spend a whole article detailing the ins & outs of ESEUTIL as well as proper database recovery practices but to be frank; ESEUTIL /P SHOULD ALWAYS BE A LAST RESORT! It is a hard recovery which essentially whacks everything out of the JET database that it doesn’t understand as valid data, in an effort to get it to mount. Ideally, if a customer’s database were in a Dirty Shutdown state & a Soft Recovery (ESEUTIL /R) failed; then the next step would be to restore the .EDB database file from backup & replay existing Transaction Logs to get the database to a current state (many Exchange backup solutions do this).

I’ve only ever had to run a /P for customers who did not have a backup & who’s only other recovery option would be manually backing up Outlook Cached mode to .PST (the ugliest of all options). Environments like these are an excellent example of customers who would be great for Office 365 because they don’t have the IT Staff to maintain a proper backup practice. Unfortunately, many individuals find themselves with a database that won’t mount & ESEUTIL /P is the first thing that turns up in their search results so they run it haphazardly. 9 out of 10 times, the database will mount & you won’t really lose much data. However, I’ve also seen a 150GB .EDB database file go down to 60GB after running a /P because an entire table or similar got whacked out of the database because it was corrupted. Bottom line, /P IS A LAST RESORT!!!!!

Back to our ActiveSync Issue. There’s one other thing that should be noted after running a /P on a database. It leaves your database in an UNSUPPORTED configuration; at least for the time being. The official word from Microsoft Support is that as soon as you run an ESEUTIL /P you should immediately run an Offline Defrag on the database (ESEUTIL /D; essentially creating a new database) & then run a New-MailboxRepairRequest on all mailboxes in it. So this really shines some light on the customer situation above. They performed a /P but performed neither of the above procedures on the database. To be honest, few customers do because the Offline Defrag is so time consuming (5-10GB/hr depending on HW) & requires downtime. This customer suffered the consequences because while their database would mount after running the /P, they still had logical corruption in the database. It just chose to adversely affect ActiveSync in this case.

Now this is where my personal practices slightly differ from those of MS Support (use at your own risk). Starting in Exchange 2010, mailbox moves are Online. So what I do is immediately create move requests for all mailboxes on the database in question to another database. The mere process of moving a mailbox should remove corruption (as seen above) & it has the benefit of allowing your users to work while the move takes place. Once the mailboxes have been moved I then run New-MailboxRepairRequest against all the mailboxes.

This isn’t always the best method, it’s just the one I use when the customer is really concerned about getting back up as soon as possible (RTO vs RPO). I’ve also seen cases where one bad database causes Store.exe to crash & bring down every other database on that server; so in that case the Offline Defrag is required. Either way, the Microsoft Support method should be your 1st choice. The important thing is to take this as a lesson of what can happen if /P is run in ignorance. It’s not the only way logical corruption can occur but I’ve seen it as the culprit more than once.

 

Additional Reference:

http://www.paulhite.com/2013/05/repairing-mailbox-corruption-in.html

Recovering a Mailbox Database after Accidental Deletion from Active Directory


Not too long ago some colleagues wanted to know how to recover an Exchange 2010 mailbox database in the event someone was to actually delete the database object from Active Directory. Kind of an odd request since the Exchange Management Shell & the Exchange Management Console prevents you from doing this on a database with mailboxes still on it but an interesting scenario to play with.

So let’s say a hapless Jr. Admin is playing around in ADSIEDIT or maybe even LDP (Active Directory for Grown Ups if you ask Exchange Principal Program Manager Greg Taylor) & accidentally deletes a Mailbox Database object from AD. Doesn’t matter how many copies of that database you have in your DAG, its lifetime is now tied to the speed of your AD replication.

So what do you do? Likely the best answer is for you to perform an Active Directory Authoritative Restore; this assumes of course that you have one & you know how to do it. It can also mean a serious undertaking if you’re in a large global environment. I suppose you could also immediately shut down your Domain Controllers in that AD Site before it has a chance to replicate the deletion to the other AD Sites.; then setup a new DC in that site ASAP. That also has its issues in terms of outages.

So I feel that in a crappy situation you should always have a clear understanding of all your options. So here’s another one that involves the old database file switcheroo & some Exchange shell commands.

So in this scenario I have a couple test mailboxes (TestUser1 & TestUser2) on a database called TestDB1 in an Exchange 2010 Environment.

a

b

The below screenshot shows via OWA that we have an email in the inbox for TestUser1 which is ultimately residing within the .EDB Jet database file for TestDB1.

2

Now let’s live life on the edge & get rid of TestDB1 via ADSIEDIT (children & the squeamish may wish to divert their eyes).

C

d

Now that wasn’t that bad. Let me just run a Get-Mailbox command to verify the mailboxes are still present in Active Directory…..

4Uh oh… I’m met with yellow text which states:

WARNING: The object ASH.COM\Users\TestUser1 has been corrupted, and it’s in an inconsistent state. The following validation errors happened:
WARNING: Database is mandatory on UserMailbox.

It’s important to note that while the Mailbox Database object has been removed from AD, the actual database files have remained untouched; so all of the mailbox data is still intact. I didn’t test this but I imagine the database itself would still remain mounted until the next time the Information Store Service was restarted or its cache of AD info expired. However, transport to/from that database or IIS would likely be the first components to fail.

So how do we fix this using a method not already listed above? Simple, just move the existing database files to a safe location (good idea to use Eseutil /mh to confirm the database is in a clean shutdown) & then create a new Mailbox Database in EMC with the same name as the one you just accidentally deleted.

Note: We’re about to create a blank database that users could be accessing & mail can potentially flow to. So to prevent data loss or Outlook .OST files being affected, go ahead & Disable/Stop the Microsoft Exchange Transport Service as well as the RPC Client Access Service so that no mail can be delivered to this database & no Outlook clients can access it.

5

Once the new blank (aka Dial-Tone) database is mounted, go ahead & dismount it.

6

Delete all the database files within this new mailbox database. Move the original .EDB file into the new database’s location (assuming it was in a Clean Shutdown state after running the Eseutil /mh command).

7

Go to the Properties of the Mailbox Database in EMC & check the box for “This database can be overwritten by a restore”. You should now be able to mount the database.

8

Go to Exchange Management Shell & run a Clean-MailboxDatabase against this database.

9

You should now see the “Inconsistent/Corrupt” mailboxes showing up in the “Disconnected Mailboxes” pane in EMC (in my example below, TestUser2 isn’t showing because I already fixed him before taking this screenshot).

10

You can now use the “Set-Mailbox” shell command to set the Mailbox to use the Database in Active Directory. This value got nulled-out after the accidental deletion which caused the object to be corrupt/inconsistent. EX: Set-Mailbox TestUser1 –Database TestDB1

11

You can now set the Information Store & RPC Client Access Services back to Auto & Start them. At this point the user should be able to login to their mailbox & access their original inbox with all data still there.

12

And that’s it. It may not be the method your organization uses to recover from this particular situation but I feel it’s important to understand your options & if anything, this serves as a demonstration of the flexibility you have with Database Portability as well as Shell. This same process can be used in Exchange 2013 with only the Management Tools being different.

Beware Full OAB Downloads After Installing 1st Exchange 2013 Server in Existing 07/10 Environment


Members of the Exchange 2013 Technology Adoption Program (TAP) have known about this issue for a while & the general public had the potential to figure it out once Exchange 2010 SP3 came out last month which allowed co-existence with 2013 in a lab environment; now the Exchange Team has been very clear about it with this recent blog post today. Actually, a Microsoft Support-led session at MEC was when I first heard about it in detail. So what’s the issue? Basically, you have the potential to experience an organization-wide full Offline Address Book download just as a result of installing the first Exchange 2013 server into your existing Exchange environment.

Background:

The Offline Address Book is used by Outlook Cached mode clients to be able to have offline access to Address Lists as well as some Group Metric data when they aren’t connected to the Exchange Server. For a very detailed explanation from Neil Hobson see the following article from him.

Issues can occur when the OAB for an organization grows to a large size, sometimes in the hundreds of MB. Things that contribute to this size are things like number of recipients in AD, number of distribution groups, populated user attributes, & certificate usage (reference ). It’s important to note that GAL Photos are NOT stored in the OAB. The OAB just includes a pointer to AD where the photo is actually stored (reference ). Fortunately, Exchange/Outlook is smart enough to only download the changes to the OAB instead of the entire thing every day. There are still some circumstances where the entire OAB will be downloaded again, which makes it very important to understand the size of your OAB so you know just how much of your networks bandwidth will be used when all Outlook Cached Mode clients perform a full download (reference ).

So as you might imagine, whether or not clients will perform a full OAB download becomes a topic of concern during an Exchange migration.

Offline Address Books in Exchange 2003/2007/2010 are associated with Mailbox Databases, specifically on the Properties>Client Settings Tab of the Mailbox Database in the associated Management Console:

OAB1

However, in the screenshot above you’ll notice that the “Offline Address Book” field is blank on this Mailbox Database. This is the case by default with all Mailbox Databases. This is not an issue because any Mailbox Database that has its “OfflineAddressBook” attribute set to $Null by default will use the Default Offline Address Book in the Exchange Organization. This OAB can be seen below:

OAB2

This means if you have configured multiple Offline Address Books in your environment then you would need to manually specify the additional OAB’s on the Databases you would want to use them; otherwise the default OAB would be used. Simply put, if the value for OAB is blank on a Mailbox Database then it will use the Default OAB. If it is hard-set then it will use whichever OAB you hard set it to. Some customers will hard set this value if they want Mailboxes on specific Mailbox Databases to use a specific OAB. Maybe an OAB that only contains a specific Address List instead of the entire GAL like the example below:

OAB3

Many organizations just have 1 OAB & as a result have never populated the Properties>Client Settings>Offline Address Book value of their Mailbox Databases. This is where a big issue can come into play during an Exchange 2013 migration, or even if you just want a single Exchange 2013 server in your environment for a test group of users.

Issue:

As the recent Exchange Team Blog post announcing 2013 CU1 states, you need to make sure all of your Exchange 2007/2010 Mailbox Databases have an actual value populated for their Offline Address Book. If they are currently blank, then populate them with your current Default OAB. Nothing will change in the environment as a result of this because they will continue to use their current OAB & continue to only download the OAB changes.

Failure to do this will result in each of these Mailbox Databases switching to use the Exchange 2013 Offline Address Book that gets created during installation of your first Exchange 2013 Mailbox Server. This will result in a Full OAB Download for all of your Outlook Cached Mode clients on these Mailbox Databases; a potentially nasty situation which could bring your network to its knees.

You can see the Exchange Team Blog post in question for steps on how to repoint these databases or just use the following commands which I have taken from the post:

“Get-MailboxDatabase | Where {$_.OfflineAddressBook -eq $Null} | FT Name,OfflineAddressBook –AutoSize”

This lists all Mailbox Databases in your environment with their OfflineAddressBook attribute set to $null.

Then run:

“Get-MailboxDatabase | Where {$_.OfflineAddressBook -eq $Null} | Set-MailboxDatabase -OfflineAddressBook (Get-OfflineAddressBook | Where {$_.IsDefault -eq $True})”

This command will grab each of these Mailbox Databases & populate the OfflineAddressBook attribute with the value of your Organizations current Default OAB. Effectively changing nothing in terms of client behavior but ensuring that when you install Exchange 2013, each of these Mailbox Databases do not switch over to using the 2013 OAB; at least not until you are ready & can stage this process, maybe one MDB at a time.

Summary:

These steps should be mandatory for any organization considering implementing an Exchange 2013 Server into their existing Exchange 2007/2010 environment.

Checking for Open Relay in Exchange 2007/2010


Scenario:

So this is a fairly common scenario & I figured I’d post an easy method to diagnose the issue. Customers will often suspect that they’re an open relay due to being placed on a blacklist or having issues sending email to certain domains. There’s some general confusion as to what constitutes as an Open Relay & even the difference between a Relay & a Submit action in SMTP terminology. Hopefully this can clear some of the confusion.

Background:

Submit = Submitting an email message to an SMTP server that is destined for a domain that exists on that server (or in that server’s environment). You’re sending it to an address that the server is authoritative for.

Relay = Submitting an email message to an SMTP server that is destined for a domain that exists in another messaging environment. You’re sending to an address that the server is not authoritative for.

So there’s nothing inherently wrong with relaying. It’s what happens if you use your Hotmail account to send an email to someone’s Gmail account. It happens every time you email someone outside of your own messaging system. The key detail is whether or not you have authenticated to the SMTP server beforehand. So when you’re using Hotmail or Exchange via Outlook/OWA then you have obviously authenticated either via an Authentication Prompt, OWA Form, or using NTLM.

So this typically comes up when a customer needs to have an application, network printer, or other device be able to send emails through Exchange (or any internal SMTP server).

So the important thing to point out here is that as long as the application/device only needs to be able to send to addresses that your SMTP server is authoritative for then it is a Submit action & not a Relay action. This just means you only need it to be able to hit a Receive Connector that allows Anonymous Submit; which is how most of the world’s SMTP servers are configured to accept email from the Internet.

However, if your application/device needs to be able to send to an address not under the authority of the local SMTP server then it will be performing an SMTP Relay action & will require additional configuration.

The recommended approach is to have the Application/Device authenticate to your SMTP server if it supports it. Alternatively, you can configure the Receive Connector (Exchange) to allow Anonymous Relaying from that Application/Device’s IP address.

For instructions please see this Microsoft Post.

http://blogs.technet.com/b/exchange/archive/2006/12/28/3397620.aspx

This is a very common issue amongst customers because they may not be familiar with how to configure this. However, unfortunately I will often see customers make an even worse mistake; allowing Anonymous Relaying from an entire range of IP Addresses or possibly the entire Internet. It won’t take long for Internet folks with malicious intent to figure this out & start using your server to SPAM whoever they wish. This typically results in your Exchange Server’s sending IP being placed on various Blacklists which can prevent you from sending to certain email domains.

Resolution:

It is ALWAYS recommended to create a separate Receive Connector for this purpose. In fact I tell customers to never mess with the Default Receive Connectors if they can get away with it. But what will ultimately happen is the customer will use the steps in the Microsoft article above to enable Anonymous Relaying on their Default Receive Connector instead, which they’re probably also using as their Internet ingress point. The problem with this is that the Remote IP range of that connector is 0.0.0.0-255.255.255.255 out of the box; meaning the entire Internet.

Another thing the customer might do is create a new Receive Connector for Relaying but instead of just having 1 IP address in there (the IP of their Application Server or Network Device) they’ll add an entire range or more IPs than are needed. This can get pretty complicated to troubleshoot if you have many different Receive Connectors on many different Exchange Servers in the environment.

So I’m hoping people can use my explanation to help them configure this properly as well as troubleshoot any issues they may have. In addition to that, here’s a very useful command to use in Exchange Management Shell to list out all Receive Connectors in the environment that have the Anonymous Relay permission enabled. Use this to track these connectors down & then verify the RemoteIP Ranges are properly scoped/configured to be as secure as possible.

Get-ReceiveConnector | Get-ADPermission -User “NT Authority\Anonymous Logon” | Where-Object {$_.ExtendedRights -like “ms-Exch-SMTP-Accept-Any-Recipient”} | Format-List Identity,ExtendedRights

Disabling Outlook Anywhere & Avoiding Unnecessary Authentication Prompts for Certain Mailboxes


So this is a complicated scenario but only because this particular customer made it that way; in fact the solution ended up being very simple.

Scenario:

One of my Consultant co-workers pinged me on an issue he was sorting through at a customer site. They were using UAG for their Outlook Anywhere endpoint, both internally & externally. They had a policy to only allow Outlook Anywhere for roughly 30% of their user base. They were enforcing this using AD group membership in UAG to block access to the Outlook Anywhere rule for all users except for those on the allowed list.

Not only was this a nightmare to manage but it also caused Outlook Authentication prompts in certain scenarios. I’ll explain:

When internal Outlook users moved between wired & wireless networks (or vice versa), Outlook would be disconnected just long enough for it to attempt an Outlook Anywhere connection over HTTPS (since the RPC/MAPI connection didn’t reconnect quite fast enough for Outlook’s liking). Well since they were using NTLM for Outlook Anywhere this didn’t really pose a problem for the users who had been allowed to use the OA rule in UAG. However, the users who had been blocked (the majority of their users) would get Outlook auth prompts.

This raised another question from the Consultant & the client; why does enabling Outlook Anywhere on your Client Access Server result in all Outlook clients being enabled for Outlook Anywhere? Shouldn’t there be a method to disable it by default & only enable it via AutoDiscover in Outlook on the mailboxes we choose? Well I’m not Microsoft so I couldn’t answer that but what I was able to do was give them a much better solution going forward which wouldn’t require the hassle of managing group membership for the UAG rule.

Background:

When you enable Outlook Anywhere on your Client Access Server (Exchange 2007/2010), AutoDiscover will then start handing out information to all Outlook Clients on how to connect via OA if a direct RPC/MAPI/TCPIP isn’t available. This allows external Outlook clients to connect to their Mailbox without the use of a VPN.

Exchange AutoDiscover hands these out using what’s called Outlook Providers. These allow Administrators & Exchange itself to differentiate between the various settings used with Outlook Anywhere VS direct RPC/MAPI/TCPIP connections.

The EXCH Outlook Provider is used to hand out settings used when connecting via RPC/MAPI/TCPIP while the EXPR Outlook Provider is used to hand out settings when connecting via Outlook anywhere (RPC over HTTPS). You can view the settings of each by running Get-OutlookProvider | Format-List.

This is the response received using the Test E-mail AutoConfiguration utility in Outlook for a mailbox after Outlook Anywhere has been enabled in the environment. This image shows the EXCH settings.
1

This image shows the EXPR settings received in the same AutoDiscover response. These are the settings Outlook will use to connect to Outlook Anywhere if it needs to. Notice here it says “Exchange HTTP” for the Protocol opposed to “Exchange RPC” in the previous image.
2

Below you’ll find the XML response from the “XML” tab of the Test E-mail AutoConfiguration utility. You can see the settings for both the EXCH & EXPR Outlook Providers.

<Protocol>

        <Type>EXCH</Type>

        <Server>CASArrayAustin.contoso.local</Server>

        <ServerDN>/o=Contoso/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Configuration/cn=Servers/cn=CASArrayAustin.contoso.local</ServerDN>

        <ServerVersion>7383807B</ServerVersion>

        <MdbDN>/o=Contoso/ou=Exchange Administrative Group (FYDIBOHF23SPDLT)/cn=Configuration/cn=Servers/cn=CASArrayAustin.contoso.local/cn=Microsoft Private MDB</MdbDN>

        <PublicFolderServer>EX10A.contoso.local</PublicFolderServer>

        <AD>ausdc.contoso.local</AD>

        <ASUrl>https://mail.ash.org/ews/exchange.asmx</ASUrl&gt;

        <EwsUrl>https://mail.ash.org/ews/exchange.asmx</EwsUrl&gt;

        <EcpUrl>https://mail.ash.org/ecp/</EcpUrl&gt;

        <EcpUrl-um>?p=customize/voicemail.aspx&amp;exsvurl=1</EcpUrl-um>

        <EcpUrl-aggr>?p=personalsettings/EmailSubscriptions.slab&amp;exsvurl=1</EcpUrl-aggr>

        <EcpUrl-mt>PersonalSettings/DeliveryReport.aspx?exsvurl=1&amp;IsOWA=&lt;IsOWA&gt;&amp;MsgID=&lt;MsgID&gt;&amp;Mbx=&lt;Mbx&gt;</EcpUrl-mt>

        <EcpUrl-ret>?p=organize/retentionpolicytags.slab&amp;exsvurl=1</EcpUrl-ret>

        <EcpUrl-sms>?p=sms/textmessaging.slab&amp;exsvurl=1</EcpUrl-sms>

        <OOFUrl>https://mail.ash.org/ews/exchange.asmx</OOFUrl&gt;

        <UMUrl>https://mail.ash.org/ews/UM2007Legacy.asmx</UMUrl&gt;

        <OABUrl>https://mail.ash.org/oab/69ed661e-c685-4ae2-a284-da308d7bd480/</OABUrl&gt;

      </Protocol>

<Protocol>

        <Type>EXPR</Type>

        <Server>oa.ash.org</Server>

        <SSL>On</SSL>

        <AuthPackage>Basic</AuthPackage>

        <ASUrl>https://mail.ash.org/ews/exchange.asmx</ASUrl&gt;

        <EwsUrl>https://mail.ash.org/ews/exchange.asmx</EwsUrl&gt;

        <EcpUrl>https://mail.ash.org/ecp/</EcpUrl&gt;

        <EcpUrl-um>?p=customize/voicemail.aspx&amp;exsvurl=1</EcpUrl-um>

        <EcpUrl-aggr>?p=personalsettings/EmailSubscriptions.slab&amp;exsvurl=1</EcpUrl-aggr>

        <EcpUrl-mt>PersonalSettings/DeliveryReport.aspx?exsvurl=1&amp;IsOWA=&lt;IsOWA&gt;&amp;MsgID=&lt;MsgID&gt;&amp;Mbx=&lt;Mbx&gt;</EcpUrl-mt>

        <EcpUrl-ret>?p=organize/retentionpolicytags.slab&amp;exsvurl=1</EcpUrl-ret>

        <EcpUrl-sms>?p=sms/textmessaging.slab&amp;exsvurl=1</EcpUrl-sms>

        <OOFUrl>https://mail.ash.org/ews/exchange.asmx</OOFUrl&gt;

        <UMUrl>https://mail.ash.org/ews/UM2007Legacy.asmx</UMUrl&gt;

        <OABUrl>https://mail.ash.org/oab/69ed661e-c685-4ae2-a284-da308d7bd480/</OABUrl&gt;

      </Protocol>

      <Protocol>

This image shows the actual Outlook Anywhere settings being configured on the client as a result of the AutoDiscover EXPR response. (File>Account Settings>Change>More Settings>Connection)
3

Resolution:

So the solutions here is actually fairly easy & oddly enough, not well known. The Set-CASMailbox command can be used to block a particular mailbox from accessing various Client Access features. In this case we can use it to block Outlook Anywhere for John’s Mailbox. (Note: This command can also be scripted or piped to take effect on any number of mailboxes in the environment).

Set-CASMailbox –Identity John –MAPIBlockOutlookRpcHttp $True

After running this command you may need to wait about 15min for AD replication to take effect as well as 15min for AutoDiscover, Outlook Anywhere, & ultimately Outlook to take the change. To speed this process up you can recycle the MSExchangeAutodiscoverAppPool in IIS as well as restart the Microsoft Exchange Service Host service on each CAS.

Now, if you run Test E-mail AutoConfiguraton you’ll see that the Outlook client doesn’t even get the EXPR response because they’ve had that feature blocked.

4

Now if you look at the Outlook Anywhere settings (below) in Outlook, they are no longer even configured/enabled. Now in my lab using Outlook 2013 I had to actually perform a profile repair to get this change to take effect immediately. You will likely either have to wait longer for it to take effect or manually repair the profile.

5

So in this customer’s case, users who were not allowed to use Outlook Anywhere would not get the Outlook Authentication prompt when moving from internal wired to wireless or vice versa because their Outlook client never attempted the Outlook Anywhere connection; they just remained in a disconnected state until the new connection came fully online.

Also, after showing the customer this command they no longer had to rely on UAG to control who could or couldn’t access Outlook Anywhere; they could now just script the Set-CASMailbox command.