New behavior in Outlook 2013 causing certificate errors in some environments

Background:

I originally discovered this issue back in early Feb & let a couple people on the Exchange Product Team know about it via the TAP but it seems to be affecting more customers than initially thought so I thought I’d share.

In Outlook 2007 through Outlook 2010 all domain-joined Outlook clients would initially query Active Directory for AutoDiscover information & ultimately find a Service Connection Point (SCP) value that would point them to their nearest Client Access Server’s AutoDiscover virtual directory. If that failed then they would revert to using DNS like any non-domain-joined Outlook client. The order of this non-domain-joined lookup is as follows:

https://company.com/autodiscover/autodiscover.xml

https://autodiscover.company.com/autodiscover/autodiscover.xml

Local XML File

http://company.com/autodiscover/autodiscover.xml (looking for a redirect website)

SCP AutoDiscover Record

Why it ever looked to https://company.com/autodiscover/autodiscover.xml I’ll never really know because honestly I’ve never come across a customer who had it deployed that way; most have https://autodiscover.company.com/autodiscover/autodiscover.xml but I imagine when Exchange 2007 was first being developed they weren’t exactly sure how customers would be implementing AutoDiscover.

Issue:

The above methods have served us well since Exchange 2007 timeframe but for some reason the Outlook team decided to try & implement some giddyup into Outlook & try to speed up the process. They decided to have domain-joined Outlook 2013 clients query both the SCP values in AD as well as the DNS records at the same time. If an SCP record was found it would still be used but in the event it failed then it would already have the DNS response ready to go. Great idea, however there’s one problem in the implementation.

If Outlook 2013 encounters any kind of Certificate error while doing the simultaneous DNS query then you will receive a pop-up in Outlook about the cert.

I actually stumbled upon this while in the middle of the scenario below:

That’s right, I actually get a certificate pop-up for my lab’s domain name (ash15.com) & not autodiscover.ash15.com like one would expect if I were to have a certificate issue on Exchange.

When Outlook 2013 does it’s simultaneous DNS AutoDiscover query the first URL it tries is https://company.com/autodiscover/autodiscover.xml, which in my lab environment resolved to my Domain Controller, which was also serving DNS, as well as a Certificate Authority. Ash15.com resolved to this server because it’s my internal Active Directory domain name & the name server entry resolves to my DC (just ping internaldomainname.local in your AD lab environment & you’ll see the same thing).

Now because I have web enrollment enabled & am listening on 443 in IIS the server responded. Also, because I did not have a cert installed on the server with ash15.com in the Subject or Subject Alternative Name then it gave the certificate error we see above.

Resolution:

The error is easy enough to get through & it only occurred on initial profile creation but this can definitely prove painful for some customers. Obviously my lab environment is a corner case but there have been several other customers report this issue with Outlook 2013 as well.

Here’s an example scenario.

Imagine you have a public website for andrewswidgets.com hosted by a third-party hosting site & you did not pay for HTTPS/443 services. However if you were to query the website using https then it could respond & obviously not return a certificate with andrewswidgets.com on it (because you haven’t paid for it you cheapskate…). Now imagine you begin deploying users using Outlook 2013 in your internal environment. In the past, they would have found the SCP record that would have pointed them to your internal Exchange 07/10/13 server for AutoDiscover & would have been happy as a clam (one Exchange Product Manager’s favorite way to describe Exchange bliss). However, now they may get a certificate pop-up for andrewswidgets.com when creating a new profile.

There are a couple ways around this. Make sure andrewswidgets.com doesn’t listen on 443, or possibly get a proper cert on your website that is listening on 443. Simply put, just make sure whatever andrewswidgets.com resolves to is something that’s not going to throw a certificate error.

I’ve heard nothing concrete or public but the Outlook team is aware of the issue & listening to customer feedback. I suggest contacting Microsoft Support if your organization is running into this issue.

 

Also, this KB offers methods to control which AutoDiscover methods are used by your Outlook clients

 

Requesting an Exchange Certificate from an Enterprise Certificate Authority using Command-Line (WebServer Template Missing in 08 /CertSRV)

Scenario:

Customer had hired a Consultant to originally setup their Exchange 2007 environment and now their Certificate had expired. Was originally setup to use their 2008 Enterprise CA so customer not only did not know how to generate the request from within Exchange but also did not know how to submit it to their own CA (I know).

Now with a 2003 CA I would just generate the certificate request from within Exchange Management Shell then Browse to http://CA-Name/CertSRV> Click Request a Certificate>Advanced Certificate Request>Submit a Certificate Request by Using a Base-64…..>Then select “Web Server” from the Certificate Template drop-down (Figure 1).

However, on a 2008 CA you do not have the option for Web Server (Figure 2)

This obviously makes it difficult to use the old familiar web-based interface to request your certificate. I believe these additional templates were removed from /CertSRV by default due to security reasons but I have yet to confirm.

Resolution:

So in this case I just needed to generate the certificate request on 2007, copy the .req file to my CA, and use the certreq.exe utility on the CA to process the request. The commands for the request are as follows:

Certreq.exe –submit –attrib “CertificateTemplate:webserver” C:\RequestFile.req NewCertName.cer

Depending on the settings of your CA this request may be auto approved (in which case the .cer file will be located in your current working directory in Command-Prompt; or just specify a path in the command) or you may need to approve it. You can do this either by launching the Certificate Authority MMC snap-in and going to “Pending Requests” or using the following command:

Certreq.exe –accept NewCertName.cer

Once you get the cert file just import it using Exchange Management Shell (if 2007; I usually recommend the GUI Wizard in 2010).

References:

Exchange 2007 Certificate Request Command

Exchange 2007 Certificate Import Command

Exchange Enable Certificate for Services

Exchange 2010 Certificate Request/Import/Assignment Process

If you choose to use the command line method on a 2003 CA then you may have to go through the following article

In searching to see if anyone else had published these steps I ran across the blog of Jeff Schertz. I’ve been to his blog before and always find great content. Here’s the referenced post but check out some of his other great articles; specifically for Lync.

Edit: Check this post if you receive a “Certificate Not Issued (Incomplete)” message via command prompt.

Problems with Federation Trust After changes to your certificate

Here is the situation and the solution

Situation

• I Had a federated trust setup in exchange 2010 SP1 (same issue can happen in RTM)
• I created it using the “UseLegacyProvisioningService” switch and so was using a 3rd party certificate
• After the trust was established I had some issues with the cert… and while it’s a long story the gist is that the cert was revoked and I received a new one.
• Well this caused an issue with my federation trust because I didn’t get the cert switched before the revocation (this can also happen if you delete the cert from the cert store or if it expires before you roll to a new one)

Symptoms: I received the following errors when I try to make any changes to the Federation trust or even try to delete it.

An error occurred accessing Windows Live. Detailed information: "The request failed with HTTP status 403: Forbidden.".
    + CategoryInfo          : InvalidResult: (:) [Set-FederationTrust], LiveDomainServicesException
    + FullyQualifiedErrorId : 84DE3E74,Microsoft.Exchange.Management.SystemConfigurationTasks.SetFederationTrust


Error:
Exception has been thrown by the target of an invocation.

 

An error occurred accessing Windows Live. Detailed information: "The request failed with HTTP status 403: Forbidden.".

The request failed with HTTP status 403: Forbidden.
Click here for help… http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.141).aspx?v=14.1.218.11&t=exchgf1&e=ms.exch.err.Ex1FCF67

Reason: the certificate that was used and is expected is no longer valid and so cannot be trusted on the live servers at Microsoft

Solution: Use ADSIEdit to change the cert to the new thumbprint

1. Add the new cert as the next cert in EMC under Federation Trusts
2. Open ADSIEDit with Domain admin Credentials
3. Connect to Configuration naming context
4. Browse to Domain –> Configuration –> Services –> Microsoft Exchange –> OrgName –> Federation Trusts
5. 
6. Rich Click on your Federation Trust in the right hand window and go to properties
7. Scroll down until you find the key “msExchFedOrgNextPrivCertificate” (this was where my solution varied from EXPTA’s)
8. Edit the key and select all the contents and copy, then close the key
9. Edit the Key “msExchFedOrgPrivCertificate” and paste in your copied contents (It may be a good idea to have a copy of this keys contents before overwriting it)
10. Close all windows
11. re-open the EMC or EMS run your failed commands again and life is grand!

Thanks to EXPTA and Gene at Microsoft for the assist in figuring this out.

Users missing from the Offline Address Book

Problem:

Source:        MSExchangeSA
Event ID:      9359 and 9323

OALGen truncated or dropped properties for entry user, windows in address list ‘\Global Address List’ because they exceeded the configured size limits for the version 4 offline address list.  The affected MAPI ids are:  8c6a 8008. – \Default Offline Address List

• 8c6a = PR_EMS_AB_X509_CERT = User certificate
• 8008 = PR_EMS_AB_IS_MEMBER_OF_DL_W  = Distribution List Membership (These can be ignored) http://blogs.msdn.com/b/dgoldman/archive/2009/12/01/please-read-events-9320-and-9359-on-new-installation-of-exchange-2010.aspx

Solution:

Problem is caused by too much data in the AD user properties, in our case there were too many certs associated with the user, another possibility is pictures or 3rd party extensions.

To verify the certs

1. Open Active Directory Users and Computers
2. Click –>  View –> Advanced Features
3. Go the the properties of the user in question
4. Click on Published Certificates
5. remove any certificates.
6. Click OK.

List of MAPI IDs

http://technet.microsoft.com/en-us/library/cc179232(office.12).aspx

Troubleshooting CRL issues in a Exchange 2010 Lab

Here is the story, I was building a Lab so I could test Domain secure  connections between exchange 2010 Orgs. I issued a cert to both servers from a CA in domain 1, and imported CA root cert to the trusted roots of both servers. Then when I tried to activate services or use MTLS on my connectors I got the following error.

The Certificate Status Could not be determined Because the revocation check failed

Here are the steps I took (with a some help) and got my servers talking and CRL checking working.

1. Verify that a CRL URL is published
• Re-issue cert if needed
2. Verify that the CRL URL can be accessed
3. Clear the URL cache
• certutil -urlcache crl delete
• certutil -urlcache ocsp delete
4. Check validity of the URLS in the cert
• certutil -verify -urlfetch C:\foobar2.cer
5. Clear and Force re-sync of cache
• certutil -setreg chain\chaincacheresyncfiletime @now
6. Clear and Force re-sync of cache and don’t use cache for 3 days
• certutil -setreg chain\chaincacheresyncfiletime @now+3
7. Installed and configured the 2008 Online Responder on my CA
8. netsh winhttp set proxy proxy-server="http=myproxy:8080;https=sproxy:8080" bypass-list= "*.foo.com"

 

 

Note: I finally I found that I had an issue with my TMG server when routing across it (even though it was supposed to not be filtered)
I moved my VM to the same networks (i.e. Both on 192.168.10.x) and then I was able to get it working…

Still need to figure out why TMG was breaking it, Conversely I did get it working with ISA 2006 without issue, I will update this post when I figure out the issue with TMG.

 

http://technet.microsoft.com/en-us/library/bb430772.aspx

http://blogs.technet.com/b/pki/archive/2007/09/13/how-to-refresh-the-crl-cache-on-windows-vista.aspx.

http://social.technet.microsoft.com/Forums/en-US/exchange2010/thread/299c8ebe-223c-43ab-8cbc-c8221991813a

Walkthrough Series: Threat Management Gateway \ Exchange publishing

Since it seems to be a popular series I wanted to consolidate links to all my TMG Publishing articles.

As you may know they are designed to be a very simple walkthrough to get you started, the in no way cover every scenario but it should be enough to get you started.Then once you have it working, backup the config and tweak to your hearts content :), Have fun securing exchange.

1. OWA
2. EWS\Outlook anywhere
   Alternate method without pre-auth: using the same listener as OWA
3. Active sync
4. SMTP 
5. Front-End \ Back-end TMG 
6. Ehlo article and Detailed white Paper From Greg Taylor.

You Had Me At EHLO… : Publishing Exchange Server 2010 with Forefront UAG and TMG

Some of you have been able to use my basic quick and dirty walkthrough for publishing Exchange2010 with TMG.

Good news! there is a very detailed whitepaper from Greg Taylor that goes in to much more detail.

Here is a link to the whitepaper as well as the EHLO article.

That being said I am still working on the:
Internet –> TMG FE –> TMG BE –> Exchange Publishing rules and hopefully will have the walkthrough done by the 23rd.

You Had Me At EHLO… : Publishing Exchange Server 2010 with Forefront UAG and TMG

Exchange 2007/2010 certificates and new Go Daddy

I found recently a situation where the Godaddy cert chain wasn’t installed and some phone clients had issues, the following is a proven request\import process to install the cert and chain.

 So I gave this process to a friend and found out that I should either rename or subtitle this post, “Getting SBS 2008 to recognige your 2048 bit certificate” – great side effect!

1. First make your cert request in powershell, it should look something like this:
   • New-ExchangeCertificate -GenerateRequest -SubjectName “C=US, O=Company, CN=mail.domain.com” -domainname mail.domain.com,autodiscover.domain.com,hostname,hostname.domain.local -FriendlyName mail.domain.com -privatekeyexportable:$true -path c:\cert_myserver.txt
   • IMPORTANT: don’t do any new cert requests or run any wizards until the cert is imported
2. Send the cert request to Godaddy as a UCC certificate
3. Import the Certificate to complete the request
   • Import-ExchangeCertificate –Path “C:\CertificateFile.cer” | Enable-ExchangeCertificate -Services pop, smtp, iis, imap
4. Export the certificate
   1. Start –> Run –> MMC –> Add Snap-in –> certificates –> Local computer
   2. Right click certificate –> all tasks –> export
      1. Include the certificate chain and private key
      2. Enter a password
5. Re-Import certificate including chain (this imports the chain certs also)
   1. Right click in a blank area of the certificate MMC –> all tasks –> import
   2. Select the certificate you exported –> import –> include the certificate chain.

 

And yes there are other was to get to the same result, this is just a simple easy to explain way

Publish Exchange 2010 with TMG (cont)

Walkthrough on publishing all roles through TMG with AD pre-auth on TMG. (Part 3/4 active sync)

Configure Active sync rule on TMG

1. Open Forefront TMG
2. Click on
3. In the Action Pane under Task click 
4. Give the rule a Name ill name mine “2010 Activesync”
5. 
6. Next –> Next
7. 
8. Internal Site Name should be your CAS server FQDN (needs to be on the cert)
9. 
10. The external name is what you use to access active sync(Also needs to be on the cert)
11. 
12. Select the Listener OA listener created on Part 2.
13. 
14. 
15. 
16. Finish
17. Now Outlook anywhere is published!

Go Back To OWA

Go Back to Outlook anywhere

Move on to SMTP

Publish Exchange 2010 with TMG (Forefront Threat Management Gateway) Series:

1. OWA
2. EWS\Outlook anywhere
3. Active sync
4. SMTP

Publish Exchange 2010 with TMG (cont)

Walkthrough on publishing all roles through TMG with AD pre-auth on TMG. (Part 2/4 EWS\Outlook anywhere)

Configure Outlook anywhere rule on TMG

1. Open Forefront TMG
2. Click on
3. In the Action Pane under Task click 
4. Give the rule a Name ill name mine “2010 OA”
5. 
6. Next –> Next
7. 
8. Internal Site Name should be your CAS server FQDN (needs to be on the cert)
9.  
10. The external name is what you use to access OA (Also needs to be on the cert)
11. 
12. Click new to make a new Listener
13. 
14. Name it whatever you want, I named mine “Basic Auth” because I am going to use it for Basic auth for OA\EWS.
15. 
16. Select one of the External IPs listed (not all IP addresses or you cant do multiple auth methods)
17. 
18. Select the certificate you imported earlier
19. 
20. Use HTTP Authentication
21.  
22. Click –> Next –> Finish –> Select the Listener.
23. 
24.  
25. 
26. Finish
27. Now Outlook anywhere is published!

• Go Back To OWA
• Now on to Active sync

Publish Exchange 2010 with TMG (Forefront Threat Management Gateway) Series:

1. OWA
2. EWS\Outlook anywhere
3. Active sync
4. SMTP