AD Certificate Services not starting due to database in Dirty Shutdown

Background

I had a customer running SBS (Small Business Server) 2011, which runs Exchange 2010, who needed to renew their SSL Certificate as it had recently expired. I have quite a bit of experience with SBS since we have a large Support customer base running it & while it can be a pain to troubleshoot because of so many moving pieces (AD/Exchange/SharePoint/WSUS/SQL/RD Gateway all on one box) there are a few cool features. One of these features is the “Setup your Internet Address” wizard. Because SBS is also its own Certificate Authority the wizard will generate a certificate for you & assign it to Exchange/IIS/RD Gateway. It will also configure all the Exchange virtual directories for you as well as create a certificate install package for you to deploy to non-domain joined systems so your Outlook Anywhere clients will trust your CA.

 

Issue

However, when going to re-run the wizard to renew the certificate I received an error regarding the Active Directory Certificate Service not running. The System event logs had a 7024 event from “Service Control Manager” stating “The Active Directory Certificate Services service terminated with the service-specific error %%939523546”.

So we were unable to request a new certificate & the customer was hoping to avoid purchasing a third-party certificate since they had been working fine (for an extremely small shop) like this for several years.

 

Solution

After researching the error, I found that the error code given pointed to the Certificate Authority database being corrupted. So I navigated to C:\Windows\System32\Certlog & I found an old friend; an ESE (Extensible Storage Engine) database file.  If you didn’t know already, ESE isn’t just used for Exchange.

AD Certificate Services, DHCP (C:\Windows\System32\dhcp\dhcp.mdb), & Active Directory itself (C:\Windows\NTDS\ntds.dit) all use ESE databases.

The caveat however is that instead of ESEUTIL, you should use ESENTUTL to work with them.

(Additional references 1 & 2)

So I ran esentutl /mh <CA Name>.edb to view the header of the database file & found that it was in a Dirty Shutdown. I then tried to run a Recovery against the database by running Esentutl /r edb but this failed.

If this were an Exchange database then this would be where I would try to restore from a backup. Unfortunately this customer did not have a backup of their CA database file (I think a lot of customers would fall into this category) so I had to move onto running a Repair which is the dreaded “/P”.

Microsoft Support offers strict guidance around running a “/p” on Exchange (like performing a Defrag or a Mailbox Move followed by an Integrity Check/Mailbox Repair immediately after having to run a /p; Also, it should be considered a LAST resort) but no such guidance exists for Certificate Services since it is a much MUCH simpler database structure. But a ‘/P” is almost always a destructive action, with associated data loss, so if you have a backup you should always pursue that option first

I ran esentutl /p <CA Name>.edb & after it completed I was then able to start the Active Directory Certificate Services Service. All the proper data (including Issued Certificates & Templates) were still there & after re-running the SBS “Setup your Internet Address” wizard the customer now had a renewed certificate.

 

 

 

Requesting an Exchange Certificate from an Enterprise Certificate Authority using Command-Line (WebServer Template Missing in 08 /CertSRV)

Scenario:

Customer had hired a Consultant to originally setup their Exchange 2007 environment and now their Certificate had expired. Was originally setup to use their 2008 Enterprise CA so customer not only did not know how to generate the request from within Exchange but also did not know how to submit it to their own CA (I know).

Now with a 2003 CA I would just generate the certificate request from within Exchange Management Shell then Browse to http://CA-Name/CertSRV> Click Request a Certificate>Advanced Certificate Request>Submit a Certificate Request by Using a Base-64…..>Then select “Web Server” from the Certificate Template drop-down (Figure 1).

However, on a 2008 CA you do not have the option for Web Server (Figure 2)

This obviously makes it difficult to use the old familiar web-based interface to request your certificate. I believe these additional templates were removed from /CertSRV by default due to security reasons but I have yet to confirm.

Resolution:

So in this case I just needed to generate the certificate request on 2007, copy the .req file to my CA, and use the certreq.exe utility on the CA to process the request. The commands for the request are as follows:

Certreq.exe –submit –attrib “CertificateTemplate:webserver” C:\RequestFile.req NewCertName.cer

Depending on the settings of your CA this request may be auto approved (in which case the .cer file will be located in your current working directory in Command-Prompt; or just specify a path in the command) or you may need to approve it. You can do this either by launching the Certificate Authority MMC snap-in and going to “Pending Requests” or using the following command:

Certreq.exe –accept NewCertName.cer

Once you get the cert file just import it using Exchange Management Shell (if 2007; I usually recommend the GUI Wizard in 2010).

References:

Exchange 2007 Certificate Request Command

Exchange 2007 Certificate Import Command

Exchange Enable Certificate for Services

Exchange 2010 Certificate Request/Import/Assignment Process

If you choose to use the command line method on a 2003 CA then you may have to go through the following article

In searching to see if anyone else had published these steps I ran across the blog of Jeff Schertz. I’ve been to his blog before and always find great content. Here’s the referenced post but check out some of his other great articles; specifically for Lync.

Edit: Check this post if you receive a “Certificate Not Issued (Incomplete)” message via command prompt.