Exchange 2013 SP1 Breaks Hub Transport service

I had an issue last night that woke me up at 2 am in the morning by the On Call phone. I feel that we might see this often when Exchange admins start to applying Exchange server 2013 SP1.

After installing Exchange 2013 sp1, MSExchange Transport service hangs at “Starting”, then eventually crashes with couple of event ID’s.

“Event ID 1046, MSExchange TrasnportService

Worker process with process ID 17836 requested the service to terminate with an unhandled exception. “

“Event ID 4999, MSExchange Common

Watson report about to be sent for process id: 2984, with parameters: E12IIS, c-RTL-AMD64, 15.00.0847.032, MSExchangeTransport, M.Exchange.Net, M.E.P.WorkerProcessManager.HandleWorkerExited, M.E.ProcessManager.WorkerProcessRequestedAbnormalTerminationException, 5e2b, 15.00.0847.030. “

Only way to get Transport service to start, is to disable all receive connectors and reboot the server. Does it sound familiar? My colleague Andrew Higginbotham  wrote this article few weeks ago. Although it was a different issue, but custom receive connectors on a multirole server is the key.

In my case, this is also a multirole server(CAS and Mailbox on one box).  Hub Transport service listens on TCP port 2525, and Frontend transport listens on TCP port 25.  There are two custom receive connectors that were created with Hub Transport role. Both are listening on TCP port 25. I’m not sure why they haven’t had external mail flow issue by now, but it sure knows how to get your attention by breaking the transport service.

If we disable both custom receive connectors, transport service starts fine. So we went ahead and changed transport role from Hub Transport to Frontend Transport on both connectors with Set-ReceiveConnector powershell cmdlet, then re-enable them to test. Hub Transport service stays up without issue. Of course, we also rebooted the server to make sure that issue is fixed.



Edit. Microsoft has released the following KB addressing the issue