There are many posts and forums questions about the reasons of why User Profile Synchronization Service may fail to start: it may stuck in Starting status or return to Stopped state after some time. Recently we faced with similar issue and the reason was quite interesting: may be you won’t face with it in your environment, but technique used for troubleshooting may be useful for you.
Ok, everything has started with unsuccessful attempt to start User Profile Synchronization Service from Sharepoint Central administration. It froze some time in Starting state and then returned to Stopped state. In Sharepoint ULS logs the following message was added:
The Execute method of job definition Microsoft.Office.Server.UserProfiles.UserProfileImportJob (ID …) threw an exception. More information is included below. Operation is not valid due to the current state of the object.
at Microsoft.Office.Server.UserProfiles.UserProfileImportJob.CreateSteps()
at Microsoft.Office.Server.UserProfiles.UserProfileImportJob.Execute()
at Microsoft.Office.Server.Administration.UserProfileApplicationJob.Execute(SPJobState jobState)
at Microsoft.SharePoint.Administration.SPTimerJobInvokeInternal.Invoke(SPJobDefinition jd, Guid targetInstanceId, Boolean isTimerService, Int32& result)
If you will google for it you may find the following frequent answer: add Read and Execute permissions to all users group or to Network Service on C:\Program Files\Microsoft Office Servers\15.0 folder. It didn’t help in our case. In the ULS there was also another error right after the message which is shown above:
SettingsProvider: System.InvalidOperationException: Adding Microsoft.SharePoint.PowerShell...
Data Source=dbserver;Initial Catalog=SharePoint_Service_User_Profile_Sync;Integrated Security=True;Enlist=False;Pooling=True;Min Pool Size=0;Max Pool Size=100;Connect Timeout=15
at Microsoft.IdentityManagement.Samples.SettingsProvider.GetDbConnection()
at Microsoft.IdentityManagement.Samples.SettingsProvider.Microsoft.IdentityManagement.Settings.ISettingsProvider.TryGetSetting(String name, String& value)
at Microsoft.IdentityManagement.Settings.ExternalSettingsManager.TryGetSetting(String settingName, String supportedParameters, String& value, String& unsupportedValue)
at Microsoft.ResourceManagement.Utilities.ExternalSettingsManager.TryGetSetting(String settingName, String supportedParameters, String& value, String& unsupportedValue)
at Microsoft.ResourceManagement.Data.DatabaseConnection.InitializePrimaryStoreConnectionString()
at Microsoft.ResourceManagement.Data.DatabaseConnection.get_ConnectionString()
at Microsoft.ResourceManagement.Data.DatabaseConnection.Open(DataStore store)
at Microsoft.ResourceManagement.Data.TransactionAndConnectionScope..ctor(Boolean createTransaction, IsolationLevel isolationLevel, DataStore dataStore)
at Microsoft.ResourceManagement.Data.TransactionAndConnectionScope..ctor(Boolean createTransaction)
at Microsoft.ResourceManagement.Data.DataAccess.RegisterService(String hostName)
at Microsoft.ResourceManagement.Workflow.Hosting.HostActivator.RegisterService(String hostName)
at Microsoft.ResourceManagement.Workflow.Hosting.HostActivator.Initialize()
at Microsoft.ResourceManagement.WebServices.ResourceManagementServiceHostFactory.CreateServiceHost(String constructorString, Uri[] baseAddresses)
at Microsoft.ResourceManagement.WindowsHostService.OnStart(String[] args)
In order to check what may be the reason of this exception I needed to find assembly with Microsoft.IdentityManagement.Samples.SettingsProvider class. In the GAC there are several Microsoft.IdentityManagement.* assemblies, but they don’t contain mentioned class. Then I went to the folder which contains executable file for Forefront Identity Manager Service (e.g. "C:\Program Files\Microsoft Office Servers\15.0\Service\Microsoft.ResourceManagement.Service.exe", but it may be different in your case. In order to check it go to Services MMC and find executable file for Forefront Identity Manager Service service on General tab of service details window). There was SettingsProvider.dll which contained interested class.
Here is the code of Microsoft.IdentityManagement.Samples.SettingsProvider.GetDbConnection() method which threw exception:
1: private static string GetDbConnection()
2: {
3: SqlConnectionStringBuilder builder;
4: Guid empty = Guid.Empty;
5: using (RegistryKey key = Registry.LocalMachine.OpenSubKey(
6: @"SYSTEM\CurrentControlSet\Services\FIMSynchronizationService\Parameters"))
7: {
8: string str = (string) key.GetValue("SyncDatabaseId");
9: if (string.IsNullOrEmpty(str))
10: {
11: return null;
12: }
13: try
14: {
15: empty = new Guid(str);
16: }
17: catch (FormatException)
18: {
19: return null;
20: }
21: }
22: string connectionString = RunPSScript(string.Format(CultureInfo.InvariantCulture,
23: "\r\n#Set-ExecutionPolicy Unrestricted\r\nAdd-pssnapin Microsoft.SharePoint.PowerShell\r\n\r\n" +
24: "$syncDB = get-SPDatabase -Identity {0}\r\n$syncDB.DatabaseConnectionString\r\n",
25: new object[] { empty })).Trim();
26: try
27: {
28: builder = new SqlConnectionStringBuilder(connectionString);
29: }
30: catch (Exception)
31: {
32: throw new InvalidOperationException(connectionString);
33: }
34: ...
35: }
36:
37: private static string RunPSScript(string script)
38: {
39: using (Process process = new Process())
40: {
41: process.StartInfo.FileName = Path.Combine(
42: Environment.GetFolderPath(Environment.SpecialFolder.System),
43: @"WindowsPowerShell\v1.0\powershell.exe");
44: process.StartInfo.Arguments = "-Command -";
45: process.StartInfo.RedirectStandardOutput = true;
46: process.StartInfo.RedirectStandardInput = true;
47: process.StartInfo.UseShellExecute = false;
48: process.StartInfo.CreateNoWindow = true;
49: process.Start();
50: process.StandardInput.WriteLine(script);
51: process.StandardInput.WriteLine("exit");
52: process.StandardInput.Flush();
53: string str = process.StandardOutput.ReadToEnd();
54: process.WaitForExit();
55: return str;
56: }
57: }
Exception is thrown on line 32 in SqlConnectionStringBuilder(connectionString) constructor call. It occurs if connection string passed to SqlConnectionStringBuilder constructor is invalid. If we will see few records up we will find that connection string is retrieved using quite interesting way: Windows PowerShell process is launched and inside it the following PowerShell script is executed:
1: Set-ExecutionPolicy Unrestricted
2: Add-pssnapin Microsoft.SharePoint.PowerShell
3: $syncDB = get-SPDatabase -Identity ...
4: $syncDB.DatabaseConnectionString
I.e. it gets reference on User Profile Sync database which id is stored in HKEY_LOCAL_MACHINE > System > CurrentControlSet > Services > FIMSynchronizationService > Parameters > SyncDatabaseId and then just calls its DatabaseConnectionString property in last script command which makes it return value of the PowerShell process which is returned to Microsoft.IdentityManagement.Samples.SettingsProvider.GetDbConnection() method.
This code doesn’t contain dependencies on other sub systems so it was quite easy to simulate it by copying to standalone console application. When I ran it I found that instead of single connection string it returns 2 lines:
Adding Microsoft.SharePoint.PowerShell...
Data Source=dbserver;Initial Catalog=SharePoint_Service_User_Profile_Sync;Integrated Security=True;Enlist=False;Pooling=True;Min Pool Size=0;Max Pool Size=100;Connect
And the problem was in the first line “Adding Microsoft.SharePoint.PowerShell...”. Because of it SqlConnectionStringBuilder() constructor threw exception because this is not correct connection string. After that I contacted guys who configured environment and they said that added Microsoft.SharePoint.PowerShell module to default Windows PowerShell by default because it is needed almost everytime (the code which adds it was added via custom profile.ps1 file in powershell.exe folder C:\Windows\System32\WindowsPowerShell\v1.0). As it turned out it was not very good idea because it caused mentioned problem with starting User Profile Synchronization Service: if you need PowerShell console with Sharepoint support it is better to use Sharepoint Management Shell which loads Microsoft.SharePoint.PowerShell automatically, while Windows shell should be left untouched. After I deleted profile.ps1 with custom code and restarted the PC, User Profile Synchronization Service had started successfully. Hope that this story will help you to troubleshoot similar problems.
