Document ID Service is OTB feature which adds Document ID field to OTB Document content type. All custom content types which inherit Document content type also will have it after correct feature activation. The main problem is that this field is added asynchronously via timer job daily (see below) and it causes a lot of confusions for developers and administrators. On Sharepoint Online it is problematic to force the process, but for on premise environments it is possible and in this article I will show how to do it and on what mechanisms it is based on. Also note that on Sharepoint Online DocId feature is only available with Enterprise plan.
Before to continue reading this article I highly recommend to check another my post Internal details of SPWorkItemJobDefinition or several reasons of why SPWorkItemJobDefinition doesn’t work, which describes details of internal implementation of work items in Sharepoint. It is needed because Document ID feature actively uses work items and we need to know how they work before to investigate Document ID feature itself. Work items which are used in the process of activation of Document ID feature are processed by the following timer jobs (you may find them in Central administration > Monitoring > Job definitions):
Job title: Document ID enable/disable
Job type: Microsoft.Office.DocumentManagement.Internal.DocIdEnableWorkItemJobDefinition
Work item type: 749FED41-4F86-4277-8ECE-289FBF18884F
Description: Work item that propagates content type changes across all sites when the Document ID feature is reconfigured.
Default schedule: Daily 21:30-21:45
Job title: Document ID assignment
Job type: Microsoft.Office.DocumentManagement.Internal.DocIdWorkItemJobDefinition
Work item type: A83644F5-78DB-4F95-9A9D-25238862048C
Description: Work item that assigns Document ID to all items in the site collection.
Default schedule: Daily 22:00-22:30
Both jobs are provisioned by DocumentManagement feature (Id = "3A4CE811-6FE0-4e97-A6AE-675470282CF2") of WebApplication scope. First job adds fields to content types, second – sets the value in added fields for existing documents. If you don't see these jobs in Central administration > Monitoring > Job definitions, activate this feature with force flag. Pay attention on work item types (749FED41-4F86-4277-8ECE-289FBF18884F and A83644F5-78DB-4F95-9A9D-25238862048C) – we will need them below.
First of all we need to activate "Document ID Service" feature in Site settings > Site collection features. Lets check what happens in feature receiver Microsoft.Office.DocumentManagement.DocIdFeatureReceiver during activation and deactivation:
1: public override void FeatureActivated(SPFeatureReceiverProperties
2: receiverProperties)
3: {
4: FeatureActivateDeactivate(receiverProperties, true);
5: }
6:
7: public override void FeatureDeactivating(SPFeatureReceiverProperties
8: receiverProperties)
9: {
10: FeatureActivateDeactivate(receiverProperties, false);
11: }
Internal FeatureActivateDeactivate() method is shown below:
1: private static void FeatureActivateDeactivate(SPFeatureReceiverProperties
2: receiverProperties, bool fActivate)
3: {
4: SPSite parent = receiverProperties.Feature.Parent as SPSite;
5: ULS.SendTraceTag(0x61326d37, ULSCat.msoulscat_DLC_DM, ULSTraceLevel.High,
6: "Document ID/FeatureActivateDeactivate: Entering with fActivate = {0}",
7: new object[] { fActivate });
8: DocumentId.EnableAssignment(parent, null, fActivate, fActivate, false, false);
9: ULS.SendTraceTag(0x61326d38, ULSCat.msoulscat_DLC_DM, ULSTraceLevel.High,
10: "Document ID/FeatureActivateDeactivate: Leaving");
11: }
Let’s check DocumentId.EnableAssignment() method:
1: public static void EnableAssignment(SPSite site, string prefix, bool fEnable,
2: bool fScheduleAssignment, bool fOverwriteExistingIds, bool fDocsetCtUpdateOnly)
3: {
4: if (site == null)
5: {
6: throw new ArgumentNullException("site");
7: }
8: if ((site.RootWeb == null) || (site.RootWeb.ContentTypes == null))
9: {
10: throw new ArgumentException("site.RootWeb.ContentTypes");
11: }
12: if (fEnable && !IsFeatureEnabled(site))
13: {
14: throw new InvalidOperationException(
15: "Assignment cannot be enabled with the feature deactivated");
16: }
17: if (DocIdHelpers.IsSiteTooBig(site))
18: {
19: DocIdEnableWorkitem.ScheduleWorkitem(site, prefix, fEnable,
20: fScheduleAssignment, fOverwriteExistingIds, fDocsetCtUpdateOnly);
21: }
22: else
23: {
24: DocIdEnableWorkItemJobDefinition.EnableAssignment(site, prefix, fEnable,
25: fScheduleAssignment, fOverwriteExistingIds, fDocsetCtUpdateOnly);
26: }
27: new DocIdUiSettings(fEnable, prefix).Save(site);
28: }
So it check whether current site collection “too big” by using DocIdHelpers.IsSiteTooBig() method. If yes, it creates new work item which will add fields to content types and will create another work item which will assign values in these fields for existing documents (see below). If site is not “too big”, it will make changes in content types synchronously and then also will create work item for filling fields for existing documents. I.e. this second work item will be created in any case, but we will return to it later. Now let’s check what means “site is too big”. Here is how DocIdHelpers.IsSiteTooBig() method is implemented:
1: public static bool IsSiteTooBig(SPSite site)
2: {
3: return IsSiteTooBig(site, 1, 40, 20);
4: }
5:
6: public static bool IsSiteTooBig(SPSite site, int maxWebs, int maxListsPerWeb,
7: int maxDoclibsPerWeb)
8: {
9: ...
10: }
I.e. site is “too big” when it contains more that 1 SPWeb, more that 40 lists or more than 20 document libraries. If one of these conditions is true for your site, content types will be changed asynchronously in work item. It is created in DocIdEnableWorkitem.ScheduleWorkitem() method as shown above. Here is how it is implemented:
1: public static void ScheduleWorkitem(SPSite site, string Prefix, bool Enable,
2: bool ScheduleAssignment, bool OverwriteExisting,
3: bool DocsetCtUpdateOnly)
4: {
5: if (site == null)
6: {
7: throw new ArgumentNullException("site");
8: }
9: string url = site.Url;
10: ULS.SendTraceTag(0x636e7037, ULSCat.msoulscat_DLC_DM, ULSTraceLevel.Medium,
11: "Document ID Enable:ScheduleWorkitem for '{0}':enable={1},schedule={2},overwrite={3}",
12: new object[] { url, Enable, ScheduleAssignment, OverwriteExisting });
13: ULS.SendTraceTag(0x636e7038, ULSCat.msoulscat_DLC_DM, ULSTraceLevel.Medium,
14: "Document ID Enable: ScheduleWorkitem called for site '{0}' by this stack: {1}",
15: new object[] { url, Environment.StackTrace });
16: XmlSerializer serializer = new XmlSerializer(typeof(DocIdEnableWorkitem));
17: StringWriter writer = new StringWriter(CultureInfo.InvariantCulture);
18: DocIdEnableWorkitem o = new DocIdEnableWorkitem();
19: o.fEnable = Enable;
20: o.fScheduleAssignment = ScheduleAssignment;
21: o.fOverwriteExisting = OverwriteExisting;
22: o.fDocsetCtUpdateOnly = DocsetCtUpdateOnly;
23: o.prefix = Prefix;
24: serializer.Serialize((TextWriter) writer, o);
25: string strTextPayload = writer.ToString();
26: writer.Dispose();
27: site.AddWorkItem(Guid.Empty, DateTime.Now.AddMinutes(30.0).ToUniversalTime(),
28: DocIdEnableWorkItemJobDefinition.DocIdEnableWIType, site.RootWeb.ID, site.ID, 1,
29: false, Guid.Empty, Guid.Empty, site.RootWeb.CurrentUser.ID, null, strTextPayload,
30: Guid.Empty);
31: ULS.SendTraceTag(0x636e7039, ULSCat.msoulscat_DLC_DM, ULSTraceLevel.Medium,
32: "Document ID Enable: ScheduleWorkitem for '{0}' leaving", new object[] { url });
33: }
Work item is added on lines 27-30 by calling SPSite.AddWorkItem() method. It adds work item which with type 749FED41-4F86-4277-8ECE-289FBF18884F which is processed by “Document ID enable/disable” job (see above). This work item propagates new Documend ID field. More accurately it propagates 2 fields are created and added to OTB Document CT: Document ID (SPFieldUrlValue) and Document ID Value (string). Both of these fields are not shown neither in Document content type details page in Site settings > Content types nor in Documents doclib columns in library settings. You will see it only when will upload some document to Documents doclib (or other doclib which uses OTB Document content type or custom content type which inherits it) and then will click View properties. Document ID will be there and it will look like link which points to http://example.com/_layouts/15/DocIdRedir.aspx?ID={docId}.
If you read the article which I mentioned in the beginning you already know that work items are stored in ScheduledWorkItems table in content database. In order to check it execute the following Sql query:
1: declare @enable uniqueidentifier
2: set @enable = cast('749FED41-4F86-4277-8ECE-289FBF18884F' as uniqueidentifier)
3: select * from [dbo].[ScheduledWorkItems] where [Type] = @enable
Query should return 1 row:
It has the following params specified in TextPayload column:
1: <?xml version="1.0" encoding="utf-16"?>
2: <DocIdEnableWorkitem
3: xmlns:xsd="http://www.w3.org/2001/XMLSchema"
4: xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
5: <fEnable>true</fEnable>
6: <fScheduleAssignment>true</fScheduleAssignment>
7: <fOverwriteExisting>false</fOverwriteExisting>
8: <fDocsetCtUpdateOnly>false</fDocsetCtUpdateOnly>
9: </DocIdEnableWorkitem>
Pay attention on DeliveryDate column. It contains the earliest time in UTC format (add 3 hours in
order to get local time) when job can be executed. But if you will try to force job running (by clicking Run now in CA) before the time specified in DeliveryDate column, nothing will happen. This is the most often reason of confusion: force “Document ID enable/disable” job run immediately after feature activation. But as you can see from listing of DocIdEnableWorkitem.ScheduleWorkitem() method, DeliveryDate is set to DateTime.Now.AddMinutes(30.0).ToUniversalTime(). Which means that you have to wait at least 0,5h or change system time, restart timer server, run the job and then change time back and restart timer service again (to be more accurate you will need to run also “Document ID assignment” job before to change system time back right after “Document ID enable/disable” job is finished. See part 2 for details).
The reason of this behavior is in [dbo].[proc_GetRunnableWorkItems] stored procedure:
1: CREATE PROCEDURE [dbo].[proc_GetRunnableWorkItems] (
2: @ProcessingId uniqueidentifier,
3: @SiteId uniqueidentifier,
4: @WorkItemType uniqueidentifier,
5: @BatchId uniqueidentifier,
6: @MaxFetchSize int = 1000,
7: @ThrottleThreshold int = 0,
8: @RequestGuid uniqueidentifier = NULL OUTPUT
9: )
10: AS
11: SET NOCOUNT ON
12: IF (dbo.fn_IsOverQuotaOrWriteLocked(@SiteId) >= 1)
13: BEGIN
14: RETURN 0
15: END
16: DECLARE @iRet int
17: SET @iRet = 0
18: DECLARE @oldTranCount int
19: SET @oldTranCount = @@TRANCOUNT
20: DECLARE @Now datetime
21: SET @Now = dbo.fn_RoundDateToNearestSecond(GETUTCDATE())
22: DECLARE @InProgressCount int
23: DECLARE @ThrottledFetch int
24: DECLARE @ReturnWorkItems bit
25: SET @ReturnWorkItems = 0
26: BEGIN TRAN
27: SET @InProgressCount = 0
28: SET @ThrottledFetch = 0
29: SET @ThrottleThreshold = @ThrottleThreshold + 1
30: IF @ThrottleThreshold > 1
31: BEGIN
32: SET ROWCOUNT @ThrottleThreshold
33: SELECT
34: @InProgressCount = COUNT(DISTINCT BatchId)
35: FROM
36: dbo.ScheduledWorkItems WITH (NOLOCK)
37: WHERE
38: Type = @WorkItemType AND
39: DeliveryDate <= @Now AND
40: (InternalState & (1 | 16)) = (1 | 16)
41: END
42: IF @BatchId IS NOT NULL
43: BEGIN
44: SET @ThrottledFetch = 16
45: END
46: IF @InProgressCount < @ThrottleThreshold
47: BEGIN
48: UPDATE
49: dbo.ScheduledWorkItems
50: SET
51: InternalState = InternalState | 1 | @ThrottledFetch,
52: ProcessingId = @ProcessingId
53: FROM
54: (
55: SELECT TOP(@MaxFetchSize)
56: Id
57: FROM
58: dbo.ScheduledWorkItems
59: WHERE
60: Type = @WorkItemType AND
61: DeliveryDate <= @Now AND
62: (@SiteId IS NULL OR
63: SiteId = @SiteId) AND
64: (@BatchId IS NULL OR
65: BatchId = @BatchId) AND
66: (InternalState & ((1 | 2))) = 0
67: ORDER BY
68: DeliveryDate
69: ) AS work
70: WHERE
71: ScheduledWorkItems.Id = work.Id
72: SET @InProgressCount = @@ROWCOUNT
73: SET ROWCOUNT 0
74: IF @InProgressCount <> 0
75: BEGIN
76: EXEC @iRet = proc_AddFailOver @ProcessingId, NULL, NULL, 20, 0
77: END
78: SET @ReturnWorkItems = 1
79: END
80: CLEANUP:
81: SET ROWCOUNT 0
82: IF @iRet <> 0
83: BEGIN
84: IF @@TRANCOUNT = @oldTranCount + 1
85: BEGIN
86: ROLLBACK TRAN
87: END
88: END
89: ELSE
90: BEGIN
91: COMMIT TRAN
92: IF @InProgressCount <> 0
93: AND @InProgressCount <> @MaxFetchSize
94: AND @WorkItemType = 'BDEADF09-C265-11d0-BCED-00A0C90AB50F'
95: AND @BatchId IS NOT NULL AND @SiteId IS NOT NULL
96: BEGIN
97: UPDATE
98: dbo.Workflow
99: SET
100: InternalState = InternalState & ~(1024)
101: WHERE
102: SiteId = @SiteId AND
103: Id = @BatchId
104: END
105: IF @ReturnWorkItems = 1
106: BEGIN
107: SELECT ALL
108: DeliveryDate, Type, ProcessMachineId as SubType, Id,
109: SiteId, ParentId, ItemId, BatchId, ItemGuid, WebId, UserId, Created,
110: BinaryPayload, TextPayload,
111: InternalState
112: FROM
113: dbo.ScheduledWorkItems
114: WHERE
115: Type = @WorkItemType AND
116: DeliveryDate <= @Now AND
117: ProcessingId = @ProcessingId
118: ORDER BY
119: DeliveryDate
120: IF @@ROWCOUNT <> 0
121: BEGIN
122: EXEC @iRet = proc_UpdateFailOver @ProcessingId, NULL, 20
123: END
124: END
125: END
126: RETURN @iRet
127:
128: GO
As you can see it returns items which have DeliveryDate <= @Now. That’s why we need to change system time in order to process work items.
You may stop here if you wanted to have quick solution for the problem of why Document ID field is not added. But if you would like to know more internal details, you should check part 2.
I could not solve my problem for 3 hours and finally found your article => solved in 5 minutes! Thanks a lot.
ReplyDeleteHow did you get the source code for "Document ID Service" feature activated event?
ReplyDeleteSonic Vader, first of all find this feature xml in Template/Features folder. In feature xml file find type and assembly of feature receiver. After that find this assembly dll on the Sharepoint server file system and load it to some .Net decompiler e.g. Telerik JustDecompile. Some Sharepoint code is obfuscated but some not.
Delete