SharePoint WebHooks

 Webhooks in SharePoint Online

In this post I’m going to try to explain what is SharePoint Online Webhooks and  in a few steps how a developer can use, debug Webwooks in SharePoint Online and explain the approach recommended by Bert Jansen and Vesa Juvonen.

Since of beginning of this year we can use Webhooks  to trigger an event in SharePoint Online List or Document Library. It enables the developers to build custom application that subscribe events that occur in a SharePoint List or Document Library. For now Webhooks covers the events  that have already happened, in other words events of type “ed” (ItemAdded, ItemUpdated, etc.).

How can a developer create a subscription in SharePoint Online?

We can use Postman to create the subscription and understand that messages are exchanged between SharePoint Online and the application that will receive the messages. Before starting with Postman we  need to have an Office 365 Subscription with SharePoint Online, register the Microsoft Azure Active Directory (AD) application to get access token to use in Postman, create project with Web API  template to handle the incoming request and finally use the ngrok to create a secure tunnel to your dev environment (localhost). We can see all the steps in Office Dev Center.

To create a subscription we need to make a POST request to “http://<Your Site Url>_api/web/lists(‘<List ID>’)/subscriptions”.

The body of POST is constitued by:

{
  "resource": "url of the list",
  "notificationUrl": "endpoint url to send notification to",
  "expirationDateTime": "the date when notification will expire, not be more than 6 months",
  "clientState": "is not a require field. It can be used for validation notifications or tagging different subscriptions"
}

Let’s prepare all the necessary things. Create our custom  Web API and configure ngrok to receive the message (notificationUrl in body request). Create Azure application to the SharePoint Online and select the permission (read and write items and lists in all site collection permissions), generate a key and configure redirect url to “https://www.getpostman.com/oauth2/callback”.

Now that we already have everything, we will open the Postman, configure the authentication access, fill the body and that’s it. Do not forget to check if localhost application is running.

If everything goes as expected we should receive a response in the next 5 seconds.

The answer is as follows:

{
 "clientState" : "",
 "id": "id of subscription",
 "expirationDateTime": "the date when notification will expire",
 "notificationUrl": "endpoint url to send notification to",
 "resource": "id of sharepoint list"
}

In subscription we can use the following tasks, try on:

• Create a new subscription –      POST         /_api/web/lists(‘<List ID>’)/subscriptions
• Get subscriptions –                     GET           /_api/web/lists(‘<List ID>’)/subscriptions
• Delete a subscription  –             DELETE   /_api/web/lists(‘<List ID>’)/subscriptions(‘<subscription id>’)
• Update a subscription  –            PATCH     /_api/web/lists(‘<List ID>’)/subscriptions(‘<subscription  id>’)

Now that we already have our subscription set, let’s add a new document in the List that we just added  the subscription and see what messages are exchange between SharePoint Online and our WebAPI. SharePoint Online only sends a message to the endpoint notifying when occurred something, a item as been changed, a item as been added, ect.

{
    "subscriptionId":"id of subscription",
    "clientState":"the option string value that can be used to used for validation",
    "expirationDateTime":"the date when notification will expire",
    "resource":"id of list where the subscription is registered",
    "tenantId":"id tenant where subscription is regitered",
    "siteUrl":"server relative url",
    "webId":"id of web where subscription is regitered"
}

This is the notification sent from SharePoint Online to the notification url (Web API). As in the creation of the subscription, if we do not have an answer in the next few minutes the SharePoint Online will try to send again, it will retry of 5 times with a 5 minute wait time between the attempts. As we can see in the notification, nothing is sent about the change made in the SharePoint Online list, we need to develop the same code to get these changes, we will use GetChanges API.

With this small example we can understand how to create a subscription and see the exchanged messages.

For production environment I recomend the implementation of Bert Jansen and Vesa Juvonen.

Bert Jansen and Vesa Juvonen implementation

This process  is complex, there is the possibility of losing messages, because the Web API did not respond in sufficient time, is not accessible or for any other reason. To prevent most problems and short time between sending and processing the message Bert Jansen and Vesa Juvonen recommend  the implementation of the figure.

In this reference implementation Bert Jansen and Vesa Juvonen proposes to use a SQL Azure AD where it saves ChangeToken of the last processed message, a Storage Queue to store all the messages sent from SharePoint Online List’s and a Web Job to processing the message and associated logic.

How does this all work?

When the subscription is created, a record is created in the SQL Azure AD with the List ID and the changetoken (CurrentChangeToken). Then whenever a message is sent through a SharePoint Online List it is added to the Storage Queue. Web Job processes the message (picks up the List Id) and get change token from SQL Azure AD,  processes the all messages referring to the List in question from that change token to the front (the change token is a string  which among other things has the date time of the change), does all the necessary logic and update the database with lastchangetoken to the current List.

Summary

Recently we have seen Microsoft work hard on making SharePoint Online to use the technologies that are in vogue, SharePoint Online Webhooks is another example of this work.

Note that for implementing a Webhook subscription in a prodution environment,  I recommend the approach of  Bert Jansen and Vesa Juvonen because we can guarantee that sooner or later all the requests will be processed without losses.

Sometimes we have the need to prevent, validate actions in the Lists before the actions are completed, events of the type “ing” (ItemAdding, itemUpdating, etc). Webhook does not permite the developer to intervent in this type of events, but remember that in that case we can use Remote Event Receiver.

References:

Bert Jansen and Vesa Juvonen