Thursday, 22 February 2018

Nintex Workflow: Parse Email Content

A - OBJECTIVE: to provide a practical example of Nintex Workflows (used in SharePoint 2013) which can be used to parse email content.

B - PROBLEM :

HTML emails sent to a SharePoint list (incoming email is enabled) in a readable format: 
Field Name 1: Field Value 1 
Field Name 2: Field Value 2 
Field Name 3: 28/02/2018 12:00:00 AM...
There are some known issues such as:

  1. A variable of type "Multiple lines of text" cannot be printed in full at Nintex for review purpose. A test done at the function Log in history list only showed a string of ~250 characters.
  2. It's not straightforward to replace a character (e.g. new line \n) with a white space.
  3. The function "Query XML" expects to have an input properly HTML-formatted.
C - SOLUTION:

Configuration-wise, 

  • the feature incoming email must be enabled so that an encrypted email (.eml) can be sent to the list.
  • the column "Body" should have the type "Enhanced Rich Text" (or else, a pair of field name & field value will not be in the same line)

Nintex-wise,

Step 1: Get the incoming email body:
Step 2: Strip all unexpected HTML tags, such as: <html>, </html>, <head>, </head>, <body>, </body>, <br>, <p>, </p>, etc ; and append the proper HTML tags to the string.


Step 3: use the function "Query XML" to extract all pairs of name & value, the output should be a collection of lines:


Step 4: count the number of pairs (in terms of the number of lines):



Step 5: at each line, we should (1) remove the special character \n , (2) split the line based on the pattern ":" and save the result in a collection, (3) process the line to find out the value is a string or a date, & (4) assign the value to the proper workflow variable:





Step 6: finally, you can use the workflow variables to create an item at other list with the mapped columns.

D - SOURCE CODE: 

Step 1: with the variable Body is multiple lines of text.

Step 2: BodyHTML is multiple lines of text:
Append the proper HTML format:

Step 5:









Thursday, 25 January 2018

SharePoint Issue (on-prem) - Document Library with incoming email

A - OBJECTIVE: 

We have a case in which emails cannot be sent from SharePoint 2010 Foundation to SharePoint 2013.

The issue can be described in several ways:

1. Document Library with incoming emails cannot receive emails.
2. Emails get stuck in the folder Drop at the server (under C:\inetpub\mailroot).
3. SharePoint cannot send emails to itself to prevent infinite loop.

B - PROBLEM:


Expected Result: In this particular case, we have 2 SharePoint environments (2010 Foundations & 2013 Enterprise) and we would like to send emails from SharePoint 2010 to the other. 

Scenario:
  • In the folder Drop of the mailroot, all emails dropped there can be processed, except those emails sent from SharePoint 2010.
  • The stuck emails were sent from an external SMTP IP address.
Some test cases have been done pertaining to email notification as follows:
  • Test 1 - SharePoint 2013 to SharePoint 2013: An simple workflow is created at another SharePoint 2013 site to send email to the expected SP2013 site. It works.
  • Test 2 - Manual Email: using a Powershell script to send an email from the SharePoint2010 server, we could receive emails properly at the SharePoint 2013 site.
# manually send emails from the SharePoint 2010 server
$fromemail = "sp2010@mydomain.com"
$users = "anthonynhn@mydomain.com","sp2013@mydomain.com" 
$SMTPserver = "202.12.34.567"
send-mailmessage -from $fromemail -to $users -subject "Manual Email Test" -BodyAsHTML -body "Message goes here" -priority High -smtpServer $SMTPserver


Troubleshooting:
  • Stuck emails were picked from the folder Drop and compared to successfully delivered emails in test 1 and test 2.
  • The issue is from the SharePoint site because the email can be delivered to the folder Drop, and we should look for the email header.
After comparing the files .eml , it turned out that the error is because of this line:





C - SOLUTION:

The issue is created by design to avoid an infinite loop in SharePoint, where you can find the code at SPEmailEngine.HandleEmailFile



string str4 = message.Headers["X-Mailer"];
if (!string.IsNullOrEmpty(str4) && string.Equals(str4, "Microsoft SharePoint Foundation 2010", StringComparison.OrdinalIgnoreCase))
{
    return null;
}

The solution is therefore to remove the X-Mailer line in the email header to intentionally bypass the emails:
  • Solution 1: to directly modify the email header at the network level (Ironport in my case).
  • Solution 2: to remove the problematic X-Mailer line soon after an .eml mail is sent to the folder Drop, following these steps thanks to this blog:

1. copy Smtpreg.vbs into the c:\inetpub\AdminScripts - smtpreg is available to download from MS.

2. create a vbs file from the script below and copy it to the AdminScripts folder (I named mine wss)

<SCRIPT LANGUAGE="VBScript">

Sub ISMTPOnArrival_OnArrival(ByVal iMsg, EventStatus )

if iMsg.Fields("urn:schemas:mailheader:x-mailer") = "Microsoft SharePoint Foundation 2010" then
iMsg.Fields.Delete("urn:schemas:mailheader:x-mailer")
iMsg.Fields.Update
end if

iMsg.DataSource.Save
EventStatus = 0
End Sub

3. run the following commands to register the script

cd c:\inetpub\adminscripts
cscript smtpreg.vbs /add 1 OnArrival DeleteMsg CDO.SS_SMTPOnArrivalSink "mail from=*"
cscript smtpreg.vbs /setprop 1 OnArrival DeleteMsg Sink ScriptName "c:\Inetpub\AdminScripts\wss.vbs"


D - REFERENCE: 

Some interesting articles pertaining to this matter:
  • An overview of incoming emails (SharePoint 2013): https://bernado-nguyen-hoan.com/2013/06/18/solving-sharepoint-2013-incoming-mails-stuck-in-drop-folder/
  • A solution to the above issue: http://sharepoint-uk.blogspot.com.au/2008/03/wss-emailing-itself-x-mailer-windows.html


Thursday, 28 December 2017

SharePoint 2013 - Usage Analytics (ViewsLifetime)

A - OBJECTIVE: 

This article is to share a case study about SharePoint usage analytics (pageviews specifically) which is captured and updated by SharePoint (out-of-the-box feature).

B - CASE STUDY & THE PROBLEM :

- Search Service Application has several interesting services, including (1) Search Analytics and (2) Usage Analytics, and we can make use of the value ViewsLifetime (a managed property in the search application) to show the pageviews at a certain item. 

- To view the value ViewsLifetime of a certain page, you can use the URL (path to be updated): http://sitecollectionURL//_api/search/query?querytext='path:http://sitecollectionURL/news/Pages/ArticleURL.aspx'&sortlist='ViewsLifeTime:descending' 



- This value can be retrieved and used in a Content Search webpart which makes use of the custom Display Items:


- You can modify the TailTrimming to get the full pageviews data at production.

- The problem happens when the value ViewsLifetime is not updated properly and no value is captured in the EventStore, and the pageviews does not work.


C - TROUBLESHOOTING & SOLUTION:

A couple of online questions have no answer, such as:

https://social.msdn.microsoft.com/Forums/office/en-US/bcbd2474-31a0-4b09-97cc-67fa19d9ef3f/usage-analytics-event-store-not-updating-popularity-reports-showing-0-delete-usage-receivers?forum=sharepointdevelopment

The value ViewsLifetime is updated by the timer job Usage Analytics which run once a day (usually at 1am) to analyse all raw usage logs captured by the timer job Usage Data Import (run once every 5 mins).

We checked the scheduled jobs related to the services, and they work:

- Further check showed that the receivers of the relevant services have worked well.

- We manually run the service Analytics, but the RequestUsage folder at C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS\RequestUsage is not updated



- We manually check the service Data Import, but the EventStore is not updated as well.



Solution: 

At the staging environment: (app & web roles are in the same server) we restarted the application server, and it worked. EventStore is updated properly again and the ViewsLifetime is updated.

At the production environment:



  • Step 3: you should both/either restart the wsstracing.exe service and/or drill down to the affected folder.
  • Step 4: we restarted the service wsstracing.exe but the problem could not be resolved.
  • Step 5: we rename the problematic folder (which seems to be corrupted) to AffectedFolder2, and re-create the same folder to let the service run. If the folder cannot be renamed (e.g. TimerJobUsage), please go inside the folder and delete all outdated files (with the last modified date is before today).



  • The issue at production is resolved!




D - USEFUL RESOURCE: 

1. https://www.linkedin.com/pulse/20140724173810-70514633-sharepoint-usage-report-in-sp-2013-is-merged-in-search-service-application

2. https://blogs.technet.microsoft.com/tothesharepoint/2014/01/23/view-and-configure-usage-analytics-reports-in-sharepoint-server-2013

3. https://blogs.msdn.microsoft.com/spblog/2014/04/03/sharepoint-2013-usage-analytics-the-story

Wednesday, 21 June 2017

Nintex Workflows 101

A - OBJECTIVE: 

This article is to review basic points of Nintex Workflows (on-premises version 2013).

Case Study: we will create a workflow to delete all items in a list.

B - TIPS:

To list all potential or encountered technical errors or operational issues (based on frequently asked issues in my experience):

1. Delete a workflow: go to Workflow Inventory page as follows

 




2.  Schedule workflows to run automatically

 



 
3.  Process Collection (i.e. list of SharePoint): you should create a workflow variable of type "Collection" to handle this:

 

4.  Assign Flexi Task: is an important control in Nintex Workflows, where you can:
  • Allow a user (e.g. approver) to have different actions (i.e. outcomes), and each outcome can have its own set of activities (i.e. unique Nintex Workflow controls)
  • Assign who can be the target audience of this task & send custom email notification to the user(s).
  • Customize a Task Form (similar like Nintex Forms) which is used by this particular target audience.
Important Note:

Lookup Fields in Flexi Task: you can create linked controls in (a) a Nintex Forms, or (b) Task Form in a Flexi Assign Task (workflow)

For example, when a user chooses a value in Publication control, the check-boxes will be updated accordingly



Issue: when it comes to workflow migration (i.e. import an exported workflow) to a new SharePoint farm, such Lookup controls will not work anymore with the following warning:



Work-around: you need to delete the Lookup control and re-configure it again. This approach is acknowledge & confirmed by Nintex technical team.

5. State machine: is a crucial concept where you can define the states of your workflow while various parties can involve in a certain state(s).




C - CASE STUDY:

We will create a simple workflow to loop through all items in a SharePoint list & delete all items in a scheduled job.



D - SOURCE CODE:


1. We will create a query list where a variable of type Collection (named colItemIDs) will be used to keep all Item IDs:






2.Create a workflow variable index to store the number of items:



 3. Create a Collection Operation to count the number of SharePoint list items:




4. Create a Run-If control to ensure that all operations are done when the list has some items:



5. Create a For-Each loop to run through all get the individual ID out of the collection of IDs, and store this value in a workflow variable strItemID:






6. Add a Delete Item control to delete an individual item:



7. Schedule the workflow using Nintex Scheduled Job.