SharePoint Online - Custom Applications & Forms with React

A - OBJECTIVE: to highlight several important notes required for custom developments at SharePoint Online using React: 

1. Useful third-party products: Nintex Forms (limited) and Nintex Workflows for Office 365. 

2. Useful framework & libraries: 

• React (with the concept of components, which can be rendered as a control, a section, or a whole webpart).
• State management with Mobx (or Redux): Mobx is preferred because the custom application at SharePoint Online is not a large-scaled application, and Mobx is more straightforward.
• Form controls (e.g. People Picker, Textbox, etc): Office UI Fabric React/Angular can be considered together with PnP SPFx Controls.
• Data Create/Read/Edit/Delete actions: SharePoint PnP Framework is a good option, or you can write your own ajax API calls (reference here).

3. Visual Studio Code is recommended as a code editor to develop custom applications.

Note: SharePoint 2010/2013 Designer Workflow (free of charge) can be used to some extent (apart from the commercial Nintex workflow) if you want to trigger special action automatically (e.g. send email).


B - PROBLEMS & SOLUTIONS:

To list all potential or encountered technical errors or operational issues:

1. Nintex Forms: if your form has a number of controls (e.g. > 30 controls) and business rules (e.g. > 50 rules), Nintex Forms will probably take about 10+ seconds to load as at December 2018 (hopefully Nintex will improve this matter in future). Therefore, a custom form (e.g. using React) is a good alternative.

2. Library Version: it is recommended to fix the UI Fabric version (in package.json) because the automatically updated version can lead to a conflict in your control code later.

e.g. npm install mobx@3.1.2 --save









3. Real-time Code Error Notification: you should always run gulp serve in the terminal so that the code editor (i.e. Visual Studio Code) can highlight any error immediately soon after you save the code.  

4. Set State (default React webpart) does not support multiple setState in the default React webpart (i.e. only the first setState call takes place). Therefore, we need to use an additional library (i.e. Mobx) to address this matter.

5. Caching (PnP Caching) can be simply used by adding .usingCaching() in the PnP functions and this temporary data can be saved in either local storage (i.e. local caching - to store all configuration data) or session (to store user credentials). However, this caching can cause an issue if the form needs to refresh data frequently (e.g. cascading dropdown list may not be updated as expected).
.

C - DEVELOPMENT & DEBUGGING:

Debugging:

a. React Extension is available at Chrome and is useful to debug the Typescript files.

b. Hosted Workbench can be configured to run and test your code at SharePoint Online workbench (separated from the deployed solution):

c. Debugging can be done from the code editor (i.e. simple mark the breakpoint and use F5):

Development:

1. Reusable modules (e.g. repeater, custom cascading dropdown, etc) should be packaged as a component which will used in the main component.

2. All declared and dependent libraries used in your code can be installed once using the command npm install in the integrated terminal.

3. When Mobx is used, the tag @action must be added if an observable object/variable is updated to keep track of the observable item.

4. ReactDOM is a useful library to manipulate DOM elements in React approach:

5. Real-time value update for a certain control in the form requires a React function componentDidMount():

Note: this React summary is useful as a reference.

D - DEPLOYMENT: 

1. During production deployment, it is best to clean the package to generate a small solution 

2. The solution is packaged as follows in the Terminal:
gulp bundle --ship
(wait)
gulp package-solution --ship


3. The SharePoint solution package .sppkg must be deployed to the library sites/AppCatalog and all users who use the solution must have the access right to the AppCatalog.

SharePoint Framework (SPFx) Introduction

A - OBJECTIVE: 

This article is to provide a summary of Microsoft guidelines about SharePoint Framework for developers at SharePoint Online (SPO).

B - PREPARATION :

           I. Installation: 

To prepare for the development you will need: 

1. A SharePoint Online tenant: developers can request for a 1-year trial at Office 365 (including SPO) at this link. 

2. SharePoint Online Management Shell: to be installed to your laptop to connect to SPO and run Powershell scripts that can automate your tasks.

3. Visual Studio Code (Free): a kind of development tool for your custom web-parts & extensions.

           II. Programming Language: 

Moving forwards, to develop custom solutions at SPO, Typescript (a superset of Javascript) is the main programming language you should get yourself familiar with.

           III. Deployment: 

Option 1 - to troubleshoot the code and work on the development at local browser (link here). 

Option 2 - to deploy the custom solutions (webparts or extensions) to SPO (link here).

Note:

* The deployment should be done via Command Prompt, for example:

gulp trust-dev-certgulp serve --nobrowsergulp bundle --shipgulp package-solution --ship

* Apart from the deployed package, you can host the associated files (e.g. css or js) to an Office 365 Content Delivery Network CDN (e.g. for webpart at this link here , or for extension at this link here), or Azure CDN (link here).

C - MAIN POINTS:

The custom solutions (webpart or extension) are designed into several blocks as follows:

1. Configuration: it's basically first level of a webpart design where you can preconfigure (link here) or validate the webpart properties (link here).

2. Properties: including the required settings for the webpart/extension properties

• *.manifest.json: to specify the default values of the properties
• interface NameWebpartProps
• protected getPropertyPaneConfiguration

3. Resources: such as language settings (example of a localised webpart), or styling

• mystring.d.ts
• en-us.js
• NameWebpart.Module.scss

4. Main Content: the function to render the webpart/extension content.

5. External Libraries:
     - A detailed guide is at this link.
     - External JS frameworks: must be installed to the solution using CMD (e.g. npm install --save jqueryui). 

In code view, we have these blocks as follows:

Let's go through some Microsoft tutorials here:

           I. Webparts:

1. Properties: 

- Webparts are allowed to be configurable (link here), and webpart icon can be updated.

- SharePoint content can be pre-loaded to the webpart properties (link here), or you can have a cascading dropdown among the properties (link here).

2. Javascript frameworks: Developing on top of other Javascript frameworks (e.g. jQuery, AngularJS, etc) is possible:

• jQuery:
• To incorporate UI to the webpart content (e.g. jQuery UI Accordion - link here).
• DataTables (link here): to build powerful data overviews of data from SPO & external APIs.
• FullCalendar (link here): to display data in a calendar view.
• AngularJS: 
• To migrate AngularJS to SPFx: https://docs.microsoft.com/en-us/sharepoint/dev/spfx/web-parts/guidance/migrate-angular-1-x-applications-to-sharepoint-framework
• To develop a custom AngularJS application at SPO: https://docs.microsoft.com/en-us/sharepoint/dev/spfx/web-parts/guidance/build-client-side-web-parts-with-angular-1-x
• JSOM: the example used here loads information about SPO lists in the current site after selecting a button.

3. Connected webparts: Client-side webparts can share data between each other (link here).

           II. Extensions: 

As of April 2018, there are 3 types of Extensions:

• Application Customizer: e.g. to extend or change the GUI of a SPO site (link here).
• Field Customizer: e.g. to customise the display of a column in a SPO list (link here), with an example of migration project here.
• ListView Command Set (aka Context Menu): e.g. to add an additional command to the list menu (link here), with an example of migration project here.

Similar to webparts, extensions have icons which can be customised (link here).


D - USEFUL RESOURCE: 

1. SPFx Microsoft https://docs.microsoft.com/en-us/sharepoint/dev/spfx/set-up-your-developer-tenant

2. SharePoint Online Powershell scripts (useful functions): http://www.sharepointfire.com/2016/01/get-sharepoint-online-web-templates-powershell/

3. Typescript Key Notes: https://blog.mariusschulz.com/series/typescript-evolution

Nintex Workflow for Office 365 (cloud)

A - OBJECTIVE: to highlight new features at Nintex Workflow for Office 365 (cloud version).

B - REASON:

In comparison with the on-premises version, Nintex Workflow for Office 365 (cloud version) will be reviewed for its new technical actions that can be used in your development.

C - SOLUTIONING:

1. Cross-tenant Operations:

• In multi-tenancy scenario, an action in this tenant (e.g. list item update) can trigger another action at a different tenant.
• This feature is useful when it comes to performing a cross-tenant data aggregation or system/application integration.
• In the example, the action Office 365 update items illustrates the case when an item at other Office365 tenant is to be updated.

2. Document (or Form) Generator: 

• In process improvement, paper forms are converted to electronic forms, but it is mostly saved in the form of list item, not an actual e-form. However, usually for legal purposes, the process owner has a need to store the softcopy of the e-form following a specific format and the static content will be changed from time to time (e.g. Term & Conditions can be updated year after year, not what the user keyed in). As a result, the data integrity will be the problem because of the changes in the static text.
• This action "Document Generation" is a good option to capture the whole format of the form (including user inputs & static words at the time the form is submitted).

• In the example, a developer can create a new template (stored at the doc library Shared Documents) associated to a specific MS Word format, and this format can be captured at the submission time. In addition, workflow variables can be used in the template to generate dynamic content. Any revision of the form in future will not affect the submitted form.

3. (External Internet-accessible) Database Connectivity service:

• This feature is similar to the Business Connectivity Services at SharePoint, which allows your application to consume any Internet-accessible database directly.

4. Integration with other service providers (e.g. Azure, Adobe Sign, DocuSign, etc).

• These features are available under the tab Integration and Provisioning at the Nintex actions.

5. Misc: other new actions should be taken into consideration to design your applications:

• New Wait Actions: the workflow will be paused until the preferred action is executed.

• User Interaction: the 2 actions Assign a task and Start a task process can be used together with Nintex Forms for Office 365.

        Important Note: by design, you cannot send the task to an external user.

• Translate Document: A workflow will pick a specific document under a selected library and translate it to the preferred language, then save it to another library

D - SOURCE CODE: 

Nil.

Nintex Workflow: Parse Email Content

A - OBJECTIVE: to provide a practical example of Nintex Workflows (used in SharePoint 2013) which can be used to parse email content.

B - PROBLEM :

HTML emails sent to a SharePoint list (incoming email is enabled) in a readable format: 

Field Name 1: Field Value 1 

Field Name 2: Field Value 2 

Field Name 3: 28/02/2018 12:00:00 AM...

There are some known issues such as:

1. A variable of type "Multiple lines of text" cannot be printed in full at Nintex for review purpose. A test done at the function Log in history list only showed a string of ~250 characters.
2. It's not straightforward to replace a character (e.g. new line \n) with a white space.
3. The function "Query XML" expects to have an input properly HTML-formatted.

C - SOLUTION:

Configuration-wise, 

• the feature incoming email must be enabled so that an encrypted email (.eml) can be sent to the list.
• the column "Body" should have the type "Enhanced Rich Text" (or else, a pair of field name & field value will not be in the same line)

Nintex-wise,

Step 1: Get the incoming email body:

Step 2: Strip all unexpected HTML tags, such as: <html>, </html>, <head>, </head>, <body>, </body>, <br>, <p>, </p>, etc ; and append the proper HTML tags to the string.

Step 3: use the function "Query XML" to extract all pairs of name & value, the output should be a collection of lines:

Step 4: count the number of pairs (in terms of the number of lines):

Step 5: at each line, we should (1) remove the special character \n , (2) split the line based on the pattern ":" and save the result in a collection, (3) process the line to find out the value is a string or a date, & (4) assign the value to the proper workflow variable:

Step 6: finally, you can use the workflow variables to create an item at other list with the mapped columns.

D - SOURCE CODE: 

Step 1: with the variable Body is multiple lines of text.

Step 2: BodyHTML is multiple lines of text:

Append the proper HTML format:

Step 5:

SharePoint Issue (on-prem) - Document Library with incoming email

A - OBJECTIVE: 

We have a case in which emails cannot be sent from SharePoint 2010 Foundation to SharePoint 2013.

The issue can be described in several ways:

1. Document Library with incoming emails cannot receive emails.
2. Emails get stuck in the folder Drop at the server (under C:\inetpub\mailroot).
3. SharePoint cannot send emails to itself to prevent infinite loop.

B - PROBLEM:


Expected Result: In this particular case, we have 2 SharePoint environments (2010 Foundations & 2013 Enterprise) and we would like to send emails from SharePoint 2010 to the other. 

Scenario:

• In the folder Drop of the mailroot, all emails dropped there can be processed, except those emails sent from SharePoint 2010.
• The stuck emails were sent from an external SMTP IP address.

Some test cases have been done pertaining to email notification as follows:

• Test 1 - SharePoint 2013 to SharePoint 2013: An simple workflow is created at another SharePoint 2013 site to send email to the expected SP2013 site. It works.
• Test 2 - Manual Email: using a Powershell script to send an email from the SharePoint2010 server, we could receive emails properly at the SharePoint 2013 site.

# manually send emails from the SharePoint 2010 server
$fromemail = "sp2010@mydomain.com"
$users = "anthonynhn@mydomain.com","sp2013@mydomain.com" 
$SMTPserver = "202.12.34.567"
send-mailmessage -from $fromemail -to $users -subject "Manual Email Test" -BodyAsHTML -body "Message goes here" -priority High -smtpServer $SMTPserver

Troubleshooting:

• Stuck emails were picked from the folder Drop and compared to successfully delivered emails in test 1 and test 2.
• The issue is from the SharePoint site because the email can be delivered to the folder Drop, and we should look for the email header.

After comparing the files .eml , it turned out that the error is because of this line:

C - SOLUTION:

The issue is created by design to avoid an infinite loop in SharePoint, where you can find the code at SPEmailEngine.HandleEmailFile: 

string str4 = message.Headers["X-Mailer"];
if (!string.IsNullOrEmpty(str4) && string.Equals(str4, "Microsoft SharePoint Foundation 2010", StringComparison.OrdinalIgnoreCase))
{
    return null;
}

The solution is therefore to remove the X-Mailer line in the email header to intentionally bypass the emails:

• Solution 1: to directly modify the email header at the network level (Ironport in my case).
• Solution 2: to remove the problematic X-Mailer line soon after an .eml mail is sent to the folder Drop, following these steps thanks to this blog:

1. copy Smtpreg.vbs into the c:\inetpub\AdminScripts - smtpreg is available to download from MS.

2. create a vbs file from the script below and copy it to the AdminScripts folder (I named mine wss)

<SCRIPT LANGUAGE="VBScript">

Sub ISMTPOnArrival_OnArrival(ByVal iMsg, EventStatus )

if iMsg.Fields("urn:schemas:mailheader:x-mailer") = "Microsoft SharePoint Foundation 2010" then

iMsg.Fields.Delete("urn:schemas:mailheader:x-mailer")

iMsg.Fields.Update

end if

iMsg.DataSource.Save

EventStatus = 0

End Sub

3. run the following commands to register the script

cd c:\inetpub\adminscripts

cscript smtpreg.vbs /add 1 OnArrival DeleteMsg CDO.SS_SMTPOnArrivalSink "mail from=*"

cscript smtpreg.vbs /setprop 1 OnArrival DeleteMsg Sink ScriptName "c:\Inetpub\AdminScripts\wss.vbs"

D - REFERENCE: 

Some interesting articles pertaining to this matter:

• An overview of incoming emails (SharePoint 2013): https://bernado-nguyen-hoan.com/2013/06/18/solving-sharepoint-2013-incoming-mails-stuck-in-drop-folder/
• A solution to the above issue: http://sharepoint-uk.blogspot.com.au/2008/03/wss-emailing-itself-x-mailer-windows.html