SharePoint Online - Azure WebJob

A - OBJECTIVE: to schedule a timer job for SharePoint Online using Azure App Services.

B - PROBLEMS :

In SharePoint Online, we need to schedule a "timer job" without any server or Virtual Machines, and Azure WebJob is the best option to schedule a SharePoint task to run automatically based on a pre-defined schedule.

C - SOLUTION:

High-level approach:

1. Download & install Visual Studio (to create a Console Application .NET Framework)
2. Subscribe to Azure App Services to enable the feature
3. Develop a console application (.NET Framework) using C# to programmatically manipulate the SharePoint list and library using CSOM.
4. Publish the solution to Azure
5. Schedule a CRON job.

D - SOURCE CODE:

The following details are to create a sample:  

1. In the newly created project, right-click to select "Manage NuGet Packages...", then add the package "AppForSharePointOnlineWebToolkit", and ensure that the 2 files "SharePointContext.cs" and "TokenHelper.cs" is generated automatically in the project.

2. Configure all configurable keys in the file App.config

3. Set up a connection from the console application to SharePoint Online: reference to Microsoft Dev #1 below.

4. To query a large list (more than 5,000 items):

        private static List<ListItem> LoadLargeList(ClientContext context, string listName)
        {
            List loadedList = context.Web.Lists.GetByTitle(listName);
            context.Load(loadedList);
            context.ExecuteQuery();

            var listItemCollection = new List<ListItem>();
            ListItemCollectionPosition position = null;

            do
            {
                CamlQuery camlQuery = new CamlQuery();
                camlQuery.ViewXml =
                    @"<View Scope='Recursive'>
                        <Query></Query>
                        <RowLimit>5000</RowLimit>
                    </View>";
                camlQuery.ListItemCollectionPosition = position;

                ListItemCollection listItems = loadedList.GetItems(camlQuery);
                context.Load(listItems);
                context.ExecuteQuery();

                position = listItems.ListItemCollectionPosition;

                foreach(ListItem item in listItems)
                {
                    listItemCollection.Add(item);
                }
            } 
            while (position != null);

            return listItemCollection;

        }

5. To get all files in a document library which has many folders:

                        //handle the Document Library
                        FolderCollection allRequestDocSets = context.Web
                            .GetFolderByServerRelativeUrl(ConfigurationManager.AppSettings["SPLibRequestDocsRelativeURL"])
                            .Folders;

                        context.Load(allRequestDocSets);
                        context.ExecuteQuery();

                        int countArchivedDocSets = 0;
                        Global.allArchivedFiles = new List<File>();

                        foreach (Folder docset in allRequestDocSets)
                        {
                            //Console.WriteLine("DocSet Processing : " + docset.Name);

                            if (!String.IsNullOrEmpty(docset.Name) && docset.Name.Split('-').Length == 2)
                            {
                                int requestID = Int32.Parse(docset.Name.Split('-')[1].Trim());
                                if (archivedIDs.Contains(requestID))
                                {
                                    countArchivedDocSets++;
                                    context.Load(docset.Files);
                                    context.ExecuteQuery();
                                    FileCollection archivedFileCollection = docset.Files;
                                    foreach (File file in archivedFileCollection)
                                    {
                                        Global.allArchivedFiles.Add(file);
                                    }
                                }
                            }

                        }

6. The console application can be published directly to Azure App Service

7. Lastly, you can use App Service Editor to modify the schedule

Note: a license is required to schedule a webjob.

REFERENCE:

1. Microsoft Dev: https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/getting-started-with-building-azure-webjobs-for-your-office365-sites

2. CSOM: https://docs.microsoft.com/en-us/sharepoint/dev/sp-add-ins/complete-basic-operations-using-sharepoint-client-library-code