Post by: Oliver Wirkus
Posted on: 3/28/2017 5:02:00 PM
Categories: Office 365;SharePoint
Description: How to overcome the list view threshold in SharePoint online by using indexed columns.
you add more than 5000 items to a list or library in SharePoint online, you are
faced with this error message Unfortunately, this list view threshold can't be changed in SharePoint online, but there are options to overcome this limitation. If you use Bing (or Google) to look for options, you will probably find this article published by Microsoft. It highlights some options to manage large lists (with more than 5000 items) in SharePoint online. One of those options is to create a view using an indexed column as a filter. Because I used this option in one of my recent projects to overcome the list view threshold in SharePoint online, I created this blog post to share my findings. SharePoint online saves all the data in a SQL Server database. One option to improve the performance and to reduce the CPU load while retrieving items is to use indexed columns. I guess Microsoft suggested to use views with indexed columns to overcome the list view threshold because of that reason. Let's have a look on how to create an indexed column first before I continue with setting up a view. To create an indexed column, you need to navigate to the list settings first. On the settings page look for the list of columns. At the bottom of the list of columns, you'll find the link 'Indexed Columns'. If you click on this link, you'll get to the list of indexed columns. Here new indexed columns can be created, but keep in mind the number of indexed columns is limited to 20 per list. In addition, not all of the existing columns can be used as indexed columns. The following table is providing detailsSupported Column TypesUnsupported Column TypesSingle line of textMultiple lines of textNumberHyperlink/PictureCurrencyAny custom field typeDateTimeCalculated fieldChoice filed (single value only)Multi-value choice fieldLookup (single value only)Multi-value lookup fieldPerson/Group (single value only)External dataManaged Metadata Here is one thing I would like to note from my personal experience, creating an indexed column based on a lookup field does not help in terms of overcoming the list view threshold. My recommendation is to replace Choice fields with managed metadata fields which will also enhance the maintainability of the data. After an indexed column has been created, it can't be used right away. It usually takes some time for SharePoint online to propagate the changes to the underlying SQL server database. If you are creating a new indexed column because you want to fix an issue with a list that has already hit the 5000 items list view threshold, then this won't work! As creating an indexed column requires SharePoint online to 'touch' every single item in the list, the list view threshold will be hit again while the index gets created, which will stop the process internally. You'll end up thinking the index has been created, but it hasn't. If you need to create an indexed column for a list that has already hit the list view threshold, you need to create an empty list first, create all the indexed columns and when SharePoint online is done with that, copy the items to the list. This is the only way I know to create an indexed column for a list that has more than 5000 items. Let's continue with creating a basic view. For this blog post, I'm creating a basic view that is using the indexed ID column to create a filter as this column is an indexed column by default. The following screenshot shows that a view based on an indexed column is able to show more than 5000 items. Let's continue with adding a sort to the view I have just created. The following screenshot show that I just added the 'End Date' column as a sorting parameter to the view. I want the view to display the items with the most recent end date first. But if I use this view, SharePoint online displays the odd error message again. Why, the view worked before, didn't it? The answer is easy! Yes, the view worked perfectly as long as it was just using the indexed column. Things changed when I added an additional column to sort the list items. This will add an additional column to the internal query SharePoint online is using to retrieve items. As this additional column is not an indexed column yet, we are hitting the 5000 item list view threshold again. Lessons learned if you want to use indexed columns to overcome the 5000 items list view threshold, you need to keep the following in mind You can only have 20 indexed columns per list. Plan thoughtfully! You can't create an indexed column if the list is already hitting the list view threshold. If you use additional sorting in a view, the sort column also needs to be an indexed column. Review the list items and check if some can be deleted or archived to another list. Although I understand why Microsoft implemented this list view threshold in SharePoint online many years ago, I don't think it is really needed anymore. Today's hardware and modern data centers should be robust enough to allow tenants to have lists with more than 5000 items. I wouldn't be too surprised if we would see changes in terms of the list view threshold in the near future. Update Unfortunately there is another limitation I wasn't aware of. This is what I found out "When you create a filtered view, make sure the first indexed column in the filter expression does not exceed the List View Threshold. SharePoint selects the first indexed column in a query. Other columns you specify in the view filter may or may not be indexed, but the view does not use those indexes, even if the result of the filtered view returns less than the List View Threshold." (https//support.office.com/en-us/article/Manage-large-lists-and-libraries-in-SharePoint-b8588dae-9387-48c2-9248-c24122f07c59). To me this means that even with indexed columns there is no viable way how to overcome the list view threshold in SharePoint online! Here is the PowerShell script I used to create list items automatically (I used an additional number field 'LVTNumber') # Add references to SharePoint client assemblies and authenticate to Office 365 site - required for CSOM
Add-Type -Path “C\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.dll”
Add-Type -Path “C\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.Runtime.dll”
Add-Type -Path “C\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.WorkflowServices.dll”
# Specify tenant admin and site URL (replace '...' with your settings)
$SiteUrl = "..."
$ListName = "..."
$UserName = "..."
$SecurePassword = ConvertTo-SecureString "..." -AsPlainText -Force
# Bind to site collection
$ClientContext = New-Object Microsoft.SharePoint.Client.ClientContext($SiteUrl)
$credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $SecurePassword)
$ClientContext.Credentials = $credentials
# Get the list
$List = $ClientContext.Web.Lists.GetByTitle($ListName)
# Loop to create list items
for ($i=1; $i -le 10000; $i++)
$ListItemCreationInformation = New-Object Microsoft.SharePoint.Client.ListItemCreationInformation
$NewListItem = $List.AddItem($ListItemCreationInformation)
$NewListItem["Title"] = "LVTTest_$($i)"
$NewListItem["LVTNumber"] = $i;
write-host "Item created LVTTest_$($i)"
write-host "Script finished!" Post by: Oliver Wirkus
Posted on: 3/24/2017 4:43:00 PM
Categories: Business;Office 365;SharePoint
Description: This blog post describes how to use calendar overlays in SharePoint
Calendars are very popular in SharePoint. They get added to many sites and subsites in various SharePoint portals, especially to team and project sites. Although calendars are widely used in SharePoint, not many site owners spend time planning how to use them. I often see subsites with a single calendar crammed with many appointments of different kinds. My usual recommendation is to use one calendar for each type of appointment, but when suggesting this improvement, many site owners are afraid it may affect usability. This is where calendar overlays come into play. What are calendar overlays in SharePoint? In a nutshell, with calendar overlays, you can merge multiple calendar views. Those additional views can come from different calendars or from just one calendar. Each overlaid calendar can be displayed in a different color. How can overlaid calendars be used in SharePoint? I see two typical use cases for calendar overlays in SharePoint. The first one is to aggregate appointments from multiple calendars into a single view. Here is an example a marketing team uses one calendar for vacations, another for stat holidays and a third calendar for managing events. With overlaid calendars, the event planning will become much easier as holidays and vacations are superimposed on the events calendar, even though they are managed in different calendars. The second use case is the popular color coding with different types of events being displayed using different colors. This can be achieved by creating a view for each type of appointment, even if only one calendar is used. Overlaying these views will result in a single calendar view displaying different types of appointments in different colors. How are calendar overlays created? Let me show you this by using an example you can recreate in your environment easily. In my demo, I have a subsite with three calendars, a primary calendar, a calendar listing stat holidays and a calendar for vacation requests. I would like to superimpose the calendar with the stat holidays and the vacation calendar on the main calendar. Both calendars are providing a calendar view called 'All Items'. Before I continue with some screenshots, I should mention that calendar overlays only work with calendar views. List views cannot be overlaid. In my example, I have added Easter Sunday to the stat holidays calendar and a vacation request to the vacation calendar. To create the overlays, I turn to the primary calendar, open the ribbon and select 'Calendar Overlays' The following dialogue shows up, and I click on the ‘New Calendar’ link I create a new calendar overlay for vacations as
shown in the next screenshot (don’t
forget to click on ‘Resolve’ to be able to select an additional calendar list) I repeat this step with the stat holidays calendar. After I have done that, the list of overlaid calendars looks like this Let's switch back to the primary calendar and see what the overlaid calendars look like Here you can see that the view from the vacation calendar (red) is added to the primary calendar and the view from the stat holiday calendar shows up in green. Keep in mind that both appointments are NOT part of the primary calendar. They are still managed in their own calendars. On the left side, SharePoint shows a legend explaining which calendar is displayed and in what color. To navigate to one of the overlaid calendars, click on the calendar name in the left navigation. As you can see, it just takes a little additional configuration, to create calendar overlays which improve usability and maintainability. That's why calendar overlays are my hidden gems in SharePoint.