Migrate SharePoint 2010 Term Store to SharePoint Online with PowerShell

Last week I worked with a customer on migrating one SharePoint 2010 site to a new SharePoint Online.

I can qualify the site as Knowledge Base designed for optimal discoverability of the documents that are uploaded. To achieve a good discoverability you will need some good metadata describing the resources. Many times the metadata that is used is actually managed metadata that needs to be migrated/recreated in SharePoint Online.

If you have 10 or 20 terms it will not be an issue to recreate them, but if you have 400 for example it will not be very practical to manually recreate all terms.

There are many powershell scripts out there to export/import terms, but the success rate and the complexity might vary. This is why I would like to share how I did it and it worked out pretty well for me.

For the purpose we are going use the custom cmdlets provided for free by Gary Lapointe.

For demonstration purposes I will export one term set group with one term set that has limited number of terms. You can check it out below, it also has some parent/child terms.

In order to export the term set group you will need to deploy the WSP that will add the custom SharePoint Server 2010 commands. By doing so you will add the additional 2010 commands directly to the Microsoft.SharePoint.PowerShell snap-in.

To export taxonomy object as xml we are going to use Export-SPTerms. You will need to supply some taxonomy object as input parameter, this will be a taxonomy session if you want to export everything, for more examples see the cmdlet help. You can see how the Legal term set group looks as  xml below.

As you can see all essential information that is needed is exported, even some that will be an issue if you are importing the terms to a different environment or SharePoint Online. This is the Owner or every attribute that represents on-prem identity that you might have. The import command will also try to set this properties with the same values and it will fail because the identity as it was exported cannot be found. The way to workaround this is just to set different value for Owner that will be a valid Online identity. Now it is up to you to decide if you want to do this tradeoff and migrate the objects with different Owner than the source. Below are two lines (3 to make it fit better) that will take the content of the exported XML and will set new Owner for each XML node where the Owner attribute is not empty and later the same XML object can be used for input of the import command.

---

[xml]$termXML = Get-Content "C:\Legal.xml"
($termXML.SelectNodes("//*")) | Where {$_.Owner -ne $null} | `
ForEach-Object {$_.SetAttribute("Owner", "i:0#.f|membership|admin@MOD******.onmicrosoft.com")}

---

To import the taxonomy objects in SPO you will need to download and install the SharePoint Online Custom Cmdlets. 

This will actually install a new module called  Lapointe.SharePointOnline.PowerShell.

The command that we are going to use for the import is Import-SPOTaxonomy. For InputFile parameter we are going to use the variable from the above lines after we have set all identity attributes. If you are importing an object that is not a top level term store you should specify ParentTermStore(can get it with Get-SPOTermStore), if not you should switch on the parameter "Tenant". Before all that, you should connect to a site in your target tenant using Connect-SPOSite. Below are the lines to import the Legal term set group.

---

Connect-SPOSite -Url "https://mod******.sharepoint.com"
Import-SPOTaxonomy -InputFile $termXML -ParentTermStore (Get-SPOTermStore)

---

And this is it. Our Legal term set group is recreated and available in the entire tenant. One nice thing is that the GUIDs will be copied as well.

I hope that this was helpful and big thanks to Gary Lapointe for writing this great tools! The same approach should work for SharePoint 2013, but I have not tested this.

Disable SharePoint Event Firing in PowerShell process

Last week I worked with a customer that is using SharePoint 2010 as part of their enterprise DMS solution. Only a small part of the users are accessing the SharePoint sites directly, but they are accessing and adding documents using 3rd party Outlook integration product and in-house legacy LOB system integrations with SharePoint. If you have dealt with such DMS solutions(which is not uncommon in some industries) you will know that during the exploitation you might end up with a complex folder structures created based on the document metadata. You might also end up with large numbers of empty folders.

The empty folders are an issue for my customer and they reached me with a request to write a PowerShell cleanup script that will run on schedule and will delete the empty folders.

The catch is that there is an ItemDeleting event receiver that is preventing the deletion of any item including folders. You can see in my dev. machine a similar event receiver in action. It is also stopping the operation when I call Recycle() on item in PowerShell.

If you are a developer, most probably you know how to disable the event firing inside an event receiver in order to prevent the firing of other event receivers. This is done by setting the value of property SPItemEventReceiver.EventFiringEnabled. We can do the same thing in powershell with below code and prevent any events from being fired.

$assembly = [Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint");
$type = $assembly.GetType("Microsoft.SharePoint.SPEventManager");
$prop = $type.GetProperty([string]"EventFiringDisabled",`
[System.Reflection.BindingFlags] `
([System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::Static)); 
#SET EVENT FIRING DISABLED.
$prop.SetValue($null, $true, $null); 
 
<#
 DO WHAT YOU NEED TO DO
#>
 
#SET EVENT FIRING ENABLED.
$prop.SetValue($null, $false, $null); 

This code will disable the event firing in the current PowerShell thread and I am able to delete/recycle any item without executing the event handler. I have tested this with SharePoint 2010 and 2013, haven't tested it on SharePoint 2016, but I assume that it will work there too.

This is very powerful technique use it carefully on your own responsibility. I hope that this was helpfull!

Unable to change the password of Managed Service Account from SharePoint

Since SharePoint 2010 we have the concept for managed service accounts in SharePoint. In order to use an account in SharePoint 2010/2013 you first need to register it as managed. After you register the account as managed, you can change the account password from the SharePoint(UI and PowerShell) you can even set an automatic password change in order to comply with the security policies of the company. It is even recommended to change the password of managed account only from SharePoint. This way the password will be changed and SharePoint will be aware of this change. Before we have the managed accounts it was real pain and a risky operation to change the passwords of the accounts used by SharePoint. However, I have seen multiple deployments (including mine done in the early days) where service accounts are created with attributes "Password never expires" and "User cannot change password". This attributes are self-descriptive enough. After some time when the environment is deployed, everything is working fine and we are all happy, comes the time where you need to change the password for security reasons as described above. To do this you go in the Central Administration or in powershell and performing just a routine change of managed account password, but you hit below "Access is Denied" errors:

In PowerShell:

---

PS C:\> Set-SPManagedAccount -identity ILABS\SP_PortalApppool -NewPassword `
(ConvertTo-SecureString "SomeNewPass1234" -AsPlainText -Force) -SetNewPassword `
-ConfirmPassword (ConvertTo-SecureString "SomeNewPass1234" -AsPlainText -Force)
Set-SPManagedAccount : Access is denied
At line:1 char:1
+ Set-SPManagedAccount -identity ILABS\SP_PortalApppool -NewPassword (ConvertTo-Se ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Share...tManagedAccount:SPCmdletSetManagedAccount) [Set-SPManagedAccount], Win32Exception
    + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletSetManagedAccount

---

Again the "Access is denied" is not telling us what is the reason behind this.
The explanation is that the password change should be done by the user that is subject of this change and if he does not have the permission to do so, the operation will fail. The permission users to change their own password is denied by the attribute "User cannot change password". So if you do not have solid reason to do so, do not turn on the "User cannot change password" attribute of accounts that are used in SharePoint.
The second error you may hit when you enter new password or decide to use SharePoint generated new password is following:

In PowerShell:

---

Set-SPManagedAccount : The password does not meet the password policy requirements. Check the minimum password length, password complexity and password history requirements At line:1 char:1 + Set-SPManagedAccount -identity ILABS\SP_PortalApppool -NewPassword (ConvertTo-Se ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidData: (Microsoft.Share...tManagedAccount:SPCmdletSetManagedAccount) [Set-SPManage dAccount], Win32Exception + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletSetManagedAccount

---

This error is more descriptive, it is telling us what is wrong.
Unfortunately, it can be a bit misleading, because you are sure that you are compliant with your password policy  and you are still getting this error. Well there is one not so obvious conditions that could lead to this error.
The reason is that the password of the account was recently changed. We all know that there is a GPO policy that after certain time will prompt the user for password change, it is a good security practice to keep this GPO policy enabled, it is called "Maximum Password age".
However, there is another policy that define the minimum password age. As you can guess it is called
"Minimum Password age" and it is enabled by default with value 1 day. This means that if a user change the password, he will not be able to change it again in the next 24 hours from the time of the last password change. If you open your Default domain policy you can find the password policy like this: Default Domain Policy -> Computer Configuration -> Windows Settings -> Security Settings -> Account Policy -> Password Policy.
The third error you may hit is really telling us all:

In PowerShell:

---

Set-SPManagedAccount : The password for the account ILABS\sp_portalapppool, as currently stored in SharePoint, is not the same as the current password for the account within Active Directory. Change the password to match the existing password within Active Directory in order to continue. At line:1 char:1 + Set-SPManagedAccount -identity ILABS\SP_PortalApppool -NewPassword (ConvertTo-Se ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidData: (Microsoft.Share...tManagedAccount:SPCmdletSetManagedAccount) [Set-SPManage dAccount], InvalidOperationException + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletSetManagedAccount

---

In this case the password of the managed account is different from the password stored in SharePoint, most probably due to a recent password change in the AD.
To correct this you first need to know what is the current password in the AD or if you do not know it you may try to reset it again to value that you will be aware of.
Then you go to the Central Administration, select change password, mark use existing password, enter it and you are ready to proceed as shown below.

Оr use PowerShell:

---

Set-SPManagedAccount -identity "ILABS\SP_PortalAppPool" -ExistingPassword (Convertto-Securestring "demo!234" -AsPlainText -Force)

---

Microsoft Dynamics CRM 11 and SharePoint 2010 "Alternate Access Mapping" error

Last week a local firm contacted us about an issue that they have with Dynamics CRM 11  and SharePoint 2010 integration, but first I will make you a brief overview of the integration between those two products.
Dynamics CRM 11 can be integrated with SharePoint in order to have centralized Document management functionality based on SharePoint.
CRM 11 On-Premise  can be combined with MOSS 2007, SharePoint 2010 and SharePoint 2013 (All editions including Online). The integration will allow us to have one location where to store the documents from the CRM entities and records, the users can work with the documents from the CRM as well in SharePoint, no documents will be stored in the CRM itself.
At this time we do not have standard support from Microsoft for Microsoft Office SharePoint Server 2007(MOSS 2007), so this article is mainly for the integration with SharePoint 2010, also the issue that is the reason for this article is with SharePoint 2010.
The integration with SharePoint 2010 is fairly simple process, since the CRM is using SharePoint as a document storage. As this is an overview of the integration I am not getting in details.
The process is very straightforward and can be done from the CRM interface. You can check following article for detailed procedure for integrating this products, Here.
The important part(but not mandatory) here is the List Component that is deployed on the SharePoint side, it gives us integrated look of file "grid" and the ability to create the libraries and folders automatically from the CRM. Below you can see how it looks with installed list component and without it.

With installed list component:

Without list component, the SharePoint page is shown as it is in iframe:

The issue of the customer started when he migrated the sql instance of the SharePoint to a different server, the customer started to receive below error when a new record is created in the CRM.

Error:

---

An error occurred while loading the page. The URL may not have been mapped in the SharePoint Server. Ask your system administrator to check the Configure alternate access mappings settings in the SharePoint Central Administration.

---

It is clear that the CRM was trying to create a new folder for the record in the Accounts  library(in my case). If the users open record with existing folder they are able to view the content correctly, they can upload a new document, but the file view fails to refresh.
My first thoughts were "Well there must be something wrong with the AAM from Sharepoint side.".
Unfortunately after a couple of hours troubleshooting and testing there was nothing wrong with the Alternate Access Mapping.
According to the customer nothing was changed except the migration of the SQL instance. I am not sure what exactly were the versions of the SharePoint and the CRM because I had limited time for access on the systems.
However, I decided that there must be something wrong with the list component. I downloaded the latest available package and deployed it in the Document location site.
Can you guess what happened? Yes everything started to work as charm!
Since I cannot confirm the exact versions of the products and what were the conditions that led to this behaviour. I think that if you get such errors it is worth trying to upgrade to the latest list component solution available.
At this time the latest package version is 05.00.9690.4159 from 7/1/2014.

You can download the solutions for SharePoint 2010 and 2013 HERE

SPDeletedSite and SharePoint Deleted Site Collections alert script

Couple of weeks ago I received a question from one of our customers if there is a way to get alert when a site collection from their SharePoint 2013 farm is deleted. This was kind of important for them because the power to create and delete site collections is delegated to project owners, team leaders etc. The customer often receives a requests for restoring a site that was deleted by mistake or they suddenly notice that a site with important information just disappears(deleted by the owner). The fastest way to restore such deleted site is to use the command Restore-SPDeletedSite, but often the request for restore comes after 30 days and the sites are permanently deleted.
As far as I know, there is no OOTB feature that can notify for deleted site collections, so my proposal to the customer was to write a PowerShell script that can track the deleted site collections and send notifications. The script is fairly simple it is just getting the current deleted sites with the command Get-SPDeletedSite -WebApplication <WebAppUrl> and it compare the result with the result from the last run and if there are new deleted sites it will send a mail with details like below one:

And since the key point in this script is the retrieval of SPDeletedSite I would like to explain a bit more for this feature and and share the history of it.

I guess that every SharePoint administrator knows about this feature in SharePoint 2013 and a big part of us are so happy that we have it.

The story of it starts with the release of the SharePoint 2010 with a capability called "Gradual Site Delete". This capability was designed in order to mitigate performance degradation or even service interruption caused by so called Lock-escalation that may happen in the content database if we attempt to delete very large site collection. This was an issue in WSS 3.0 and Microsoft SharePoint Server 2007(MOSS 2007).

The short story around "Gradual Site Delete" is that when a user deletes site collection, the site collection is not instantly deleted, it is marked for deletion and the content becomes unavailable. Then for the actual deletion is responsible a timer job definition called "Gradual Site Delete" that is executed on daily schedule by default(configurable).

This timer job deletes the site collection from the content database in small enough portions to prevent lock escalation. If you want to read more on the subject, please check the post from Bill Baer.

Then with SharePoint 2010 SP1 we received 3 additional cmdlets that are allowing us to work with site collections that are "marked" for deletion, but not deleted yet. They are the same as in SharePoint 2013 - Get-SPDeletedSite, Restore-SPDeletedSite, Remove-SPDeletedSite and Move-SPDeletedSite (Only in SharePoint 2013).

And now with the cmdlet Restore-SPDeletedSite we can restore site collection that was deleted by the end user.

As I said eventually this 'deleted' sites should be permanently deleted by the "Gradual Site Delete" timer job. The exact time how many days the site collection should be kept in "Gradual Site Delete" phase as SPDeletedSite object (marked for deletion) before permanent deletion, depends on the time settings of your site collection recycle bin that is set on Web Application level.

By default the Recycle Bin is enabled and the retention period for the deleted items is 30 days. This means that if a site collection is deleted it will stay as "restorable" SPDeletedSite object for 30 days.

Be aware that when you delete site collection in powershell with the command Remove-SPSite, you need additionally to specify parameter  -GradualDelete in order to use the Gradual Site Delete. 

If this parameter is not used the site collection will be instantly deleted, it is strongly recommended to use Gradual Site Delete for large site collections.

Download the script from: Technet Gallery