Attach additional event listener to out of the box SharePoint ribbon control

Have you ever had a use case where you must do something additional when an out of the box ribbon control is clicked, like "Open With Explorer" or "Export to Excel"? Maybe not, until now?

What could make this use case valid is the fact that the ribbon is slowly being phased out and it is no longer available in the modern experience, in addition not all features available in the ribbon are supported by all browsers and clients.

Having something additional being executed when a button from the ribbon is clicked can help you collect usage data for this features and better plan the move to modern experience or (like it or not) making Edge default browser, which does not support "Open with Explorer".

I am not aware of any other way to collect this information in SharePoint on-prem or Online.

Here is an example script that can be added as Scriptlink user custom action and it will attach additional click event listener on the "Open With Explorer" button. Everything that the function will do is to log the action, user name and the document library title in the console, but you can do whatever you find for useful, like logging it in the ULS or calling an API that will record the event.
I have tested it in SharePoint 2016 but it should work in SharePoint 2013 and SharePoint Online classic experience lists, it requires JQuery.

The Script:

Build trust for federated search between two SharePoint Server farms

Federated search is when you aim to receive search result from separate SharePoint (on-premises) by performing a search query in a separate on-premise SharePoint farm.
If you have done such configuration probably you have seen the official documentation for setting it up. This procedure will work in most of the cases.
However, this will not work if you do not have outbound connectivity from the remote farm that will receive the search query (ReceivingFarm) to the farm that is sending the query (SendingFarm).
In that case the federated search will be possible as long as the SendingFarm can access the ReceivingFarm, vice versa is not required, but you should take a bit different approach when building the trust since the SendingFarm web app metadata end point will not be available.
The first thing that needs to be done is to export the root and the token signing certificates from the SendingFarm and also get the Issuer Name (NameIdentifier) of the SendingFarm STS .

---

## Export Root Certificate
$rootCert = (Get-SPCertificateAuthority).RootCertificate
$rootCert.Export("Cert") | Set-Content "C:\SendingFarmRoot.cer" -Encoding byte
 
## Export Signing Certificate
$stsCert = (Get-SPSecurityTokenServiceConfig).LocalLoginProvider.SigningCertificate
$stsCert.Export("Cert") | Set-Content "C:\SendingFarmSTS.cer" -Encoding byte
 
## Get the STS Issuer Name
$issuerName = (Get-SPSecurityTokenServiceConfig).NameIdentifier

---

The difference from the official procedure will be how we are going to create the trusted token issuer and the trusted root authority in the ReceivingFarm, this is step 3 in the official procedure.
First copy the SendingFarm certificated to the ReceivingFarm.
Having above done you can create the trusted security token issuer and the trusted root authority  in the ReceivingFarm.

---

## Read SendingFarm Signing certificate
$stsCert = Get-PfxCertificate "C:\Install\Certs\SendingFarmSTS.cer"
## Read SendingFarm root certificate
$rootCert = Get-PfxCertificate "C:\Install\Certs\SendingFarmRoot.cer"
# Create a trusted security token issuer
$i = New-SPTrustedSecurityTokenIssuer -Name "SendingFarm" `
                                      -Certificate $stsCert `
                                      -IsTrustBroker:$false `
                                      -RegisteredIssuerName "<SendingFarm IssuerName>"
 
# Configure trust of the token-signing certificate'
# by adding the trust used to sign oAuth tokens'
# to the list of trusted root authorities'
# in ReceivingFarm
New-SPTrustedRootAuthority -Name "SendingFarm" `
                           -Certificate $rootCert

---

Now, you can continue with the trust configuration as it is described in the documentation.

I hope you found this helpful!

Search result security trimming for File Share content source with ADFS users

Indexing of file shares is a common requirement if you have  legacy file share that hasn't been migrated to SharePoint or you are using file share for archiving purposes. SharePoint Search can provide this functionality.

SharePoint also support search result trimming for file share content. That means that if the user does not have permission to a certain content on the file share, the user will not see the content appearing in the search results.

If you are using Windows integrated authentication the security trimming does not require anything special, it will just work. This is not the case if your users are using ADFS to authenticate against SharePoint. If you are using ADFS it is mandatory to have two more claims in order to make the security trimming working.

Those claims are Primary SID and Primary Group SID. In some articles you can find that the Primary SID is required in S2S authentication scenario, but nothing about the Primary Group SID. The Primary SID is the User object SID and the Primary Group SID is the SID of the Domain's primary group

In this post I will demonstrate how to setup it up in ADFS and SharePoint. I have tested it with ADFS 4.0 and SharePoint Server 2016.

On the ADFS side you will need to create two Issuance Transformation rules using template "Pass Through or Filter an Incoming Claim".

You can use below rules to append your rule file and import it to your SharePoint Relying Party Trust(s).

But first, you will have to export your current rules by using below command.

---

$sprp = Get-AdfsRelyingPartyTrust -Name "<SharePointRP_Name>"
$sprp.IssuanceTransformRules | Out-File "C:\IssuanceTransformRules.txt"

---

Append the file with below rules for Primary SID and Primary Group SID or any additional rules you might want.

---

@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass Primary Group SID"
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarygroupsid"]
 => issue(claim = c);
 
@RuleTemplate = "PassThroughClaims"
@RuleName = "Pass Primary SID"
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid"]
 => issue(claim = c);

---

Now, import  the file containing your old and newly added rules.

---

Set-AdfsRelyingPartyTrust -TargetName "<SharePointRP_Name>"`
 -IssuanceTransformRulesFile "C:\IssuanceTransformRules.txt"

---

On the SharePoint side you will have to create the claim type mappings for the two new claims. You can use the example script below.

---

Add-PSSnapin *SH*
 
$sts = Get-SPTrustedIdentityTokenIssuer
 
$sts.ClaimTypes.Add("http://schemas.microsoft.com/ws/2008/06/identity/claims/primarygroupsid")
$sts.ClaimTypes.Add("http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid")
 
$sts.Update()
 
$map = New-SPClaimTypeMapping `
-IncomingClaimType  "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarygroupsid" `
-IncomingClaimTypeDisplayName  "Primary group SID" -SameAsIncoming
$map2 = New-SPClaimTypeMapping `
-IncomingClaimType "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid" `
-IncomingClaimTypeDisplayName "Primary SID" -SameAsIncoming
 
Add-SPClaimTypeMapping -Identity $map -TrustedIdentityTokenIssuer $sts
Add-SPClaimTypeMapping -Identity $map2 -TrustedIdentityTokenIssuer $sts

---

And that's is all you need to do. If everything is fine you will see values for the two new claims and the security trimming should work for the ADFS users.

If you are wondering how to see the claims, I am using one of the many SharePoint Claims Viewer web parts found on the internet. I am also using LDAPCP for claims provider. Above requirement and scripts will be the same if you are using the OOTB claims provider.

I hope you found it helpful!

Change Site Policy Deletion notification email template in SharePoint

The Site Policies in SharePoint are information management tool that helps you implement some site life cycle management. Whether this is dictated by internal house keeping rules or some external regulations that apply to your organisation, the Site Policy is the out of the box way to go if you want to "close" a site, delete it or both, automaticaly after certain period of time.

With site policies you have the option to notify the site owners in advance before the site is deleted. The mail looks like the one below.

The information in this email might not be suitable for your organization.

Fortunately there is a way to change the default Site Policy notification email body template and the email subject. This is not done in some XML template file like the Alerts template, maybe there is one, but I have not found it. There is SSOM and CSOM API that you can use to set custom email body template per policy.

The documentation of this is very poor and the best resource on this is the article Site Policy in SharePoint.

Unfortunately I have not managed to make this work server side or using PowerShell. I have tried with SharePoint 2013, SharePoint 2016 and SharePoint Online ssom and csom as well.

The only way I found it working is from console application using the CSOM approach.

The site policy post above is good and the code should work as it is, but it has some gaps.

There are three placeholders that we can use, placeholders for Site Url, Deletion Date and Mailbox Id.
However the placeholders with curly braces that are demonstrated in the post will not work.

I would like to save you some time testing especially if you are targeting SharePoint Online, as there you cannot manually run the "Expiration Policy" timer job.

The correct placeholders are below, without curly braces or any spaces.

---

SiteUrl: <!--SiteUrl-->
Deletion Date: <!--SiteDeleteDate-->
Mailbox ID: <!--TeamMailboxID-->

---

I hope you found this helpful!

Disable SharePoint Event Firing in PowerShell process

Last week I worked with a customer that is using SharePoint 2010 as part of their enterprise DMS solution. Only a small part of the users are accessing the SharePoint sites directly, but they are accessing and adding documents using 3rd party Outlook integration product and in-house legacy LOB system integrations with SharePoint. If you have dealt with such DMS solutions(which is not uncommon in some industries) you will know that during the exploitation you might end up with a complex folder structures created based on the document metadata. You might also end up with large numbers of empty folders.

The empty folders are an issue for my customer and they reached me with a request to write a PowerShell cleanup script that will run on schedule and will delete the empty folders.

The catch is that there is an ItemDeleting event receiver that is preventing the deletion of any item including folders. You can see in my dev. machine a similar event receiver in action. It is also stopping the operation when I call Recycle() on item in PowerShell.

If you are a developer, most probably you know how to disable the event firing inside an event receiver in order to prevent the firing of other event receivers. This is done by setting the value of property SPItemEventReceiver.EventFiringEnabled. We can do the same thing in powershell with below code and prevent any events from being fired.

$assembly = [Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint");
$type = $assembly.GetType("Microsoft.SharePoint.SPEventManager");
$prop = $type.GetProperty([string]"EventFiringDisabled",`
[System.Reflection.BindingFlags] `
([System.Reflection.BindingFlags]::NonPublic -bor [System.Reflection.BindingFlags]::Static)); 
#SET EVENT FIRING DISABLED.
$prop.SetValue($null, $true, $null); 
 
<#
 DO WHAT YOU NEED TO DO
#>
 
#SET EVENT FIRING ENABLED.
$prop.SetValue($null, $false, $null); 

This code will disable the event firing in the current PowerShell thread and I am able to delete/recycle any item without executing the event handler. I have tested this with SharePoint 2010 and 2013, haven't tested it on SharePoint 2016, but I assume that it will work there too.

This is very powerful technique use it carefully on your own responsibility. I hope that this was helpfull!

Error when access Nintex Live Management page on fresh Nintex Workflow 2013 installation.[Tip]

Today I hit a strange issue with a fresh installation of Nintex Workflow 2013 with Nintex Live.
Nintex Live solution was deployed by the installer. However when I tried to access the Nintex Live Management page in the Central Administration I received below error.

---

The resource object with key 'LiveAdmin_Page_Management_Title' was not found.

---

If you have done everything correctly so far the solution is very simple.
Open SharePoint Management Shell as Administrator and launch the command Install-LiveService , test if the page is now available, perform IISRESET on your CA server(s).

Query Usage and Health Data collection database for slow requests execution

Here is another article I am going to dedicate to the SharePoint performance monitoring.

For me one of the most useful out of the box tools to gather information for the farm performance is the Usage and Health data collection done by the Usage and Health Data Collection Service Application. This service application can be easily provisioned and set if you have installed your SharePoint farm with AutoSPInstaller.

I highly recommend to plan,provision and use this service application in your SharePoint farm!

If you have not done this yet, you can see how to provision the application here. I recommend to specify at least the logging database name in the New-SPUsageApplication  command, for further configuration see following  article, there you will find how to set up various properties like log location, what to monitor and the retention of the logs. Here is an official resource from TechNet.

Above I mentioned the logging database, this is the database used by the Usage and Health Data Collection Service Application to store all the logs. The logs are collected on each SharePoint server and stored in the log location you have set. This logs are then pushed into the logging database by a timer job called "Microsoft SharePoint Foundation Usage Data Import ", by default it is running every 5 minutes.

The logging database is the only database in SharePoint where running SQL queries is supported in order to analyze our logs. See some official guidance View data in the logging database in SharePoint 2013.

The easiest way to get some information from the logging database is from the CA interface described here. There you can see some statistics for the Slowest Page request and you can filter by Server,Web Application, maximum range of 100 rows and predefined timeframe by day, week, month, as you can see on the screenshot below.

We are getting essential information from this page, but in my case I needed to track some occasional slow request reported by the users. Here are some disadvantages of the result that we get from "View Health reports" page.

1. This is the major one, because it is limiting the amount of information we can receive. The results are aggregated averages for a certain request. We can have one slow request or we may have many. In this case we are going to receive Average, Minimum, Maximum values for Duration for example, we cannot see when a peak had occurred, the requests are grouped.

2. The columns we can see is predefined. This is again because of the request grouping. If you do a simple Select Top 100 for example from one of the dbo.RequestUsage (this is the table where the requests are logged) partitions you will see that we have quite a lot of columns and some of them can be really useful for our troubleshooting process.

3. The timeframes are predefined, the result is limited by 100.

This is why I created a SQL query that will output all request for a certain timeframe that is configured by you, it can be less than a second or days. The output will show you all requests in that timeframe, they will not be grouped, you can add additional conditions and columns to be displayed and the most important, you can see the time that the request was logged. I find this extremely helpful for troubleshooting, you can narrow down the results for a very small timeframe, you can filter by user login for example and track the request in the ULS logs by Correlation Id. See how it looks from the screenshot below and you can compare it with the result from the Slowest Pages report.

[*UPDATE*]: I have published a PowerShell version you can find it and read more HERE

The screenshot above is an older version, in the current version that you can download below I have included filtering by UserLogin. As I mention you can modify the script to show you whatever column  you want (include them in SELECT statement and the GROUP BY clause) and include additional filters. I have put example values for some of the variables, many of them can be Null and some useful comments as well.

A common use case will be if you receive a notification that the SharePoint was "slow" for a certain user in a certain time. You can use this script and it will show you all requests for this user, this information will help you in your investigation of potential performance issue.You can then export this result in CSV and play with it in Excel for example.

Keep in mind that this query can put some load on your SQL server, the timing is in UTC and the duration is in seconds.You should run the script against your Usage and Health (logging) database. The script is tested on SharePoint 2013/SQL 2012.

 Download the script from: Technet Gallery

(You may experien issue with Chrome, use Firefox/IE to open the link)

Load Testing SharePoint 2013 with Visual Studio 2013. Gotchas for SharePoint Administrators

Last week I had to do load test for one of our customers SharePoint 2013 environment, so I decided to share with you some gotchas that might not be obvious for SharePoint Administrators that are new to Visual Studio(VS) and do not have experience with load testing with VS, but want to try or use it. 

If you happened to be a fulltime developer or maybe QA, please do not laugh out loud and feel free to leave your comments below this post.

This article is not a walkthrough on how to build load test solution with VS2013. There is a great SPC 2014 session from Todd Klindt and Shane Young, it is very informative and so much fun.

So, in the test I created for my customer I have 3 Web Tests, two with read profile browsing different sites and testing search and one that has a write profile. The write activity is to create an item in out of the box Contact List and attach a small file to the item. The first thing that failed was in the "write" Web Test.

File Upload is Failing - Now in the new versions of Visual Studio this should not happen, the feature was introduced back in VS2010. However it happened with me when I did this the first time. The entire web test failed because it was unable to find the file that should be attached in the list item. The strange thing is that in the next day I recreated the test and everything worked fine. I tested this with Visual Studio 2013 Update 1 and Update 4. The normal behavior is when you record your web test Visual Studio is intelligent enough and knows what file you are uploading, it will copy that file to the load test project root folder and will set the post request for the test to take the file in your project folder. However this might not happen and your web test will fail, like I have shown on the picture below.

[Solution 1] - This is the quick and dirty solution and since I am pro I went right for it :). When we are adding new item to a list we have a form that we should fill and then by clicking Save we send POST request with the parameters of the form, the file path  that should be uploaded is parameter as well. The POST request parameters are recorded by Visual Studio and you can edit them. You have just to find the correct request, expand it, look in to the "Form Post Parameters" like it is shown below.

Next you have to locate "File Upload Parameter" that has some value for FileName and the value is just the filename of the file you want to upload. It is possible to have empty File Upload Parameters or many different parameters. When you find the correct  File Upload Parameter you just put the literal path of the file as FileName property, like on the picture below.

Now, this is not very good because when you move your Load Test project to a different computer you have to move the file and again edit the File Upload Parameter(eventually). For a better solution keep reading/scrolling.

Masking Unwanted/Failed dependent requests - The dependent requests are the requests that you make when you hit a page and load additional resources. For example, you test request to http://sharepoint.contoso.net, but in the landing page there are multiple files referred like CSS, JavaScript, pictures etc. The requests to this files are dependent requests and they are taken into account in your web test results if they are successful or failed.

This became kind of a deal for me because I was testing highly customized site and several dependent requests for JavaScript files (.js) were failing with status 404 and this will not look good in your load test results. After a short googling I found this free eBook. There on page 189 is discussed exactly this issue. The solution to mask failed dependent request is to use web test plugin. Below you can see how a web test result looks like with failed dependent request. I have simulated this in my lab by referencing a JavaScript file located in the same site and then I deleted it. You can see that the missing file currentU.js even is making one of my primary request to fail, although everything else is fine with it.

In this book there is source code of a web test plugin that can filter dependent requests that are starting with certain string. This will be fine if your dependent request were from different domain like some CDN or caching solution. If I use this plugin as it is, it will filter all dependent requests from my SharePoint site. This is why I modified the code to filter dependent request that are ending with certain string. You can read how to develop and use your own plugin here. See below how it looks.

I have highlighted some of the important elements from the picture(hope you see it well), however I want to stress on two things. After you build your web test plugin project it is just a class library(dll), in order to have your new plugin available in your web tests you have to reference the dll in your Load Test project. The second thing is the configuration of the ending sting for filtering. This is first configured when you add the plugin to the web test, you add the string as value for property FilterDependentRequestsThatEndsWith, in my case it is ".js". If you like my plugin you can download the project (builded dll is in the bin/debug folder) here.

The third gotcha I will share took me 5 minutes to figure it out. It is not so much, but this issue popped out right after I launched the entire load test for first time. This was important because I was testing Production(like a boss) and I was going to test the limits of the Farm so the timing was negotiated with the customer as maintenance window with possible service degradation and/or outage.

Configure Load Test run location - This setting specify whether you will use Visual Studio Online or the computer you are running the test from. As I wrote above I have tested this with VS2013 Update 1 and 4, so I will assume that from Visual Studio 2013 Update 1 the default configuration is Visual Studio Online. You can change/check this in test settings of your solution it is in the General section as it is shown below.

If you click on Deployment section you will see that there is an option to add additional files that should be deployed with the solution.

[Solution 2 - File Upload is Failing] - Add the file as deployment in the test settings and if needed edit the post request form parameter to refer to the file only by filename. This solution is way better.

Finally, this post became too lengthy for blog post, but I hope it was informative!

User Profile Incremental Synchronization Timer Job - Access is denied Error

I stumbled on this issue in one of our customer's SharePoint 2013 environment and I came across many forum posts and article, but did not found the complete solution, so I decided to share it with you.

The background story is as follows. We have a SharePoint Server 2013 hosted on Windows Server 2012 environment that is using User Profile Synchronization(leveraging FIM). We have one User Profile Service application.

The User Profile Synchronization service is running on the server where the Central Administration is hosted.

So far so good, we also have the User Profile Service running on two servers, in order to have a redundancy for this service, for short I will call this servers APP01 (CA,Sync Service, User Profile Service) and APP02(User Profile Service). The User Profile Synchronization service is successfully started and running on APP01.

The issue comes when you want to go and implement the principle of least privilege. This will require to remove the Farm account from the local Administrators group on the server where the User profile Synchronization is running (FIM). You need the Farm Account to be Local Admin in order to successfully start User Profile Synchronization.

You remove the the Farm account from the local Administration group.

Time goes by and you start to receive an error Event ID: 6398 on APP02 with following description:

---

The Execute method of job definition Microsoft.Office.Server.UserProfiles.UserProfileImportJob (ID 7acb171d-5d7d-4601-ba08-85f24c0969b4) threw an exception.More information is included below.

Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))

---

This description tells us that the "User Profile Incremental Synchronization Timer Job" timer job is failing on APP02. To be sure you can use Get-SPTimerJob and for Identity parameter to use the GUID from the error message. This is the Timer Job responsible for the User Profile Synchronization. You check the Timer Job history for this job and it is running on APP02 and it is Failing.

It is possible to see this job to fail every minute one day it can run on APP02 and next run to be on APP01, or to see ongoing synchronization in the Central Administration, but when you open the miisclient.exe you see that there is no synchronization operations running, this however may prevent new synchronisations to start.

But User Profile Synchronization service is running on APP01 not on APP02!

Now all over the forums, folks are posting question like "The Synchronization timer job is running on the wrong server and it is failing". Actually this is not true!

Remember that we are running User Profile Service instance on both APP01 and APP02. The "User Profile Incremental Synchronization Timer Job" can run on both servers and it should run in success.

So the timer job is executed by the Timer Service and the Timer Service is running under the Farm account as the Sync Service. From the event log error it is obvious that the Farm account needs some permissions in order to execute the job successfully. When the account is in the local Administration group it "have it all".

The First thing to check is following MS Article. There is a section called "Remove unnecessary permissions". In this article it is explained how to grant Remote Enable permission over the Microsoft FIM 2010 WMI namespace MicrosoftIdentityIntegrationServer (on the server running FIM). This permission is needed for the Farm account to be able to run the job successfully from server that is not running User Profile Synchronization Service (FIM). Remember to do IISRESET if you are running the Sync. service on a server that is also hosting Central Administration.
Of course I did the procedure above, but does not worked out!
Apparently the job needs to interact with the Microsoft FIM 2010 WMI namespace. I am not sure for this but I think that the timer job is invoking Execute method on MIIS_ManagementAgent class.
Out of curiosity below you can see how the MicrosoftIdentityIntegrationServer (MIIS) WMI namespace looks like. I have based my User Profile Synchronization reporting script on MIIS WMI.

Now under the hood WMI is using DCOM, so in order a non-Administrator account to interact with WMI remotely it needs permissions over DCOM on the server where the Sync service is running at.

The Second things to check/change are the permissions over DCOM server on the machine that is running User Profile Synchronization Service. The Farm account needs to have Remote Access, Remote Launch and Remote Activation permissions. To do this use the article  Securing a Remote WMI Connection . Now this is a bit tricky and confusing for me. There is a simple way to check if the Farm account has remote access to WMI. By using below PowerShell command with farm account credentials from remote computer you should be able to retrieve the synchronization operations history.

---

Get-WmiObject -Class MIIS_RunHistory -Namespace root/MicrosoftIdentityIntegrationServer -ComputerName <YourSyncServer> -Credential <Domain>\<FarmAccount>

---

If there is an issue with the permissions you will receive some Access Denied error like this.

The DCOM permissions can be tricky after doing a change there close and open new PowerShell console and test again the remote WMI with the script above. If you still have Access Denied error locate Windows Management and Instrumentation DCOM application and give needed permissions to the Farm Account.

As you can see there are lots of moving  parts Timer Jobs, WMI, DCOM and so on. But there are countless reason for fail. Use the PowerShell line above. If there is an issue with the WMI remoting it will show it. There may be firewall issue it will show you something like "RPC service unavailable" error. If you do not receive a result from the remote WMI test you have some issue.

This is how I solved the issue. I am hoping that it is helpful for you!

Unable to change the password of Managed Service Account from SharePoint

Since SharePoint 2010 we have the concept for managed service accounts in SharePoint. In order to use an account in SharePoint 2010/2013 you first need to register it as managed. After you register the account as managed, you can change the account password from the SharePoint(UI and PowerShell) you can even set an automatic password change in order to comply with the security policies of the company. It is even recommended to change the password of managed account only from SharePoint. This way the password will be changed and SharePoint will be aware of this change. Before we have the managed accounts it was real pain and a risky operation to change the passwords of the accounts used by SharePoint. However, I have seen multiple deployments (including mine done in the early days) where service accounts are created with attributes "Password never expires" and "User cannot change password". This attributes are self-descriptive enough. After some time when the environment is deployed, everything is working fine and we are all happy, comes the time where you need to change the password for security reasons as described above. To do this you go in the Central Administration or in powershell and performing just a routine change of managed account password, but you hit below "Access is Denied" errors:

In PowerShell:

---

PS C:\> Set-SPManagedAccount -identity ILABS\SP_PortalApppool -NewPassword `
(ConvertTo-SecureString "SomeNewPass1234" -AsPlainText -Force) -SetNewPassword `
-ConfirmPassword (ConvertTo-SecureString "SomeNewPass1234" -AsPlainText -Force)
Set-SPManagedAccount : Access is denied
At line:1 char:1
+ Set-SPManagedAccount -identity ILABS\SP_PortalApppool -NewPassword (ConvertTo-Se ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (Microsoft.Share...tManagedAccount:SPCmdletSetManagedAccount) [Set-SPManagedAccount], Win32Exception
    + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletSetManagedAccount

---

Again the "Access is denied" is not telling us what is the reason behind this.
The explanation is that the password change should be done by the user that is subject of this change and if he does not have the permission to do so, the operation will fail. The permission users to change their own password is denied by the attribute "User cannot change password". So if you do not have solid reason to do so, do not turn on the "User cannot change password" attribute of accounts that are used in SharePoint.
The second error you may hit when you enter new password or decide to use SharePoint generated new password is following:

In PowerShell:

---

Set-SPManagedAccount : The password does not meet the password policy requirements. Check the minimum password length, password complexity and password history requirements At line:1 char:1 + Set-SPManagedAccount -identity ILABS\SP_PortalApppool -NewPassword (ConvertTo-Se ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidData: (Microsoft.Share...tManagedAccount:SPCmdletSetManagedAccount) [Set-SPManage dAccount], Win32Exception + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletSetManagedAccount

---

This error is more descriptive, it is telling us what is wrong.
Unfortunately, it can be a bit misleading, because you are sure that you are compliant with your password policy  and you are still getting this error. Well there is one not so obvious conditions that could lead to this error.
The reason is that the password of the account was recently changed. We all know that there is a GPO policy that after certain time will prompt the user for password change, it is a good security practice to keep this GPO policy enabled, it is called "Maximum Password age".
However, there is another policy that define the minimum password age. As you can guess it is called
"Minimum Password age" and it is enabled by default with value 1 day. This means that if a user change the password, he will not be able to change it again in the next 24 hours from the time of the last password change. If you open your Default domain policy you can find the password policy like this: Default Domain Policy -> Computer Configuration -> Windows Settings -> Security Settings -> Account Policy -> Password Policy.
The third error you may hit is really telling us all:

In PowerShell:

---

Set-SPManagedAccount : The password for the account ILABS\sp_portalapppool, as currently stored in SharePoint, is not the same as the current password for the account within Active Directory. Change the password to match the existing password within Active Directory in order to continue. At line:1 char:1 + Set-SPManagedAccount -identity ILABS\SP_PortalApppool -NewPassword (ConvertTo-Se ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidData: (Microsoft.Share...tManagedAccount:SPCmdletSetManagedAccount) [Set-SPManage dAccount], InvalidOperationException + FullyQualifiedErrorId : Microsoft.SharePoint.PowerShell.SPCmdletSetManagedAccount

---

In this case the password of the managed account is different from the password stored in SharePoint, most probably due to a recent password change in the AD.
To correct this you first need to know what is the current password in the AD or if you do not know it you may try to reset it again to value that you will be aware of.
Then you go to the Central Administration, select change password, mark use existing password, enter it and you are ready to proceed as shown below.

Оr use PowerShell:

---

Set-SPManagedAccount -identity "ILABS\SP_PortalAppPool" -ExistingPassword (Convertto-Securestring "demo!234" -AsPlainText -Force)

---

SPDeletedSite and SharePoint Deleted Site Collections alert script

Couple of weeks ago I received a question from one of our customers if there is a way to get alert when a site collection from their SharePoint 2013 farm is deleted. This was kind of important for them because the power to create and delete site collections is delegated to project owners, team leaders etc. The customer often receives a requests for restoring a site that was deleted by mistake or they suddenly notice that a site with important information just disappears(deleted by the owner). The fastest way to restore such deleted site is to use the command Restore-SPDeletedSite, but often the request for restore comes after 30 days and the sites are permanently deleted.
As far as I know, there is no OOTB feature that can notify for deleted site collections, so my proposal to the customer was to write a PowerShell script that can track the deleted site collections and send notifications. The script is fairly simple it is just getting the current deleted sites with the command Get-SPDeletedSite -WebApplication <WebAppUrl> and it compare the result with the result from the last run and if there are new deleted sites it will send a mail with details like below one:

And since the key point in this script is the retrieval of SPDeletedSite I would like to explain a bit more for this feature and and share the history of it.

I guess that every SharePoint administrator knows about this feature in SharePoint 2013 and a big part of us are so happy that we have it.

The story of it starts with the release of the SharePoint 2010 with a capability called "Gradual Site Delete". This capability was designed in order to mitigate performance degradation or even service interruption caused by so called Lock-escalation that may happen in the content database if we attempt to delete very large site collection. This was an issue in WSS 3.0 and Microsoft SharePoint Server 2007(MOSS 2007).

The short story around "Gradual Site Delete" is that when a user deletes site collection, the site collection is not instantly deleted, it is marked for deletion and the content becomes unavailable. Then for the actual deletion is responsible a timer job definition called "Gradual Site Delete" that is executed on daily schedule by default(configurable).

This timer job deletes the site collection from the content database in small enough portions to prevent lock escalation. If you want to read more on the subject, please check the post from Bill Baer.

Then with SharePoint 2010 SP1 we received 3 additional cmdlets that are allowing us to work with site collections that are "marked" for deletion, but not deleted yet. They are the same as in SharePoint 2013 - Get-SPDeletedSite, Restore-SPDeletedSite, Remove-SPDeletedSite and Move-SPDeletedSite (Only in SharePoint 2013).

And now with the cmdlet Restore-SPDeletedSite we can restore site collection that was deleted by the end user.

As I said eventually this 'deleted' sites should be permanently deleted by the "Gradual Site Delete" timer job. The exact time how many days the site collection should be kept in "Gradual Site Delete" phase as SPDeletedSite object (marked for deletion) before permanent deletion, depends on the time settings of your site collection recycle bin that is set on Web Application level.

By default the Recycle Bin is enabled and the retention period for the deleted items is 30 days. This means that if a site collection is deleted it will stay as "restorable" SPDeletedSite object for 30 days.

Be aware that when you delete site collection in powershell with the command Remove-SPSite, you need additionally to specify parameter  -GradualDelete in order to use the Gradual Site Delete. 

If this parameter is not used the site collection will be instantly deleted, it is strongly recommended to use Gradual Site Delete for large site collections.

Download the script from: Technet Gallery

Bulk Upload and Update User Profile Photos in SharePoint 2013

This post will be about a script I wrote and some interesting things I learned to write it. This script lets you upload and update multiple user profile photos for multiple user profiles in SharePoint 2013. There are many post on how we can rewrite the PictureURL property in the user profile object and assign it a custom value, but my script does not work that way. This script is emulating on what is happening in one popular out of the box scenario for importing user profile pictures, from AD.

The inspiration for this script came from an issue (of course) that one of my colleague have with one of our customers. The customer has some complex domain structure. They have around 16 domains and even federation. So the customer of course wants their SharePoint users to be able to have profile photos in their SharePoint profiles, so we have User Profile Synchronization in place and it is synchronizing the Picture user property from the AD attribute thumbnailPhoto. The users however can edit their profile picture and they upload new photos.

The issue is that since some CU installation this year, I am not sure, but I think it was April 2014, when a full synchronization is launched in the User Profile Application, the pictures that are uploaded by the users are lost. So my colleague ran the synchronization first on Staging environment that is almost identical to the Production, it has similar configuration of the User Profile Service Application and of course we lost the pictures that the user have uploaded. The Production environment however was not touched, but soon or later we had to apply some property mapping change and run full synchronization on Prod. too. We needed a way to get the profile pictures from Prod. reupload them to Staging for the correct profiles, then run full synchronization on Prod. and reupload again the user profile pictures.

And here I come with my powershell skills and some free time. As I said above I came across many scripts for editing the PictureURL property of the UserProfile object, but this option was not acceptable for me because in SharePoint we have 3 sizes of user profile photo thumbnails.

I first created  the script to download the biggest available picture thumbnail and save them under unique name that can later help me to determine to which account the picture belongs. I did this by getting the values of the PictureURL and the LoginName of the profiles and then simply download the largest thumbnail of the photo. You can get all the profiles in a User Profile Application with the snipped below. You will need a site collection URL to determine the service context and run it under Farm account. Of course the things are a bit different in scenario with multiple User Profile Applications or if the application is Partitioned(for multi tenant deployment).

---

$SiteURL = 'http://mysite.contoso.com/'
Add-PSSnapin Microsoft.SharePoint.PowerShell
$Site = Get-SPSite -Identity $SiteURL
$context = Get-SPServiceContext -Site $site
$upm =  New-Object Microsoft.Office.Server.UserProfiles.UserProfileManager($context)
$AllProfiles = $upm.GetEnumerator()
ForEach($profile in $AllProfiles)
{
$profile
}

---

And now I had all the profile pictures from Prod. and we had to figure out  how to upload them on Staging and "connect" them to the correct account.
And here comes the interesting part. So normally the pictures for the user profiles are imported from the thumbnailPhoto, but we have a URL to picture for value of the PictureURL property and the thumbnailPhoto, contain just a picture as binary. As we know after synchronization we need to run the out of the box command Update-SPProfilePhotoStore. 
So I found what is happening between the import and thumbnail generation.
When the import/sync from the AD happens the thumbnailPhoto value is saved as .jpg image in the same picture library with profile photos that also contains the thumbnail versions of the profile pictures. For example if you have My Site host http://mysite.contoso.com, the URL of the library in SharePoint 2013(I forgot to mension that we are working on SP2013 farm) will be http://mysite.contoso.com/User Photos/ and the pictures are in folder Profile Pictures. The thumbnails are named Account_LThump.jpg for large, Account_MThump.jpg, for midsize and Account_SThump.jpg for small. The midsize picture is used as profile photo.
So the initial picture are also saved in this library but under a special name like this 0c37852b-34d0-418e-91c6-2ac25af4be5b_RecordID.jpg. Here is the moment to say that we have only one, nonpartitioned User Profile Service Application. The first part if the picture name the guid (exactly this guid) is the guid of the default partition of nonpartitioned UPSA. It is the same on all SharePoint 2013 farms I have seen. You can check it by doing SQL query on the Profile DB, but only on non-production Farm because doing query on this DB is not supported. It is a column of every profile entry in the Profile DB.

The second part, the RecordID is a unique ID for every user in the partition. It is just a number not a guid. You can also see it in the picture above, you can also get it with powershell as property of the user profile object. But, so far I haven't found a way to see the PartitionID of the default partition of nonpartitioned UPSA in PowerShell.
And when you run Update-SPProfilePhotoStore with the corresponding parameters, it reads the available not converted to thumbnails .JPGs read the users RecordID, creates the thumbnails, add the url of the picture in the user profile and in the general case delete the original(depends on the parameters).
So this was our answer on how to upload the pictures from Prod to Staging and get all three thumbnails variants and map the PictureURL property. One additional feature to this method is that for PictureURL is mapped the url of the biggest thumbnail version(not the midsized as if it was imported from AD) and you will have pretty profile pictures with good resolution.
Here came the idea for this script. What if there are SharePoint Admins/Customers that do not want to get the pictures from the AD, but they have all the profile pictures and want to mass upload them as if the users have uploaded themselfs.
And I came up with this script. You need to run it two times(As Administrator under the Farm account) first time will be generated a CSV file with the RecordIDs,LoginNames and picture paths. The picture path will be empty you should fill it with the local or the network path to the picture. The picture can be whatever resolution and almost any format (JPG,PNG,GIF,BPM,TIFF), the script will convert it to jpeg and upload it under corresponding name. And when you have filled the picture paths you want(it is not mandatory to fill the path for all accounts), run again the script with appropriate parameter for upload and update.
You can download the script from the link below, for instructions and examples see the help of the script like this Get-Help .\Upload-ProfilePhotos.ps1 -full . I always put some fair amount of help topics in my scripts.

Download From: TechNet Gallery

Scripts to Add,Remove and and view SharePoint 2013 Maintenance Windows

My colleague shared a MSDN Blog post about new class in SharePoint 2013. With this class you can configure message that will appear on the sites informing the users for upcoming maintenance activity, this is done on Content DataBase level. The message can look like the picture below. The messages are predefined for several scenarios you can find samples at the mentioned MSDN Post. You can specify Maintenance Start Time,End Time,Notification Star Time,End Time, link with information about the maintenance and read only period. This is very cool feature, but I don't like the script that is shown in the post, so I wrote two scripts for Adding,Removing and View of the maintenance windows. You can choose individual Content DataBase,Site,Web App or All Content DataBases in the Farm. You can use pipeline to pass objects form other cmdlets, the script will ask you for confirmation before adding or removing the maintenance windows. For more information see the Help in the script there are many example how to use it. To run the scripts you will need Microsoft.SharePoint.PowerShell PS snap-in loaded.

                            Download the scripts form: TechNet Gallery 

Site collection Term Set Groups and how to reconnect Term Set Group to site collection.

When we go to the Term Store manager in SharePoint 2013 Central Administration we can see a bunch of Term Set group with various term sets in it. This term set groups are visible in the entire farm, so far so good.
But we can have a term set group that out of the box will be visible/usable by the the site collection where it is created, this groups are sometimes referred as site collection term set group or local term set group. We can have such term set group if we activate the "SharePoint Server Publishing Infrastructure" site feature or by some manual method like in this Article. As I said this term set group, the term sets and terms in it will be initially visible/usable only for the site collection and this was related to the problem I had this week and of course the solution.
As you have noticed in most of my posts I pretty much use PowerShell and the SharePoint Object Model, so I will show you how this site collection term set group looks in PowerShell and what we can do with it.
For demo purposes in this post I am going to use two HNSC. One based on TeamSite (http://termsetT.auto.l) with no term set group and second (http://termsetP.auto.l) is with publishing features activated, so the second one comes with site collection term set group.
First we will see how to find the site collection term stet group. This can be achieved with below snipped.

---

Add-PSSnapin microsoft.sharepoint.powershell
$site = Get-SPSite http://termsetp.auto.l
$taxonomySession = Get-SPTaxonomySession -Site $site
$termStore = $taxonomySession.TermStores["Managed Metadata Service"]
$pubSiteGroup = $termStore.GetSiteCollectionGroup($site)

---

First we will get an instance of our publishing site. Then we are getting an instance of SPTaxonomySession, this is done with out of the box cmdlet Get-SPTaxonomySession, this object will give us access to the term stores of the web application where our site is deployed or to to the site subscription in multi tenant scenario. In our case this is our Managed Metadata Service application. The taxonomy store has method called GetSiteCollectionGroup,  we are giving  our site as argument and here is the output:

We have two interesting properties to look at. The first one is IsSiteCollectionGroup, it has value True, so this group should be site collection term set group for some site. Second one is SiteCollectionAccessIds, this id is actually telling us the ID of the site collection that can see, use the term sets and edit them. This is the ID of our publishing site. The third one is SiteCollectionReadOnlyAccessUrls, in my case this is the url of my team site that has no site collection term set group, now I will be be able to see the group from my team site, use the terms, but I will not be able to edit the content of the group from the Term Store Management Tool in the Team Site, only read only access. You can actually give read only access from the Term Store Management Tool of the site collection that has full access, when you select the term set group on the bottom you can see section called Site Collection Access, there you can add the URLs of the sites you want to grant read only access.
Last week I receive an email from one of our customers, they needed to do a restore of one of their sites using Restore-SPSite cmdlet and after it their site collection term set group disappeared from the term store and they heavily rely on managed metadata to tag documents and to navigate in huge libraries. They wanted to get back the old group with the old terms because there was a big number of documents tagged and if they create new tags or import them somehow they will be with different IDs, tagged documents will be with invalid tags and navigation and filtering will not work.
The issue here was that they had overwritten the old site with the restore and now the site was with different ID. The old term set group was there, but it was not associated to the site or the correct way here to say is that the new site ID had no access to the old term set group. This can be solved by granting access to the new site ID. I did the same restore operation to my publishing site to reproduce the issue and below is the sniped I used to grant access to the new ID. Well if you do this with publishing site you may receive "error to load the navigation because the term set was not attached correctly...". You can fix this by switching to Managed to Structured navigation and then again to Managed with associating with the correct term set. This however was not the case with the customer and here is the sniped that granted access to the new site ID.

---

$site = Get-SPSite http://termsetp.auto.l
$taxonomySession = Get-SPTaxonomySession -Site $site
$termStore = $taxonomySession.TermStores["Managed Metadata Service"]
$group1 = $termStore.Groups['Site Collection - termsetp.auto.l-1']
$group1.AddSiteCollectionAccess($($site.ID))
$termStore.CommitAll()

---

The issue was clearly solved. However I had a similar case with different customer, the customer claimed that the site collection term set group just disappeared. I was able to get the old group in PowerShell but nothing helped me to "bring it back to life" , so I created new group and instead to try to do something with the group I just moved the term sets that were in it to the new group via PowerShell and everything worked fine. I created the new site collection term set group in PowerShell. Lets look at the method we used to get the group for certain site in the first sniped GetSiteCollectionGroup. If you give another argument that is boolean (True/False), the method will check if there is site collection term set group and if you give True and there is no such group it will create it with the out of the box naming. If somewhere in the term store you have some group with the same name it will put a number at the end of the name. Remember that after doing some changes in the term store you should use CommitAll() method to save the changes in the database.

---

Add-PSSnapin microsoft.sharepoint.powershell
$site = Get-SPSite 'http://termsetT.auto.l'
$taxonomySession = Get-SPTaxonomySession -Site $site
$termStore = $taxonomySession.TermStores["Managed Metadata Service"]
$pubSiteGroup = $termStore.GetSiteCollectionGroup($site, $true)
$termStore.CommitAll()

---