Showing posts with label Script. Show all posts
Showing posts with label Script. Show all posts

Saturday, 19 August 2017

Build TreeView with XML data in PowerShell [Tip]

In one of my recent scripts I worked on, I had to visualize XML data in a tree view manor.
I achieved this by using the TreeView Windows Forms control and I think that the result is good and can be used as an example if you have to do something similar.
I tweaked the function and made it a standalone "XML Browser" script that is visualizing the XML document and if you double click on the element you will copy the Outer XML text to the clipboard.

PowerShell XML Browser

In order to visualize xml you need to supply the path to it and the starting element. On the screenshot above I have a web.config file loaded and the starting element I want to visualize is "configuration". You can find the code and the example below. I hope you find it helpful!


Thursday, 6 April 2017

Nintex Workflow UDA Usage report script

Yesterday I worked with a client that have many Nintex workflow published with heavy usage of UDAs(User Defined Actions). I wanted to get a detailed report on the UDA usage. Unfortunately I am not aware of any  out of the box Nintex tool that can do that. The Analyze button can give you some information, but you need to click on the workflow to find out where it is located, you need to be in the scope where the UDA is published, you can get information for one UDA at a time and the information is not really "exportable".
This is why I created a powershell script that will give you information for the UDA usage across the farm on all levels. It will give you useful information like UDA Name, Workflow Name, Defined At, List, Web, Site, WebApplication, WorkflowType, Author, UDA Version Used, Workflow Id.
There are two "modes" of the script, the default will give you just the GUIDs of the list,web,site and the web apps. If you want to get the name of the list and the URLs you need to use the second mode that will require more time to complete but will give you nice looking URLs instead GUIDs. If you want to get the URLs just use switch parameter GetUrls. The result can be saved in CSV format or it can be outputted in powershell. If you give value for CSVPath the Grid View will open at the end to visualize the data. The main source of information is the Nintex Configuration database and you can use the script with SQL authentication if you have an account with enough permissions and your SQL supports it.
I have tested the script with SharePoint 2016,2013 and 2010 and the oldest Nintex Workflow version I tested was 2.3.7.0.
You can see the code and the output examples below. I hope you find it useful!

Output with URLs retrieved:
Nintex Workflow UDA Usage report with URLs

Quick output with GUIDs:
Nintex Workflow UDA Usage report with GUIDs

Monday, 15 February 2016

Copy List Views in SharePoint and SharePoint Online with PowerShell

In the last couple of weeks I am working with a customer that mainly uses SharePoint Server as DMS(Document Management System). I had to move a large number of documents from one library to another   due to  corruption in some of the files caused by excessive use of unique permissions (~ 32 000).
In this post I will not talk about why you should limit the usage of unique permissions especial in big libraries, it's a long story.
As part of the work I had to copy many list views to the new library. I am not a fan of "Save as Template" approach, my solution was to use a PowerShell script and copy the views programmatically.
The script did its job and I thought that it will be nice to have something like this for SharePoint Online.
I was unable to find any ready script that can do this, so I wrote one.
Both scripts are doing basically the same thing, getting the source and destination webs, getting the source and destination lists, getting the source and destination View collections and create new view in the destination using properties from the source.
It was a bit tricky to load what I need and not making the script extremely slow with CSOM, since we cannot use simple lambda expressions syntax in PowerShell. My solution to this was to use a function written by the SharePoint automation superstar  Gary Lapointe. You can check it out in his article for ItUnity, where he explains how to workaround the limitation in PowerShell concerning the lambda expressions. I highly recommend to read the entire series dedicated on using PowerShell against SharePoint Online.
You can download both scripts (on premises and online) below. Please, Test, Rate and use Q&A section in the Gallery!


 Download On Premises script from: TechNet Gallery

Download SharePoint Online script from: TechNet Gallery

Saturday, 17 October 2015

Capture SQL IO latencies for a period of time - The PowerShell Script

In this post I am going to share a PowerShell script that is not directly related to SharePoint, but can be a powerful tool for troubleshooting SharePoint performance issues.
Earlier this year I published an article called Test and Monitor your SQL storage performance and some good practices for SharePoint, in this article I showed some tools I use to troubleshoot SQL Server storage performance. One of the tools I mentioned there was a SQL script written by Paul Randal from SQLskills.com, it will let you capture the IO latencies for a period of time.
As with my previous post I wanted to transform the SQL script to PowerShell, so it can be used in more scenarios, get the result directly and present the result as objects in PowerShell.
I contacted Paul and he gave me permission to write and publish this script. Note that the original script(this one too) is copyrighted and SQLskills.com have all rights reserved. You can see the original copyright in Paul's post, in my script and in the TechNet Gallery post. Respect it!

Since we've said that, here are a few words about my powershell script. It is really simple, just issuing a T-SQL commands against the SQL instance. You do not have to be on the SQL server as long as you have connectivity and appropriate permissions. The script should work against SQL Server 2005 and newer. You can use Windows integrated authentication, where the identity of the account that is running the script will be used to connect to the sql or you can use SQL Authentication.
The output can be in powershell as System.Data.DataRow type or in CSV file that will be displayed in GridView at the end.
I think that this script is a good example and you can use it as reference to transform some of you SQL scripts to PowerShell. If you look at the original SQL code you will notice that it is one script and in my script I have 5 SQL commands. This is because 'GO' is not T-SQL statement and it will not work directly with the .Net SqlClient. This is why I am executing every batch as separate command.
You can see how the result looks and a download link below. For more information see the help section in the script, ask a question in the Q&A section in Technet Gallery and if you like it Rate it.

SQL Storage IO latencies result



 Download the script from: TechNet Gallery
(You may experience issue with Chrome, use Firefox/IE to open the link)

Tuesday, 11 August 2015

Discover what permissions user has in SharePoint Farm with PowerShell

Permissions and access control is essential part of Information management and governance in a successful SharePoint implementation. As everything in SharePoint, user permissions and access require significant planning in order to prevent future headaches.
However, very often for one reason or another, there is no significant permissions planning, or if there is, it was not implemented correctly or the business users, site administrators and content authors haven't received proper training required to maintain a well organised permission structure. With the introduction of  the "SHARE" button all over SharePoint 2013(most probably this will be valid for SharePoint 2016 as well), it became even easier for the users to break permission inheritance and grant another user or group with "Edit" permission to site,list or item, when only Read access was needed.
Time goes by and you onboard a new customer that has no clear concept for permission management in SharePoint, everything is great the sun is shining and then you receive a query from the customer asking you to give them information on what permissions a user has in their SharePoint. As mentioned the customer has no concept for permission management and they have nice mix of SharePoint groups, AD groups, permission levels, many object with unique permission shared with individual users and so on. You try to figure out what permissions the user has, you dig deeper in the content and eventually end up in below situation.

SharePoint Admin Mind Blown

This is why I did something I had to do a long time ago, I wrote a powershell script that can get all permissions granted to windows user. This includes Farm level, Web Application User Policies, Site level(Owners/Admins),Web,List,Folder and Item. The script is creating a CSV file with all permissions and at the end it is loading the file in GridView for ad-hoc review. See how it looks like below.

Permission Report Gridview

The main goals for the script is to cover as much scenarios as possible. As you can see the script is covering different levels, not only permissions over securable objects. It is working and was tested in SharePoint 2010/PowerShell 2.0 and SharePoint 2013/PowerShell 3.0. It is working with Windows Classic authentication and Windows Claims.You can select the lowest object level that you can scan for permissions, starting with Web Application to Item. The script is showing if the permissions are granted directly or inherited by a Domain group, and if they are inherited it will show you the name of the group.
To achieve this I had to spare a lot of my free time. At the end this became one of the most complex scripts I have ever written and I am able to share. I hit a lot of rocks and did a lot of testing until I decided that it is good enough to be shared. There are many similar scripts, but so far I haven't found any that can cover this many scenarios. This is also a call for everyone that will download and use the script to give me a feedback and if there are any issues in different setups, I highly doubt that the script can break something.
With this post I also want to share some of the interesting PowerShell techniques I have used in the script.
Above I mentioned that the script is not getting permissions from the securable objects only. This are the objects that can be secured with permissions and permission levels, they are SPWeb,SPList and SPItem. Getting the permission information for securable objects is a big part of my script, you can read how to get permission information for securable object in a post by Gary Lapointe. In Gary's post you can find a script very similar to mine, unfortunately I found it when my script was almost ready.
There is one issue with the permission info for securable objects. In a scenario where the user is member of AD group and this group is member of SharePoint group there is no way to find from which AD group the user is inheriting the SharePoint group membership.Also consider a scenario where AD group is granted with permissions in Web Application with User Policy, you will need login/claim of the group to see if there is a policy. This is why my script is getting user's AD group membership, if there is a permission that is coming from SharePoint group the script will get the members of the SharePoint group and will see if the user is member of the SharePoint group or some of its AD groups is member and it will give you AD group name.
Getting AD user/group information is easy when you are on a Domain Controller, it is actually not very hard to do it from any domain joined computer. I have done it by using the Type Accelerator [adsisearcher], you can read about it and how it is working from this article.
Here is another issue I dealt with, the primary domain group is not listed in the attribute memberof. Imagine that the primary domain group is not "Domain User", you will need to know it in case there are some permissions granted in SharePoint. There is a KB that has example how to do it with nasty old VB script :). How to do it in PowerShell is a bit easier. Every AD user has attribute "primarygroupid" that matches exactly with the group object attribute "primarygrouptoken" of the primary group, this ID is also the last digits from the group's SID.
There are some more techniques and workarounds I have used in the script, but I am risking the post to become very lengthy and boring, so download the script to see them.
You can download the script from below link. Please, test,use and rate it in Technet Gallery. For examples and prerequisites see script's Help section!



 Download the script from: TechNet Gallery
(You may experience issue with Chrome, use Firefox/IE to open the link)

Thursday, 2 July 2015

Migrate all SharePoint databases to new SQL Server with minimum clicking and a few words about my new job!

Here we are, a new post after a long pause. I will try to change this in future and blog as much as I can. If you are not visiting my blog for first time I guess you have noticed that the branding is slightly different. The change is mainly in the bottom right where is located the logo of my current employer.
At the moment I am writing this post I am in my fourth week as Senior SharePoint Engineer for bluesource.I think that I made a good decision and this change is a step forward, something that I needed and wanted.
Bluesource is a great company, not very big(at the moment), but currently present on three continents Europe, Australia and North America. It has wide portfolio of technologies and services.As a Senior SharePoint Engineer I will be the main person involved in the ongoing SharePoint support of our customers and our internal SharePoint infrastructure.
I am working with many customers and this is a great opportunity to see different solutions, implementations, requirements and requests. I will support not just SharePoint 2013, but 2010 and Online. A lot of the customers are looking at fast,cheap and supportable solutions, this is why they are heavily using Nintex Workflow, Nintex Forms, InfoPath and more. I am especially excited about Nintex Workflow, this is the perfect tool/platform for developing  no-code, robust workflows for SharePoint.
This is why you can expect more diverse post for interesting and useful(I hope) real-world examples from my day-to-day job. 
One of my first tasks was to plan and execute the migration of our public/extranet SharePoint 2010 farm to new SQL Server instance as part of internal infrastructure restructuring.
As I mentioned this farm is hosting our public and extranet sites, so no downtime was allowed, this required the migration method to be database backup/restore instead of detach/attach. There are many resources how to do this, one of the best overviews of the process can be found in the posts of Thuan Soldier and Todd Klindt.
To be exact, stopping all SharePoint services is mentioned in both posts, this means complete outage, I did it only with complete content management change freeze and stopped Timer Services in order to prevent any timer job execution, and components that can write in the databases or in the local configuration cache. This is not the best way, but I and my stakeholders took the risk to do it that way and the migration went smoothly without content outage and issues.
To minimize the risk I had to do the backup and restore as fast as possible. I had to do backup/restore for up to 40 databases with combined size of 70 GB. Database backup and restore is a click intensive, repeatable operation and people tend to do mistakes when a lot of clicking and repeating is involved, computers can do better repeatable,boring things.
My solution to automate this process was a great powershell script from Chrissy LeMaire  (PowerShell MVP). I just cannot express how good that script is! Although I haven't tested it in all possible scenarios that can be used in, I think it will work great just how it worked in my migration.
The script is called Start-SqlServerMigration and you can check it out and download below.
Here are some of the features: support detach/attach and backup/restore for db migration, can use Windows and SQL authentication, migrate Windows and SQL logins(with correct SID), supports old SQL Server versions, migrate jobs and SQL server objects, sets the initial source DB owner, export/migrate SQL global configuration and many more. See some screenshots of backup/restore migration in my dev. environment.



A video from Chrissy:



                             Download the script from: TechNet Gallery

Monday, 11 May 2015

Write and Get User Profile Properties in SharePoint Online with CSOM in PowerShell. The Scripts!

Last year (1st December) I published an article called Write and Get User Profile Properties in SharePoint Online with CSOM in PowerShell . In this article I demonstrated the latest capabilities that were introduced with the latest CSOM SDK. There were new APIs that made possible to work with User Profile properties in SharePoint Online(for now). I published PowerShell snippets for writing user profile properties and retrieving them. At that time I was unable to figure out how to write MultiValue properties, the snippets were just for the demo, with hard coded property and they were not very reusable.
Since this article became very popular and people were happy that they can do this from PowerShell, I decided to write generalized script for writing any Single or Multi value user profile property.
The Name(not the DisplayName) of the property to be edited is provided as script parameter. As I said the script is generalized, not tested with all available out of the box properties in SharePoint Online. Some of the properties have specific requirement for the value. For example the About Me property should be html(<br> instead new line), the Birthday should be some parsable datetime format and so on. If the format is not correct in most cases you will receive some meaningful exception from CSOM.
The script I will provide requires SharePoint Server 2013 Client Components SDK(x64) or SharePoint Online Client Components SDK(x64) that are released September 2014 or newer.
The script is written like native cmdlet, it accepts pipeline values for the AccountName, you can combine it with a CSV or simple text file that contains a list with account names, include it in your own scripts and modify it as you find for useful. Test the script to account that is not C-level manager :-) and if it works use it for bulk property updates for "live" accounts for example. 

To demonstrate how the script for writing user profile properties is working I am going to change the profile of  Anne Wallace (AnneW@spyankulov.onmicrosoft.com). She is the President of Contoso. I am going to change her Work Phone(Single Value), her Skills(Multi Value) and her Birthday(short date Single Value). See how her profile page looks like before changes.


Below are the PowerShell lines I have used to change this properties, I have made it in couple of lines in order to fit it better in the blog. Fore more examples see the Help section of the scripts.


########################## Change Work Phone ###############
.\Set-SPOUserProfileProperty.ps1 -PropertyName WorkPhone `
-AccountName 'AnneW@spyankulov.onmicrosoft.com' -Value "+44 555 555 55" `
-SPOAdminPortalUrl 'https://spyankulov-admin.sharepoint.com' `
-UserName ****@spyankulov.onmicrosoft.com -Password *******

########################## Change The Skills ###############

.\Set-SPOUserProfileProperty.ps1 -PropertyName 'SPS-Skills' `
-AccountName 'AnneW@spyankulov.onmicrosoft.com' -Value "SharePoint","PowerShell","Tech Support"-SPOAdminPortalUrl 'https://spyankulov-admin.sharepoint.com' `
-UserName ****@spyankulov.onmicrosoft.com -Password ******* -MultiValue
######################### Change The Birthday ###############                                         
.\Set-SPOUserProfileProperty.ps1 -PropertyName 'SPS-Birthday' `
-AccountName 'AnneW@spyankulov.onmicrosoft.com' -Value "5/9/1988" `
-SPOAdminPortalUrl 'https://spyankulov-admin.sharepoint.com' `
-UserName ****@spyankulov.onmicrosoft.com -Password *****


Lets see how Anne's profile looks like below, after the change.



We can see that the properties are changed accordingly. How cool is that :).

In the package you can download below I also included a script that can retrieve the user profile properties in PowerShell. Please, download the script,test,use and rate!
I am writing a fair amount of Help to my scripts, please read it.

 Download the script from: TechNet Gallery
(You may experience issue with Chrome, use Firefox/IE to open the link)

Thursday, 9 April 2015

Script to get All Webs in SharePoint Online site collection

Do you ever wanted to have a SharePoint Online cmdlet to get all webs in a site collection? Something like this can be useful if you want to see all webs and if there is something that you do not want or need. Or if you want to get all webs as original type and do further operations with them. Well as you know there is no such out of the box cmdlet in SharePoint Online Management Shell.
We however can use the dlls from SharePoint Server 2013 Client Components SDK and write our own powershell script to do that.
There however are many scripts from the community that are described as "get all webs in site collection". One of the most popular scripts that is popping out if you do some googling, is the script of Corey Roth (MVP). The script is nice but actually getting all webs in a SharePoint site collection with CSOM is a bit tricky, since we do not have (at least I do not know) a shortcut for getting really all webs, something like SPSite.AllWebs when you run powershell in on-premises.
What Cory's script is doing is to get the top level web from the client context, then get its sub webs and then it is returning the Title and the Url of the subwebs like I have shown in the screenshot below.


As you can see we are not getting much. The site I am doing this demo on is a site with two subwebs, below them we have some more subwebs, below them more and we cannot see them.
So I create a script that can output all webs in SharePoint Online site collection very similar to what SPSite.AllWebs is doing in on-premises and the output is Microsoft.SharePoint.Client.Web. See how it looks like.

All SharePoint Online Webs

As you can see the output is different. I have quite a lot of webs in my site, there is even an AppWeb.
See part of my code.


BEGIN{
function Get-SPOSubWebs{
        Param(
        [Microsoft.SharePoint.Client.ClientContext]$Context,
        [Microsoft.SharePoint.Client.Web]$RootWeb
        )


        $Webs = $RootWeb.Webs
        $Context.Load($Webs)
        $Context.ExecuteQuery()

        ForEach ($sWeb in $Webs)
        {
            Write-Output $sWeb
            Get-SPOSubWebs -RootWeb $sWeb -Context $Context
        }
    }
    Add-Type -Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.dll" | Out-Null
    Add-Type -Path "C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\Microsoft.SharePoint.Client.Runtime.dll" | Out-Null
}
PROCESS{

    $securePassword = ConvertTo-SecureString $PassWord -AsPlainText -Force
    $spoCred = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $securePassword)
    $ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteUrl)
    $ctx.Credentials = $spoCred

    $Web = $ctx.Web
    $ctx.Load($Web)
    $ctx.ExecuteQuery()

    Write-Output $Web

    Get-SPOSubWebs -RootWeb $Web -Context $ctx
}



All I have done is to get the root web, return it as result, get the subwebs, then get the subwebs of each subweb that is found (and so on) and then return each web as result. The key job is done by a function defined in BEGIN blok (not very pretty, but works) Get-SPOSubWebs . The function is running itself for each web found, I think that this can be defined as recursion.
At the end we have all webs in the site collection. In order to run the script you will need SharePoint Server 2013 Client Components SDK (x64) installed and optionally if you want to see the data in powershell console loaded type display template(included with the script). For more information see my article Display CSOM objects in PowerShell.

 Download the script from: Technet Gallery
(You may experien issue with Chrome, use Firefox/IE to open the link)

Monday, 16 February 2015

SharePoint Weekly Backup cycle script.

In this post I am going to share a script for SharePoint backup I wrote for one of our customers.
Since we had no budget for a 3rd party backup solution I decided to use the native SharePoint backup capabilities in a script.
For me the SharePoint Farm backup is a good solution if you do not have any advanced 3rd party tools like DocAve for example or your Farm is not very big. Amongst DPM,SQL backups, File System backups the SharePoint farm backup is the best solution in case of disaster and you have to restore the entire Farm. You can see the following article for details on the different methods.
The drawback with the SharePoint Farm backup is that it is not very granular and it can be a bit slow in large environments. For example in Web Applications the lowest level object you can restore is Content Database. However, you can restore a content database and then use Recover data from an unattached content database and restore site, web or list. Still, you will not be able to directly restore an item for example.
Be aware that if you try to restore a content database from Farm backup to a live Farm you may receive error similar to the one below.

Restore failed for Object 2K8R2_SP_Content_Portal_Bla (previous name: 2K8R2_SP_Content_Portal) failed in event OnPostRestore. For more information, see the spbackup.log or sprestore.log file located in the backup directory.

And when you look at the restore log you will find something like this:

FatalError: Object 2K8R2_SP_Content_Portal_Bla (previous name: 2K8R2_SP_Content_Portal) failed in event OnPostRestore. For more information, see the spbackup.log or sprestore.log file located in the backup directory. SPException: Cannot attach database to Web application. Use the command line tool or Central Administration pages to attach the database manually to the proper Web Application.

In this case I am restoring the old content database 2K8R2_SP_Content_Portal to a database with name 2K8R2_SP_Content_Portal_Bla. However SharePoint will try to attach the restored content database to the Web Application and it will fail because we already have a content database with that ID. However the restored database is completely okay and you can see it in the SQL Server Management Studio.
The customer required to have a backup once a day and to keep the backups for one month. We were also disk space constrained.
There are many SharePoint backup scripts, but I wanted to have a different folder for every week and also to have one full and the rest of the backups to be differential in order to save some free space. Additionally you can enable SQL Server backup compression to save more disk space. 
The script I wrote is doing this, also it can backup only the configuration.
The script is doing a 7 day cycles, when a new cycle starts it will create a new folder that resembles the start, end and the type of the backup. For example if you start a full backup cycle on 14.02.2015 the script will create a folder with name 14022015_20022015_F (Dates are in EU standard ddMMyyyy). Since it is new cycle the script will start with Full farm backup. Then all the backups until 20.02.2015 will be differential. Here how it looks one weekly cycle.

SharePoint backup cycle


The script will also do a clean up. You can specify how many cycles you want to maintain and it will keep them and the old backups will be deleted. Do no combine this script with full database backups because it will break full-differential sequence.  


 Download the script from: Technet Gallery
(You may experien issue with Chrome, use Firefox/IE to open the link)

Monday, 27 October 2014

SPDeletedSite and SharePoint Deleted Site Collections alert script

Couple of weeks ago I received a question from one of our customers if there is a way to get alert when a site collection from their SharePoint 2013 farm is deleted. This was kind of important for them because the power to create and delete site collections is delegated to project owners, team leaders etc. The customer often receives a requests for restoring a site that was deleted by mistake or they suddenly notice that a site with important information just disappears(deleted by the owner). The fastest way to restore such deleted site is to use the command Restore-SPDeletedSite, but often the request for restore comes after 30 days and the sites are permanently deleted.
As far as I know, there is no OOTB feature that can notify for deleted site collections, so my proposal to the customer was to write a PowerShell script that can track the deleted site collections and send notifications. The script is fairly simple it is just getting the current deleted sites with the command Get-SPDeletedSite -WebApplication <WebAppUrl> and it compare the result with the result from the last run and if there are new deleted sites it will send a mail with details like below one:

Alert for Deleted SharePoint site collection

And since the key point in this script is the retrieval of SPDeletedSite I would like to explain a bit more for this feature and and share the history of it.
I guess that every SharePoint administrator knows about this feature in SharePoint 2013 and a big part of us are so happy that we have it.
The story of it starts with the release of the SharePoint 2010 with a capability called "Gradual Site Delete". This capability was designed in order to mitigate performance degradation or even service interruption caused by so called Lock-escalation that may happen in the content database if we attempt to delete very large site collection. This was an issue in WSS 3.0 and Microsoft SharePoint Server 2007(MOSS 2007).
The short story around "Gradual Site Delete" is that when a user deletes site collection, the site collection is not instantly deleted, it is marked for deletion and the content becomes unavailable. Then for the actual deletion is responsible a timer job definition called "Gradual Site Delete" that is executed on daily schedule by default(configurable).

Gradual Site Delete Timer Job

This timer job deletes the site collection from the content database in small enough portions to prevent lock escalation. If you want to read more on the subject, please check the post from Bill Baer.

Then with SharePoint 2010 SP1 we received 3 additional cmdlets that are allowing us to work with site collections that are "marked" for deletion, but not deleted yet. They are the same as in SharePoint 2013 - Get-SPDeletedSite, Restore-SPDeletedSite, Remove-SPDeletedSite and Move-SPDeletedSite (Only in SharePoint 2013).
And now with the cmdlet Restore-SPDeletedSite we can restore site collection that was deleted by the end user.
As I said eventually this 'deleted' sites should be permanently deleted by the "Gradual Site Delete" timer job. The exact time how many days the site collection should be kept in "Gradual Site Delete" phase as SPDeletedSite object (marked for deletion) before permanent deletion, depends on the time settings of your site collection recycle bin that is set on Web Application level.

SharePoint 2013 Recycle Bin Settings

By default the Recycle Bin is enabled and the retention period for the deleted items is 30 days. This means that if a site collection is deleted it will stay as "restorable" SPDeletedSite object for 30 days.
Be aware that when you delete site collection in powershell with the command Remove-SPSite, you need additionally to specify parameter  -GradualDelete in order to use the Gradual Site Delete. 
If this parameter is not used the site collection will be instantly deleted, it is strongly recommended to use Gradual Site Delete for large site collections.

Download the script from: Technet Gallery

Friday, 10 October 2014

Create ZIP Archive in PowerShell and SharePoint IIS and ULS archiving scripts

In my career as Windows Web hosting administrator and now as dedicated SharePoint Administrator I have always needed a way to clean up the IIS logs since the IIS does not provide such functionality.
I guess that this is an issue for everyone involved in some sort of application management that is hosted on IIS.
Most of us leave Log Rollover settings to Schedule, Daily and we end up with bunch of files in our log directory. The size of the logs depends on how many hits the application is receiving. If the application is some public website the logs can become quite big and the volume where the logs are stored  can run out of space (in any case you do not want the IIS logs stored on System drive or on drive with application files).
So, the most recent experience I had was with one of our customers that has their public websites hosted on SharePoint 2013. The sites are quite busy and we have huge 1GB logs almost everyday and with NTFS compression the files are still around 300MB size on the disk.
Since it is a good practice to keep this log files as long as we can, we can have two options.
Option one is to zip(or use our archiving tool of choice) all log files,put it away and delete the original file in order to save some space and option two to write or find a script to do this for us.
I found many scripts for archiving IIS logs. Most of them are using 7-zip to do the zip archives or the cmdlet Write-Zip that comes from the PowerShell Community Extensions. Both tools are using some sort of a 3rd party compiled code that I do not want to run on the servers that are hosting the public sites of our high profile customer.
At the moment we don't have version of PowerShell that has native cmdlets for working with .zip files. The new PowerShell version 5 will have native cmdlets that can create and extract zip files, but it is still in Preview version and it is not recommend for production environments. If you want to check it out there is a preview of the Windows Management Framework V5 available for download.
So, I needed to write my own script to do the work without using any 3rd party tools.
With some research I found some classes that are added to the System.IO.Compression Namespace in .NET 4.5 that might do the work. The .NET Framework 4.5 is a prerequisite for every SharePoint 2013 so I do not need to install anything additional.
Below you can see the function that I have used in the IIS Log Archiving script.


Function New-LogArch{
    [CmdletBinding()]
    Param (
        [parameter(Mandatory=$true)]
        [System.IO.FileInfo]$file,
        [parameter(Mandatory=$true)]
        [string]$Location
    )
    PROCESS{
        Add-Type -Assembly System.IO.Compression.FileSystem
        $tmpFolder = $Location + '\' + $File.BaseName
        New-Item -ItemType directory -Path $tmpFolder | Out-Null
        $destfile = $Location + "\" + $File.BaseName + ".zip"
        $logPath = $file.FullName
        $zippedName = $file.Name
        $compressionLevel = [System.IO.Compression.CompressionLevel]::Optimal
        [System.IO.Compression.ZipFile]::CreateFromDirectory($tmpFolder,$destfile,$compressionLevel, $false )
        $archive = [System.IO.Compression.ZipFile]::Open($destfile, [System.IO.Compression.ZipArchiveMode]::Update )
        [System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile($archive, $logPath, $zippedName, $compressionLevel) | Out-Null
        $archive.Dispose()
        Remove-Item $tmpFolder -Force

    }
}


The function takes two parameters, the information for the file where it can get the literal path to the log file and the location where the archive will be saved.
The first thing we do with the System.IO.Compression classes is to define the compression level in the variable $compressionLevel, I am setting Optimal in order to get smallest file size possible.
Next thing is to create an empty zip archive from a temporary, empty directory I have created earlier. I was unable to find how to create the archive directly from the log file.
When we have the zip file created with the correct Location,Name and Compression Level I "open" it in the variable $archive.
Once we have the archive file opened we use the method CreateEntryFromFile in class System.IO.Compression.ZipFileExtensions, with this method we can add a file to zip archive and we add the original log file to the archive.
Finally we invoke Dispose() method on the $archive, because when we open the zip archive with Update argument the file is locked.
Now when I have the key component that will let me zip the log files I created the IIS log Archive and Cleanup script.
It will find the log location of all IIS sites, will check when they are modified and if they are older than the retention period that you have defined as parameter it will archive the log in the destination you have specified and will delete the original file. You will have a zip file with the name of the original log file that contains only one log file. The script will also check for old archive files and if they are older than the value you have specified it will delete the zip file.
I also wrote a script that can archive the SharePoint ULS logs, because it is a good practice to have them in case you need to do performance analysis and troubleshooting for example.



Download "IIS Log Archive and CleanUp Script" from: Technet Gallery

Download "ULS Log Archive Script" from: Technet Gallery

Monday, 25 August 2014

Bulk Upload and Update User Profile Photos in SharePoint 2013

This post will be about a script I wrote and some interesting things I learned to write it. This script lets you upload and update multiple user profile photos for multiple user profiles in SharePoint 2013. There are many post on how we can rewrite the PictureURL property in the user profile object and assign it a custom value, but my script does not work that way. This script is emulating on what is happening in one popular out of the box scenario for importing user profile pictures, from AD.
The inspiration for this script came from an issue (of course) that one of my colleague have with one of our customers. The customer has some complex domain structure. They have around 16 domains and even federation. So the customer of course wants their SharePoint users to be able to have profile photos in their SharePoint profiles, so we have User Profile Synchronization in place and it is synchronizing the Picture user property from the AD attribute thumbnailPhoto. The users however can edit their profile picture and they upload new photos.
The issue is that since some CU installation this year, I am not sure, but I think it was April 2014, when a full synchronization is launched in the User Profile Application, the pictures that are uploaded by the users are lost. So my colleague ran the synchronization first on Staging environment that is almost identical to the Production, it has similar configuration of the User Profile Service Application and of course we lost the pictures that the user have uploaded. The Production environment however was not touched, but soon or later we had to apply some property mapping change and run full synchronization on Prod. too. We needed a way to get the profile pictures from Prod. reupload them to Staging for the correct profiles, then run full synchronization on Prod. and reupload again the user profile pictures.
And here I come with my powershell skills and some free time. As I said above I came across many scripts for editing the PictureURL property of the UserProfile object, but this option was not acceptable for me because in SharePoint we have 3 sizes of user profile photo thumbnails.
I first created  the script to download the biggest available picture thumbnail and save them under unique name that can later help me to determine to which account the picture belongs. I did this by getting the values of the PictureURL and the LoginName of the profiles and then simply download the largest thumbnail of the photo. You can get all the profiles in a User Profile Application with the snipped below. You will need a site collection URL to determine the service context and run it under Farm account. Of course the things are a bit different in scenario with multiple User Profile Applications or if the application is Partitioned(for multi tenant deployment).

$SiteURL = 'http://mysite.contoso.com/'
Add-PSSnapin Microsoft.SharePoint.PowerShell
$Site = Get-SPSite -Identity $SiteURL
$context = Get-SPServiceContext -Site $site
$upm =  New-Object Microsoft.Office.Server.UserProfiles.UserProfileManager($context)
$AllProfiles = $upm.GetEnumerator()
ForEach($profile in $AllProfiles)
{
$profile
}

And now I had all the profile pictures from Prod. and we had to figure out  how to upload them on Staging and "connect" them to the correct account.
And here comes the interesting part. So normally the pictures for the user profiles are imported from the thumbnailPhoto, but we have a URL to picture for value of the PictureURL property and the thumbnailPhoto, contain just a picture as binary. As we know after synchronization we need to run the out of the box command Update-SPProfilePhotoStore. 
So I found what is happening between the import and thumbnail generation.
When the import/sync from the AD happens the thumbnailPhoto value is saved as .jpg image in the same picture library with profile photos that also contains the thumbnail versions of the profile pictures. For example if you have My Site host http://mysite.contoso.com, the URL of the library in SharePoint 2013(I forgot to mension that we are working on SP2013 farm) will be http://mysite.contoso.com/User Photos/ and the pictures are in folder Profile Pictures. The thumbnails are named Account_LThump.jpg for large, Account_MThump.jpg, for midsize and Account_SThump.jpg for small. The midsize picture is used as profile photo.
So the initial picture are also saved in this library but under a special name like this 0c37852b-34d0-418e-91c6-2ac25af4be5b_RecordID.jpg. Here is the moment to say that we have only one, nonpartitioned User Profile Service Application. The first part if the picture name the guid (exactly this guid) is the guid of the default partition of nonpartitioned UPSA. It is the same on all SharePoint 2013 farms I have seen. You can check it by doing SQL query on the Profile DB, but only on non-production Farm because doing query on this DB is not supported. It is a column of every profile entry in the Profile DB.


The second part, the RecordID is a unique ID for every user in the partition. It is just a number not a guid. You can also see it in the picture above, you can also get it with powershell as property of the user profile object. But, so far I haven't found a way to see the PartitionID of the default partition of nonpartitioned UPSA in PowerShell.
And when you run Update-SPProfilePhotoStore with the corresponding parameters, it reads the available not converted to thumbnails .JPGs read the users RecordID, creates the thumbnails, add the url of the picture in the user profile and in the general case delete the original(depends on the parameters).
So this was our answer on how to upload the pictures from Prod to Staging and get all three thumbnails variants and map the PictureURL property. One additional feature to this method is that for PictureURL is mapped the url of the biggest thumbnail version(not the midsized as if it was imported from AD) and you will have pretty profile pictures with good resolution.
Here came the idea for this script. What if there are SharePoint Admins/Customers that do not want to get the pictures from the AD, but they have all the profile pictures and want to mass upload them as if the users have uploaded themselfs.
And I came up with this script. You need to run it two times(As Administrator under the Farm account) first time will be generated a CSV file with the RecordIDs,LoginNames and picture paths. The picture path will be empty you should fill it with the local or the network path to the picture. The picture can be whatever resolution and almost any format (JPG,PNG,GIF,BPM,TIFF), the script will convert it to jpeg and upload it under corresponding name. And when you have filled the picture paths you want(it is not mandatory to fill the path for all accounts), run again the script with appropriate parameter for upload and update.
You can download the script from the link below, for instructions and examples see the help of the script like this Get-Help .\Upload-ProfilePhotos.ps1 -full . I always put some fair amount of help topics in my scripts.

Download From: TechNet Gallery

Thursday, 14 August 2014

Scripts to Add,Remove and and view SharePoint 2013 Maintenance Windows

My colleague shared a MSDN Blog post about new class in SharePoint 2013. With this class you can configure message that will appear on the sites informing the users for upcoming maintenance activity, this is done on Content DataBase level. The message can look like the picture below. The messages are predefined for several scenarios you can find samples at the mentioned MSDN Post. You can specify Maintenance Start Time,End Time,Notification Star Time,End Time, link with information about the maintenance and read only period. This is very cool feature, but I don't like the script that is shown in the post, so I wrote two scripts for Adding,Removing and View of the maintenance windows. You can choose individual Content DataBase,Site,Web App or All Content DataBases in the Farm. You can use pipeline to pass objects form other cmdlets, the script will ask you for confirmation before adding or removing the maintenance windows. For more information see the Help in the script there are many example how to use it. To run the scripts you will need Microsoft.SharePoint.PowerShell PS snap-in loaded.

                            Download the scripts form: TechNet Gallery 



Sunday, 27 July 2014

SharePoint 2013 Service Account creation script and how I write scripts.

In this post I am going to share a PowerShell script that I wrote called Create-SPServiceAccounts.ps1 for creating the accounts needed for a SharePoint Server 2013 installation. There are many similar scripts, but I don't like how they are written. Also I have added an additional options in my script for Input file with the data for the accounts (account name,description and password), it can read CSV,XML and Excel files for input. I have also added the ability accounts to be created remotely by running the script from computer in the domain or not. I think that the script is well written and have some useful patterns and techniques for writing a readable, reusable PowerShell scripts. In this post I will do an overview how I am writing scripts and will try to explain some of the techniques I used in the Service Account creation script.

What tool  I am using for writing scripts?

Most of the time I am using a free third party ISE (Integrated Scripting Environment) tool called PowerShell Plus Professional by Idera . It has some great feature like code highlighting and it is easy to run and debug scripts. A great feature is that you can copy the code as HTML. I find this very useful because I don't use any code highlighters in this blog, instead I just copy my code from the tool and paste it in the HTML source of the posts. The tool comes with big script library for Windows,AD,SharePoint,SQL and many more. But it has some limitations and I also use the native PowerShell ISE and of course the powershell console to run and debug.

How I am reading three different types of Input files?

A bit of a background. One of the best practices for PowerShell scripting is modularity. Every piece of code with different  functionality that will be reused is organised in functions with own parameters and scope. The code from this script that do initial checks and logic is only 25 rows, but the functions are 222 rows.
So I have three different functions for reading different type of Input files. The script will check what is the file extension and will decide what function to use for reading the input file.

The function for CSV:

function Read-CSV{
 [CmdletBinding()]
Param (
    [parameter(Mandatory=$true)]
    [string]$Path,
    [parameter(Mandatory=$false)]
    [string]$DefaultPass
)
$csvImput = Import-Csv -Path $Path
$OutputAll = @()
ForEach($row in $csvImput)
{
    $Output = @{
        "AccountName" = $row.AccountName
        "Description" = $row.Description
        "Password" = $row.Password
    }
    If($DefaultPass){
        $Output['Password'] = $DefaultPass
    }
    If (Test-AccountData -Data $Output){
        $OutputAll += $Output
    }

}
Write-Output $OutputAll
}

The key for reading csv files is the native cmdlet Import-Csv. It will read the file and will create an object for each row(without the column names) in the csv with properties named as the csv columns and corresponding value. Then I am building a hash table for each object and the function returns a collection of hash tables for every account that needs to be created.

The function for XML:

function Read-XML{
 [CmdletBinding()]
Param (
    [parameter(Mandatory=$true)]
    [string]$Path,
    [parameter(Mandatory=$false)]
    [string]$DefaultPass
)
[xml]$xmlInput = Get-Content $Path
$OutputAll = @()
ForEach ($xElement in ($xmlInput.ServiceAccounts.Account))
{
    $Output = @{
        "AccountName" = $xElement.AccountName
        "Description" = $xElement.Description
        "Password" = $xElement.Password
    }
    If($DefaultPass){
        $Output['Password'] = $DefaultPass
    }
    If (Test-AccountData -Data $Output){
        $OutputAll += $Output
    }

}
Write-Output $OutputAll
}

For reading XML I am creating new XmlDocument object by reading the content of the xml file. Then we can work with every xmlElement that describes an account and take it's properties. Again as in the other functions for every account we are creating hash table and when we read all the information the function will return us a collection of hash tables.

The function for Excel files:

function Read-Excel{
 [CmdletBinding()]
Param (
    [parameter(Mandatory=$true)]
    [string]$Path,
    [parameter(Mandatory=$false)]
    [string]$DefaultPass
)
$objExcel = New-Object -ComObject Excel.Application
$objExcel.Visible = $false
$WorkBook = $objExcel.Workbooks.Open($Path)
$WorkSheet = $WorkBook.sheets | Where {$_.Index -eq '1'}
$intRowMax = ($WorkSheet.UsedRange.Rows).count
$OutputAll = @()
for($intRow = 2 ; $intRow -le $intRowMax ; $intRow++)
{
    $Output = @{
        "AccountName" = $WorkSheet.Range("A$($intRow)").Text
        "Description" = $WorkSheet.Range("B$($intRow)").Text
        "Password" = $WorkSheet.Range("C$($intRow )").Text
    }
    If($DefaultPass){
        $Output['Password'] = $DefaultPass
    }
    If (Test-AccountData -Data $Output){
        $OutputAll += $Output
    }
}
Write-Output $OutputAll
$objExcel.Quit()
(Get-Process -name excel -ErrorAction SilentlyContinue | Sort-Object StartTime)[-1] | Stop-Process -ErrorAction SilentlyContinue
Remove-Variable objExcel
}

For reading Excel file(.xlsx) we will need Excel office application installed. The script is creating a COM Excel application instance, opening the file, choosing the first workbook. Then unfortunately we need to read the content of each cell, as far as I know there is no way to treat every row as an object or something similar so we can take its content like in the CSV. So we are taking the count of the rows, skipping the first and we read every cell and creating a hash table. This is done in For loop where it depend what is the number of the row we are reading. And again the function will return a collection of hash table. All function will output the same object(s), so after the processing the Input file we will not care what type it was we will just process every hash table in our collection.
For the record, killing the Excel.exe process is not the correct way of removing COM object instance, but I had some weird behavior when I used below code to get rid of the COM instance.

[System.Runtime.Interopservices.Marshal]::ReleaseComObject($x)

All three functions are very similar, they have similar parameters and outputs, so we we will not care for input/output when we read different input file.
For more information on PowerShell function you can read about_Functions, about_Functions_Advanced , about_Functions_Advanced and all about files from the PowerShell Help.

Why so many Try/Catch constructions?

Try/Catch/Finally constructions are preferred way for error handling in PowerShell. This statements were introduced in v2.0, before that there was only available Trap statement. The error handling is big subject in Powershell, I will try to explain how Try/Catch/Finally works in example for creating new AD user.


Try{
    New-ADUser -SamAccountName "JohnD1111111111111111111" -Name "John Doe" -AccountPassword $secPass -DisplayName "John Doe" -ErrorAction Stop
}
Catch [System.Exception]{
    Write-Host "Unable to create Account, folowing exception occurred: $($_.Exception.Message)" -ForegroundColor Red
}

In this sample we will run the block in Try statement. If there is an error/exception the block in the 'Catch' statement will be executed. I have not included 'Finally' statement, but if there was its code will be executed anyway no matter if we have exception or not and it is not required. In our example the New-ADUser will fail because the value for SamAccountName is with length more than 20 symbols. However the 'Catch' block will be triggered only if the error produced in the 'Try'  is Terminating. Most of the PowerShell commandlets produce non-Terminating errors, that means that if the command fails the script/loop/pipeline will continue.Our cmdlet for creating new AD user also produces non-Terminating error, this is why we have explicitly set ErrorAction to Stop, this will make the error Terminating, The whole idea of error handling is to identify if we have error and do something. In this sample we will not see the error that the cmdlet produces, instead we will execute Catch and it will show us customer message that contains the message of the original exception and will tell us what is wrong, without seeing the entire exception from PowerShell. And because the error is in Try/Catch construction the whole script will not stop and will continue.

How locally defined function is executed on remote computer?

You can launch the account creation script from the DC that you want to create the accounts in or you can run it from completely different computer and create the accounts in the target DC remotely.
This is achieved very easy, I am just executing the function that creates the accounts locally on our remote DC and also remoting the parameters that I will use. The line looks like this:

Invoke-Command -Session $session -ScriptBlock ${function:Create-UsersLocal} -ArgumentList $Hash,$OUnit

Our function for creating accounts when the script run on the DC is Create-UsersLocal, I am just promoting it via the ScriptBlock and its parameter values via the ArgumentList.
And now explained. Let say that we have a functions called Hello-World, a very simple function that just say hello from the computer it runs in different color. The color of the text can be set as parameter. If we load the function it goes to a PSDrive called 'Function'. We can see on the screenshot below that this PSDrive contains our function. If we supply just the path to the function it will execute it on the remote computer as it is and we will fail if the function is not defined in the remote session. Our goal is to get the definition of the FunctionInfo object, this is done by adding dollar sign when supplying the script block for the remote session, this will force reading the definition of the function and convert it to useful script block for the remote session we are also transferring a variable with the same name as the parameter of the function and this is how the magic happens. You can see on the screenshot below.



For more information on PowerShell scripting I highly recommend to visit http://powershell.org/ especialy their Free eBooks section.

You can download my script from the    Technet Gallery