When to perform a Search full crawl in SharePoint 2013?

Search is indeed a mission critical component in SharePoint 2013 and it’s very important that it functions properly so that you get the desired results. As we all know, the search results and their relevancy is directly proportional to how often your content sources are crawled and what sort of crawling you’re running in your SharePoint farm (i.e. full crawl, incremental crawl and continuous crawl). So, in this post I’m not going to discuss about the different type of search crawls or the SharePoint 2013 search architecture, perhaps I would be discussing on when and under what circumstances should a SharePoint administrator perform a full search crawl. The reason for me picking up this topic is because I see a lot of misconception among SharePoint administrators in understanding when the Search full crawl has to be performed. For the most part, I’ve seen many folks turning on full crawl when it’s not required at all and before doing so we need to understand that turning on Search full crawl is going to consume a lot of your server’s resource and at worst case it could even make your SharePoint farm go to an unresponsive state and hence it’s very important that we do this only when it’s required.

Alright, let’s get into the details ….

Listed below are the reasons why and under what circumstances should a SharePoint farm Administrator perform a full search crawl:

1.You just created a new Search Service application and the default content source (i.e. Local SharePoint sites) that gets created along with the newly created Search service application hasn’t been crawled yet.

2. You recently added a new content source and it hasn’t been crawled yet (Note: This is applicable for all the types of content sources (i.e. Local SharePoint sites, File shares, Exchange public folders and External line of business data)

3.When there has been, a change made to the existing content source (meaning, when you’re trying to edit the existing content source for making some changes)

4.When you’re patching your SharePoint 2013 farm by installing a Cumulative update, Service packs and hot-fixes etc. For some reason I see a lot dilemma on this specific point because it brings up a question on why should a full crawl be performed post the patching .The reason for this is really simple , if you read my article on patching a SharePoint farm you would notice that I’ve mentioned a step where you need to suspend the search crawl before patching your farm and the reason for mentioning that is because it’s quite possible that when you check the crawling schedule before patching you farm there may not be any instance of crawl running. However, if a crawl is triggered by schedule which occurs during the installation, the search application may crash or lead to inadvertent results. In worst case, you might end up rebuilding the entire search application. Hence, as a best practice it’s very important that you suspend the search service application before patching your farm and once you’re done with patching your farm please go ahead and resume it and run a full crawl.

5.When changes have been made to managed properties in search. A full crawl of all affected content sources is required for the new or changed managed property to take effect.

6.If you want to detect security changes that were made to local groups on a file share after the last full crawl of the file share

7.When the incremental crawl keeps failing continuously. If an incremental crawl fails many consecutive times for any content, the system removes the affected content from the search index. In such case, please look into the search crawl logs and try to identify the issue and fix it after which you need to run a search full crawl so that the failed content gets updated in the search index.

8.If you have made some changes to the search Crawl rules such as adding, deleting or modifying the crawl rule.

9.When your search index gets corrupted you need to perform a search index reset after which you need to run a full search crawl. Please check my article on search index reset to understand how to perform an index reset and under what circumstances should you be performing a search index reset.

10.The permissions given to the default content access account has been changed.

11. Apart from the above mentioned one’s the system by itself would be performing a search full crawl even when an incremental or continuous crawl is scheduled under the following circumstances:

a)The SharePoint administrator stopped the previous crawl.

b)A content database was restored, or a farm administrator has detached and reattached a content database.

c) A full crawl of the content source has never been done from this Search service application.

d)The crawl database does not contain entries for the addresses that are being crawled. Without entries in the crawl database for the items being crawled, incremental crawls cannot occur.

Thanks for reading this post. Happy SharePointing!!!

Demystifying MinRole in SharePoint Server 2016:

MinRole – I hope everyone would agree with me when I say that “MinRole” has become a buzz word among many SharePoint folks ever since Microsoft released SharePoint Server 2016. I myself have personally read many articles/blogs and viewed some videos on it to understand in detail about MinRole and how to make use of it. However, there has been times where I couldn’t really understand it completely and I had to work with many SharePoint experts in the industry to understand in detail about what MinRole is and how it works. But still I can sense a lot of uncertainty among few SharePoint folks in understanding MinRole and how to make use of it. Hence, in this article I’ll be explaining in detail about the below mentioned points….

What is MinRole?
How to deploy a SharePoint 2016 farm using MinRole topology?
Different server roles in MinRole
Different type of MinRole topologies
MinRole -Before and after Feature Pack 1
The benefits of using MinRole
MinRole Administration
MinRole compliancy
Opting out of MinRole
How/where to deploy 3^rd party apps while using MinRole?

Alright, so let’s get started …

What is MinRole?

To put it in very simple words, MinRole is a new farm topology based on a set of predefined server roles which got introduced in SharePoint Server 2016. Unlike the old traditional SharePoint farm topologies where you add a server to a farm and then configure it, here you can select the role of a server when you create a new farm or join a server to an existing farm and SharePoint will automatically configure the services on each server based on the server’s role. SharePoint Server 2016 by default has been optimized for the MinRole farm topology.

So, the point here to understand is, with MinRole you don’t need to add servers to a SharePoint farm and then configure each server in the farm as WFE, APP, Search etc.… MinRole will do that magic for you. Once you add a new SharePoint 2016 server to a farm and run the configuration wizard you would get a screen as shown below which asks you to choose the appropriate role .Once you select the appropriate role ,SharePoint will automatically turn on and configure the necessary services based on the server’s role.

Now that we have understood about MinRole, let’s understand how to deploy a SharePoint 2016 farm using MinRole topology.

2.How to deploy a SharePoint 2016 farm using MinRole topology?

Before I go ahead and discuss about how to deploy a SharePoint 2016 farm using MinRole topology, let’s refresh ourselves by taking a glance at the default SharePoint 2013 streamlined topology which we’re already used to. Let’s look at the image below to understand about the default SharePoint 2013 streamlined topology…

So as shown in the image above, in SharePoint 2013 when you create or add a new server to the farm you have to manually go to the “Manage services on server “section on Central administration site and turn on the required services after which you would be configuring the required service application (Ex: Search Service Application, Managed metadata service application, User Profile service application & Distributed Cache service application etc.…)

However, the good news with SharePoint 2016 you don’t need to spend time on turning on the required services under “Manage services on Server “. You just need to focus on choosing the required role on the “Specify server role “window which I just described above and SharePoint will take care of the rest for you. Hang on, let’s be clear here …. SharePoint will only take care of automatically turning on the required services but the service application has to be configured by you as an admin. I guess while reading this, you must have this question in mind … “Well this is cool, but how does SharePoint manages to do this by itself? “…The answer to this follows, when you create a new farm or join a machine to an existing farm, SharePoint starts the base set of service instances that are required for the server’s role. It also detects which additional services have been enabled in the farm and starts the matching service instances as appropriate for the server’s role. Finally, it detects which service applications have been created in the farm and which services are necessary to support those service applications. Those service instances will be started as appropriate for the server’s role, as well.

MinRole management of service instances doesn’t happen only when you join a server to a farm. As you enable or disable services in the farm, or as you create and delete service applications in the farm, MinRole starts and stops service instances on the existing servers in the farm. This ensures that each server in your SharePoint farm is running exactly the services it needs.

So, the end result is, you as a SharePoint farm administrator can only focus on what services you want to run in your farm and not worry about where they’re running. The MinRole topology in SharePoint will take care of the rest.

Also, let’s take a look at the image below which illustrates how the SharePoint services are scattered between these different server roles while using MinRole topology.

All the user interactive scenarios would be running on the WFE role, all the background tasks such as Search, UPS etc. would be running on the APP role and finally the caching services would be running on the DC role .

Well, hang on …. I still didn’t tell you how to deploy a SharePoint 2016 farm using MinRole. There’s two variants to do this … 1. Using the SharePoint product configuration wizard 2. Using PowerShell.

Using the SharePoint Configuration Wizard:

So, you can choose the role of a server while adding it to the farm using the below mentioned screen which you get while running the product configuration wizard.

Using PowerShell:

Now that we have understood how to deploy a SharePoint 2016 server /farm using MinRole, let’s try to understand the different roles available in MinRole topology.

Different server roles in MinRole:

The below mentioned image from one of my presentations on SharePoint 2016 clearly illustrates the different roles that are available in MinRole.

So, based on your need/architecture planning you can choose the appropriate role. However, this architecture might sound quite costly because with MinRole you can’t add two application roles together like how we used to do in SharePoint 2013 for small farms with 4 to 6 servers, meaning you don’t get to enjoy the privilege of having Search and Managed metadata or may be Search and User Profile service running on the same server. In MinRole if you do so then that particular server would be marked as non-compliant. But Microsoft has listened to its customers about this and has made some changes to the MinRole feature in Feature Pack 1 release for SharePoint 2016 and I’ll be talking in detail about that later on this article.

Note: The concept of Service packs is gone in SharePoint 2016 and is now replaced with Feature packs. You don’t get to see Service packs anymore at least on SharePoint 2016. Also, the Feature packs won’t be as separate packages like your service packs which gets released separately( i.e. once in 12 months as a separate package ). A particular month’s CU/PU would be called as a feature pack where Microsoft would ship all the fixes/new features and that month’s CU would be called as Feature Pack. Till now Microsoft has release Feature Pack 1 (i.e. Nov 2016 CU) and you can find the details about that in this link below . So, a specific month’s CU would be released as a FP hereafter .

https://support.microsoft.com/en-us/help/3127940/november-8,-2016,-update-for-sharepoint-server-2016-kb3127940

Microsoft was quite ahead of their schedule while they released FP1 as the original release date was planned on 2017 .However they managed to release that on Q4 of 2016 itself .

This image below depicts the roadmap for SharePoint Server 2016 :

Alright , let’s jump into the different type of topologies in MinRole .

Different type of MinRole topologies :

Now that we have seen a lot about MinRole , I guess it really begs the question of how to choose the appropriate SharePoint topology while using MinRole . Well , let’s go and take a look at it . Shall we ?

A typical SharePoint 2013 Topology :

This is how a typical SharePoint 2013 Topology would look like . Please check the image below .

In this case the SharePoint Administrators manually configure services on each server to fit these roles and in addition that as features and services are added, administrators have to determine where these components should run based on best practices, current server load, etc.

But this is not the case with SharePoint 2016 MinRole Topology , since this is a role based architecture you can directly choose the role you want to deploy and MinRole will take care of the rest . Please check the image below which depicts a SharePoint 2016 MinRole topology architecture .

SharePoint 2016 MinRole Topology :

MinRole topology.PNG

As shown in the image above, you need not less than 4 servers to deploy a SharePoint 2016 farm. If you’re including SQL then in that case you need at least 5 servers for MinRole. Also , Minimum configuration does not have any resiliency.

Let’s see how this works when you want to plan a SharePoint 2016 HA farm with MinRole topology .

So, as you can see in the image above , two servers are required for each role . When it comes to Distributed cache three servers are required in a cluster quorum . We also need SQL availability groups to achieve HA in the SQL layer. So, in total you might require 13 servers altogether if you’re also adding Office Online server in HA .

However , this count may vary based on your architecture and planning . Please check the image below where I’ve designed a HA SharePoint 2016 farm with proper planning .In this case the total number of servers required is 18 .So the point to note here , based on your business needs you can scale out the total number of servers for HA .

Custom 3 Tier MinRole Topology:

This is how a custom 3 tier MinRole topology looks like. The front-end servers are benefited from MinRole. The custom server role is used to configure custom servers to run majority of SharePoint service applications and reduce the number of servers. Unlike MinRole, the services have to be manually configured on the custom server role. It’s the job of the SharePoint Administrators to configure the required services on the custom server.

custome 3 tier.PNG

Custom HA Topology with Search:

This is how this architecture has been planned,

Two load balanced servers with Front-end role.
Two custom servers running distributed cache, User Profile Sync, Secure Store.
Two servers with Search server role.
SQL servers configured with always on availability groups.

5.MinRole -Before and after Feature Pack 1:

Now, if you see the complete overview of MinRole you might understand that you need high budget to implement this due to the total number of servers required. Unlike SharePoint 2013, you don’t get to add the roles together in a single server (i.e. Custom Role) while using MinRole topology and this might increase the budget and many customers have reported the same concern to Microsoft. As always, Microsoft listened to their customer’s feedback and they’ve made some changes to this in Feature pack 1. Let’s look at that in the image below.

I guess the image above gives a detailed explanation about the changes to MinRole topology post FP1 . So, post FP1 you can add two roles together which will reduce the total number of servers required to build a SharePoint 2016 farm using MinRole.

post FP1.png

If you’re interested in knowing more about the new features that was introduced in Feature Pack 1, please take a look at the link below.

https://blogs.office.com/2016/09/26/announcing-feature-pack-1-for-sharepoint-server-2016-cloud-born-and-future-proof/

The benefits of using MinRole:

Listed below are the benefits of using MinRole.

Simpler Deployments
Improved Performance and Reliability
Simpler Capacity Planning and Farm Scalability.

Simpler Deployments:

SharePoint Administrators no longer need to worry about which services have been enabled on which servers.
Administrators can reduce the risk of slight misconfigurations during installation by leveraging a template-type deployment.
Administrators can focus on what functionality to enable in the farm and let SharePoint take care of the rest.

Improved Performance and Reliability:

Microsoft has been operating SharePoint online since 2011 and has analyzed key performance characteristics of operating SharePoint at an internet scale such as CPU, Memory, disk I/O and network latency.
SharePoint has been optimized for MinRole topology based on all that analysis /learning which Microsoft learned from operating SharePoint Online in their own datacenters for years.
Improved service application load balancer services requests from local service instances instead of going across the farm to another server.

Simpler Capacity Planning and Farm Scalability:

In SharePoint 2016, Microsoft bases capacity planning on the MinRole topology.
Leverage predictable and perspective capacity-planning guidance by deploying a farm based on the MinRole topology.
As SharePoint needs grow, easily add more servers to the farm and SharePoint will automatically configure the additional servers.

MinRole Administration:

You can administer MinRole from the Central administration site and also via PowerShell

Using Central Administration site:

You can change the role of a server after it’s deployed and also check whether the server is complaint from the central administration site itself. The same can be achieved using PowerShell as well. A server in the farm which was acting as WFE today can be made as a APP tomorrow and once you change the role SharePoint will automatically turn on and off the required services .

Using PowerShell:

Note: There’s some bugs that has already been identified and reported to MS while toggling the role of server from the Central Administration site and hence it’s better to use PowerShell to change the role of a server

8.MinRole compliancy:

Once a Server’s role is configured, only those services appropriate for that role can run on that server.
SharePoint 2016 has a new set of Health Analyzer rules and timer jobs to identify when a server isn’t MinRole complaint.
If a service is accidently turned on and shouldn’t be running on that server, SharePoint will automatically turn it off.

9.Opting out of MinRole:

As a SharePoint administrator, you can always say no to MinRole if you’re not planning to use it. This can be achieved by assigning some/all the servers in the farm to the custom role and then manually manage service instances on these servers. Also, you need to consider using “ServerRoleOptional” parameter when creating a new SharePoint farm if existing deployments script needs to remain intact.

10.How/where to deploy 3^rd party apps while using MinRole?

Well, the answer to this simple. Yes, you guessed it correctly, so it’s the “Custom Role” that you need to choose while deploying any third-party applications such as (Ninetex Workflows, AvePoint etc.). In addition to that, services like PerformancePoint, PowerPivot etc. would best fit in the custom role.

MinRole is truly phenomenal and would definitely reduce the risk and the time taken by a SharePoint administrator to deploy a SharePoint 2016 farm. Microsoft has done an awesome job in introducing MinRole on SharePoint 2016 which would definitely reduce all our burdens as SharePoint administrators. Thanks for reading this post …. Happy SharePointing!!!

What is Secure Score in Office 365?

Secure-Keyboard-Hero

This post is on a new service which was introduced by Microsoft couple of months back called as “Office 365 Secure Score “. If you’ve ever wondered how secure your Office 365 tenant really is, then it’s time about time now to stop wondering because we have “Secure Score “now to take care of that. So, what’s this new service called as Office 365 secure score? What does it do? How do I make use of it? …. Well, I’m going to answer all those questions that you have in your mind about Office 365 secure score in this article and you will also learn about how to make use of this service to enhance your business with Office 365. Alright, let’s get started …. Shall we?

What is Office 365 secure score?

This is how Microsoft defines Office 365 secure score … “The Secure Score is a security analytics tool that will help you understand what you have done to reduce the risk to your data in Office 365, and show you what you can do to further reduce that risk “. To put it in very simple words, it’s a tool that runs on the background and checks the security standards of all the service used by you as an organization (i.e. SharePoint Online, Exchange Online, Skype for Business Online, Azure AD etc. …) and assigns a credit score.

What’ the idea behind Office 365 secure score?

The approach by Microsoft to this experience was very simple. First, they created a full inventory of all the security configurations and behaviors that customers can do to mitigate risks to their data in Office 365 (there are about 77 such things in total). Then, they evaluated the extent to which each of those controls mitigated a specific set of risks and awarded the control some points. More points means a more effective control for that risk. Lastly, they measured the extent to which the service has adopted the recommended controls, add up the points, and present it as a single score.

How to use Office 365 secure score?

The first thing you would notice once you login to the secure score portal is the welcome screen (check the screenshot below) which gives you a small definition about Office 365 secure score. In the below mentioned screenshot I’ve logged into the secure score portal of my Office 365 tenant by accessing this URL (i.e. https://securescore.office.com/ ) and I get this screen which gives me a welcome message about Office 365 secure score.

Note: If you already logged into your tenant you can directly access the Secure Score URL which I mentioned above and it will allow you inside the portal without prompting for your credentials once again.

2. Once you read all the welcome messages about Secure score you will get two different tabs as shown in the image below.

i)Dashboard.

ii )Score Analyzer.

3. The first tab which says “Dashboard” is where you get to see the secure score summary. This panel gives you your current Secure Score, and the total number of points that are available to you, given your subscription level, the date that your score was measured, as well as a simple pie chart of your score. The denominator of your score is not intended to be a goal number to achieve. The full set of controls includes several that are very aggressive and will potentially have an adverse impact on your users’ productivity. Your goal should be to optimize your action to take every possible risk mitigating action while preserving your users’ productivity. The below mentioned screenshot depicts the secure score summary of my Office 365 tenant where I’ve scored 61 out of 344 as on May 27, 2017.

4. The next section on the “Dashboard” tab after the “Secure score summary” section would be the section which tells how to improve your score. It gives you the targeted score that you can achieve for your tenant and lists out the action items to improve your score. You can make use of the slider to preview your improved score as shown in the image below.

5. The next section will list out all the pending action items that I’m supposed to complete to achieve the maximum score.

6. Now, let’s look at few pending action items to see what it means and how it would impact my Secure score in Office 365.

i) Designate less than 5 global admins:

This one says that I should designate less than 5 global administrators for Office 365 tenant and in my case, I’ve breached it by making it as 6. Hence, it’ asking me to correct it and it also gives me an overview about the score I would get by doing so.

ii) Enable MFA for all global admins:

This one says that I have to enable Multi factor authentication for all my 6 global admin accounts as none of accounts have that enabled and this is considered to be a security breach. It also tells me that I can achieve a score of 50 by doing so.

7) The next section under the “Dashboard” tab is the “Risk Assessment “section which gives me an overview about the top threats in my tenant. It is very important that Office 365 global administrators should read this and understand the risks they are mitigating every time they take an action.

Let’s look at the “Account breach” scenario here and see the details about the risk.

Compare your score:

The Office 365 Average Secure Score is calculated from every Office 365 customer’s Secure Score. You can use this section to understand how your score stacks up against the average score.

Note: The Average Secure Score only includes the numerator of the score, not the denominator. So, the average points may be higher than you can achieve because there are points in controls associated with services that you have not purchased (meaning , you might be using a different plan such as E3 whereas other customers might be using E5 or other plans) .

Alright, now let’s look at the “Score Analyzer” tab in the Secure Score portal.

Score Analyzer:

As of now, it’s only the global administrators who have access to the “Secure Score “portal and in the future, it would be made available to other administrators as well such as SharePoint Online administrator, Exchange Online administrator & Skype for Business administrator. However, in the interim you can use the “Score Analyzer “tab to export the secure score results and share it with your executives or stakeholders or other administrators (i.e. SharePoint Online, Exchange Online etc.) so that they’re aware of the progress that’s made on risk mitigation in Office 365. The Score Analyzer experience allows you to review a line graph of your score over time, to export the audit of your control measurements for the selected day to either a PDF or a CSV, and to review what controls you have earned points for, and which ones you could act on.

The below mentioned image depicts the “Score Analyzer” tab of my secure score portal.

2. I can make use of the “Export “button on the top right corner to export these results in PDF & CSV format.

3. It also gives you an overview of all the “Complete “and “Incomplete” actions and the scores associated to those action items as shown in the image below.

4 .The “Complete “and “Incomplete” actions are classified based on three different categories as you see below (i.e. Account, Data & Device)

5. Finally, I can make use of the “Export “button which I mentioned above to export the results to a PDF/CSV Please check the image below to see a sample report.

So finally, to conclude, the Secure Score is indeed a great tool to keep your Office 365 tenant as secure as possible and at the same time you need to be aware that the Secure Score does not express an absolute measure of how likely you are to get breached. It expresses the extent to which you have adopted controls which can offset the risk of being breached. No service can guarantee that you will not be breached, and the Secure Score should not be interpreted as a guarantee in any way.

Resources to know in detail about Secure Score in Office 365:

Microsoft Mechanics video on Office 365 Secure Score: https://youtu.be/h__nxWlm5Nc

Office 365 Secure Score API: https://blogs.technet.microsoft.com/office365security/using-the-office-365-secure-score-api/

You can also check my Webinar recording on Office 365 where I’ve shown a small demo on Office 365 secure score. Here’s the link to that: https://youtu.be/HYcfXWN30O0

Thanks for reading this post …. Good luck with Secure Score in Office 365!!!