SharePoint is like a chainsaw!

bestThere are a couple of things I have said about SharePoint now for many years, which i think still hold true. SharePoint is like a chainsaw – its a tool – A chainsaw is great for cutting down trees, but misuse it and you can cut both your arms off. This is true of SharePoint.

Microsoft have done an excellent job at selling SharePoint, and given that investing in SharePoint as a platform can be a significant investment I have seen some companies insist on its use even what it doesn’t necessarily make sense. I am an architect, a developer, and a SharePoint advocate but i am also an IT Consulatant, which means sometimes even though i have several nice shiny SharePoint platforms i will say no if i don’t think SharePoint will benefit a project. When architecting any SharePoint solution you should be thinking about how to leverage the out of the box functionality, rather than redesigning it, which can be a costly exercise in time and money; things can get complicated fast. As a SharePoint consultant i tend to focus on the data, the information flow and the business case and recommend not only how to customise SharePoint but also how to change a way of working; customers wont always want to change their workflow but its something that should be considered. A few days back a customer who had development done by a third party said to me –

“I looked at our site, i looked at MOSS 2007 and it was completely different. Why didn’t anyone tell us we were being so heavily customised?”

With the economy being the way it is i am seeing large companies wanting to not customise and enter into long development projects, but to use out of the box functionality more and more often; and its an approach that makes sense – leverage out of the box 90% and customise the last 10% – if you need to customise at all. If you are thinking of customising any more than that stop and ask yourself really hard why you have chosen SharePoint; of course the benefits may still outweigh the negatives, just don’t take the decision blindly.

Microsoft has put a lot of effort into making SharePoint easy to customise by designers; theres a trend to move UI development away from code development on all manner of platforms and the Microsoft takes that approach as well – you don’t need specialist SharePoint skills to design a site layout in SharePoint 2013. Prior to that customising design and CSS could be a real nightmare. Their approach to branding now lends itself well to mobility, and customising look and feel is significantly less challenging than it was

Of course, designing a large scale SharePoint solution is about a lot more than that; to do it effectively you need to be knowledgeable about many things, including architecture, the business needs, security, development, infrastructure, and maintenance. Ive seen a lot of bad SharePoint implementations out there that have been because not all people that call themselves SharePoint developers understand SharePoint. A person who is good at .net may have a head start when it comes to SharePoint development but doesn’t know the theory behind it, may not be aware of everything the framework gives you, or may work against it. Ive seen solutions out there that have done some really bad things – like changing configuration files directly in _layouts – the solution will appear to work great but as soon as you decide you want to have several solutions deployed on the same environment suddenly things start to not work as well as expected.

Its worth also noting that not all .net developers make good SharePoint Developers, even though they both code using C#. From a resourcing perspective I’ve been asked before how long it will take a .net developer to become a SharePoint developer, and its a question that drive me nuts for a few reasons – Firstly people learn in different ways and different rates, secondly understanding a little infrastructure for a large scale project i have found to be fairly important, and a few other skills that not all developers have or want. I cant just put a .net person on a training course and assume they will come back share point competent. Third is because developers aren’t familiar with the framework they tend to default into what they know. Ive seen projects before that have used databases rather than lists – not because of any specific design reasons (because  there are definitely good reasons to do this) but because they knew SQL, and just weren’t happy with Queries and CAML. The chances of successfully converting .net people  to SharePoint people are radically increased with mentoring, and leveraging experience and the SharePoint way of thinking is important. There are many good SharePoint developers out there, but i think its the fact they think from a SharePoint mindset that makes them good rather than just a pure ability to code.

A SharePoint Solution Architect has an obligation to make sure that a development team is working in the correct fashion; it is not enough to design a solution and assume developers are thinking of the same approach as he is; even if not  directly involved in the projects, the approach to the solution design should be clearly documented.

SharePoint is an awesome product that can be an excellent return of investment, and it may seem from what i have written that i think SharePoint is a dangerous technology – and to an extent is is; but then to an extent all technologies are! In life its possible to abuse just about anything, but if you don’t realistically look at the risks, you will never avoid them!

 

A practical approach to SharePoint application management – Part III

Continuing on a theme – in part II i started to describe some of the key documents that should be considered when running a service the list wasnt complete so i will continue here.

System Testing protocols and reports

The protocol is the set of rules that you follow to successfully test. the system test report is a report that details the results from when a test has been executed.

I find that in a service management situation a couple of types of system test are really beneficial; of course we need to see the detailed test results that have been performed by the developers buts I also like to have a smaller system test protocol for when i am performing installation qualification work. The smaller test is for anyone who does any reconfiguration work, or even a reboot. I have seen often that someone from a platform team does an installation and then verifies the installation was correct by just navigating to a site – if the home page is up things are great! In reality though that’s not good enough if you are trying to run a high quality proactive service. I normally have a small test protocol to test key elements of the site. on a sharepoint site it’s fairly obvious if your database has gone down in the earlier versions, but with sharepoint 2013 you have the possibility to specify a backup database; You should check the health status of things, you should ensure that if your site is using business connectivity all those connections are up. Its better that you find a problem than your customer.

Of course in some cases such a document isn’t required because your active maintenance of is site may be extensive. If you are using third-party products to actively monitor health of your farm then it may be immediately apparent if a web service is down on a big monitor in your office. Like all the documentation i have mentioned there is an art to writing good effective documentation, and the key is normally to think about what it is your document needs to achieve and who its target audience are.

Installation Qualification

All documentation should be under change management; the installation qualification is about properly documenting installations and changes to the system. Each time a new solution or patch is deployed an installation qualification document is published; It typically details

• The reason for the change – maybe a change request number
• The procedure to follow in order to perform the change
• The roll back procedure for the change
• The procedure for ensuring the change was completed successfully
• The prerequisites for the change.

normally speaking the IQ is fairly static. To minimise the work i do i have one IQ per service;witihn the configuration documentation i would have a table of variables which get referenced in the IQ

So the IQ may read: Login to <SERVER> with user <USER>, and in the documentation under my project name, under my environment (System Test or Live for example); in this way most of the time when a deployment i can pretty much copy and paste most of the document – and just fill in some key details in the header. Doing things this way means i need to make sure my configuration documentation is tied in with the release of the software.

Conclusion

This is an overview only and many people out there will do this a different way, but this way of having a core structure of documents works for me. Of course in many cases you will need further documentation – the systems architecture documentation – such as extra documentation on interfaces etc, subsystems, load testing – documentation strategy should be thought through – you should not blindly follow a documentation template just because its how its been done in the past. Good documentation supports your working processes and information requirements from different areas of your organisation; it should be written and structured in a way which as far as possible avoids need for repetition. There are some cases where you may find it necessary to repeat information – for example some configuration information may be needed by your operations staff – but they may not have security rights to see all configuration documentation. such cases where you have such needs there should be a clear process in place to ensure that changes are applied in all areas of the organisation they are required. Documentation in my mind is the key to effective governance because in planning your documentation you are forced to consider your workflow from many different perspectives (such as security, DR, business, maintenance and user requirements). When you start to plan this documentation all manner of questions are often raised; and in answering them you are usually improving your service.

Related articles

• A practical approach to SharePoint application management – Part I (sharepoint.owenrichardson.com)
• A practical approach to SharePoint application management – Part II (sharepoint.owenrichardson.com)

 

A practical approach to SharePoint application management – Part II

Its important to note we are talking about application maintenance here, not application development, although application developers will be responsible for writing some of the key documentation (Such as the Operations Guide) – documentation requirements for both Application developers, and for your Server Operations team are going to vary. I will make up an example service just to illustrate various points here. Rather than repeating information in documents i tend to refer to other documents, and make sure the information i am referring to is in a sensible place. With all documentation its important to have change control – a version number and at minimum if a document is draft or approved – i like to retain a full document history; I also create document templates where possible but in some cases they are not practical – a poorly thought out template can cause more harm than good.

These are the core documents i think you need to have in place to effectively run a Application Management (AM) service.

Service Level Agreement

Details the service level with the customer. It includes things such as:

Service Catalog – Describing which services are defined.

Service Level Categories – Priorities have to be clearly defined as to hours of operation and response times. For example priority A issues may be defined as “System completely down, multiple users effected”. Everyone in the world thinks their own issue is priority A, so defining it is important!

Financials – The service costs, and the penalties incurred when you fail to meet your service level objectives.

Systems Architecture Document

This document describes the system in terms of its architecture. It is describing generically any instance of your solution. Its aim should be to give any architect an overview of the solution so that when changes need to be made to your enterprise architecture your architect can just pick up this document. It includes things such as:

• An overview? What is it? what does it do?
• What technology does it require.
• Requirements for external systems
• Scope – What is and is not included as part of this service. For example are external services included as part of this solution or not?
• Logical Architecture
• Business context of the application – who uses it? is it part of a larger system?
• Minimum Requirements for running
• Security requirements such as authentication requirements and information classification
• Design considerations – is the solution built to be scalable? redundant? cheap? Does it meet disaster recovery requirements?

Its fairly common for people to include a physical architecture here but i resist doing this because when designing documentation for an enterprise i want to avoid repeating myself.

to illustrate my point a bit. The systems architecture document tells our reader that our site is an intranet site for an insurance company. It tells us the intranet site requires connectivity to our Business intelligence (BI) platform to consume sales data. It has a nice description of the wsp solution that has been built – including listing its features, and content types, and showing a logical information layout for the site, and detailing the security requirements. It would also establish the different roles that need to be played for a sharepoint 2010 site in our example -we have a Web Front End Role (Its a SP2010 Solution), Central Administration Role, Database Role, and an Indexing role – Fairly standard sharepoint roles. We dont put specific configuration information in here because we want to centralise that information. By having a configuration document we  can avoid specifying this information in many places, and again, reduce work.

From this document if an architect wants to move the site he has all the information he needs- it would tell him instantly if the proposed move is viable – for example if we were consolidating websites onto one server he would see if there are capacity issues, if security means this cannot be done (maybe data is classified and you aren’t allowed to have external connections, or maybe from the new server its not possible to establish a connection to the BI layer)

Disaster Recovery Plan

Depending on scale disaster recovery can sometimes be part of the operations guide, but often warrants a document of its own. The disaster recovery document is a product of a business continuity plan – business analyses risks (Risk Analysis), assign a cost to the business for each disaster scenario (Business Impact Analyse) and then builds a Business Continuity Plan on top of this. In going through this process the business in hand with the technical team establish a set of requirements that have to be met in your disaster recovery plan. There are many factors to be considered, including SLA adherence, restore time objectives (RTO), restore point objectives – BCP and DRP requirements may be complex – you can buy whole books on the subject.

The aim of the disaster recovery plan is to have a document that in an emergency can be referenced to bring your systems back up. Its common to name people in this document, but i don’t like to do that. In every document i write i like to normalise information as far as possible. In the DRP i may say contact the technical manager, or contact the service manager – referring to people by their role – and i would have the DRP document reference my roles and responsibilities document.

Roles and responsibilities document

Is a simple document that lists:

• Job Role
• Name
• Contact information

By keeping all this information together i can save a lot of work – I’ve lost count of the times I’ve opened a little used DRP or SAD to discover a reference to a manager that left years back. By referring to people by their roles and referencing this document you  have contact information in one place. Which is great especially for your BCP and DRP; in an emergency you don’t want to be tracking people down.

Configuration Document

This is where we get into specifics on how our systems are configured. The SAD defined what our solution looks like, the configuration document provides the information on each of the specific instances. I will typically create one configuration document that covers multiple platforms (Systems Test, Acceptance Test, Integration Test, Live)

So for each platform we describe the specific information required for it. Firstly i define the roles – In our example our production intranet site may have 3 web front ends – each one would have an associated ip address or host header, alternate access mappings and other information that is important to capture.

Operations Guide

An application is like a black box to the people who maintain your server platform and they need to know how to interact with it. Its important to remember that all documentation is required to fulfil a function and we shouldn’t have to reinvent the wheel. Before starting to write this its important to establish your target audience expertise – if you have a platform maintenance team who has some familiarity with sharepoint you should not have to document common tasks, unless for some reason those tasks deviate from the normal. Ive seen in the past for example a sharepoint solution that had third party software in it which needed to be restarted by hand. – in this case the operations guide needed to have a server restart procedure.

If your application logs to files you need to make the operations team aware of this – or if you have requirements for log file retention that are non standard. If you know your logging is particularly verbose for some reason the attention of the server team needs to be drawn to that; similarly you can do a lot to reduce help-desk overhead by monitoring your logs automatically; if your webservices have a tendency to crash and a third party needs to be notified to restart them, significant time can be saved by having the logs monitored and an automatic email dispatched. Determining the requirements of the operations guide can and should involve collaboration between operation management and application management teams; its important though to note your roles and responsibilities – as a representative of an applications management team have been surprised how often i get asked provide documentation for things i am not responsible for – its important to have balance; the operations guide should not be a guide to how to operate SharePoint but a guide to how to operate the application.

Above i have only started to touch on some of the core subjects; I have more to say on the documentation subject in part 3!

Related articles

• A practical approach to SharePoint application management – Part I

 

A practical approach to SharePoint application management – Part I

Running a SharePoint Application Maintenance team can be tricky. You end up with a diverse set of people with a diverse set of competencies whose aim should be quite simple – to provide a service to a customer as specified by the service level agreements. Ive been doing this for quite a while now and i thought i would share some observations. I could probably fill a couple of books on this subject so i will do a little bit and if anyone wants more, i will add more.

The team maintaining the servers and platforms are not necessarily the same team or the same company that maintain the applications – this is increasingly common now the cloud is gaining momentum. Ive seen now in a few services a fuzzy line between what the server teams are responsible for and what the application maintenance teams are responsible for as soon as SharePoint is involved – This is because there has in the past been two perspectives that i observed. Typically a server team would think of sharepoint as an application – because its not Windows and it isn’t SQL. The application team would see sharepoint at part of the platform; because they install their applications on top of it. In truth, SharePoint is the responsibility of both Server Operations and Application Operations teams. Delineating responsibilities is therefore important. As a rule of thumb in terms of day to day doing things anything that requires configuration in central administration i leave as a responsibility of the Server team, and anything thats done as a site admin from the SharePoint interface i leave to the application teams. Thats not to say that the server team has total responsibility. When building a sharepoint application consideration should also have gone into architectural design; how databases are configured is a great example of where some collaboration needs to exist. A companies business continuity plan may well dictate how the content databases and site collections are spread out over a sharepoint farm (so in an emergency essential services can come up first). Disaster recovery planning and business continuity planning are something that need to be done with the customer – its too common that IT people decide how to backup their environments without taking the customer business requirements into consideration. Business Continuity Planning and disaster recovery planning are considered as afterthoughts, which is crazy because they can radically impact on how you organise your information architecture.

Change management and controlled access to server environments is important – virtually nobody should have access to system usernames and passwords. If your application service team need access to a server, put them in a security group (Server maintenance teams should be doing the same thing). If your application team needs access to a server, give them just the privs they need – so they can log on and have read only access to logfiles and events. Letting people login as system admins is not only bad practice but it screws your ability to audit access to your environments; in an emergency its useful to be able to see the username of the person that changed a file – not so you can punish them but so you can quickly get to the bottom as to why a change was made. The server person that made the change should have an installation qualification document that not only is there a document of what change was made, theres also a rollback procedure. The application team tells people how to install an application, the server team does it.

Server teams should have their own best practice – a document that describes what the application team can and cannot do on the servers – for example common no no’s include:

• Modifying any of share points configuration files in the sharepoint hive
• Running FTP services rather than using proper services
• Running application pools with administrative rights

… The list goes on!.

As a bare minimum for implementing a sharepoint maintenance service with a customer i would suggest the following platforms exist – integration test platforms are also good to have where needed:

Development – Each user should have their own development environment or be on a shared one – either way they should be able to do anything they want with it, as long as they follow company policy on security. Development is the only platform that isnt under change and release controle.

Systems Test – Owned by the maintenance team i would let devs have admin access but would still configure things as close to the customer environments as possible. Its VERY important that you dont do all your testing as administrative users.

Acceptance Test – Owned by the customer, or at least customer facing- before putting code into acceptance I like to see code, architectural design, security, and documentation reviewed. Strict change control is essential.

Live – The actual production servers are under strict control and monitored.

Application teams should have their own best practices and governance too. They should be using virtualisation, should be following configuration best practise in their test environments for configuration and development. There should also be policies for code reuse, and code review.

The key to effectively managing a service is good documentation and good interaction between platform and application teams. Documentation is often seen as a burden but the truth is if you do it properly and effectively plan it, its a way to not only safeguard yourself in a disaster (assume everyone can die) it also lets you be fairly lazy on a day to day basis. If you have a good document you can avoid having to answer the same questions a thousand times.

A great example – Your server teams see sharepoint as a black box – as a development team you would know the common errors that you would be produced by your application, or may be generating your own error logs. The server team need to know what to look for- if for example your application needs to send information periodically via a webservice and if tails, your server team can set up a process to monitor your logs looking for the failure and then send an SMS to you if it does. Suddenly, your in a position where you can start to be proactive – all you need to do is formally tell them, and have a process to manage the change when you get a new phone….

Related articles

• A practical approach to SharePoint application management – Part II