It is also important to note that patching must not be limited to main product only, but include all inter-dependant components such as backup software, Java components, etc. to ensure compatibility.
The general recommendation is to maintain an N to N-1 software and hardware update strategy policy. N in this case is the latest service pack, patch, major update, maintenance release, driver, firmware version, etc.
Keeping software up-to-date is even more critical if you have cloud or hybrid environment. For example, hybrid configuration with Microsoft’s Office 365 requires that all participating servers MUST run the latest or at least the prior version of Exchange to be supported and compatible with Office 365.
In addition to building a change management process, it is also important to ensure that the planned change is validated in the test lab. We understand that maintaining a test lab today is not practical and may not even be possible, especially taking into consideration that not only main components but also 3rd party servers and applications must be included in testing. However, since most datacenters are fully virtualized today, there are tools available to quickly and efficiently provision an exact replica of your production environment.
Another common issue we see is bundling multiple changes together in a single change request. Making multiple changes at once when troubleshooting an issue should be avoided by any means. First, if the issue gets resolved, you do not know which particular change resolved the issue. Second, it is entirely possible the changes may aggravate the current issue.
The more complex the hardware or software architecture, the more unpredictable failure events can be. Managing failure at scale requires making recovery predictable, which drives the necessity for predictable failure modes.
We recommend our clients to keep the environment as simple as possible. The key elements of achieving simple, redundant environment which is optimized for performance is standardization and following best practices.
It is important to note that most often the vendor’s recommendations and best practices come from analysis of data from a number of clients, product feedbacks and field observations. That is why if the vendor tells you to configure X or update to version Y, chances are they are telling you for a reason, and you would be wise to follow that advice.
Another important step that is often forgotten is continuing to collect and analyze data after deployment is completed and adjusting it if changes occur. Frequently we see customers implement an architecture and then question why the system is overloaded. One should remember that environment is constantly changing. A good example of this is bring you own device which was not an option in many customer environments, whereas, now it is becoming the norm. As a result, customer’s messaging environment changed – users required larger mailbox quotas, the proliferation of devices, the capabilities within the devices, etc. These changes affected messaging environment design and consumed more resources. In order to account for cases like this, you must baseline, monitor, and evaluate how the system is performing and make changes as required.
We hope this post will uncover potential issues that may affect your environment and give you some food for thought.