![one_does_not.jpg]()
Folks,
We all know that sooner or later our beloved SAP systems are bound to have some sort of an upgrade: support packs, enhancement packs and such. In the SAP world, the upgrade is about as inevitable as death. And there is frequently just as much fear, confusion and uncertainty associated with the upgrade process.
This blog is not about the technical details of the support pack installation or SPAU activities (plenty of information about that on SCN, here is one good example) but rather about managing and surviving an upgrade project.
My personal upgrade experience happened to be with the medium size SAP customers, but you will likely find many similarities with smaller and larger installations. Of course, just like with any advice you find on the interwebs “your mileage may wary”.
As an example, I will use a fictitious company ACME Corp. (any resemblance to real organizations is purely coincidental) with the headquarters at an undisclosed US location. The company’s name and business specifics are not really relevant here – the process is quite similar for many SAP customers.
To Upgrade or Not to Upgrade?
Some companies upgrade their SAP systems at regular intervals and some only when enough “business steam” builds up. The upgrade can also be driven by the legal requirements (this is frequently the case for the customers using SAP HR module). Personally, I believe that any frequency is fine and every customer is perfectly capable of choosing the right option just for them.
More regular / frequent upgrades keep the SAP system as current as possible and improve your SAP team’s expertise thus reducing each upgrade’s impact. Test scenarios and documentation get better. The business users get into the routine and don’t act surprised when they need to test something.
Less frequent upgrades can be more disruptive because more changes are made at the same time and there is not much experience with handling them. But at the same time – look at all the savings! (Especially if you manage to stretch it for a few years.)
In case of the ACME Corp., one of the SAP ECC 6.0 systems has not been upgraded for several years and EHP6 was applied due to the need to align with other SAP systems at the sister companies.
Fail to Plan, Plan to Fail
The saying “good, fast, cheap – pick any two” applies to the SAP projects as well. If you can throw tons of resources/money at the project, it might be done fast and good. Otherwise be realistic. “Good” is definitely not the one you’d want to sacrifice here.
Start planning for the upgrade as soon as possible, ideally when making plans for the upcoming year. Pick the potential go-live date and then plan the schedule backwards from there. Try to do as much as possible preparation in advance. (Really, this is not much different from the party planning. And, as we’ll see, the guest list is just as important!)
From my experience, about 2-3 months from start to end is a decent schedule that allows completing the upgrade comfortably on a low budget with internal resources. Stretching this for much longer can do more harm than good. Completing it in less than 2 months might require getting additional resources. As noted above, the project length also depends on how frequently the upgrades are performed (more frequent upgrades are usually shorter).
Some very helpful documents can be prepared before the upgrade (ideally, these should always be available, but we all know how it goes).
- The test scenarios or at least the business process documentation. It does not have to be elaborate, but you will need at least some simple end-to-end scenarios for testing. Enter order, confirm availability, deliver – that kind of thing.
- The “interface inventory” and documentation. Also no need for anything complex here. A simple spreadsheet worked well for several ACME’s projects: interface name, direction (to/from SAP), yes/no column for the designated test environment, and connection method (RFC, FTP, IDoc, etc.).
- Any documents left from the SAP implementation or previous upgrade (“lessons learned”, go-live issue log, modification list, etc.) can be useful.
Sadly, documentation and project planning can also be some of the biggest time wasters in an upgrade project. The KISS principle works well here. Don’t build a huge MS Project plan for something that can be tracked in a simple spreadsheet. One page specification template is more likely to be filled in than an 8-page one and is, in fact, more useful.
Resources
The actual upgrade needs to be performed by a qualified Basis… err… Netweaver Administrator. If your SAP system is hosted externally make sure to coordinate the upgrade with the service provider well in advance. If there are no in-house Basis resource, a consultant might be needed for situations when (not “if”) some task comes up that is not covered by the service agreement. (I’ve had some “no idea what I’m doing, but this is my best guess” moments, it’s rather unpleasant.)
You will also need an ABAPer to re-apply the modifications and assist with the interface testing and possible fixes. The functional folks are usually not involved in the early upgrade stages but may need to assist with testing.
If you need to bring in the consultants, it’s not a good idea to use the new ones. Most of the work usually involves decent knowledge of the company’s inner workings and all kinds of skeletons in your SAP closet. The consultants who never worked in your system before might take the whole duration of the project just to understand what’s going on.
And, of course, you will need the business users for testing. If there is a strong community of intelligent and engaged SMEs available then consider yourself very lucky. I cannot stress enough the importance of the business user involvement in any SAP project. The right people will know what to do, when and how. The wrong people will drag their feet, whine about how SAP “does not work” and then will end up doing sloppy testing. If you are not confident in the business users – add at least an extra week to the project.
Maintenance in the Time of Upgrade *)
Most SAP landscapes consist of the DEV, QA, and PRD systems. Naturally, the DEV and QA systems will need to be upgraded first. But this will make them incompatible with the production system for a while. So, how do you support the live system in the mean time?
Technically, you could declare a complete development freeze and simply have PRD drift on its own while DEV and QA systems are being upgraded and testing carried out. But I find that this simple and cost-effective option is rather unpopular with the management.
On practice, many customers choose to set up a “parallel universe” of DEV and QA system clones (or even just one combined DEV/QA environment) for the production support while the real DEV/QA systems are being upgraded and testing is performed.
The biggest challenge with such setup is making sure that all the changes made there do not get lost or conflict with the upgrade-related changes. Unfortunately, there are no perfect tools for this at the moment. My suggestion would be to keep the changes to the bare minimum and postpone any non-essential work until after the upgrade.
Make sure to document and track all the changes being made during the upgrade (simple Excel spreadsheet works here too). To some extent, the changes can be moved between the parallel systems using transports. However, someone would still need to make sure the changes do not cause any conflicts.
If the number of changes is small, they can be simply copy-pasted manually in the upgraded DEV system. This will also create a chance to review them at once.
We don’t Need no Modification
The DEV system has been upgraded and now you need to decide how to handle “the ghost of Christmas past”, i.e. all those modifications popping up in SPAU/SPDD transactions.
This is when a scene from the old comedy “Liar, Liar” comes to mind. The main character (played by Jim Carrey) is a lawyer who suddenly loses ability to lie and has to respond to a call from a client asking for a legal advice after robbing an ATM. Jim Carrey takes the phone and, holding it at arm’s length, yells: “Stop breaking the law, a*hole!”. This is exactly the way I frequently feel when I see SPAU/SPDD results: “Stop making those bleeping modifications!”
Yes, there are still situations when a modification need to be made, I won’t pretend it’s not true. And in some cases, such as our beloved SD user exits, it is even “approved by SAP”. But the stuff that can be easily accomplished by a configuration change, user exit or not even needed should not make it into the modification to begin with. Just a few random examples of what I removed in the past: business logic put in a standard program for no reason (moved to a legitimate user exit); a work-around per obsolete SAP note; modification in the standard search help to add functionality no one asked for or was even aware of. The list goes on.
Before escorting the modifications out of your system make sure to document them though. If a modification turns out to be needed after all (this happened too!), it can be very helpful. I simply take the screenshots and put them in a Word document together with a note how it was handled and documents supporting the decision.
Note on the Side-effect Notes
One word: don’t. (Or is it technically two words? English is confusing.) In one of the previous upgrades ACME Corp. was practically bullied by the consultants into applying a hundred or so side-effect notes. Unfortunately, many of them involved making manual modifications, which will have to be removed in the next upgrade. Gee, thanks.
The reason the consultants were so insistent: without those notes the SAP system would be “unstable”. But in reality, with or without the side effect notes at any given point the SAP system is just as “stable” as it gets. New notes and support packs are released all the time and then even more notes on top and then more notes to fix the previous ones, etc. This is a never-ending process (I’ve heard this is called “agile development” or something).
Instead of spending time on the side-effect notes, rather add more time to the testing phase. If there are some notes that are truly needed – it will come out in testing. Also the tools like ANST make the note search much easier these days and further reduce the need for the side-effect note “carpet bombing”.
In the most recent upgrade at ACME, none of the side-effect notes were applied initially. The testing found that two notes from the list had to be applied (minor fixes) and also discovered an error that required a correction by the new SAP note. Yeah, so much for the “stability”.
What do we Test and Is There an App for That?
The million dollar question usually is: “How do we know what exactly was changed in this upgrade?” The short and straight-forward answer is: you don’t. But don’t panic (yet).
Of course, there is the whole cottage industry offering special “solutions” that are supposed to make your upgrade a breeze, save dollars and hours, mitigate risks, etc., etc. While such tools have their merits, I feel none of the currently available ones offer significant ROI for the small to medium size SAP customers. For them it makes more sense simply to run “end to end” testing.
Just think about it – let’s say you’ve invested in some tool that tells you “nah, that VA01 transaction wasn’t changed, you don’t need to test it”. Would you be willing to rely on that statement? Would you not make sure that at least a sales order can be placed after the upgrade? And wouldn’t you need that order anyway to test the consequent documents? Then why pay for that tool? Before searching for information think about what you would do with it.
I’ve been involved in the upgrades that utilized costly commercial software, a home-grown program (pushed by a consulting company), and no special tools at all. Out of those the home-grown program was the worst by far. Not only it turned out to be useless in identifying potential pitfalls, but it also raised too many false alarms. This spooked the management causing a delay to explain there was, in fact, nothing to worry about.
If you do decide to use some upgrade / testing assistance software, make it a really good one or none at all.
Getting Support from SAP
In the unfortunate event you run into an error in the standard functionality that is not resolved by any of the existing SAP notes, you will need to enter an incident with SAP. Unless you can wait a month or more for the issue to be resolved, don’t bother using Medium priority in the incident. If the error can delay the whole upgrade project, don’t be shy and use quite appropriate in such case High priority. And give the Customer Interaction Center (CIS) a ringy-ding-ding if your ticket is not getting the adequate attention. The phone numbers for CIS can be found in the Contact Us section on the SAP Support portal.
U-A-T! U-A-T!
As soon as the QA system has been upgraded and any additional transports (containing SPAU/SPDD adjustments, changes from the “parallel universe”, etc.) have been applied, it’s time to get down and dirty with User Acceptance Testing (UAT).
Without exaggeration, UAT is the most important part of the upgrade project. Allow as much time as possible for this stage (don’t forget that this will also include “fix and re-test” for any identified errors). However, don’t go overboard either and keep it simple and practical. Make sure that the test scenarios cover all the vital processes, including the fiscal period closing.
One item frequently overlooked in UAT is the output. If your business still relies on the paper copies, make sure that all such documents are actually printed and look right. Demand the papers as proof of testing (preferably signed in blood). During the last upgrade at ACME the users thought that print preview is an acceptable substitute for printing. And then after the go-live in Production some barcodes went MIA all of a sudden. Not a pleasant experience.
Testing coordination is very similar to herding the cats. I find that telling people specifically “this is when we need you to be available and this is what you will need to do” works better than just emailing out the spreadsheets and expecting everyone to “do the needful”.
If all the business users are available in one location, then locking them in a room and feeding occasionally can work wonders for successful and timely UAT completion. If you are dealing with the multiple and/or remote locations, it’s a good idea to assign a team leader / coordinator on site.
As much as I hate the meetings and conference calls, the UAT phase is when some status checking is needed at least every other day. (Unless you work with unbelievably awesome people.)
Interface Testing
The SAP systems usually interface with other applications and you would be wise to test those interfaces after the upgrade. At ACME Corp., the interface testing is a separate line on the project plan and is performed mostly by the IT resources, usually even before UAT.
If the interface testing will require assistance from the external vendors, they need to be informed well in advance. I found that sometimes the vendor’s key employees happen to be irreplaceable. Goodness forbid you schedule the interface testing when such “superstar” is on vacation.
Upgrade Hits the Fan AKA The Go Live
While I’m not a fan of the very detailed project plans, the go-live does deserve more attention than the rest of the project.
Here I use the same approach as with the party planning. For the stuff that needs to be done on the day of the event I write down every single step, in sequence. When you need to bake the cake, there is nothing more frustrating than finding that the butter needed to be taken out of the fridge hours ago to be softened. (Although whacking the cold butter stick with a rolling pin can be quite therapeutic.) No matter how small the step is – add it go the go-live plan.
Make sure to retain a pre-upgrade system clone for at least a month after the go-live. (This could be a temporary QA environment that was used for the production maintenance during the upgrade.) In my experience, about 50% of the “issues” reported after the upgrade have nothing to do with it.
But it is important to encourage the users to report any suspicious behavior and address their concerns. A delay in reporting an actual glitch would be more costly than spending few minutes in an old system confirming that something worked the same way before.
Ounce of prevention
Keeping your SAP system healthy at all times will make the upgrade process simpler. Avoid modifications (see above) and maintain reasonable ABAP guidelines for the custom development. For example, using some shady unreleased function module may seem like a cool idea for a minute, but then down the road it could cause a major delay while the whole team searches why an interface stopped working even though no changes were made in any programs. Well, guess what - SAP changed the function module and since it was not released they did not have to tell anyone.
Maintain useful and short documentation. Document any upgrade-related peculiarities for the future projects. For example, ACME Corp. experienced some issues with the Firefighter access after one upgrade and then the same happened in another system. Simple document can be a life-saver in such cases.
Conclusion
While the SAP customers fear the upgrades, some SAP employees are known to recommend casually to just upgrade to EHP7, as if it’s as simple as installing a smartphone app (and even those can go terribly wrong).
I find that truth is somewhere in the middle of the wide spectrum between the "easy button" and "mission impossible". There is a Russian saying “the eyes are afraid but the hands are doing”. An SAP upgrade can be very similar: you might not know what exactly was changed and what you are getting into. But you prepare a reasonable plan, assemble the right people and just get on with it. Then suddenly it’s done! Until the next upgrade reaches for your neck with its long, cold hands... Bwahaha! ![]()
This blog describes the new and improved tools in the SL Toolset 1.0 with 
