‘Testing’ is one of those terms that is so familiar that everyone uses it quite comfortably, certain that they know what it means. You can send your car for an MOT to make sure that everything works as expected; you can test a paint colour to make sure that it contrasts well with your woodwork. When talking about the development of a piece of software it is easy to assume that one of these everyday definitions of testing will apply. The reality though is quite different and often a misunderstanding of the nature and importance of software testing can put a whole project at risk.
When your car goes in for an MOT, the mechanic checks the state of all of the standard components to make sure they are tuned according to their specifications and none of them need to be replaced. The nearest parallel in the world of software is a standard off-the-shelf application like Microsoft Word. Every copy of Word (for a particular operating system) is essentially identical, and unlike a car its component parts don’t wear out because they consist only of digitally-stored information. It can be mis-configured, and you can imagine testing your installation of Word to resolve a configuration issue. I have never heard of anyone testing Word after installation though to make sure that it works correctly: that is the big advantage of buying an off-the-shelf application – it is usually safe to assume that all the major problems have already been found and resolved. Contrast that situation with a bespoke development and things are rather different. Except at the most basic level, every part of a bespoke development is new and unique. There is no history of use in which many thousands of people have given over thousands of hours of effort identifying bugs, reporting them and finding solutions. That all has to be done systematically as part of the development project.
Today there are many applications available for free on the Internet. If you have a problem that can be fixed using software, one strategy is to try several downloadable applications until you find one you like. That is similar to testing paint colours: they are already there, and you just have to choose one. All of the chemistry that went into developing the paint has already been done, and so has all of the engineering work that went into developing the colour mixing machine. In a software development project on the other hand, the equivalent of the chemistry and engineering has to be done during the project itself and every change to the design costs substantial time and effort.
Testing in software development is mainly aimed at identifying problems as early as possible. If they can be identified before any code has been written, they can be changed quickly and easily. Likewise once the process of writing code begins, the earlier in the process any changes can be made, the less effort is wasted. Getting the commitment from the client to be available for testing throughout the project and especially in the early stages can help to keep the project under control. That makes it much more likely that everything will run to time and to budget, and everyone is in favour of that…
Perhaps a better analogy for software testing is something that you don’t usually think of as ‘testing’ at all. Few people have items of clothing tailor-made on a regular basis, but you can imagine how it works: you don’t go along to the dressmaker or the tailor one day to give them your requirements and then come back a month later to pick up the final product. What if something was not quite right? Would it be reasonable to make corrections to the cut or the fit on the day you expect to take your new garment away? Rather, you would expect to go in several times during design and development to evaluate and approve work done so far and to guide the course of the rest of the project. A more familiar situation might be getting your hair done. Typically people don’t go to the hairdresser and say “I want it like this; wake me up when you are finished”. Instead you have a dialogue with the hairdresser about the progress of the work based on the constant feedback that you get from watching your image in the mirror. True, some hairdressers can be a little bossy, and it is still possible to walk out with an unsatisfactory haircut. A good software development team is not bossy though, and relies on the client to guide the work as it progresses. Successful projects are those in which the client and the development team work closely together throughout. That is the way to make sure that the engine runs as expected, the fit and appearance are satisfactory, and no-one goes away feeling like they got a bad haircut.
In practice, there are some specific testing activities where participation from the client is vital. Once the specification of the final product has been agreed between the client and the team, development usually progresses as a series of prototypes each one more advanced than the preceding ones. From the client’s perspective, the purpose of testing is to make sure that a prototype correctly implements part of the specification as agreed. On the one hand, the earlier a problem is identified, the less effort is required to fix it. On the other, if everything works as expected everyone can move forward in the project with confidence in the progress to date. Sometimes a prototype may implement a feature purely as an experiment to see how it might fit with the rest of the application. That is an opportunity for debate between the team and the client about the future course of the design. In both of these cases, good engagement from the client is essential for a good outcome. Without help from the client, the development team would have to rely on their own interpretation of the specification which can be wildly different from what the client intended. At the end of a project, there is typically a final round of testing in which the client makes sure that all the essential features from the specification have been correctly implemented. If everything works successfully, the project is signed off and the product handed over – hence the name ‘acceptance testing’.
To deliver a successful result, it is almost as if the client is part of the development team, always on hand to provide feedback and guidance when needed. Then the project is more like an extended conversation between the developers and the client, each taking turns as appropriate. From that perspective, testing is not something special that happens just once or twice. Rather it is the client’s contribution to the conversation – something which happens regularly all the way through. The client can have a huge impact of the success or failure of a project. My best advice is to make sure that you get to know the rest of the team, talk to them regularly and keep the conversation going until the job is done.
