Automate Web Service Testing the Right Way

Web services Web services are perfect candidates for automated testing. Compared with validating a UI, Web services Web services testing is quite simple: send a request, get a response, verify it. But it's not as easy as it may look. The main challenge is to identify the formal procedures that could be used for automated validation and verification. Other problems include dynamic data and differing request formats.

In the project our team was working on when this was written, there were many different Web services in need of testing. We needed to check not only the format of the request and response, but also the business logic, data returned by the services and the service behavior after significant changes were made in the architecture. From the very beginning we intended to automate as many of these tests as possible. Some of the techniques we used for automated verification of Web services functionality were as follows:

1.Response schema validation.
This is the most simple and formal procedure of response structure verification. There are free schema validators out there that can easily be built into automated scripts. However, schema validation is not sufficient of you also need to validate the Web services logic based on data returned from a service.

2.Comparison of the actual response with the expected response.
This is a perfect method for Web service regression testing because it allows for checking of both response structure and data within the response. After the successful manual validation, all valid responses are considered correct and saved for future reference. The response of all next versions of that Web service will then be compared with these template files. There is absolutely no need to write your own script for XML file comparison; just find the most suitable free tool and use it within your scripts. However, this method is not applicable when dealing with dynamic data or if a new version of Web services involves changes in the request structure or Web services architecture.

3.Check XML nodes and their values.
In the case of dynamic data, it can be useful to validate only the existence of nodes using XPath query and/or check the node values using patterns.

4.Validate XML response data against original data source.
As a “data source” it’s possible to use database, another service, XML files, non-XML files and in general, everything that can be considered a “data provider” for Web services. In combination with response schema validation, this method could be the perfect solution for testing any service with dynamic data. On the other hand, it requires additional skills to write your own scripts for fetching data from data source and comparing it with service response data.

5.Scenario testing.
This is used when the test case is not a single request but a set of requests sent in sequence to check Web services behavior. Usually the Web services scenario testing includes checking one Web service operation using another. For example, you can check that the CreateUser request works properly by sending a GetUser request and validating the GetUser response. In general, any of the above techniques can be used for Web services scenarios. Just be careful to send requests one after another in the correct order. Also be mindful of the uniqueness of test data during the test. The service might not allow you to create, for example, several users with the same name. It’s also common for one request in a sequence to require data from a previous response. In this case you need to think about storing that data and using it on the fly.

All of these techniques were implemented using a home-grown tool which we were using in our day-to-day work. Using that tool, we are able to automate about 70 percent of our test cases. This small utility that was originally created by developers for developers to cover typical predefined test conditions and gradually evolved into rather powerful tool. Having such a tool in your own organization could help your testing efforts well into the future.

Source: http://www.stpmag.com