I have been discussing about testing service oriented solutions through a lot of blog posts as well as this book. I just want to summarize some key points, specially since I switched from SOA middleware testing to service-oriented solutions testing.
You should not even think about QA approach without automation when it comes to service oriented solutions. I observe three distinct entities involved in service oriented solution testing approaches.
Services Testing
End-to-End Testing
UI Testing
Individual services can be tested as they make available for testing. In case, they are not ready or under implementation and QA is blocked to verify integration flows, you can use service simulation mechanisms. soapUI mock services will be handy in this. Testing the functionalities of individual services should be 100% automated and continuously deployed to the release/testing branches. Since SOAP web services are already dead, there is no need to spend time on experimenting various testing tools. RESTful services can be tested using simple HTTP clients and can easily be integrated into main code branches without much efforts.
End-to-End testing should also be automated as much as possible. An 80% goal may be realistic and achievable. End-to-End testing should also be done by by-passing front-end and combining invocations of various services.
UI testing will also be equally important based on the nature of the solution. The UI regressions are tedious and must be automated using a web test framework such as Selenium web driver. However, the focus on UI automation should be the last out of all automation approaches since the maintainence burden of UI automation is high comparing to the rest.
Even if you do majority of automated testing, you cannot replace human brain when releasing software. With a large percentage of automated tests, the testers could spend a lot of time analyzing user behavior to derive exploratory tests.
This is not about TDD (Test-driven development). But everyone in the team should equally responsible for the quality of the final deliverable. Therefore, each service developer must work with QA teams to design high quality test plans and review the test cases since the initial phases of release cycles.
According to my understanding, the best way to derive test scenarios is studying your users closely. See how they use the previous versions of your service oriented solution. If it is a brand new version, study similar/competitive products. Then, derive test scenarios which mimics the real world use cases.
Automation is the key
You should not even think about QA approach without automation when it comes to service oriented solutions. I observe three distinct entities involved in service oriented solution testing approaches.
Services Testing
End-to-End Testing
UI Testing
Individual services can be tested as they make available for testing. In case, they are not ready or under implementation and QA is blocked to verify integration flows, you can use service simulation mechanisms. soapUI mock services will be handy in this. Testing the functionalities of individual services should be 100% automated and continuously deployed to the release/testing branches. Since SOAP web services are already dead, there is no need to spend time on experimenting various testing tools. RESTful services can be tested using simple HTTP clients and can easily be integrated into main code branches without much efforts.
End-to-End testing should also be automated as much as possible. An 80% goal may be realistic and achievable. End-to-End testing should also be done by by-passing front-end and combining invocations of various services.
UI testing will also be equally important based on the nature of the solution. The UI regressions are tedious and must be automated using a web test framework such as Selenium web driver. However, the focus on UI automation should be the last out of all automation approaches since the maintainence burden of UI automation is high comparing to the rest.
Even if you do majority of automated testing, you cannot replace human brain when releasing software. With a large percentage of automated tests, the testers could spend a lot of time analyzing user behavior to derive exploratory tests.
QA should not be an after-thought
This is not about TDD (Test-driven development). But everyone in the team should equally responsible for the quality of the final deliverable. Therefore, each service developer must work with QA teams to design high quality test plans and review the test cases since the initial phases of release cycles.
Study your users
According to my understanding, the best way to derive test scenarios is studying your users closely. See how they use the previous versions of your service oriented solution. If it is a brand new version, study similar/competitive products. Then, derive test scenarios which mimics the real world use cases.