Test automation design (Chain of verifiers)

Today, I like to write something related to the test automation design. I called it "Chain of Verifiers". It would be nice if I can get some feedback on this design. I've got good feedback on this design from one test architect but that does not mean this is good. Let me know if you have any comments.

Motivation for pattern is to come up with simple, robust and yet flexible and scalable test automation design especially in SOA test automation context. Basic idea for Chain of Verifiers pattern is separating one test case execution into three logical pieces. Those are “Test Operation”, “Gathering verification source” and “test verification”. 

Test Operation

Test Operation is activity or activities that are required to do for each test case (i.e. creating a request, send request, get response)

All the logic to construct the request, sending request and parsing response, and etc. belongs to this part

Gathering Verification Source

Gathering verification source part is responsible for gather all the data or information that will be used in verification.

This part also should be able to take care of parsing data. If logs are generated during the operation and it can be parsed in certain object or data. (It is better to do here so that verifiers do not have to parse the log or convert to certain class type it needs) 

The expected results are not necessarily required for this part. It could be if some input used for creating request is expected value. However, main focus is to gather data or info that will be used in verification 

Create one container (i.e. VerifySource) that contains all the data gathered from the operation (req/res pairs, inputs used, expected result(in some cases), parsed logs) 

Executing Test Verification

Executing Test Verification is the part that contains logics that what verification need to run (based on each test case’ purpose) and actually verifying the results

Selecting verifiers for verification logic and executing verifiers are decoupled. (will explain more details later on)

VerifierFactory: This is responsible for gathering verifiers and creating chain for certain aspect of verification. (It will be refined based on each test case needs)

Verifiers: This is responsible for verify one logical part that each test case needs.

Overview

More Details of Executing Test Verification

• VerifierFactory holds the logic of gathering set of verifiers that each test case intends to verify, which mean VerifierFactory logic need to be interated through out the test automation development process. 
• VerifierFactory can return two ways of verification process (similar to Decorator pattern)
• Sequential verifier chain: previous verifier failure cause the test failure immediately (i.e. cases where following verifiers are meaningless to perform like response contains error when it is not expected)
• Distinctive verifier set:  execute all the verifiers in the chain and return the result of “AND” operation of each verifier result. This is used when previous verifier failure not necessarily casue the failures of following verifiers
• Sequential and distinctive set of verifiers both can be used in the test code

Benefits of Chain of verifiers pattern

• It is follows Open-Close principle (Open for extension and close for modification)
• Each verifier can be implmented and run independently.
• Independent implementation effort can be easily distributed to different testers, which means even new person join the team, he/she can easily write Verifiers for given part without truly understanding of entire automation or project
• Each verifiers can be used for hybrid test execution (manual execution + automated verification)
• It’easy to add, remove, change verifiers in the factory along with change.
• If verification logic detail changes, you can only change the Verifier (less side effect of code change)
• If verification logic changes, you can only change the VerifierFactory.
• It’s easy to test one verifier (newly added ) by itself by changing VerifierFactory