Test automation purpose and trade-offs

Today, I like to write about misconception and purpose of test automation.

Now days, test engineers' programming capability is crucial in the software industry. I guess there is already an occupation like a Test Architect. And considering scale of the project, we cannot test applications without test automation. I guess I can safely say that test engineers' programming skill are better than like 5-6 years ago.

What is concerning to me is that as test engineers writing more codes, some of them are loosing focus. Test engineers understands design patterns, code quality, OOP concepts etc. I think that is very good thing happened in our test community. However, I really want to say that test automation framework code is not the same as production code. Test automation framework exist to "TEST" applications.

Here are some trade-off for test automation

• Simpleness over performance: One of the biggest difference between test automation code and production code is that there is a team to test production code while there is no other team to test test automation code. So then, we, test engineers, are responsible for testing our test automation framework as well. There are two things we can do about this: unit testing and simple design. You can read a lot about unit testing by searching online. I want to focus one simple design. Simple can be pretty relative term, but I like to define simple as many separate helpers doing one thing. I also want to define simple as avoiding a long chain of dependencies or deep hierarchy. Simple also can mean you do not spend too much time on debugging. Simple means no multiple nested if statements as well. So your test framework will consist of test procedure codes and bunch of small helpers. So you can easily find where the test failed and why the test failed. The usage of many small helpers will not be very efficient or fast. However, this simplicity reduces time for debugging and time for fixing test code bug.

• Readability over duplicate code: Everyone knows that duplicate codes are bad. No one goes against avoiding duplicate code. We also believe that refactoring is virtuous thing in software development. Why duplicate codes are bad? Yes, of course, if you have the same logic of codes written in many places in the application, it will be hell to find all the usage and update them. That's why we want to refactor them and change the codes in one place. I think avoiding duplicate codes principle applies also when we are writing test helper classes. Now, let's closely look at test procedure codes. In one TestCase class, there are several test methods representing testcase1-1, testcase1-2 and testcase1-3. Let's say, each test case differs in inputs. Then codes for input are different, but rest of the codes are pretty much identical. I can see the duplicate codes in three methods. Again, we need to be very careful about refactoring these duplicate codes. Each test method contains test procedure codes, which shows steps to be taken(code-wise) to verify failure or success of that specific use(test) case. This steps should not be over refactored. Refactoring means one more stack trace to debug. And also hides some steps from each test method(since it will be refactored to somewhare outside of the method). Let's not refactor code for the sake of refactoring. 6 months later, it will really hard for you to figure out what you did. And, it will be even harder for other testers to read your test case or add new test case.