Wednesday, December 1, 2010

Assertion in test automation

I think we(software workers) love to use the expression "it depends" a lot when we explain things. And it sounds right. I thought of one topic "When and where to put assertions", but I think the answer covers most of stuff is "it depends." But I really don't want that expression on this topic.

Let's talk about assertions. I think assertions has different meanings in test code compared to dev code. I don't think dev want to use assertion in production code. We want to handle error case more gracefully than failing the execution. I guess dev can use assert while developing application, then disable it when it goes to production.

Test code cannot exist without assertion. That's what we(test engineers) do. So I've been thinking about best practice for assertion usage.

Let me throw some questions that I had.

"Do I need to use only one assertion per test cases?"
"Too many assertion in one test case is bad?"
"Does many assertions in one test case means too many logic is executed in one test case?"
"Is that mean I should have smaller tests?"
"Does helper classes need to have assertions?"
"If assertion is in helper class, then does it mean helper class is part of the test?"

What's your answer? "IT DEPENDS???"

I don't think number of assertions in one test case really does not matter. What it really matters is that one test case is verifying one logical thing. It could be multiple things to check to pass the test or there are multiple steps need to be taken to execute the test case.

But one important thing to remember is that assertion should stay in test procedure code not in helper class code. Because helper class, as its name says, helps test execution. Assertions should be executed before or after helper code executes and keep the generic nature of helpers.