Leadership for a tester - part 1

Today, I like to write about leadership.

For those genius techie testers, leadership might not be very attractive topic. You already getting respect for your technical skills and delivery so you don't have to worry about being a manager.

But seriously, do you only need a leadership skill when you become a manager? I've seen lots of leaders who don't have manager or senior title. Wether you're planning to go managerial route or individual contributor route, you need leadership skill. Leaders are influential. And good companies know that and use if for measure of promotion.

There are tons of books and articles about leadership. And those are mostly written by business people (e.g. MBAs) and for the business people (e.g. MBAs). Lots of them are the same concept with a little different flavors. I found a good one yesterday. And I've been thinking about it all day to relate to my lovely work, software testing.

Here is the simple quote that explains.

"A good leader has three kinds of leadership, and he/she should be excellent at all of them simultaneously. Those are Strategic leadership, Operation leadership and People leadership."

Let's relate this to our testing world. And I want to point out one more time, good at all those three simultaneously. 
  

Strategic leadership

It's about the decision on how you're going to test your software. It requires a good understanding of what you have, what you can control and what you can do. And you make decision and execute on that. This applies to the one who is responsible for entire test organization of the company, the one who is responsible for one testing department, or the one who is responsible for a story testing. And you can guess that the decision will impact based on what you're responsible for.

I really think context-driven testing make sense in this leadership because what you have, what you can control and what you can do differ from companies to companies. Even if two companies producing similar software product, testing strategy cannot be similar. Your company/organization will have people who has different skill set. Company/development culture can be different. Process adoption and maturity are different. You might or might not be able to work with offshore. Testing strategy the company has been using might be different. And so on and on..

However, there are things we can do to make sure testing strategy works.
First, you need to communicate your testing strategy. If you're a test director or manager, you need to communicate your strategy to all of your subordinates and make them understand. Why we're doing this way and what we're expect to get out of our current strategy. If you're an individual contributor tester, you need to communicate your testing strategy with your scrum/project team. "This is how I'm going to test this story and this is why." By doing so, you are informing your plan and open  for feedback. A great strategic leader make sure others understand what're he/she is trying to do and why. And provide a way to get feedback. If you as an IC and feel that your test manager/director just care about numbers and graph, you need to challenge them to provide strategy.

Second, you need to have a mechanism that shows progress or results of your decision. This happens naturally if you open your feedback channel. For test directors and test managers, you need to check with your direct reports to understand how the strategy is working. Often times, you'll see issues, overhead or redundancy in your strategy. And for ICs, you can see how your strategy is working by going through several iteration of work(sprints/milestone). I know measuring testing progress or success is hard. (I'm planning on writing one post about this) Sometimes it's not about numbers or graph, people notice the benefits and drawbacks. You need to check your strategy progress/result and make adjustments to make perfect fit for your situation.

Third, you need to study, research and experiment a lot on strategy. Innovation does not come out of nowhere. Hear other companies stories, your subordinate's experiences, and feedback. Have a meeting with your directs to make better. Work with PMs and developers to understand their views and progress. Continuous improvement is part of strategy effort. And make sure you're doing the right thing, not doing things right.

And lastly, once you're clear on strategy, you should believe in it and execute the hell out it.    

I'll write operational and people leadership in following post.

Systems thinking and test automation

Today, I like to share some of thoughts around test automation related to Systems Thinking.

You as an SDET, it is very important to make your team understand what your test automation strategy is. When it is clear, your team can support you and align their work with your automation strategy.

OK. let's start. To be sure, I am specifically writing about functional test automation here. And, in fact, I consider test automation as regression tool not as testing activity. I like James Bach's term 'checking' to describe automation. However, I do believe that test automation as regression tool has great value.

My test automation strategy is based on 'Systems Thinking'. Dr. Russel Ackoff came up with this idea and there there has been lots of conference around this.  I posted his video on my blog, so you can watch his video.

There are two principles that I like to mention from his talk.

First, when it comes to the system, as a whole, it creates a unique behavior or property that none of its parts have.

Second, System's unique behavior or property is created by the interaction among individual components(parts) NOT by sum of individual parts behavior.

As Dr. Ackoff explained, let me take a CAR for example.
What is CAR's unique behavior/property? That is carrying you(person) one place to another. It is only unique as a system. Let's take a look at each parts of the car. Engine, transmission, door, battery and so on. None of them can carry you one place to another. But that unique property cannot be created without parts. More specifically, the interactions between parts creates that property. You can easily understand that simply having the parts without interaction is not contributing to produce a unique system's property.

The point I'm applying to my test automation strategy is the INTERACTION.

Here is one more car example. Let's say, your car stopped while you're driving. And let's assume, it is because the engine is stopped. Yes. it is obvious that broken engine caused car to stop. But if you think a slight different perspective, car actually stopped because no force or thrust was delivered to the wheel. In other words, the interaction was broken.

And now, let's say your car is stopped, but you don't know what caused it. What do modern car manufactures do to identify issue?  Of course they are using some sort of sensors and chips. And it alerts when any sensor detects something that is not normal or when certain criteria they set is not met. And then mechanics are going after actual root causes based on that information.

I think SDETs job is to put sensors all over the code using automation. Sensors that detects all the small functionalities  that comes from interaction between components and modules. Sensors that detects component/sub-component behaviors that comes from interactions of classes and methods (unit tests - this is normally written by dev).

What this thorough automation provides a simple value. Make easy/straight bugs to be found easily and hard/mysterious bugs to be found hard.  Sounds simple, but this will give tremendous trust to your work and your test automation.

Testers are like caddies in some sense

Today, I like to describe testers with some odd(?) metaphor.

I think testers are like caddies in some sense. More specifically, the guy who walks with Tiger Woods or Phil Mickelson on the field of Professional Golf Tournament. The caddies of PGA tours. 

Why? I'm going to explain why.. 

First off, Caddy is an occupation. Golf player is an occupation as well. Caddie might not get attention as much as the players, but still he/she plays very important part of professional golf tournament. It sounds like testers in software development, doesn't it?

Here is one thought on being testers. I've seen so many testers who complains a lot about the company not treating them as important as developers. Being treated like second class citizens or getting paid less. Again it's an occupation. You choose to do it. If you care about being treated like developers, then you should be a developer. If you do not enjoy testing, you should not do testing. Perception is really hard to change. If you care what others' think about you, well..I don't know. I think you should be yourself.

 However, I truly respect those testers who has been fighting for the importance of testing in software industry over the years.  

OK.. let's start.

Caddies understand the game of golf as much as players on the field. They know general information about the game like the golf course, golf rules, difficulties of each holes, distance of each hole, location of hazard, trees around the hole, bunkers and roughs. And they also know what affects each shot. They understand things that affects the shot like winds, uphill, down hill, rough, bunker, reading greens, soft or hard green, wet grass or dry grass and etc. Coming back to testers. They should also know about general information about the product/service they're testing. What kind of product it is (web, mobile, desktop and etc.), who are the users, what kind of technology the team using, what features are important for the business, what development process is used (agile, kanban or waterfall) and so on. And they also understand what affects the developments like software architecture, code review, check-in process, bug triage, regular build process, regression strategy, reporting mechanism and so on. This is really basic stuff. Good caddies and good testers take seriously about the basics of their work.

Caddies provide good information for each shot players make. You may have seen this on TV. PGA golf players always discusses with a caddy before he/she makes shot. Caddies and players both check the distance, hole location, hole surroundings,  ball location, wind, grass, slope, curves, trees and so on. That brief conversation before the shot is the result of quick analysis. Testers are similar in that sense. Test plan/strategy should show the analysis of the feature development. Whether it is formal or informal, testers should analyze the feature that is going to be developed and discuss with developers. Here are some examples. User impact, complexity of the component, impact to existing code, risk, timeline, algorithms or implementation plan, testability and so on. This requires a lot of studying and experiments. Testers should continuously learn how to do better analysis and improve.

Caddies know players more than players know themselves. Professional caddies know his/her players so well. Strength and weakness. Ups and downs of emotion. Distance of each club. Fade and hook tendencies. And they know how to communicate with those things with players. Sometimes caddies insist on certain club because they can predict what his/her player will hit. This is really important part for testers as well. Testing is not just having a list of things and execute them. Knowing the context is the key. That's why I love the context driven school guys. Testing never starts from nothing at the beginning. Product has history. It has business environment and developers. Testers should try to understand developers' coding style and thought pattern. It really requires a close attention to each developer, but it really pays off. Thought patterns are amazingly accurate to predict. Sometimes you can even guess what a developer would  implement this particular class or functions. Bug trends as well. Who makes what kind of bugs can be understood by testers.

Well. I'll stop here.. But there is actually a big difference between caddies and testers. Actual testing part. Testing performed by testers are not there in caddie's work. I'll address that later..

               

A good test automation framework - Part 2

I previously wrote about test automation framework with details related to coding. (I would call it part 1)

Today, I like to mention a little bit more technical explanations of a good test automation framework.

I guess you (test engineers) have worked on test automation framework that was written by someone else and also worked on designing it from scratch. Have you feel some fuzzy or itchy feeling about the test automation you're working on? Something seems to be wrong but hard to explain? Does it take hours just to add a new test case? Do you feel test execution is heavy? I have the same feeling as well. I've been seriously thinking about that and I came up with following technical details. I don't think it is just because of  different coding style nor different thinking process. Hopefully, this blog actually scratch those itchy feelings. :)

1. Goal setting: This is about initial design of test automation framework. You need to set two goals. If you can satisfy both goals from framework user (testers average 2-3 years on automation), I would say your design is successful. The first goal is the framework should be easy to understand, easy to use and easy to implement. And the second goal is you need to design test framework in a way that volume of the code increases but complexity of the code does not. This seems to general and yes it depends on the context of your application. But if you write test automation framework from scratch, please do not simply start with drawing diagrams. Think about consumer of your framework. And also how the framework will grow and not become collection of giant and complex code base.

2. Do not re-factor test execution step code. This is for the user of test framework. I mentioned from my first blog to be cautious about it. Now I change my mind. Again, Do not re-factor test executions step code.  It really has many side effects.

First, your coding is restricted by re-factored code. You framework API provide several functionality and normally those are the components that supports your test execution steps. Your test execution code should be just mix and match of those components. Once you re-factor the execution steps, you are losing your flexibility of test case implementation. It's really silly to be proud of test execution step code  to be one function and it takes 6-7 parameters. Also, you lose the readability of test execution steps. Now how can you replace your test case document with your code? It's so hard to read. DRY (Do not Repeat Yourself) does not applied to test execution step code.

Second, you are adding unnecessary complexity by re-factoring code. I have a good metaphor to back up this argument. Let's say you want to test a car and you have test system that control every part of the car. But that system provides only one button for control. With one button click you need to come up with a way to start the engine, step on gas, break, changing gear, steering wheel control, lights, radio and etc. How would you control that with one button. Yes, you need to come up with some sort of Morse Code to differentiate the operations. "bip bip wait wait bippppp bip wait bippppp" that means turn the head light on. How about you have buttons for each operations. Engine button, break button, accelerator button, light button, changing gear button and etc.

Third, you don't gain much anyways. In test automation framework, what part of the codes are most frequently and most likely added and modified? It is test executions step code. Test cases are keep added and modified as the application grows sprint by sprint. Let's say you have 8 distinct test case which exercise different aspects of the application. Let's say test case 1 and 2 have 70% of the same code. And test case 2 and 3 share 60% of code a slight different way. Test case 3 and 4 shares 65% of code but also slightly different way and so on. It looks like a lot of codes are repeating. Now you start re-factoring. I can tell you; it not easy. What do you end up having? Excessive object composition or excessive class hierarchy of code. Or. you can see the same name of re-factored code with less parameters all over the place. Base class or helpers. Now look at your code after re-factoring. Lines of code you save is not much. And guess what new test case comes in and you need to re-re-factor your code.

Forth, it's not uniformly re-factored and not re-factored by responsibility or business logic. Most of test execution step code re-factoring happens due to repeating code. Not like automation framework code. Framework code is re-factored to achieve Open-Close principle or component decomposition. And it is normally uniformly re-factored because it has its own purpose. Refactoring is not simply putting repeating code in one place. Refactoring makes the code more maintainable and extensible. Existing test execution step changes in several different way, I tell you I've experienced..... It's hell.

Fifth, putting repeating code to base class causes weak cohesion and giant class (3000 lines of code kind of giant class). I'll explain this more in details later. I personally do not recommend people to use TestBase class because once you open the door, it's open for any code to go in. It stars with really essential pieces of code for bootstrap. And later add some more helpers. And later people start putting repeating test execution step code. And people start to add another base class that inherit from TestBse class. Adding if statements and switch statements. Where is the cohesion? What does this base class specifically do? Everything!


A good test automation framework continues....
 

Defining Agile Testing

Today, I like to define what exactly agile testing is. And also I like to mention drawbacks of agile testing and how to accommodate that. If you find something missing please reply on the comment section.

Following seems to be the unique parts of Agile testing. It's an interesting mind shift if you've been working as a tester in Waterfall environment.

Testing effort on defect prevention.
This is one of the fundamental differences between Agile testing and testing in Waterfall. In Agile testing, test team puts their effort on creating good quality builds before it gets to QA team (before deployment) whereas Waterfall testing puts efforts on after it gets to QA team (after deployment). This seems just a small difference but it has quite valuable implications. First, for test team to find issue before dev code gets deployed to QA environment, there should be a good communication channel between dev/pm/test and even business. Test team can help dev team not to make any assumptions about unclear spec. Test team can provide potential impact to the application due to current development(risk assessment/analysis). Test team can also provide some scripts or test data that dev team can run and use on their local environment.  Dev team can do a simple demo to the test/pm/business to verify his/her progress. All these to be happened, barrier among all disciplines has to be removed and all has to simply work as a team. I've seen a lot of issues are found during those informal discussions and demos. And those issues are quickly resolved and not logged as bugs, which can reduce the time and effort on filing a bug (repro steps, screenshots and all that stuff), triaging the bug, and dev team reproing the bug and etc. Also, enforcing writing and reviewing unit tests and checking on coverage metrics are also contributing to the defect prevention. There are a lot of things you can do to prevent defects in your software. It's a good discussion to have with your dev/pm/tests and business. You'll normally have multiple checks before code gets deployed. And you'll actually find less bugs after deployment happens. But now when you find a bug on QA environment, you can think about how you could prevent that and add more checks. It's kind of addicting. I would say..

Speed
Speed is another the key part of Agile testing. Fast feedback on developer's new check-in.  Think about the time it takes where a new mistakes goes into the check-in and those mistakes are found in testing. duration between dev puting mistakes in his/her code and finding that mistake from test. Lots of articles and papers mention about pyramid and what kind of tests take place in that pyramid. It actually has to do with speed of test execution and creating targeted and controlled tests. Each level of pyramid defines purpose of its own. Another way of looking at this is to check the  Think of find the same bug in selenium UI tests vs. fast and furious unit tests. So if your test team’s primary automation strategy is something like selenium, think again. Seriously

Consistent feedback mechanism
No doubt about this. Continuous Integration! Every check-in of the code gets tested thoroughly from unit tests to UI tests.  Gated check-in using unit test coverage percentage and promotion model of build increase the quality of the build. Test team can contribute to reviewing unit tests, writing fast component-wise testing, UI testing using mocking or spoofing.

Skilled testers.
Agile testers do not only think like users. They do think like users and engineers. They are beef-up with tools. They can pick and choose whatever tool is necessary for the execution of test of any kind. Skilled testers not only tess the code but also make good recommendations to the dev team. Testability is one of the key components of good design of the application.

Now, let’s take a look at how to accommodate drawbacks of agile testing.

Context-Driven testing
One of the tendencies of agile testers is to think a lot on mechanics of the code. Functions and components of the application. But there are places where testing strategy matters. Because each and every testing problem has to do with its context. What are the best testing strategy for calculation heavy feature? (like data driven testing) What are the best testing strategy around state based application? (like model based testing). What are the best testing strategies around infinite inputs and outputs permutation applications? (random sampling, equivalent classing, pair-wise testing) Coming up with best testing strategy around given context actually matters a lot.

Systems thinking
 Since agile testing is working on small features and make iterative progress. Testers sometimes lose the big picture of the application. Systems thinking is a great way to look at your testing. Check this video out. https://www.youtube.com/watch?v=OqEeIG8aPPk  If you trying to improve the application by taking parts separately from the system, you’ll not be successful. Watch this video, you’ll get it.