Leadership for a tester - part 2

Operational leadership

You can easily imagine that great strategy with poor execution cannot lead to a good result. However, great execution in software testing is somewhat hard to define.  We can do all sort of things as part of testing but we cannot clearly know all the effort actually 100% contributed to achieve this big ambiguous goal, 'excellent quality of software.'

To give you some idea, let me give you a question. Which tester do you think is more productive or effective? One who finds lots of good bugs or the one who can prevent from bugs being created? The one has in depth coding knowledge of the application or the one who deeply understands and knows users/customers usage and expectation? One who uses test process/methodology that has been working successfully or one who wants to try unproven new process/methodology that possibly benefit testing effort? One who found 20 pri2 bugs or the one who found 2 critical/pri1 bugs?

What do we do to measure the testing effectiveness? I've seen many times that test managers seriously going through irregular bug burn-down chart categorized by priority/severity or test story points fluctuation to get some sort of meaning out of it. I know data will not lie. But do those data actually represent the effectiveness of testing? I would say no.

Operational leader executes and delivers based on testing strategy. 

I like to emphasize one more time on "simultaneousness" Poor strategy ruins excellent execution. This is like building an excellent ship to go over a mountain. Poor execution makes excellent strategy a day dream. It's like a innovative and passionate person leading a team of complainers "that does not work in this company. you just want to do this to get visibility from senior management....."

Operation leader understands the strategy clearly and execute towards that. And operational leader helps to form excellent strategy.

Based on assessing the context of product/team/timeline/etc, if the testing strategy for current product is more towards CI and pyramid(ton of unit tests,  many integration tests, a few UI/E2E tests),  she will generate code coverage data from unit tests, come up with gated check-in strategy, gather feedback cycle and trends of integration tests and UI tests. She will continue to monitor the progress and tries to optimize the testing effort. Staffing will be leaning towards to the strategy as well. She will clearly deliver the importance of the strategy and ask her team to be best at those areas.  If the testing strategy is more towards exploratory execution and user-centric, she will discuss with the team to come up with session-based exploratory testing plan, ad-hoc testing plan, think outside of box exercise, crowd sourcing, dog fooding and etc. She will generate data relevant to exploratory testing area, buggy feature, or qualitative confidence. She will also categorize the user based on sex, demographic, educational level, and etc to demonstrate the exploratory nature of the testing effort.


Operational leader respects professionalism.

Operational leader put her heart on everything she is doing. As a professional test engineer, she will do her best to accomplish any task. Brainstorming on test cases. Writing bug report. Writing single line of automation code. Discussion during test plan/strategy meeting. Discussion with Dev team and PM team. Solid execution and delivery. No BS. That's an operational leader.  She is also looking to improve the process or methodology continuously. Just set herself to be the best in the industry, just a pure professionalism.

Often times, operational leader loves the work. Sometimes she dreams about work. This might sounds crazy but actually that happens. It's not like getting stress from work. Imagine, you are fall in love with video game and playing all day. And you're stuck at level 12 and you need to go home. Would you think about it? Dream about it? Do you want to be the best tester? then you should love the testing work. In any occupation, one that loves work always out perform one that works hard.  Furthermore, testing is a special occupation. If you don't value your work or don't find interest, your life is hell in the company.

So far, we talked about strategic leadership and operational leadership. I'm a little tired. Will do the people leadership on the next blog.

Leadership for a tester - part 1

Today, I like to write about leadership.

For those genius techie testers, leadership might not be very attractive topic. You already getting respect for your technical skills and delivery so you don't have to worry about being a manager.

But seriously, do you only need a leadership skill when you become a manager? I've seen lots of leaders who don't have manager or senior title. Wether you're planning to go managerial route or individual contributor route, you need leadership skill. Leaders are influential. And good companies know that and use if for measure of promotion.

There are tons of books and articles about leadership. And those are mostly written by business people (e.g. MBAs) and for the business people (e.g. MBAs). Lots of them are the same concept with a little different flavors. I found a good one yesterday. And I've been thinking about it all day to relate to my lovely work, software testing.

Here is the simple quote that explains.

"A good leader has three kinds of leadership, and he/she should be excellent at all of them simultaneously. Those are Strategic leadership, Operation leadership and People leadership."

Let's relate this to our testing world. And I want to point out one more time, good at all those three simultaneously. 
  

Strategic leadership

It's about the decision on how you're going to test your software. It requires a good understanding of what you have, what you can control and what you can do. And you make decision and execute on that. This applies to the one who is responsible for entire test organization of the company, the one who is responsible for one testing department, or the one who is responsible for a story testing. And you can guess that the decision will impact based on what you're responsible for.

I really think context-driven testing make sense in this leadership because what you have, what you can control and what you can do differ from companies to companies. Even if two companies producing similar software product, testing strategy cannot be similar. Your company/organization will have people who has different skill set. Company/development culture can be different. Process adoption and maturity are different. You might or might not be able to work with offshore. Testing strategy the company has been using might be different. And so on and on..

However, there are things we can do to make sure testing strategy works.
First, you need to communicate your testing strategy. If you're a test director or manager, you need to communicate your strategy to all of your subordinates and make them understand. Why we're doing this way and what we're expect to get out of our current strategy. If you're an individual contributor tester, you need to communicate your testing strategy with your scrum/project team. "This is how I'm going to test this story and this is why." By doing so, you are informing your plan and open  for feedback. A great strategic leader make sure others understand what're he/she is trying to do and why. And provide a way to get feedback. If you as an IC and feel that your test manager/director just care about numbers and graph, you need to challenge them to provide strategy.

Second, you need to have a mechanism that shows progress or results of your decision. This happens naturally if you open your feedback channel. For test directors and test managers, you need to check with your direct reports to understand how the strategy is working. Often times, you'll see issues, overhead or redundancy in your strategy. And for ICs, you can see how your strategy is working by going through several iteration of work(sprints/milestone). I know measuring testing progress or success is hard. (I'm planning on writing one post about this) Sometimes it's not about numbers or graph, people notice the benefits and drawbacks. You need to check your strategy progress/result and make adjustments to make perfect fit for your situation.

Third, you need to study, research and experiment a lot on strategy. Innovation does not come out of nowhere. Hear other companies stories, your subordinate's experiences, and feedback. Have a meeting with your directs to make better. Work with PMs and developers to understand their views and progress. Continuous improvement is part of strategy effort. And make sure you're doing the right thing, not doing things right.

And lastly, once you're clear on strategy, you should believe in it and execute the hell out it.    

I'll write operational and people leadership in following post.

Systems thinking and test automation

Today, I like to share some of thoughts around test automation related to Systems Thinking.

You as an SDET, it is very important to make your team understand what your test automation strategy is. When it is clear, your team can support you and align their work with your automation strategy.

OK. let's start. To be sure, I am specifically writing about functional test automation here. And, in fact, I consider test automation as regression tool not as testing activity. I like James Bach's term 'checking' to describe automation. However, I do believe that test automation as regression tool has great value.

My test automation strategy is based on 'Systems Thinking'. Dr. Russel Ackoff came up with this idea and there there has been lots of conference around this.  I posted his video on my blog, so you can watch his video.

There are two principles that I like to mention from his talk.

First, when it comes to the system, as a whole, it creates a unique behavior or property that none of its parts have.

Second, System's unique behavior or property is created by the interaction among individual components(parts) NOT by sum of individual parts behavior.

As Dr. Ackoff explained, let me take a CAR for example.
What is CAR's unique behavior/property? That is carrying you(person) one place to another. It is only unique as a system. Let's take a look at each parts of the car. Engine, transmission, door, battery and so on. None of them can carry you one place to another. But that unique property cannot be created without parts. More specifically, the interactions between parts creates that property. You can easily understand that simply having the parts without interaction is not contributing to produce a unique system's property.

The point I'm applying to my test automation strategy is the INTERACTION.

Here is one more car example. Let's say, your car stopped while you're driving. And let's assume, it is because the engine is stopped. Yes. it is obvious that broken engine caused car to stop. But if you think a slight different perspective, car actually stopped because no force or thrust was delivered to the wheel. In other words, the interaction was broken.

And now, let's say your car is stopped, but you don't know what caused it. What do modern car manufactures do to identify issue?  Of course they are using some sort of sensors and chips. And it alerts when any sensor detects something that is not normal or when certain criteria they set is not met. And then mechanics are going after actual root causes based on that information.

I think SDETs job is to put sensors all over the code using automation. Sensors that detects all the small functionalities  that comes from interaction between components and modules. Sensors that detects component/sub-component behaviors that comes from interactions of classes and methods (unit tests - this is normally written by dev).

What this thorough automation provides a simple value. Make easy/straight bugs to be found easily and hard/mysterious bugs to be found hard.  Sounds simple, but this will give tremendous trust to your work and your test automation.

Testers are like caddies in some sense

Today, I like to describe testers with some odd(?) metaphor.

I think testers are like caddies in some sense. More specifically, the guy who walks with Tiger Woods or Phil Mickelson on the field of Professional Golf Tournament. The caddies of PGA tours. 

Why? I'm going to explain why.. 

First off, Caddy is an occupation. Golf player is an occupation as well. Caddie might not get attention as much as the players, but still he/she plays very important part of professional golf tournament. It sounds like testers in software development, doesn't it?

Here is one thought on being testers. I've seen so many testers who complains a lot about the company not treating them as important as developers. Being treated like second class citizens or getting paid less. Again it's an occupation. You choose to do it. If you care about being treated like developers, then you should be a developer. If you do not enjoy testing, you should not do testing. Perception is really hard to change. If you care what others' think about you, well..I don't know. I think you should be yourself.

 However, I truly respect those testers who has been fighting for the importance of testing in software industry over the years.  

OK.. let's start.

Caddies understand the game of golf as much as players on the field. They know general information about the game like the golf course, golf rules, difficulties of each holes, distance of each hole, location of hazard, trees around the hole, bunkers and roughs. And they also know what affects each shot. They understand things that affects the shot like winds, uphill, down hill, rough, bunker, reading greens, soft or hard green, wet grass or dry grass and etc. Coming back to testers. They should also know about general information about the product/service they're testing. What kind of product it is (web, mobile, desktop and etc.), who are the users, what kind of technology the team using, what features are important for the business, what development process is used (agile, kanban or waterfall) and so on. And they also understand what affects the developments like software architecture, code review, check-in process, bug triage, regular build process, regression strategy, reporting mechanism and so on. This is really basic stuff. Good caddies and good testers take seriously about the basics of their work.

Caddies provide good information for each shot players make. You may have seen this on TV. PGA golf players always discusses with a caddy before he/she makes shot. Caddies and players both check the distance, hole location, hole surroundings,  ball location, wind, grass, slope, curves, trees and so on. That brief conversation before the shot is the result of quick analysis. Testers are similar in that sense. Test plan/strategy should show the analysis of the feature development. Whether it is formal or informal, testers should analyze the feature that is going to be developed and discuss with developers. Here are some examples. User impact, complexity of the component, impact to existing code, risk, timeline, algorithms or implementation plan, testability and so on. This requires a lot of studying and experiments. Testers should continuously learn how to do better analysis and improve.

Caddies know players more than players know themselves. Professional caddies know his/her players so well. Strength and weakness. Ups and downs of emotion. Distance of each club. Fade and hook tendencies. And they know how to communicate with those things with players. Sometimes caddies insist on certain club because they can predict what his/her player will hit. This is really important part for testers as well. Testing is not just having a list of things and execute them. Knowing the context is the key. That's why I love the context driven school guys. Testing never starts from nothing at the beginning. Product has history. It has business environment and developers. Testers should try to understand developers' coding style and thought pattern. It really requires a close attention to each developer, but it really pays off. Thought patterns are amazingly accurate to predict. Sometimes you can even guess what a developer would  implement this particular class or functions. Bug trends as well. Who makes what kind of bugs can be understood by testers.

Well. I'll stop here.. But there is actually a big difference between caddies and testers. Actual testing part. Testing performed by testers are not there in caddie's work. I'll address that later..

               

A good test automation framework - Part 2

I previously wrote about test automation framework with details related to coding. (I would call it part 1)

Today, I like to mention a little bit more technical explanations of a good test automation framework.

I guess you (test engineers) have worked on test automation framework that was written by someone else and also worked on designing it from scratch. Have you feel some fuzzy or itchy feeling about the test automation you're working on? Something seems to be wrong but hard to explain? Does it take hours just to add a new test case? Do you feel test execution is heavy? I have the same feeling as well. I've been seriously thinking about that and I came up with following technical details. I don't think it is just because of  different coding style nor different thinking process. Hopefully, this blog actually scratch those itchy feelings. :)

1. Goal setting: This is about initial design of test automation framework. You need to set two goals. If you can satisfy both goals from framework user (testers average 2-3 years on automation), I would say your design is successful. The first goal is the framework should be easy to understand, easy to use and easy to implement. And the second goal is you need to design test framework in a way that volume of the code increases but complexity of the code does not. This seems to general and yes it depends on the context of your application. But if you write test automation framework from scratch, please do not simply start with drawing diagrams. Think about consumer of your framework. And also how the framework will grow and not become collection of giant and complex code base.

2. Do not re-factor test execution step code. This is for the user of test framework. I mentioned from my first blog to be cautious about it. Now I change my mind. Again, Do not re-factor test executions step code.  It really has many side effects.

First, your coding is restricted by re-factored code. You framework API provide several functionality and normally those are the components that supports your test execution steps. Your test execution code should be just mix and match of those components. Once you re-factor the execution steps, you are losing your flexibility of test case implementation. It's really silly to be proud of test execution step code  to be one function and it takes 6-7 parameters. Also, you lose the readability of test execution steps. Now how can you replace your test case document with your code? It's so hard to read. DRY (Do not Repeat Yourself) does not applied to test execution step code.

Second, you are adding unnecessary complexity by re-factoring code. I have a good metaphor to back up this argument. Let's say you want to test a car and you have test system that control every part of the car. But that system provides only one button for control. With one button click you need to come up with a way to start the engine, step on gas, break, changing gear, steering wheel control, lights, radio and etc. How would you control that with one button. Yes, you need to come up with some sort of Morse Code to differentiate the operations. "bip bip wait wait bippppp bip wait bippppp" that means turn the head light on. How about you have buttons for each operations. Engine button, break button, accelerator button, light button, changing gear button and etc.

Third, you don't gain much anyways. In test automation framework, what part of the codes are most frequently and most likely added and modified? It is test executions step code. Test cases are keep added and modified as the application grows sprint by sprint. Let's say you have 8 distinct test case which exercise different aspects of the application. Let's say test case 1 and 2 have 70% of the same code. And test case 2 and 3 share 60% of code a slight different way. Test case 3 and 4 shares 65% of code but also slightly different way and so on. It looks like a lot of codes are repeating. Now you start re-factoring. I can tell you; it not easy. What do you end up having? Excessive object composition or excessive class hierarchy of code. Or. you can see the same name of re-factored code with less parameters all over the place. Base class or helpers. Now look at your code after re-factoring. Lines of code you save is not much. And guess what new test case comes in and you need to re-re-factor your code.

Forth, it's not uniformly re-factored and not re-factored by responsibility or business logic. Most of test execution step code re-factoring happens due to repeating code. Not like automation framework code. Framework code is re-factored to achieve Open-Close principle or component decomposition. And it is normally uniformly re-factored because it has its own purpose. Refactoring is not simply putting repeating code in one place. Refactoring makes the code more maintainable and extensible. Existing test execution step changes in several different way, I tell you I've experienced..... It's hell.

Fifth, putting repeating code to base class causes weak cohesion and giant class (3000 lines of code kind of giant class). I'll explain this more in details later. I personally do not recommend people to use TestBase class because once you open the door, it's open for any code to go in. It stars with really essential pieces of code for bootstrap. And later add some more helpers. And later people start putting repeating test execution step code. And people start to add another base class that inherit from TestBse class. Adding if statements and switch statements. Where is the cohesion? What does this base class specifically do? Everything!


A good test automation framework continues....