5 minute software testing tips

Today, for the first time, I create a video about software testing on Youtube!
I don't know how this will work out, but hey.. why not.

5-minutes software testing tips.. :)
Will post more and more~~

Dealing with emotion, people, and yourself in software testing

Today, I like to write about stuff around work place and somewhat related to testing.

We love testing! Don't you love doing testing all day long? I do. I really do. Well, but since it is a job, we all have crappy days and good days. Have you experience these? Developers resolve a bug by-design even if you mentioned all the details of the issue? Someone keeps bouncing bug back to you even if that's not your issue? Someone blames you on something via email and cc his manager, your manager, and some other important people?  You made a huge mistake, but your mistake gone away due to other big issue? You are just totally lost and simply take all the blame? Find a huge bug by accident? You've been keeping test automation or test case really well, but just one time when your manager sees your work, everything went wrong? Troubleshoot issue in a few second and find the root cause? Someone you believe not a very good developer/PM, but they save you from disaster? Give or take some condescending comments? You will be so happy if that guy does not work here. You'll be so happy if we don't need to do this? You get so angry about peer performance review?

I think we're so human being. Don't you think? It's not only you. It's everybody.. Right?
Here are things I think you might consider when you're dealing with people and emotion.

Do not criticize anyone publicly
Nobody likes criticism. Even if you're 100% absolutely right, do not criticize anyone in public. The other person will never appreciate your criticism. He will become very defensive. He'll try to find justification for himself and other to believe. And one day he will criticize you in public in return. I had several incidents around this. I had someone criticized me and cc'ed my manager or group alias. Oh man. It was hard to take. I came up with 100 reasons why that happens to justify the issue to myself. But I also have criticized some people with email. I thought I was cool. I email back with my awesome criticism to senior engineer's original email and yes REPLY ALL. I thought I had confidence in myself and opinions about things.. Like philosophy in testing. As a result, I've got very bad review that year. I can sense people did not like me when I raise my voice to debate. I'm not saying you need to be nice to other people to get good review. I'm saying you being confident/strongly opinionated person and you criticizing other people publicly are different thing. You can be confident and have strong opinion about everything without hurting others' feeling. When you receive dumbest email or unreasonable claim or blame from others, think how you can point out the issue without hurting others' feeling.

Leave your ego under your desk when you argue over a bug
I still get surprised by how people interpret and react to bugs. Some fight over the priority. Some doubt about severity of the bug. I've fought over bugs many many times. I have used "user perspective" cards or "terrible potential risk" cards to convince people. And yeah... I argued sometimes just for my EGO. I respectfully(?) respond to "You tester, stop breaking things!" with "I don't break things. I find things YOU broke!" But I think bugs are not our baby. It's a statement that indicate the issue which may or may not be serious impact to product/customers. It can definitely be interpreted differently to different people. Finding a bug will be our job, but how to react to the bug will be project team's responsibilities. Project management/business perspective has to be understood and development impact or effort should be understood as well. When I talk about the bug, I always try to remember what James Bach said in his talk. "Hey developer, don't think bugs as your mistake. We, testers, do not fight for bug to criticize your work. We want you to shine. It's like mother saying 'you have mustard on your mouth' when you leave the house for a date. We care about you. We want you to wipe out mustard on your mouth and shine." If you leave your ego under the desk, discuss facts around the issue and try to understand other people's perspective, it gets much smoother and easy.

Think big. Think big.
When I was a junior SDET, I was really scared of making mistakes. I get offended by people keep pointing out my mistake. I made some  automation framework changes to make it much more effective. Overall, it was great change, but it was not perfect. There is always someone point out flaws and criticize my work. I was a bit discourage to try new initiatives or change. Sometimes I got to the point "why bother." But as I gain more experience, I started to ignore those criticism. I've done many more presentations even if there are always people find flaws in my idea. I've brought so many new successful initiatives at my work even if there are people in doubts. Dr. Russell Ackoff, who is my favorite modern theorist in Systems thinking, mentioned about errors of commission and errors of omission. Error of commission is a mistake that consists of doing something wrong. Error of omission is a mistake that consists of not doing something you should have done. At work, errors of commission are visible, but errors of omission are not visible at all. But I think we should more concern about errors of omission. We should think big. Do not worry too much about small mistake you make when you doing great work. And don't worry about people's doubt on your idea if you believe it would work. We are living in this advanced and modern world, not because of people who are in doubt, but because of people who challenged status quo.

     

How to think while we're testing

Today, I like to write about something I read recently and address how it might impact our testing work.

First, I want to talk a little bit about how our brain works. Well.. Yes.. I'm no expert in neuroscience and my knowledge is limited to couple of books I read and some Googling. For the reference, what I'm about to explain came from this book, "Thinking, Fast and Slow" by Daniel Kahneman.

I wrote a blog post about assumptions and how it affects testing. link And I used James Bach's calculator testing example. I want to go a little deeper than explaining how assumptions affects testing. I want to understand why we make assumptions.

According to the book, "Thinking, Fast and Slow", humans have two different type of thinking systems. ....(BTW, I'm not saying contents of this book are all 100 % correct. It is based on credible research, and I don't think he won Nobel prize by providing BS) One is called "System 1" and the other one is "System 2". System 1 is so-called heuristic brain and System 2 is so-called rational brain. System 1 is quick, responsive, and sometimes fallible. System 2 is slow, lazy and requires energy and focus.

Here are some examples. System 1 is used when you're doing things like calculating 2+2, driving home from work (you've done many times before), brushing teeth, understand facial expression, instantiating array or string in your most comfortable programming language and etc. Basically, you've known and done so much that it does not require you to think much.

And System 2 is used when you're doing things like calculating 27*69, driving on not familiar road (without GPS), solving puzzles, writing letters, design the architecture of network system and etc. You can imagine these activities require your attention, deriving from some basic knowledge, connecting several dots of your knowledge (active interaction among neurons).       

And usage of system 1 and system 2 is very optimized. When you see 27*69 your system 1 recognize this cannot be done and ask system 2 to take over to process that. And after repetitive usage of task from System 2, System 1 can take some of the task. For example, learning chess game requires System 2 at the beginning, but once you are really good at chess game, lots of move you make can be done through system 1.

So why am I talking about this?
The calculator problem (from this blog) is basically a check for the consequence of fallible System 1. If System 1 process the problem and reaches conclusion, the problem will not move to System 2 unless you're forcing it to. And we, as test engineers, need to practice pushing our thought to System 2 while we're working. Being critical all the time is very exhausting. It requires discipline and lots of, lots of energy.

Sometimes, being on the same project for so long can prevent you from finding important bugs. Your System 1 does not give chance to System 2 to process thoughts without letting you know. You are a domain experts. You've seen lots of bugs along the way. But it can be dangerous.

So how do we force ourselves to use System 2?
I have a couple of suggestions

1. Call out your assumptions. 
It's an interesting check point. Let your brain to process flow of information about the application you're about to test first. You will think about use cases, testing strategies, test cases, test executions, report and etc. And then going over your documents or thought process again with critical mind. Try to find any assumptions you made. You're already using System 2.  

2. Focus on how rather than what.
Build your System 1 with "I need to think how to test this first, not what to test." Yes, it is important that you execute various categories of testing. Functional testing, localization testing, usability testing, load testing, performance testing, test automation, BVT/Smoke test and etc. are all important. And it seems very structured. However, it restricts you thought process to be in that category. Think about what's important, what's critical, what's necessary and how to address those with my testing. You make good use of System 2.  It will help you test is more effectively. And it will be just too fun. 

3. Be a critical thinker.
Here are three question.
"Huh?", "Really?". "So..." http://www.youtube.com/watch?feature=player_embedded&v=8TX6rzz60xQ

Problematic nature of software development

Interestingly, the nature of software development process imposes a very unique difficulty compared to other engineering process. And it is generally overlooked. That is incremental engineering effort on previous product. In other words, ‘versioning’ And I believe this particular fact actually causes lots of trouble to software development teams. Throughout years of experience, software project teams have come up with several good solution for these. I’ll explain how this trouble leads to those solutions later point. There are many books and articles about building a good software product from architectural perspective. And often times we can see that authors use metaphors like building skyscrapers and bridges to explain their points. However, building software is still significantly different from building skyscrapers or bridges even if we consider planning, designing, engineering effort, duration of work and delivery are conceptually the same. If I were to use metaphors to explain software development process, I will describe it as following with a little bit of exaggeration to point out the difference. ‘Software development is like finishing Empire State Building today and adding 50 more stories two years later. Software development is delivering Golden Gate Bridge today and adding 16 more lanes next year.’ Do you see the trouble here? Let me explain with some more specific example. Let’s say Boeing delivered ‘787 Dreamliner’ airplane a few years ago. And they are planning to build ‘797 Wright Brothers’ as the next generation of the product line. The requirements for this airplane is basically adding 250 more seats inside airplane. But Boeing engineers only allow to build 797 by physically modifying existing 787 airplane. What has to be changed? I don’t know much about plane, but I can imagine that they have to decide if 797 will be fatter or longer, which changes aerodynamics. It will also affects the size of wings and horizontal/vertical stabilizer. It will changes the weight of the airplane, so engine has to be changed. Then it might require extra gas tank. It requires more power supply and oxygen. It may require longer runway for taking off and landing. They also need to consider hitting by lightening or dealing with turbulence. The need to consider not only the airplane structure but also travelers. Extra bathrooms, emergency exits, flight attendant station have to be changed. Seating for first class and business class might be changed as well. I guess we can go on and on. And remember we’re modifying existing 787 physically. This is the nature of software development. There is no hard end product like buildings or bridges in software world. It is just the nature and we'll continue to do this. Reasonably small requirement for business can impact engineering effort significantly. Adding or modifying existing feature exposes various risks in the software product. Some feature may not be needed any more or doesn’t make sense any more, but it’s not easy to take out those dead code or features. Let’s think about how we can minimize these risk. On the next blog. :)

Testers! Earn respect, not demand

I’ve seen many cases where testers complains about how they are treated in the project team. “We are NOT a second-class citizen in this project team!” “They don’t know how difficult it is to test software!” “Testing is not an activity of try and see if it works!” Sometimes there are some conflict between QA/test team and development team. QA team present estimate for testing some feature in the meeting and get offended when development team asks, “Would it take that long? Can't you automate that?” Or QA team shows all the bugs they found and questioning quality of the code.

I believe some sort of stereotype or perception actually comes with occupation. Maybe history of software testing discipline was not well received from the beginning. The point that I’m trying to make here is that we should not blame perception about the occupation. What matters is how you perform your job. And actually you can take advantage of the perception and use it for your own benefit.

Let me give you an example. What is your perception about a gardener? You would think that a person who works as a gardener mostly is not very educated, and it’s kind of work you don’t do, not because you cannot do but because you don’t have time to do or don’t want to do. I don’t think you would respect your gardener for the occupation, but you would respect him as a human being. You still have perception about the occupation.

Let’s say one day your gardener came to you and provide you following information. He measured PH of your garden soil of front and back yard and asks you to let him apply some nitrogen to prevent clover or lawn fungus. And provide you exact dose of nitrogen he will apply based on square footage of your yard. He also provides you the information about the growth of each garden tree along with the climate of your city, and informs you that the branch of front yard tree will block certain percentage of sunlight to your living room in next summer. And he shows 3D pictures of possible garden looks with some flower beds, new trees or relocation of your existing trees and asks you if you would like to try. He always come to your house on time and do the best garden work. He cleans up every single corner of the house and always nicely fold your water hose and sweep the front side of your house. Your garden grass, flowers and trees are all healthy and well maintained, and your garden looks the best in your neighbourhood. People visiting your house always mentions how nice your garden is.

Now, would you respect his passion and professionalism as a gardener? Would you continue to have him maintain your garden? Would you recommend this guy? Would you stand when someone bad mouth about gardener being lazy and not educated?

Respect is not something you demand and get. You EARN respect from others. What is important in your testing job is doing your work with your passion and professionalism. You can differentiate yourself from others by continuously improving yourself and trying innovative approach. When you do that, perception of the QA role become meaningless. People will respect your passion and professionalism. They already have low expectation, but you exceeded far above of their expectation. Guess who gets the promotion?