Today, I like to talk about applying risk analysis in software testing
By the way, risk analysis is quite a big word. It is not specific to software industry, and depending on what's you're looking for you can get very different results.
So, let me define what the purpose of conducting risk analysis in software testing is. My 2 cents definition would be "activity of understanding the impact of malfunctioning of the application." And the severity of the impact of each malfunction scenario will determine priority and intensity of the testing.
One thing I like to call out is that application malfunctioning is not only about use cases failures defined in spec. There are always implicit requirements that is not written in spec.
But normally, I start with use cases defined in spec.
Go through each use cases and find out the impact of malfunction. For example, you are testing login page. There are several use cases for this.
- Login with activated user with valid user name and password
- Login with invalid username
- Login with invalid password
- Login with not activated user name and password.
- Forgot password
- Register link
Now let's define risk and impact of each malfunction
- Failed to login with activated user name and password (high impact)
- Login success with invalid user name(high impact)
- Login success with invalid password (high impact)
- Login success with non-activated username and password (medium impact)
- Forgot password not working (medium impact)
- Register link is broken (high impact)
This is just an example of this. You can argue with different level of impact on each malfunction. But you have an idea about this approach. Here is another post for checking business impact and fault likelihood. Let's go into more details
- Login failure message not specific enough (medium impact)
- Already logged user going to login page and still require login (medium impact)
- Login process taking too long (medium)
- Logout link does not work (high)
- Login username and password text field is too small (low)
- Password text field showing the password not dots (medium)
- on and on.....
Now you have three different buckets of test cases (high-medium-low). And then you decide what to test first and how intensively test those test cases.
Sunday, May 29, 2011
Sunday, May 8, 2011
Skills required for a test engineer
Today, I like to talk about the skills need for a test engineer.
1. Inference
This is the most important skill you need for being a test engineer. This make software testing more like Art than Science. It requires creativity, intuition, experience, analytical reasoning and extra. What can you infer from requirements or spec? I would say that spec is explicit requirements for the project. However there are always implicit requirements for the given project. For a simple example, "how would you test a soda pop vending machine?" Beside putting money and getting the right soda pop, there are a lot of things you need to consider. Like, how to check the temperature of the soda pop, how to make sure to notify office when power is out on vending machine, how to notify office when soda pop is out, how to make vending machine energy saving, on and on and on.
2. Prioritization
This the second most important skill you need. After you come up with all the possible test cases for the given project, you should be able to put each test case into priority bucket. Risk analysis would be a good tool for you. Based on the risk (what and how big is the impact of this test case failing), you can categorize all the test cases in to priority bucket. As we all know, we have limited time and budget, we need to make conscious decision not to test based on the risk.
3. Troubleshooting
This is another core skill for test engineer. Test engineer should be very good at finding root cause of the problem. Can you eliminate the noise related to the problem? Can you find out the pattern from certain condition? Time is clicking. When you have production issue, how good your troubleshooting skill is directly affecting company's loss
4. Simple and robust coding skill
Now days, software is very complex and large scale. We need to use program to test program. There are a lot of testing tools that helps testing. But fundamentally, there is no other test team that test your testing application. You need to beware of the fact when you write automation code. Besides unit testing, you need to be able to write simple and robust code. Less if statements in your code, less switch statements, avoiding nested if statements. These are all helping you not to make automation bugs.
5. Big picture in mind
If you are a test engineer, you should have better understanding of the system than developer. Developers are normally focused on what they are developing. You need to have end to end mind, system overall operation. Help developer understand how the feature they're developing fits with existing application.
6. Multi-tasking or rapid switching one task to another
Multi-tasking/switching one task to another is another skill you need. People notify testers for so many different problems they're having. Lab environment, live production issue, project testing,and etc. You normally handle several projects at a time and you should be ready for handling production issue, support issue and etc. This can be very frustrating if you are not used to it.
1. Inference
This is the most important skill you need for being a test engineer. This make software testing more like Art than Science. It requires creativity, intuition, experience, analytical reasoning and extra. What can you infer from requirements or spec? I would say that spec is explicit requirements for the project. However there are always implicit requirements for the given project. For a simple example, "how would you test a soda pop vending machine?" Beside putting money and getting the right soda pop, there are a lot of things you need to consider. Like, how to check the temperature of the soda pop, how to make sure to notify office when power is out on vending machine, how to notify office when soda pop is out, how to make vending machine energy saving, on and on and on.
2. Prioritization
This the second most important skill you need. After you come up with all the possible test cases for the given project, you should be able to put each test case into priority bucket. Risk analysis would be a good tool for you. Based on the risk (what and how big is the impact of this test case failing), you can categorize all the test cases in to priority bucket. As we all know, we have limited time and budget, we need to make conscious decision not to test based on the risk.
3. Troubleshooting
This is another core skill for test engineer. Test engineer should be very good at finding root cause of the problem. Can you eliminate the noise related to the problem? Can you find out the pattern from certain condition? Time is clicking. When you have production issue, how good your troubleshooting skill is directly affecting company's loss
4. Simple and robust coding skill
Now days, software is very complex and large scale. We need to use program to test program. There are a lot of testing tools that helps testing. But fundamentally, there is no other test team that test your testing application. You need to beware of the fact when you write automation code. Besides unit testing, you need to be able to write simple and robust code. Less if statements in your code, less switch statements, avoiding nested if statements. These are all helping you not to make automation bugs.
5. Big picture in mind
If you are a test engineer, you should have better understanding of the system than developer. Developers are normally focused on what they are developing. You need to have end to end mind, system overall operation. Help developer understand how the feature they're developing fits with existing application.
6. Multi-tasking or rapid switching one task to another
Multi-tasking/switching one task to another is another skill you need. People notify testers for so many different problems they're having. Lab environment, live production issue, project testing,and etc. You normally handle several projects at a time and you should be ready for handling production issue, support issue and etc. This can be very frustrating if you are not used to it.
Subscribe to:
Posts (Atom)