Sunday, May 29, 2011

Applying Risk Analysis in test planning

Today, I like to talk about applying risk analysis in software testing

By the way, risk analysis is quite a big word. It is not specific to software industry, and depending on what's you're looking for you can get very different results.

So, let me define what the purpose of conducting risk analysis in software testing is. My 2 cents definition would be "activity of understanding the impact of malfunctioning of the application." And the severity of the impact of each malfunction scenario will determine priority and intensity of the testing.

One thing I like to call out is that application malfunctioning is not only about use cases failures defined in spec. There are always implicit requirements that is not written in spec.

But normally, I start with use cases defined in spec.
Go through each use cases and find out the impact of malfunction. For example, you are testing login page. There are several use cases for this.

- Login with activated user with valid user name and password
- Login with invalid username
- Login with invalid password
- Login with not activated user name and password.
- Forgot password
- Register link

Now let's define risk and impact of each malfunction
- Failed to login with activated user name and password (high impact)
- Login success with invalid user name(high impact)
- Login success with invalid password (high impact)
- Login success with non-activated username and password (medium impact)
- Forgot password not working (medium impact)
- Register link is broken (high impact)

This is just an example of this. You can argue with different level of impact on each malfunction. But you have an idea about this approach. Here is another post for checking business impact and fault likelihood.  Let's go into more details
- Login failure message not specific enough (medium impact)
- Already logged user going to login page and still require login (medium impact)
- Login process taking too long (medium)
- Logout link does not work (high)
- Login username and password text field is too small (low)
- Password text field showing the password not dots (medium)
- on and on.....

Now you have three different buckets of test cases (high-medium-low). And then you decide what to test first and how intensively test those test cases.

2 comments:

Anonymous said...

This is a well explained way to think about test cases and the priorization of those, but Risk Management is more of the sort like this one here:

We found a late defect on our test environment. Deployment is planned for tomorrow, now lets find out what is the impact of the required fix for the rest of the components. Can we risk this change with minimal regression testing or is it a change impacting many parts of the system.

Risk Management is about making decisions on a higher level. Do we risk to get an angry customer because we have another delay or do we risk to get an angry customer because he gets a broken release that stops him from working...

Cheers
TOJZ

Jae-jin said...

Would risk management and risk analysis serve the same purpose?

I have not given much thought on risk management. But I think those are different.