Dynamic risk based testing helps enhance quality of solutions

©Daniel Kaesler via 123RF

The challenge with this approach is that it is typically static in nature and does not allow for risk to be reassessed based on real-time test results. The upshot is that once risk is defined, it is considered to be the same for the testing lifecycle of the product - something that may not necessarily be true. The quality of the different modules of an application under development tend to change according to a number of variables, which means that test cases may become inappropriate.

The value of testing

In order to bring in a factor of continuous improvement and maximise the effectiveness of the Quality Assurance process, it is critical to develop a more dynamic approach to RBT. Risk factors should be periodically reassessed based on the actual results of the test cases being executed to ensure the optimal set of test cases continue to be applied throughout an application's lifecycle.

The value of testing is something that is increasingly well understood, as it has been demonstrated to have a direct impact on brand value, market share and revenue generation. In addition, testing provides not only quality assurance but business assurance too and ultimately results in operational benefits such as shorter time frames for project completion and reduced costs. However, since time is often limited, it is essential to ensure that the right amount of testing is performed at an optimal cost, with high levels of quality. Traditional approaches such as RBT need to be enhanced and re-engineered with a focus on insight driven optimisation, to address dynamic, variable and changing application quality and stability across modules and releases; in short, the Quality Assurance process needs to factor in a quantum of intelligence. The concept of Dynamic RBT has emerged to address this challenge.

Key parameters for assessing risk

Dynamic RBT considers two key parameters for assessing risk. The first, the business impact, is based on the criticality of a component or module in the application and is identified based on application-specific knowledge. This is the traditional basis for all RBT. The second factor is the probability of failure, which is where the dynamic component comes in, as this is continuously reassessed based on actual test results. Each of these risk factors has three identified levels: high, medium and low. This provides a total of nine different risk profiles, instead of just three, to which test cases can be assigned.

Risk appetite varies across different applications, modules and releases, depending on the phase of the project or the type of release, and is considered when selecting the suite of test cases to be executed. Additionally, the decision of when to conclude the testing phase can be dynamically determined during execution, depending on the pass rate of high-risk test cases as well as risk appetite.

Selecting the optimal set of test cases

A clear methodology for the implementation of dynamic RBT can be identified. Firstly, it is essential to assess the business impact of a test case on the application in question. Next, the probability of failure is assessed by analysing the history of test execution results, based on the extent of actual failures observed. This probability should be re-assessed periodically so that the influence of actual quality, revealed via the failures, is reflected in the risk of a particular test case.

Based on the level of these two factors, the risk profile of a test case can be identified, and the optimal set of test cases selected. After the optimal test suite is selected, a second level of optimisation can be applied based on the actual quality of the build being tested, enabling dynamic decision-making to be implemented as to when to conclude testing.

Automating the testing environment

Dynamic RBT incorporates set roles for defining business rules and the generation of an optimal set of test cases and a variety of methods that help to automate process within the testing environment. These include automated risk profile re-assessment based on real-time test results and automated generation of an optimised set of test cases based on defined rules. In addition, it includes methods for decision-making on test execution completion based on a defined set of rules, as well as integration with a test management system, to access the requirement details, test case details and uploading of the generated set of test cases.

This approach has a number of key benefits, including reduced regression-specific defects as a result of period reassessment of risk profiles, improved software quality and an enhanced regression test suite that is well-maintained and up to date with requirements.

Ultimately, the quality of applications relies on adequate and optimised testing practices. Dynamic risk assessment and RBT enable organisations to automate the selection of the right test cases, reducing manual effort and subjective error. The resulting benefits include savings in terms of both time and money, as well as reduced cycle times which assists with improving the agility and speed with which releases can be brought to market. Dynamic RBT in effect creates a constant feedback mechanism for continuous improvement, for on-going quality assurance and operational efficiency.