Modern Test Strategies

TestUnityJanuary 10, 20181 Comment0263

A test strategy is the guidelines to determine the road map to follow for the test types and cases defined in the testing plan. It is a holistic view to achieve highest possible quality at the end result of a project. A typical modern test strategy deals with –

Test plans
Test objectives,
Testing approach
Executing environment
Automation test strategy and tools
Prioritization and risk analysis
Agile test strategy

From broader perspective test strategy can be viewed from three angles in a software test life cycle (STLC) –

Test Devlopment
Delivery
Overall factors related to entire Test Strategy

Is the test strategy template final answer?

Every company follows their own test strategy template. But it is obviously not a good approach to blindly follow a standard. Every project and client has their own requirements. Moreover, every project may deal with different challenges. Hence, proper test strategy should act as a contingency plan. Let’s consider the two scenarios –

Let say, you are going to test a website with rich UI and an external web service which is maintained by some external vendor. Here you are going to deal with two entirely different components. Hence the test strategies will be different. For UI part it could be a blend of manual and automation testing. Whereas for web service part you need to first check the overall functionality of the service and then you can set the automation process. Now, when it comes to system testing, scenarios must be determined based on customers’ perspective and not developers.
Another common scenario is the mismatch of budget and time when dealing with the quality issue. Every testing project has its pre-defined fixed budget and timeline. But the timeline like start date can skip due to development issue, resource crunch or environment issues. How will you deal in such jinxed situation despite a pre-defined testing strategy? Will you compromise the budget or timeline or quality of testing? In each case, you are going to lose the game. Hence, better not to scapegoat yourself and define the proper strategy on the go to overcome the issue.

Modern test strategy mostly deals with prioritization and risk analysis and not just waterfall model of STLC.

Common test strategy approaches followed in Waterfall Model

Quality based strategy

In this strategy, stress is given on the right quality level of the products or services. Here all testing areas may not be covered. This means you can cut down testing in less important areas. But here one should consider the relationship between qualities with financial aspects of a business. Although focusing on quality can dramatically improve the technical performance of a product but on the other side, it may fail to address many fundamental issues that may impact a company’s long-term marketing and financial performance.

For example Y2K problem. Here, a product may fail to work on 29th February 2000 which is still acceptable. Although if it would fail to process the order after 1st Jan 2000 that was not acceptable.

Priority-based strategy

Here testing is done for the most important functions and its critical in overall business. Once you find out the worst areas of development and design, you can find out more serious defects. When the possibility of potential business loss is identified through such strategy, management will be motivated to postpone the release. This will allow more testing time. The strategy is based on risk analysis where factors are scaled from 1 to 5 and the highest point is given to the worst factor.

The factors are defined based on the possible defect areas –

Critical areas
Visible areas
Usage frequency
Complex areas of the software/development
Changed areas which can be defected generator in future
Impacts due to use of new technology and methods
Resource quality and numbers
Time criticality
Localization

In each areas probability of failure is determined with calculation.

Cutting down the cost of automation

Although you can not 100% automate overall testing but automation testing is a good solution if you want to make testing cheaper. The possible risk lies here – if you are adopting automation first time and skilled resources.

Define the entry criteria for testing

To reduce the testing burden sometimes it is required to define few entry criteria of the product from the quality perspective. Entry criteria can include –

Minimum reviews having been done before starting of testing
Minimum level of test coverage performed in unit testing
Certain level of product reliability

Test strategy followed in Agile Framework

Agile model demands stepwise releases. Hence the best strategy for an agile model is to test the most important functionalities of the application while delaying the others. This helps in identifying the defect-prone areas of a product. Moreover, agile model testing is performed in short sprints. Hence, strategies are focused on few requirements and user stories. Below diagram shows the test quadrants for agile testing where for each quadrant strategy must be made based on 5W1H criteria, i.e. WHY, WHO, WHAT, WHEN, WHERE and HOW. One of the popular agile testing frameworks used in the market is ATF.

Test strategy for DevOps

DevOps is an extension of Agile methodology which combines development and operations within the same cycle.In agile development it is continuous integration between development and QA operation, in DevOps, the code is deployed in the production after QA operation for each sprint. Hence, every action in this process needs to be fully automated to stress upon code quality. Same applies to the testing operation and test cases must be automated and achieve 100% code coverage.QA must assure an automated deployment. Hence a more matured automated testing framework is applied for DevOps scenario. Running all the regression test cases is not the strategy applicable for DevOps. Rather automated cross-platform and cross-browser testing for important scenarios are stressed upon. The overall process demands a close coordination between QA and development team. Prompt and proper reporting on the test results is essential to ensure overall quality delivery.

Automation Test strategy

One of the core testing methodology for today’s agile development is automation testing. Few thumb rules are followed in test automation like –

Prevention rather than defect detection
Quality over Quantity
Quick feedback
Frequent regression testing through automation which may involve Smoke regression pack, functional pack, E2E pack
Automated Unit test
Automated integration test
Application test

The following are the building blocks of automation test strategy

- Selection of right automation tool for application – It is one of key to the success of any automation testing.
- Short listing of test cases – Through analysis of test plan is essential to shortlist the test cases which are most effective for automation.
- Setting up test environment – This is another building block to success. The rule of thumb is the test environment configuration must match with stage and production environment.
- Maximize the use of automation test scripts – This will lower the cost of testing

Clear instruction on execution and management of test cases

Focus on result – Outcome of each automation test case should be measurable and matrix-based to qualify the standard of the automation process.
Use of automation maturity model – There are different maturity models used to revamp the test strategy like –

Initial Phase
Managed Phase
Define Phase
Measured Phase
Optimize Phase

Select the best automation framework for the project – Once the test cases are finalized for automation testing, the next important step is to select the right fit from the common automation testing framework. It could be any from the below list based on the project requirement –

Data Driven Framework
Keyword driven framework
Modular based framework
Hybrid framework

Mobile Testing Strategy

Mobile application testing needs more intensive attention as it runs on multiple OS, multiple platforms and on versatile mobile features like screen resolutions, screen sizes, OS versions, country locations and much more. To cover up all the criteria either mobile testing lab should be established otherwise testing in a cloud environment is the best idea. A cloud environment testing platform such as Crowd Source enables an application to be tested by numerous testers across the globe. Thus it covers all possible OS criteria, device features, locations and most importantly load testing similar to a real-time scenario.

Conclusion: Test strategy is not something that has to be strictly followed as documented. Rather it an overall test execution process and must be defined based on several criteria as discussed in the article. However, a proper test strategy plan and execution of that can definitely improve the testing process and quality of the software.

TestUnity

Testunity is a SaaS-based technology platform driven by a vast community of testers & QAs spread around the world, powered by technology & testing experts to create the dedicated testing hub. Which is capable of providing almost all kind of testing services for almost all the platforms exists in software word.