What is the Underlying Logic of Software Testing? (Part 2)

3. The Underlying Logic of the Test Process

The test process conforms to the general engineering project process, including the phases of analysis, planning, design, implementation, and evaluation. No link can be missing, and each link is important. However, the preceding link will affect the subsequent link, so the earlier the phase, the more critical it is. Test analysis is the foundation, followed by design, implementation, and evaluation, forming a pyramid model.

Another underlying logic of the test process is to form a closed loop. If problems are found in the test process after evaluation, it is necessary to re-analyze, modify the plan, and change the design. After a complete process, a new closed loop is formed. From the perspective of test process improvement, it also needs to form a closed loop like PDCA (Plan-Do-Check-Act). From the perspective of DevOps today, testing is meant to make users more satisfied, but at the same time, it is necessary to conduct user surveys and collect user feedback to form a closed loop, similar to the closed-loop model I drew 16 years ago. In terms of the cost of defects, the earlier the test, the better, because inferior quality costs grow exponentially over time.

To sum up: Testing runs through the entire R&D cycle, forms a closed loop, and continues to improve.

4. The Underlying Logic of Test Analysis

The underlying logic of test analysis is based on systematic thinking and structured thinking. It requires thinking systematically from various aspects such as project background, product structure, and quality requirements. Details cannot be ignored. The test scope is presented completely step by step, various test risks are identified, and finally, the test items and their priorities are clarified.

Systematic thinking allows us to clearly see the inputs/outputs, pre- and post-conditions, surrounding environment, and various scenarios faced by the object.

Structured thinking helps us formulate more effective testing schemes and strategies, such as layered testing, interface-oriented testing, and so on. At the same time, testing is always risky, so a risk-based test strategy must be adopted during test analysis. The 80/20 principle is applied to determine where the 20% most serious risks are concentrated, which functions constitute the 20% most commonly used by users, and which test items make up the 20% of key testing.

• The first underlying logic of test analysis: Test analysis is a gradual and in-depth system analysis process. Systematically analyze the tested objects from different dimensions, such as business requirements, user behavior, system functions, and application scenarios, to finally determine what to test.
• The second underlying logic of test analysis: Test analysis is also a process of gambling, selection, and balance. It requires concentration and insight to make trade-offs, such as using the 80/20 principle to grasp the main risks, acknowledging that sometimes it is necessary to abandon some secondary risks.
• The third underlying logic of test analysis: Start with the end in mind; start from the test goal and finally return to the test goal. For example, considering how to measure the requirements of test adequacy, the final analysis result—the completion of each test item—should meet those adequacy requirements.

5. The Underlying Logic of Test Design

Test design involves using appropriate methods to complete the design of test data, test scenarios, or test cases based on the results of test analysis. According to engineering thinking, there is always more than one solution. Multiple solutions should be designed to select the better or best one.

The essence of test design is to cover test requirements more effectively and select one or more coverage strategies from different options, such as scenario coverage, logic coverage, path coverage, and data coverage. Test design is also a gradual process of continuous improvement.

Test design is a dialectical and unified thinking process, which includes both strict logical thinking and jumping, divergent creative thinking. It is not only the unity of opposites between Black-box testing and White-box testing methods, and the integration of static and dynamic testing, but also the integration of active and passive testing. Only in this way can we meet the design requirements more thoroughly and complete testing faster to achieve the test goal.

The underlying logic of test design: Test design is an art, but it also requires innovation and integration.

6. The Underlying Logic of Test Automation

Test automation is about giving full play to the role or value of tools. For example, tools can fully execute commands and work without complaint. Therefore, automatic testing is suitable for mechanical and monotonous testing work, such as regression testing, performance load testing, stress testing, compatibility testing, BVT (Build Verification Testing), etc.

The development and execution of test automation scripts are based on test analysis and design. If there are problems in test analysis and design, relying on tools cannot solve them. With better test analysis and design, there can be better automated testing. Therefore, it is very important for us to understand the relationship between test analysis/design and automated testing.

The development and use of tools and scripts are all done by people, so people still come first, and tools come second. Test automation is also affected by culture and process. Whether test automation can succeed is not a technical problem. Today, there are no real technical obstacles. Obstacles often appear in the corporate culture, R&D process, and development quality (such as software implementation specifications and testability).

• The first underlying logic of test automation: Tools are important, but humans are the deciding factor.
• The second underlying logic of test automation: Automated testing is fundamentally based on test analysis and design.
• The third underlying logic of test automation: All test work suitable for automation should be automated as much as possible while creating an environment conducive to automated testing.

7. The Underlying Logic of Testers

Last but not least, let's talk about the underlying logic of testers. The value of a tester does not depend solely on their current working attitude, knowledge, and skills, but on the progress rate of their attitude, knowledge, and skills. We can't change the past, but we can change the future. As long as you continue to learn and reflect, you can quickly complete your evolution and grow up rapidly, and no one can stop your bright future.

If we master the underlying logic of software testing, we can see the essence of software testing dynamically and continuously only by exploring the invariance within all changes. By seeing the bottom cards of software testing, we can always navigate it like a duck to water.

The underlying logic of testers: As long as you keep learning and reflecting, no one can stop you from becoming a testing expert.

8. Summary

The core underlying logic of software testing is to obtain necessary quality information as soon as possible and reveal quality risks.

To this end, the extended underlying logic of software testing includes:

• Forming a closed loop throughout the whole R&D cycle and continuously improving the test process.
• Risk-based testing strategies are essential.
• Starting with the end in mind and systematically analyzing the test requirements to find a balance between resources and test objectives.
• Recognizing that test design is an art, but also innovation and integration.
• Automating testing as much as possible on the basis of solid analysis and design.
• Telling the test story well and working together with all parties.

------

Click here to read What is the Underlying Logic of the Software Testing? (Part 1)