Common software testing methods and tools

Industry standard load testing tool LoadrunnerLoadRunner is a load testing tool to predict system behavior and performance. By simulating the concurrent load of tens of millions of users and real-time performance monitoring to identify and find problems, LoadRunner can test the entire enterprise architecture. By using LoadRunner, enterprises can minimize test time, optimize performance, and accelerate the release cycle of application systems. AutoRunnerAutoRunner, an automated functional testing tool, is a black-box testing tool, which can be used to complete functional testing, regression testing, nightly construction testing and automatic regression testing. It is an automated testing tool with scripting language, which provides perfect script tracing and debugging functions, supports IE testing and Windows native testing, and is the best banking testing tool in China at present. The global test management system TestdirectorTestDirector is the first test management system based on Web in the industry. It can manage global testing inside and outside the company. By integrating all parts of test management into an integrated application system, including requirements management, test planning, test execution and error tracking, TestDirector greatly speeds up the test process. Test Center, a test case management tool, is a powerful test management tool, which realizes test requirements management, test case management, test business component management, test plan management, test execution, test result log viewing, test result analysis and defect management, supports the correlation between test requirements and test cases, and can index test cases through test requirements. TARTAR is a terminal automation test tool, which is suitable for standard application systems such as VT 100, VT220, and supports command line mode and window mode (applications written by cursor). Support automatic recording of terminal applications. Support continuous recording and separate window recording. Supported window components: fields, tables, dialog boxes, windows, etc. Rational Robot Borland Silk Test 2006, a functional testing tool, belongs to the software functional testing tool and is one of the software quality management solution suites proposed by Borland Company. The tool adopts wizard setting and automatic test execution, and both novice programmers and experienced experts can quickly establish functional tests and analyze functional errors. Performance testing tool is a set of tools developed by Microsoft website testers, which is specially used to test the actual website pressure. Through this powerful stress testing tool, you can use a small number of client computers to simulate the possible impact of a large number of users online on website services. Automatic white-box testing tool JtestJtest is a java language automatic white-box testing tool introduced by parasoft company. It improves the reliability of the code by automatically implementing java unit testing and code standard checking. Parasoft also produces C++ test, a C/C++ white box testing tool. JMeterJMeter is a functional and performance testing tool and an open source project organized by Apache. It is a tool for function and performance testing, and 100% is implemented in java. The performance test and analysis tool WEBLODEwebload is a performance test and analysis tool launched by RadView company, which allows web application developers to conduct stress tests automatically. Webload tests the performance of the Web by simulating the operation of real users and generating pressure load. WinRunnerMercury Interactive's WinRunner is an enterprise-level functional testing tool, which is used to test whether the application can achieve the expected function and run normally. By automatically recording, detecting and replaying user's application operations, WinRunner can effectively help testers to test different release versions of complex enterprise-level applications, improve their work efficiency and quality, and ensure trouble-free release and long-term stable operation of cross-platform and complex enterprise-level applications.

Test manager and PM review TC:

Overview of agile testing process:

Among agile methods, XP emphasizes the importance of testing in the whole project development process. Agile testing for agile development methods is different from previous testing for traditional development patterns. In an agile team, testing is the "headlight" of the whole project team. It tells everyone where they are now and which direction they are going. Testers provide a wealth of information for the project team, so that the project team can make correct decisions based on these reliable information. Not only the testers should ensure the quality, but everyone in the whole project team should be responsible for the quality. Testers don't pester developers' mistakes, but help them find their goals and strive to achieve the ultimate goal of the project. Agile testing also requires highly iterative work, frequent feedback from customers, and dynamic adjustment of test plan and test execution. In addition, agile testers participated in more agile production activities, which had a positive impact on the decisions and plans made by the team.

According to the agile development mode currently adopted by the company's projects, it is suggested that the corresponding agile testing process is as follows:

1. Verify the requirements and design.

Requirements and design generally include: (1) functional design specification written by the project manager according to the requirements text; (2) Implementation design instructions written by developers according to functional texts include (architecture documents, project scope statements, use cases). As a tester, the focus of the audit is to check the completeness and rigor of the text definition of user requirements and the testability of functional design.

At the beginning of testing, testers should learn to do static testing, and do a good job in demand analysis and design logic analysis. Testers should pay more attention to the realizability of requirements and actively participate in the requirements analysis, design and development of projects and systems as the first user. Actively participate in the preliminary work, and quickly feedback its static test results to the design and development. Start testing as soon as possible, don't wait until the function is completely completed.

Output: the test needs to submit the review result document, which can make the test more involved in the review of DB design and framework.

2. Test plan, test case

2. 1 Write plans and test cases

In the process of agile development, time is estimated according to each user story. The developer will evaluate the complete user story required for this iteration. Developers can communicate directly with customers to determine the priority of each user story.

Benefits:

Customers can clearly understand which user story takes a long time and their priorities.

Question:

Developers often underestimate the time estimate of user stories. Version cannot be published on time or must work overtime to publish.

Analysis:

Because the version is updated quickly, the task time is estimated in hours. Developers generally ignore the time outside of development, such as the time when they encounter problems in development, the time of meetings, the time to help other members, and so on.

For example:

The developer estimates that it takes 1.5 days to encode a user story, and the developer estimates that the other time is half a day. So the estimated time given by the developer is 2 days. The actual time spent in the development stage is as follows, and the time spent in the regular meeting every day. Other members of the project need technical support in regular meetings. So I spent three hours helping. Some unexpected problems were encountered in the development process, which took 4 hours to solve. (Maybe more). Need to deal with some unexpected affairs of the company and so on. Therefore, it is strongly recommended that you fully consider other factors when estimating the time. According to an XP-related book, the best time to estimate the time is 2-3 times of the coding time. It sounds scary, but it really takes so much time to actually operate.

Testers prepare test plans and design test cases according to the approved requirements and designs. Among the three texts mentioned above, the functional design text is the main basis. The two texts of the test should also be reviewed by the project manager and developers.

2.2 Audit of test cases

In order to enable developers to participate in the evaluation of test cases, ensure the quality and feasibility of TC, ensure the smooth progress of the test, and enable developers to quickly understand the key points of the test and give corresponding opinions and suggestions, testers should also publish a TC_Matrix (test case tracking matrix) to indicate which features TC has covered. Specify the number of TC corresponding to each feature, so that the test manager and PM can see the coverage of TC at a glance when reviewing TC, and give suggestions in time for areas with insufficient coverage (such as insufficient test cases for a key feature).

In addition, in the morning meeting every morning, testers can briefly talk about the key parts of the day's test and what serious bugs exist in the project, so that developers can understand what the key points of the day's test are, how to conduct the test, and put forward their own opinions and suggestions. This strengthens the communication between developers and testers, and makes the testing work more effective and smooth.

Testing should take time to update test cases at the end of iteration.

3. Implement operational tests

In agile methods, there are two kinds of tests: unit test and receiving test. The unit test is completed by the developer, and the receiving test is completed by the customer representative.

Because our customers can't be on site, so we adopted, developers do unit tests, testers do verification tests, and finally customers do acceptance tests. Before each version is released to customers, it must be tested by testers. After the release, customers will do acceptance tests and put forward the places that need to be revised. What needs to be revised will be completed in the next version.

unit testing

Before the daily build version test, the developer should do unit test first, inform the weak links in the software in advance, and help testers adjust the test focus. unit testing

The advantage of unit testing is that it can improve the version quality, reduce the workload of testing, and reduce the incidence of shallow bugs, so that testers can devote more energy to finding deep bugs.

Confirmation test

Generally speaking, the verification test of testers is the process of putting the test cases designed in the previous step into practice as planned. The test at this stage must be carefully planned. This kind of planning is first reflected in the coordination and cooperation between development and testing. According to the dependence of product architecture and functional modules, we will jointly promote it according to the overall project planning. Judging from the process of testing, the test execution can be aimed at some functions at first, and then it can be gradually expanded. Then, the iterative process is used to complete the test task, that is, the test task is divided into multiple cycles. First of all, you can do some key functional tests, and the reusable parts (components, parts) in the code can be completely tested. Edge function test and other tests can be done in the next iteration cycle, and the last few iterations should be used for regression test and key performance and stability test.

3. 1 Provide bug trends every day.

In order to measure the progress of the project, the test can provide the bug trend of the test after the completion of the test every day, that is, the number of new bugs and the number of bugs solved every day, marked as a trend chart. Generally, the curve of the number of new bugs will show an upward trend in the early stage of the project, and the curve of the number of solved bugs will tend to rise in the middle and late stage of the project, while the curve of the number of new bugs should decline. At the end of the project, both curves tend to zero. PM will continue to observe this chart to ensure the healthy development of the project and predict the project bugs through analysis.

For each version of bugs, developers should think about why such problems occur, especially very low-level bugs, and whether similar bugs can be avoided.

Testing needs to be considered: the writing of exploratory test cases

3.2 Maintenance of test cases

In the testing phase, testers need to maintain the existing test cases in time. Usually, some test cases need to be added in the following two situations: first, for areas with incomplete test design at the beginning, and second, for external bugs (such as bugs reported by Beta customers), which are not covered by existing test cases. When the functional design of the product changes (the functional design in agile projects often changes), the test cases involved should be modified accordingly to keep the test cases in sync with the existing functional requirements.

3.3 according to the project constantly add common sense.

During the project, testers need to accumulate experience and constantly supplement and improve the common sense standards of various projects. For example, the common sense of American standards summarized by CTTS projects should be strictly followed in future American projects to ensure that the mistakes that occurred before will not recur in future project tests. While ensuring the quality of the project, we will constantly accumulate new experiences.

3.4 Control Intermediate Version

In order to better ensure the quality of software and avoid risks, it is necessary to strengthen the control of intermediate versions. For example, if a customer requests or plans to submit a version on Friday, then he must submit an intermediate version for testing on Wednesday, that is, control the intermediate version to avoid completing all the work in the most urgent time later. Before the project, the project was easy in the early stage, and there were more and more bugs in the later stage. Developers and testers are very busy and often work overtime. In order to reduce the later workload and avoid risks, it is suggested to develop Daliy build, or build it once after completing a feature, so as to effectively avoid more and more bugs, reduce the later workload accordingly, and ensure the quality of the project.

3.5 Write release notes before releasing the version.

Before each release, testers should write release notes according to the version to be released, so that customers can see the released version at a glance. The release notes mainly include three aspects: fixed, new features and known problems. Among them, the fixing part indicates which big bugs existed in the previous version were fixed by this version; The new features section explains what features have been added in this version; In the part of known problems, what are the major problems in this version that need to be improved in the next version? Or list the places where the requirements are not clear, wait for the customer to give a clear answer, and complete it in the next version.

4. Supervision and regulation of demand

Customers often change their requirements in projects that adopt agile development mode. Therefore, demand management is very necessary and important. During the whole project process, the ever-changing requirements must be tracked, and each demand change must have a corresponding historical record to facilitate the later management and maintenance. Every change can be recorded in the requirements tracking document, and the document can always keep the latest update status to keep up with the changes in requirements.

Question:

The customer may modify the requirements back and forth at a function point, and may need a function at first, but after finishing it, he feels bad and puts it down. Later, for some reasons, it is necessary to add something. It may have been revised many times during the whole process. Then be sure to record the content and date of the change. Maybe later, customers will think that a function takes so much time. Didn't you already do it? At this time, these records are the best proof to solve customers' doubts. To put it bluntly, there is evidence that we have made many changes. You may think there is such a problem. In fact, if a project lasts for more than half a year, everyone's memory may be unreliable (:P)

Suggestion:

At present, vss tool is used to back up the demand changes mentioned in daily mail, and the document is named after the date when the mail was received that day.

5. Carry out "bug cleaning" at the end of project development.

At the end of the project development, you can carry out "bug cleaning" activities. Set aside a special time period, during which all the people involved in the project concentrate on searching for bugs in the project. Please note the following points:

(1) Although this is a testing activity, participants are not limited to testers. Project managers, developers and even senior managers should participate, just like the national mobilization. The purpose is to brainstorm; (2) Cross-searching in various departments and fields should be encouraged, because new ideas and perspectives usually help to find more bugs. (3) In order to arouse enthusiasm and improve interest, competition mechanism can be introduced appropriately. For example, at the end of the activity, individuals who found the most bugs and the most serious bugs will be judged and given material and spiritual rewards. (4) Special topics can be developed, such as security, user interface usability, internationalization and localization.