Acceptance Testing Acceptance testing The complete system, including documentation, training materials, installation scripts, etc. is tested against the requirements by the client, assisted by the developers. Each requirement is tested separately. Scenarios are used to compare the expected outcomes with what the system produces. Emphasis is placed on how the system handles problems, errors, restarts, and other difficulties. Is the system we have built, the system that you wanted? Does it meet your requirements? CS 5150 3 Acceptance Testing in the Waterfall Model
Requirements Feasibility study Requirements Design System design Implementation Program design Implementation (coding) Testing Acceptance & release Operation & maintenance CS 5150
4 Acceptance Testing with Iterative Development Iterative development If the client is properly involved in the development cycle: The client will have tested many parts of the system, e.g., -> At the end of each iteration -> During user testing Problems and suggestions for improvement will have been incorporated into the system BUT: There must still be an acceptance test of the final system before it is released. CS 5150 5
Acceptance Testing with Incremental Development (Sprints) Incremental development Each sprint should be a complete development process, ending with acceptance testing by the client. If several sprints build on each other, each sprint may need to repeat the acceptance tests for earlier sprints to check that they are still met. CS 5150 6 Acceptance Testing Closed box: by the client without knowledge of the internals
The entire system is tested as a whole The emphasis is on whether the system meets the requirements The tests should use real data in realistic situations, with actual users, administrators, and operators The acceptance test must be successfully completed before the new system can go live or replace a legacy system. Completion of the acceptance test may be a contractual requirement before the system is paid for. CS 5150 7 The Testing Process Acceptance Testing is a major part of a software project
It requires time on the schedule It may require substantial investment in test data, equipment, and test software. Good testing requires good people. Help systems and training materials are important parts of acceptance testing. CS 5150
8 Techniques for Release The transition from the previous version of a production system to a new release is challenging. Parallel Testing: Clients operate the new system alongside the old production system with same data and compare results Alpha Testing: Clients operate the system in a realistic but non-production environment Beta Testing: Clients operate the system in a carefully monitored production environment CS 5150 9
Release: Parallel Testing Parallel Testing For data processing systems, such as financial systems, payroll, etc., the old and the new systems are run together for several productions cycles to check that the new system replicates the functionality of the old. Requires two sets of everything (staff, equipment, etc.). Requires software to control changeover (e.g., do not mail two sets of payments). Requires automated scripts to compare results. Parallel testing may take several months. If possible, the new system will be brought into production incrementally. CS 5150 10
Release: Alpha and Beta Testing Alpha and Beta testing must be managed If you simply make versions of the software available to clients: Many clients will never use it. Other clients will use a few features. Only a few clients will test it systematically. Only a few clients will report problems systematically. What incentives can you give clients to test your software for you? Financial (e.g., discounts on products) Prestige (e.g., recognition, publicity)
CS 5150 11 Delivery: Summary A good delivery package results in: happy client happy users
less expense in support and maintenance But many projects rush the packaging, help systems, and training materials, give them to the least experienced members of the team, do not test them properly, and generally neglect this part of the software process. CS 5150 12 Delivering the System Different categories of software product need different packaging and delivery procedures. Packaging, support, maintenance, and major failures are expensive. Trade-offs must be made.
Pressures to get products to market and in operation often lead to bad decisions. (In my experience, the pain of being late is often less than the pain of putting a bad system into operation.) Services, such as installation, training, configuration, etc. may be paid for separately. CS 5150 13 Delivering the System The best is the enemy of the good No system is ever perfect. If you insist on perfection: -> you will never release anything -> you will still never reach perfection (because
the ultimate test comes from the users) Learn what is essential to satisfy the client and focus on it. CS 5150 14 Delivery of Software Shrink-Wrapped Package Installation scripts -- automatic -- varieties of hardware and operating systems -- uninstall, reinstall, etc. Support (very expensive when it requires staff) -- staff training -- documentation (user, system administrator, expert user) Maintenance
-- client does not have source code -- no bug fixing except with new release CS 5150 15 Delivery of Software Data Processing System Acceptance -- acceptance period may cover several months -- client should be comfortable with complete system Support -- client should be self-sufficient -- documentation and training for system administrators and operators -- well organized source code for maintenance -- maintenance and support contracts
CS 5150 16 Delivery of Software Embedded System Acceptance -- hardware and software developed together -- acceptance tests are a combination Maintenance -- bug fixes may require servicing the hardware -- errors may be expensive or dangerous Support -- training for support personnel -- documentation and training for users CS 5150
17 Training Time and money spent on training is usually well spent: one-on-one in-house training training courses distance education online tutorials
Development team needs to provide training materials: CS 5150 users (perhaps several categories) system administrators system maintainers trainers 18 Training and Usability A well-designed system needs less training good conceptual model
intuitive interfaces Different skill levels need different types of training skilled users work from the conceptual model less-skilled users prefer cookbook sets of instructions occasional users will forget complex details, but remember general structure CS 5150 19 Help Systems Resources A good help system is a major sub-project (time-consuming, expensive) A good help system saves user time and support staff (timesaving, cost-saving) Help system design
Users need many routes to find information (index by many terms, examples, mini-tutorials, etc.) Help systems need to be tested with real users CS 5150 20 Documentation Online documentation Much cheaper than print Fewer restrictions on numbers of pages, colors, etc. Easy to update (e.g., over the Internet) but... Cannot be used if the user's system is down CS 5150
21 Categories of Documentation Software development Requirements, design Source code, test plans and results User Introductory (various audiences) User manual Web site of known problems, FAQ, etc. System administrator and operator System manuals Business License, contract, etc. CS 5150
22 Installation Tools Creating installation scripts may be a major sub-project Different scripts, tools and procedures for different categories of software Testing must be extensive with real users in their own environment CS 5150 23 Delivery: Maintenance Most production programs are maintained by people other
than the programmers who originally wrote them. (a) What factors make a program easy for somebody else to maintain? (b) What factors make a program hard for somebody else to maintain? CS 5150 24 CS 5150: What materials should you deliver? When you leave, all that the client has is your software and your documentation. Imagine that you work for the client and are asked to take over this system. What would you want? Different projects will have different deliverables Materials can be in any format, need not be huge, but must
be current. Place your materials on a project site web site. Discuss delivery with your client CS 5150 25 Delivery: Check List for CS 5150 Projects Documentation Requirements, updated to reflect delivered system
System & program design, updated to reflect delivered system Instructions for: users, administrators, operators Presentation slides, updated to reflect delivered system Business documentation, e.g., copyright license System Source code and matching binary for all programs Installation scripts, etc. Test scripts, test data, and test reports CS 5150 26
Table d'hôte is a set-price menu, usually cheaper than the à la carte menu, although you pay the full menu price even if you do not have all the courses. Two to five courses are usually on offer, with a...
Flowering plants reproduce sexually, so if it has a flower on it, it has to have undergone sexual reproduction or will do so soon. The food crops we are studying have small flowers on them that are so inconspicuous (small)....
Ben je ooit al naar het NFF geweest? NFF. Nederlands Film Festival. In 2016 de 36e editie. Nederlandse films. Korte en lange films. ... Heb jij ook wel eens een doel gehad dat je wilde bereiken? Hoe heb je dat...
1 Cor 10:11-13 "these things happened unto them for ensamples…for our admonition…. Wherefore let him that thinketh he . standeth. take heed lest he fall. There hath no temptation taken you but such as is common to man:
In this training module, we will discuss how to use the Data Entry Interface (DEI) to enter student responses on paper-based accommodated test forms for the ELPA21 Summative Assessments. This module covers information that Test Administrators (TAs) should be aware...
Mononuclear Non-Heme O 2. Activation. Aromatic Amino Acid Hydroxylases Use a Pterin Cofactor that Provides Two Reducing Equivalents. Pterin itself reacts with oxygen to make a 4a-hydroperoxypterin species (2-electron reduction of O. 2) that interacts with the iron center to...