©2004 Christopher League
Printer-friendly edition
Fall 2004

Assignment 4: Testing

Due: Wed Dec 15 in class.

This assignment involves a refreshing bit of role reversal: students have the opportunity to find faults in a program written by the professor!

You will write a test suite (a set of test cases) for a multi-precision calculator program, which I provide below. Then, you will execute your test cases and report on any faults that you found with the program.

To run the program (and use the coverage tools), you will log in to the comsci Linux server, by accessing this URL: http://comsci.liu.edu/ssh/ (your web browser must support Java applets). When the login window pops up, hit enter when presented with the host name, then use the login name “calctest”. The password will be sent to you via the mailing list. Upon logging in, the following commands are available:

  make                  Build the calculator program.
  calc                  Run the calculator program.
  gcov calc.cpp         Generate the coverage report.
  less calc.cpp.gcov    Inspect the coverage report
                          [use space bar to view next page, or q to quit].
  make clean            Reset the coverage statistics.
  help                  Print this help message.
  exit                  Log out of the comsci server.

You may run the calculator program as many times as you want. After that, running gcov will tell you what percentage of the source code was covered by your tests. By inspecting the coverage report, you will see the code indented, and the left column will tell you how many times each line was executed. Look for a row of hashes (#####) in the left column; this means the line has not been executed yet by any of your tests. When you are finished, be sure to type exit before closing the window.

Here are a few test cases, to show you the format and to get you started.

In the input column of each test case, I gave the keystrokes, exactly as they should be typed. In test 3, for example, you first type 4 then hit return (enter) to specify the precision. Then you enter 999.1+1=, and the result should overflow the 4 digits of precision that were available. Since, in this example, the calculator reports “OVER”, it passes this test.

Each time that the program does not produce the expected output on a particular test, you should mark that as a failure in the result column. Furthermore, you should explain the failure as clearly as possible—so that a developer has enough information to begin debugging. For test 4, the calculator produced the result “123.0” which technically is a correct answer, but ideally it would be expressed as just “123”. (Whether this is really a failure may be open to debate, since there is no requirement that says results must be expressed in as few digits as possible.)

I expect you to design a fairly comprehensive test suite—hopefully, one that exposes all the faults in my program. I cannot say how many test cases will be needed; it depends on how creative and careful you are. Three dozen test cases might be no better than just one, if they are all in the same equivalence class.

Requirements

Rationale: most physical calculators have just 10 digits of precision. Even on a 32-bit computer, you can only calculate precisely numbers up to 2 billion something (2,147,483,647 to be exact). Bill Gates cannot even keep track of his fortune this way! Thus, he needs a multi-precision calculator application. Think of it as a standard calculator, where the display can be stretched to whatever size is needed. Following is a list of requirements for a multi-precision calculator.

1. The calculator must support arithmetic on numbers up to 74 digits.
2. Upon starting the program, the user may be asked to specify the maximum precision needed, and then it will be fixed until the program is restarted.
3. If a user tries to enter more digits than fit on the display, the extra digits will be quietly discarded.
4. The calculator must support addition, subtraction, multiplication, and division, specified in infix notation, as in “3*5”.
5. Multiple operations may be chained together before giving a result, as in “2+10+21”. The user must press “=” to see the final result.
6. The calculator must respect the usual order of operations (precedence): multiplication and division first, followed by addition and subtraction.
7. The user must be able to use the backspace or delete key to correct any number as it is entered.
8. The user may press the ‘c’ key to reset the display to zero and cancel the current computation (if any).
9. The user may press the ‘q’ key to quit the calculator.
10. If any operation will “overflow” the number of digits available in the display, the calculator should abort computation and indicate the overflow error in its display.
11. The calculator need not support scientific notation, but it must maintain the exact precision in each computation. It may not “round off” the answers to requested computations so that they fit in the display.
12. The calculator must be able to hold at least one number in auxiliary memory. It will be kept there until another number is stored, or until the user quits the application.
13. The user may press the ‘m’ key to store the displayed value to memory.
14. The user may press the ‘r’ key to recall a stored number into the display.
15. If the memory is holding a non-zero value, this should be indicated by an ‘m’ near the display.
16. The calculator need not support the mouse or have a graphical interface.
17. The calculator application must run on Windows XP, Windows 2000, and all Unix variants.
18. Each computation, on numbers up to 74 digits long, must be completed within one second.
19. The calculator application should tolerate invalid input—entering invalid commands should not cause it to crash or to stop operating properly.