Class TestSuite

Object

TestSuite

public class TestSuite
extends Object

Contains stand-alone test cases, written as static methods, that are written against the classes for an assignment. Important notes:

1. This class must contain in import statement for the classes to be graded for this assignment. Thus, this class must be compiled.

2. Methods must be static

3. Method names must contain the string "test" somewhere in the name, all others are ignored. If a test needs a helper method, then the helper method name should not contain the string "test".

4. Tests methods are called alphabetically, thus the order they occur in this file is not important.

5. In the output that is produced, tests are numbered 1, 2, ..., regardless of the test method name. For example, if you have these test methods in this order:

   
      test_w()
      test_b()
      b_helper()
      a_test() 

   then they will be run in this order (and output labeled as shown)

   
      a_test()  --> Test 1
      test_b()  --> Test 2
      test_w()  --> Test 3 

   Note that b_helper will not be run (because it doesn't contain the string "test" in its name. It is recommended, for clarity, to list the methods alphabetically, for example: test01, test02, etc

6. Each test method is a test case and contains 1 or more items (parts) to be graded. A test case has a "worth", the number of points the test case is worth. For example, if a test case is worth 10 points and there are 4 parts, then each part is worth 2.5 points. Thus, if a student solution gets 3 of the 4 parts correct for this test case, then they earn 7.5 out of 10 points (More information follows).

7. Every test method MUST return an ArrayList<String>. There can be no line breaks (\n) in any string in the list. Every string in the list must have this format:

   
      Element Contents
      ----------------
      0       Description of test
      1       Points this problem is worth
      2       Item to be graded
      3       Item to be graded
      ...	... 

   Thus, each item to be graded is an individual string in the returned list. (More information follows).

8. Two types of items can be graded: strings and doubles. Here, we consider only strings, in the next item we consider doubles. A string item to be graded would typically be composed of text, integers, and/or booleans. It would not contain any double results. For example two items to be graded might look like this:

   
      r2.compareTo(g1)=-3
      r2.getId()=3, g1.getID()=6 

   Which might be produced by code like this (results is the list that will be returned):

   
      results.add("r2.compareTo(g1)= " + r2.compareTo(g1));
      results.add(String.format("r2.getId()=%d, g1.getId()=%d", r2.getId(), g1.getId())); 

   The output of the program will show the expected answer and the actual (student) answer side-by-side. Thus, it is useful to make the items to be graded to read as clear as possible so that a student can understand (some) of where their program failed. For example, the output of the program might look like this:

   
      Test 4 - compareTo() and getId()
      Incorrect - Expected: r2.compareTo(g1)=-3 | Actual: r2.compareTo(g1)=24
      Correct - Expected: r2.getId()=3, g1.getID()=6 | Actual: r2.getId()=3, g1.getID()=4
      Summary: 1 out of 2 answers correct: 5.0 points out of 10 

   String items are graded by string comparison (equals) meaning that the expected answer string is compared to the actual answer string. It is important to keep that in mind. For example, that is why you would not want any doubles in such a string as there could be small differences that don't invalidate a student's answer. For example: x=3.1415926 vs. x=3.1415927. However, there may be cases where you could include a double rounded to a certain precision. For example, if you were dealing with currency, you possibly could round so that you were comparing against a string like this: "total deposits=$123.45". Again, we deal with doubles next.

   Finally, the designer of tests will want to be careful as to how many "answers" there are in a string that is being graded, and any dependencies between them so that you don't over-penalize errors. For example, suppose you have a class with methods to calculate the surface area and volume of a 3-d object. If you wanted to weight these differently (different max points for each) you could have two separate test methods, one to test the area, and one to test the volume. Or, it you want the max points to be the same for each, you could have a single test with two parts: one part for the area and one part for the volume.

9. Doubles are also converted to strings in test methods; however, they are marked up to indicate that they are a double and then followed by markup to indicate either the absolute tolerance or tolerance as a percent. When the grading program encounters this markup it converts the value to a double and compares it to the expected double using the tolerance. This is the idea: A part is correct if: Math.abs(exp-act)<maxError. (More on this below). For example, the output of this program for a test that contains doubles might look like this:

   
      Test: 16-Testing double result
      Correct - Expected: Average ID=8.833333333333334 | Actual: Average ID=8.833333333333334
      Actual Error=0.0<0.08833333333333333=Max Error
      Incorrect - Expected: Average ID=8.833333333333334 | Actual: Average ID=8.333333333333334
      Actual Error=0.5>0.01=Max Error
      Summary: 1 out of 2 answers correct : 2.5 points out of 5.0 

   Notice that the results still show the expected and actual strings, but they also show actual and max error when comparing the doubles. The answer strings that are contained in the return of the test method would look like this:

   
      %d 8.833333333333334 %tp 1.0 Average ID=8.833333333333334
      %d 8.833333333333334 %ta 0.01 Average ID=8.333333333333334 

   Thus, the format for indicating a double is:

   
      %d value %t[p|a] tol informative_string_with_double 

   Only one double can be contained in a string. The only thing that is graded is the double itself, not "informative_string_with_double", this is used for display. Note:
   • %tp - means you specify the tolerance as a percent of the expected value. For example: %d 1000.0 %tp 1.0 means that the max error is: 1000*(1/100) = 10 and any actual answer such that: Math.abs(1000-act)<10 is correct. Note, percent is specified as a percent, not a fraction.
   • %ta - means you specify the tolerance as an absolute number (which is the maximum error). For example: %d 1000.0 %ta 5.0 means any actual answer such that: Math.abs(1000-act)<5 is correct.
   Finally, the code to produce the strings above might look like this:

   
      results.add("%d " + avg + " %tp 1.0 Average ID=" + avg);
      results.add("%d " + avg + " %ta 0.01 Average ID=" + avg); 

10. Each "part" (answer, string, item to be graded) within a test method should be surrounded with try/catch so that if a runtime error is generated the grading code can take appropriate action so that the program can continue and mark that part as incorrect. For example, consider these two strings as items to be graded in a test method:

    
             r2.compareTo(g1)=-3
             r2.getId()=3, g1.getID()=6 

    we would use code like this:

    
             try{
                 results.add("r2.compareTo(g1)= " + r2.compareTo(g1));
             }
             catch(Exception e) {
                 results.add(e.toString());
             }
             try{
                 results.add(String.format("r2.getId()=%d, g1.getId()=%d", r2.getId(), g1.getId()));
             }
             catch(Exception e) {
                 results.add(e.toString());
             }  

    Thus, an expected answer of say: "r2.compareTo(g1)=3" would be compared to the exception string which of course would be incorrect and having the added bonus of showing that the student code generated an exception and its type.

Constructor Summary

Constructors

Constructor and Description
TestSuite()

Method Summary

Modifier and Type	Method and Description
static ArrayList<String>	test01()
static ArrayList<String>	test02()
static ArrayList<String>	test03()

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TestSuite

public TestSuite()

Method Detail

test01

public static ArrayList<String> test01()

test02

public static ArrayList<String> test02()

test03

public static ArrayList<String> test03()