Test Me

A (Set of) Nifty Assignment(s)

Summary Test Me -- is a suite of programs which claim to conform to various contracts.  It is up to the student to determine whether or not these claims are justified.
Software Testing, programming by contract
CS0 to CS2, depending upon the topic of the program; most in this suite are CS0-CS1ish
Fairly easy, but difficulty can trivially be ramped up.
Students do NO CODING!  Forces students to read documentation and build test cases in a creative manner.  The assignment can be used to provide "introductions" or "pre-labs" for many traditional lab assignments.  Can serve as an introduction to testing through JUnit, etc.
It is easy for students to do a poor job.  It can also be frustrating to find obscure bugs.
Requires an understanding of English and the problem domain.  For students to do well on this type of assignment, they need to be exposed to the ideas of formal testing.
Several variants are provided.  Once the concept is clear, it is reasonably simple for instructors to build their own.  Note that the materials at this site use graphical interfaces, but text-based versions can be generated more easily and have many of the same benefits.

The General Assignment

    The student is provided with a specification for a program and a link to a web page containing an applet that purports to meet the specifications.  The student is to run the program and find the conditions, if any, under which the program fails.  (The student is not allowed to inspect any source code.)  Depending upon the desires of the instructor, the student can be required to identify/design the test cases in advance or the design can be done "on the fly".  The student turns in a report describing the testing that was done as well as the results of that testing.

Materials Provided To Instructors

    This web site contains an index page referencing six programs.  Each of the programs has a specification (written in "programming contest style") and one to four additional web pages containing applets that purport to meet those specifications.  One of the applets does meet the specifications; the other applets have some bug or set of bugs.  A second link (labeled "description") with each version will open a panel explaining the bugs, if any, in that version.  (Note:  As of 9/9/04, the "bogus" applets for the APFT program are not submitted; they are done, but are currently "misplaced".  It is possible/probable that there will be a seventh  program before the first of the year.)

Materials Provided To Students

    Normally, an instructor would take only one of the versions and the description and link them off of a web page explaining the deliverables.  For our first assignment, we tell the students that there are indeed bugs so that they don't run it once and certify it; after that first experience, they are warned that it is possible that the program in question might be correct! 


    The assignment is simple enough that it can (and should be) repeated.  Programmers learn to test code thoroughly by testing code often, under different circumstances.  The first assignment can be given with no preparation, and then discussions can be used to help students learn the concept more thoroughly.

Some Variations On The Theme Using Traditional Assignments

But It Isn't Automated Testing

    The use of GUIs in the suite means that it is difficult to do the type of automated testing that JUnit and its ilk support so well.  Students who will eventually use such a tool can still learn the principles of testing through this suite.  When they are ready to graduate to a more sophisticated tool, the concepts learned here will transfer directly.




Extra info about this assignment: