Technology-Based Testing

Members

John Schroeder, CGCC (Chair)
Chito Hernadez, MCC (1st Semester Co-Chair)
John Payne, MCC
Holly Antosz, PC
David Nachman, PC
Cindy Leshin, DIST
David Platt, MCC
Hal Swanson, PC
Diana Tomanek, ASU

Introduction

This committee was given the following charges:

What should a tech-based testing system include?
What is available now?
What is it going to take to develop one (or should we)?
What are the implications for teaching and learning?
What are the related Issues?

As a new committee, we tried to tackle all of the charges, making more headway on some than on others. The committee concentrated on features that technology-based testing systems should include, and on what was currently in use. Other charges were touched on, but not studied in depth.

Discussion disclosed that there were two different schools of thought driving the interest in technology-based testing.

Computer-administered testing is needed to better prepare students for the computerized tests that are used (or will be in the near future) by state or federal licensing authorities. Nursing, dental, legal, aviation, and EMT are some of the areas of perceived need.
Computer-generated tests are needed for areas dealing with hundreds of students in multiple sections needing standardized minimum performance level outcomes. MAT077 classes are an example, but other courses might have the same potential application.

General Features

Support text, graphics, sound, and possibly video; a support organization to provide upgrades; support various question types (not just multiple choice,); support Scantron for hard copy tests; and keep records of responses for reliability / validity purposes; and support time inputs for timed tests, utilization records, etc. The test engine should not be discipline specific, but usable by all interested disciplines.

Installation

Must be networkable with strong security features, compatible with existing machines, and available for multiple platforms. (Mac, IBM, and Unix)

Student Use / Flexibility

Easy for students to use, both for remedial work or review as well as taking the test directly on the computer; maintain records of student responses to help them select areas needing further review.

Instructor Use

Simple for instructors to add questions, change explanations, update, etc. It should create tests by criterion-based random selection or by selecting specific questions. Questions should be coded indicating subject, question type, difficulty, etc.

Other

This system must support explanations during review -- why an answer is correct or incorrect. It should randomly sort the choices in multiple-choice or matching type questions. It must support use of color and graphics. It should allow for "pointers" from a response to a related question, and allow the instructor to select single chance or multiple chances on an item. Finally, it would be nice to see support for voice or touch screen input for physically-challenged students.

What is available now?

There are several applications currently in use within the District:

Nursing at GCC uses a Windows based program for testing their students. It is written with ToolBook and does not yet support review. While still early in its use, it is perhaps the most mature application developed within the district.
Nursing at Phoenix College (PC) is using "RN Challenge" as a review vehicle. It is a 1989 copyright, character-based program that is very interesting but running on slow machines. This application does not appear to have any options for additional or updated questions.
The Legal department at PC has some similar applications for their students.
Some areas are using NovaNet, especially in student support / learning resources. This involves ongoing connect costs and outdated tests in some areas.
Aviation Maintenance Technology at Chandler Gilbert Community College (CGCC) has text files of 2100 questions for faculty use only.
Some individual faculty have experimented with LXR as a test database engine for written tests, but we did not locate any widespread use or sharing of question databases.

There are some applications in use around the world, but most are not widely known or meet the features list desired. Some that we have run across include:

University of Texas chemistry department is reported to be using a purpose built testing application that creates similar level but different exams.
University of Western Ontario French department is using a purpose built testing application that is text only, multiple-choice only, for initial course placement.
An individual in White Rock, BC reports a system written in SuperCard that supports real time simulation, etc. in use at his school.
The National Center for Software Technology in Bombay, India has a ASCII only system running on UNIX that they use for a large number of multiple choice tests.
LXR is a Mac based test database engine. When upgraded, it will support Scantron, graphics, multiple fonts, etc. Its strength is in developing paper tests. While it can be used for students to test on the computer, the screen design is very busy and potentially confusing for students. The upgraded program is approximately $900 for an individual copy.

What is it going to take to develop one (or should we)?

Currently, there is nothing commercially available that does everything (or even most) of what we feel is needed. If the District is going to get involved in funding or development, it would seem that the cross-platform issue is critical. There are database development tools available that are cross-platform, and it appears that a fast database would be the logical choice. FoxPro2.5 is currently available for DOS and Windows, with Mac due in October and Unix reportedly to follow next year. It provides the tools and speed if someone is willing to tackle the project. While Glendale's nursing software is well along in the development cycle, it is developed in ToolBook which makes it relatively difficult to move to other platforms.

What are the implications for teaching and learning?

Concerns about computer phobia do not seem to be a major hurdle, provided the software uses a consistent interface. The ability to return to a question/mark a question for review is seen by some to reduce student concerns. Clearly, anything that makes test item analysis easy has the potential to lead to better teaching and learning.

Review capabilities with explanations on a large selection of test items will allow the students to cover the material in depth before they are evaluated. This will enable the students to solidify their understanding of the material and will lead to broader understanding.

Related Issues

The technology for essay questions is lacking. The technology for multiple choice and matching is simple. Short answers can be handled with more effort from the programmer and the question writer. In general, however, available technology is not an issue.

A larger issue is access to hardware. At most of our colleges, the available hardware is in use, and we don't have the extra capacity to handle a large influx of computer-based tests. Keeping the test databases current will demand time from the instructors, offsetting the time that might be saved in developing and grading paper tests.

Finally, the "information commons"/"computer center" may not be able to provide the secure and appropriate testing environment needed. Some colleges already have a testing center that could accommodate computer based testing. It would seem that a move to testing centers would accompany widespread use of computer-based testing.

At this time, there does not appear to be any widespread use in educational facilities although business and government have used it for years. Several schools and individuals on Bitnet are asking for information, but are not flooded with responses. This could be an opportunity for MCCCD to leverage our investment in technology, and show that technology based testing is just as valid for schools as it is for industry.

Index

Ocotillo Report '93

back