Packing Slip 
item |
Usability Testing to Improve Online Writing Classes |
contact |
Susan K. Miller (Mesa Community College) susan.miller@mcmail.maricopa.edu |
credits |
Shelley Rodrigo, Mesa Community College Naomi Story, Mesa Community College Ken Costello, Mesa Community College Bob Farwell, Mesa Community College |
college(s) |
Mesa Community College |
discipline(s) |
English, Writing, Distance Learning |
summary |
We used the results of a usability testing study to revise online sections of ENG 102 (First-Year Composition). In addition, we wrote two sets of guidelines to share with faculty interested in developing distance learning courses and conducting usability testing to improve online course design. |
details |
During the spring semester, the applicants conducted usability testing on their existing online ENG 102 courses. Overall, 12 students were videotaped during the testing sessions for approximately 1 hour each completing tasks in our online writing courses. In addition, students completed a questionnaire about the course, their use of materials in the course, and their backgrounds (both in terms of computer/internet usage and writing proficiency). This learning grant provided the opportunity to use the data collected to revise our courses to improve student learning and to develop guidelines to share with colleagues engaged in online instruction. During the time covered by this learning grant, we completed the following tasks: --Watched and analyzed the videotaped data and the collected questionnaires. During this portion of the project, we: · Took detailed notes to analyze each student's interaction with the course; and · Took detailed notes to analyze the processes and methodologies of usability testing, noting what worked well and what could be improved. --Next, we cross-analyzed the collected results. During this portion of the project, we: · Cross-analyzed the results of student interactions during the usability testing sessions; and · Cross-analyzed the collected responses and observations regarding usability testing processes. --After completing the analysis, we designed and developed course revisions and implemented those course revisions to improve the design of our courses and positively affect student learning. --Finally, we revised, reorganized, and rewrote our analysis of usability testing processes into two sets of guidelines that could be shared with faculty interested in designing effective online courses and conducting usability testing to improve their course design and delivery. We will present this work at a Faculty Symposium at the Center for Teaching and Learning, and we are also using this work as the basis for a proposal to edit a special issue of the journal _Computers and Composition_. How did the project improve, advance, or enhance student learning? Please provide specific examples. Online distance learning courses are usually designed, developed, and structured in a manner that "makes sense" to the instructor. A vast majority of the time the instructor not only has more experience with the course content material, but also with the course delivery material (web pages, courseware, discussion boards, etc.). Therefore, the instructor has the advantage of a more detailed understanding of the "logical" organizational patterns between the course content and course delivery interface. When a new online distance-learning student accesses the course, however, both the course content material and the course delivery environment may overwhelm him or her. Without knowledge of the content (which is the point of him or her being in the class, to learn the content) or experience with the specific online delivery interface (even if s/he is an "experienced" online learner, s/he may be new to different types of environments), the student is at a distinct disadvantage in starting the course. We were able to have actual students participate in assessing online courses with usability testing methods, and therefore we could identify many of the spots in the course that students have difficulty accessing and/or understanding. This grant allowed us to take the time to carefully analyze the results of the usability testing sessions conducted in the Spring of 2003 and implement the needed changes into our online ENG 102 courses, helping future students quickly settle into our classes. For example, we found that a simple link on the course homepage asking "What Do I Do First?" guaranteed that students would normally look for directions for starting the course there and wouldn't get lost in the layout of the course homepage the first time they visited it. Furthermore, we will be able to share the results of our research with colleagues, and that will ultimately affect students beyond our own classes. By also publishing in national venues, we will further affect student learning through participating in scholarly discussions about usability testing, distance education, and its effects on student learning. How did you evaluate your project's success? What did it tell you? We are still in the process of evaluating the success of our project. First, we will be able to conduct a future round of usability testing on the same online distance-learning courses, discovering whether or not the changes we made are beneficial to students. Whereas the vast majority of these revisions are concerned with course design and layout, we simultaneously continue to make revisions based on course content. Since form can not be completely separated from content, the content revisions will always be in need of future rounds of course design and development testing and revision. In addition, we will evaluate the success of our project results by sharing them with colleagues at MCC and asking whether or not the guidelines we have written are helpful to them in designing their courses and conducting usability testing. Third, by submitting articles for publication on our research to peer-reviewed journals, we will be able to benefit from having our colleagues read our research and comment on its strengths, while providing suggestions for revision and improvement. Finally, usability testing itself is a formative evaluation of the design, development, and implementation of online distance learning courses. By conducting these tests and using the results to revise our own courses and help others to evaluate their courses, we are modeling a way to evaluate the success of distance learning and its potential effects on student learning. What is your plan to share this project with others? We will share the results of this project in a variety of ways with colleagues both in the district and nationwide. At the college level, we will share our results through the Center for Teaching and Learning by leading a Faculty Symposium on Usability Testing on September 24th. In our discipline (English studies and specifically Computers and Writing), we will share the results of our research through conference presentations and submitting our work for publication in peer-reviewed journals. Susan has already presented part of this research at the Computers and Writing conference held this summer at Purdue University. In addition, we are planning to propose a special issue of the journal _Computers and Composition_ that will focus on usability testing. What, if anything, will happen in the future with this project? We will be presenting the results of this project (the two sets of guidelines) at a faculty symposium sponsored by MCC's Center for Teaching and Learning on September 24th. In addition, we are writing a proposal to edit a special issue of the journal _Computers and Composition_, dedicated to usability testing. We have discovered that there is a great deal of interest in the work that we have been doing, and we would like to encourage others who are studying usability testing to share the results of their research and work. What, if anything, would you do differently? We found that watching and analyzing the videotaped testing sessions was time consuming, and we might try to streamline the process that we present to faculty so that others who conduct usability testing sessions have some options that will be more efficient. Note! As a professional courtesy to the owner of this package, if you use some aspect of this package or have some thoughts about it, please share your feedback via the comments form below. |
supplements |
Guidelines for Conducting Usability Assessments (document) Guidelines for Developing and Designing Online Courses (document) Note! As a professional courtesy to the owner of this package, if you use some aspect of this package or have some thoughts about it, please share your feedback below. |
shareback |
A "Shareback" cites the places on the web that mention, reference, or use this MLX package, and "shares" that information back here (more about shareback...) Sharebacks can be generated automatically by weblog tools. |
extra |
Last modified: Sep-10-2003 Date created: Sep-10-2003 Visitor count: 4004 Dublin Core Metadata record  This package is included in the Maricopa Learning Grants special collection. |
Maricopa Learning eXchange
http://
