ocotillo central

A&E of Technology... "Assessment and Evaluation" - looking at the impact of technology on learning

A&E of Technology
about
  timeline
  June 1998 Forum
  July 1998 Forum
resources
surveys

Ocotillo
ocotillo central
roundtables
retreats
spotlight

Assessment & Evaluation Ocotillo Forum
June 25, 1998 at District Office room 311

Participants

Welcome and Introductions

Why Are We Here?

• When the Ocotillo chairs discussed 5 main topics that they felt were important, Assessment and Evaluation was the number one issue.
• At the May Retreat this topic kept blossoming into larger issues of A&E and became harder to separate, a continuing discussion was proposed.
  (see http://www.mcli.dist.maricopa.edu/
  ocotillo/retreat98/r1.html)
• We need to involve more people in this issue.
• What are some things we need to do at the district level and what needs to be done at the college level to help resolve this issue?

Remarks from Dr de los Santos
(Vice Chanceller, Student and Educational Development)

• The Arizona Academic Program Articulation Steering Committee created the "Arizona General Education Curriculum" - which is 35 hours that students need to take in order to be eligible to attend a University. A recommendation from this committee was that the Community Colleges track these students.
• Foreign Language task force is defining state level competencies.
• There are definitional problems, parameter problems, methodological problems etc. with trying to assess student learning. How do we address NCA, Governing Board end statements and state-wide competencies?

Remarks from Doug Sawyer
(Ocotillo Chair from SCC and co-leader of A&E forum at 1998 Ocotillo Retreat)

3 Highlights from the Ocotillo Retreat A&E conversation(s)

• Inventory Assessment [i.e. How many students per workstation? How many faculty per desktop? How many staff per workstation? etc.] compared to national standards [inventory that is easy to do].
• How to measure effectiveness of technology teaching and learning.
• Do we have proper expertise to perform these assessments?

General Discussion

• Do we need to convene a larger group that can break up into committees to discuss each branch of this issue.
• We need to figure out what kinds of data we need.
• The bond as a motivator.
• What is the value of data that might be easy to collect? [i.e. numbers of computers]
• For each computer, how many student hours are being used? Even though this may not be important information for us necessarily, the community might think it is.
• But is the average number of hours more important than the outcome of that technology?
• Possible Categories of Investigation [Andrea Greene]:
  • Assessment student learning in regards to technology.
  • Program evaluation.
  • Cost benefit analysis.
  • Describing the inputs and the infrastructure.
  >We have a lot of faculty that may have expertise in one or more of these categories.
  
  
• We can count the number of computers, the number of students using those computers, and how much is costs to upgrade or replace those computers. What we don't know is what difference it makes in terms of our students and our programs.
• There is no known tool to assess how technology truly impacts student learning.
  
  
• What is it that we want students to know about technology?
• How are students technologically literate?
• What is the standard for information access and computer literacy?
• Do students have computer literacy skills based on pre-determined outcomes of what we think a computer literate person should be or does technology in the classroom help to define the student learning?
• The outcome that students need to be computer literate - that is what we measure.
  
  
• We need to assess student performance in general instead of discipline specific.
• There are so many different ways of teaching a course that there is no way we can evaluate all of them.
• The Boards end statements are now aligned with the general education curriculum criteria and should be aligned with the competencies of all AA degrees.
• Don't do an individual technology A&E, include it in an overall learning assessment.
• Before we can start taking about program level assessment, we need to have some expected outcomes for students to complete an associates degree.
• Cecelia Lopez [NCA] puts assessment simply, you do something, you look at its effectiveness, and then you change it accordingly. A simple feedback loop.
• NCA says we should limit our goals to 1 or 2, so they are achievable.
• The ongoing process of evaluation is more important than any instrument or result.
• In regards to Angelo and Cross' study on classroom research - When you teach a unit or class, you ask 5 or 6 questions of the students, then you adjust your class based on what the students said. Simple questions that are easy to respond to and evaluate. Can we do this for technology? We could have 5 simple questions and a 6th that dealt with special categories like Nursing or Allied Health Programs.
  
  
• We need to have baseline, pre- and post- tests, and after certain experiences measures. How do we do this?
• We need to begin with benchmark data and a year from now we need to figure out how to move forward.
• If the community has several important things it is looking for in a worker, then maybe we should assess that?
• How effective are we as teachers and how do we assess that?
• Where are the faculty in terms of using technology? We should determine where they are and try to move them forward. Work with faculty first - we can't measure student outcomes with out faculty input.
• We should assess attitudinal date from the students and faculty in regards to technology. How do we start?

Summary

• Outcomes Assessment is a huge undertaking; work is underway at the DI level [October event at SCC].
• Ocotillo groups should be connected with larger A&E issues but can address the A&E of technology.
• Develop a set of "easy things" for colleges to inventory that would provide useful data for a 3 year plan and support the bond.
• Develop a set of questions as "faculty" survey of technology use [is this done already]?
• Develop a set of useful questions to survey student use of technology [look at Flashlight inventory?].