QUESTION:
What are examples of relatively "easy" to obtain measures of technology use that provide meaningful data that support future planning and institutional goals? These should be things that can be done on a consistent basis.

e.g. inventories of technology, computers-student ratios, percentage of classrooms equipped with technology, hours of availability of computer labs, measures of software usage

IDEAS:

1. Survey students, faculty & staff - include focus groups
2. Comparison - experimental vs. controlled (technology vs. no technology) - pre & post results
3. Interactive assignments (web-based) in class vs. independent
4. Build technology measure into the institution wide assessment plans
5. Develop measure at learning success (i.e. job placed) certifications earned, retention, pre-post competency-based testing) as compared to technology investments (budget, space, level of technology, # of units)
6. Best practices of similar metrics @ other institutions
7. Technology staff retention vs. #7 units, budget)
8. Faculty customer satisfaction survey and follow-up
9. FTSE to technology $ investment ration; exit reviews, student and staff
10. Need a standard test of computer literacy
11. How effective is particular technology to a learning situation - must be well designed and matches the competencies
12. Do a traditional research study of comparing classes - look at retention and completion, achievement of competencies, portfolios, esp. time to learn concepts - w/out technology
13. Use attitude surveys/questionnaire to assess students feelings about impact of technology on their learning
14. Do " Pat Cross" type of classroom research
15. Present student evaluations - integrate questions regarding technology
16. Focus groups using students in technology-enhanced classes
17. Student surveys or student perceptions of the use of technology
18. In order to measure the impact of technology, the variable of technology has to be isolated
19. Student surveys
20. Comparison students
21. Pre-assessment = prescription
22. What students are using; tracking progress with technology to see
23. Post-assessment = achievements
24. ASU students - DO students survey; Also ASU faculty
25. Look at student learning at program level
26. Measure technology literacy on program and employment level
27. In appropriate use of technology hinders learning
28. Create objectives - measure learning against objectives
29. No difference from assessing delivery of learning in any other way
30. This question assumes all learners are able and all uses of technology are the same
31. Maybe we should be assessing technology
32. Maybe we should be assessing learning
33. What ever facilitates learning should be encouraged and supported
34. Feedback from students - pre/post testing, surveys, performance-based " what tool has been most useful to enhance your least useful education?"
35. Survey potential employers of students " What skills do students need for their future work?"
36. Student portfolios - use outcome of process
37. Define learning - it' s a poorly phrased question
38. Classroom management - any tool is only as good as Ôhow' you use it
39. Instructor - appropriate of accessing resources; - measure/compare results of students vs. types of resources used
40. Technology is the independent variable, not dependent variable. Curriculum still the guiding direction - technology is the tool
41. Surveys
42. Focus groups
43. Classroom research (Quantitative and Qualitative measures)
44. Comparison between groups - users vs. nonusers
45. Comparisons across campuses
46. Measure learning - define then discuss assessment
47. Control group study - proficiency assessment
48. Correlate with learning style
49. Student self-assessment
50. Cost-effective measures
51. Define technology to be assessed - isolate variables
52. Focus on short-term interventions might increase ability to control uses in study
53. Longitudinal assessment - apply technology initial class - assess enhanced skills in later classes
54. After initial exposure to tech, how many students above subsequent technology during classes
55. We don' t think this can be Ôexperimental' research; it has to be Ôreal world'
56. Ask students and instructors - systemically look at the feedback and change what you' re doing accordingly
57. Surveys of student experiences/perceptions - Look at the research
58. Comparative studies using different types of technology, or technology/no technology
59. Categorize learning and ways to evaluate
60. Standard, define end-product as evaluation tool
61. Make sure technology is being used properly
62. Assessments of graduates comparatively with those who have used technology with those who have not. Continuous assessment of new students as they progress through the institution
63. Add a question regarding the use of technology to the semester or course evaluation
64. Formally assess learning outcomes via a systemic tool such as Flashlight
65. Identify profile of successful " ?" learner using distance learning
66. Make critical analysis of the use of technology vs. traditional methods
67. Learned is really hindered when the technology does not work
68. Technology must be a means not and end
69. Research based outcomes assessment using historical data
70. Competency in the class that used technology vs. the one that didn' t - common pre-test & final exam
71. Use technology to enable something that could not have been done otherwise. Aim for the technology based course to be better than the other one.
72. (FORCE) - sample of test used at SMCC - standardized tests to access learning
73. Survey students - how do they feel about the technology - beneficial?
74. Assess students ability to teach one another
75. Collect usage data from computer services and compare this data with longitudinal classroom assessment data. An example in using links to do research in English (research skill).
76. Using discussion groups and compare with chat room groups
77. Ask the students
78. Short in-class assessment of technology as it is used in the classroom - classroom research
79. Add technology evaluation to adjunct faculty evaluation tool
80. Use technology to INFORM students of their progress - new INFORM system
81. Look at national research
82. Faculty feedback
83. Instructional Design is crucial - instruction must drive technology, not vice-versa
84. Can we finally stop addressing the issue - since the American culture requires technical literacy, we should STOP beating ourselves up with this question
85. If we must assess technology use qualitative
86. All research is flawed in some way
87. Define technology in learning - What do we mean?
88. Should be a look at what improves instructions, not just technology for technologies sake
89. Pre/post in some situations especially where prior knowledge not required
90. Ask students, faculty, support staff - survey and focus groups
91. Ask whether it takes less time to learn
92. Compare student scores, knowledge, retention in technology courses vs. non-computer courses
93. Talk with students and faculty to obtain verbal reactions to use of computers; also talk with employers
94. Look at cost vs. benefits of technology in classrooms
95. Assess students ahead of time to determine learning styles