Evaluating Instructional Technology



My blogs will include content and commentary for that relate to the each section of our course. Though this is meant for use by students, former students will have access to this site and I will update the blog regularly to allow others to keep up in the of instructional technology in nursing. Please know that the section objectives, learning activities and required assignments are only on the ACE course site, NUGR552 for students. Other interested individual can contact me if they would like to review this material. I hope the blog site will provide you with additional information, hints and resources for you to be successful in the course and it will be a resource for you after you finish the course. I have had many students ask about products that we had used in the past and so now they will be able to go back to refresh or pull information from this blog. 
There is not much in the literature on formal methods of evaluating instructional technology. There is even less research studies in the area of educational technology. What I have found are articles about evaluation or closely related to the subject and placed them in the readings under Lessons for this section.
Over the last few years I have used an evaluation template that works well as educators try to come to conclusions about use of particular technology. There are common criteria, I think, that we can all use so that not only can we be thorough in our evaluation but then we have a common way of communicating these evaluations. The current document, EvaluationCriteria for Educational Technology is the same link as I have on the ACE site for current students. As the class updates the criteria, the link will change. Students in the class will make edits in a wiki and then I’ll update this document when we are finished with the assignment. We have several categories defined that seem to be common in evaluation of educational technology: explaining the technology to others, active learning strategies, learner characteristics, approval, development, instructor characteristics, and cost/benefit.

Explaining the technology to others
To evaluate the technology under investigation one must know it well enough to explain to others. In most cases that means the educator will need to learn the technology well enough to demonstrate to others how it can be used by other educators. This is one of the biggest stretches some students have to take in the masters class. Learning new technology for some is very stressful. The current state of educational technology is such that many resources are free or at least inexpensive. You can easily go onto the internet and explore new developing technologies. If there is a substantial cost involved most programs have free trial versions for a month. Be careful with giving your charge card information as it is easy to forget to cancel if you decide to continue the service. In this category, it is important to identify a champion who would be willing to spend time learning the technology and then helping to implement.

Active learning strategies
Technology needs to add to learning not distract the learner with bells and whistles or needless searching. The easier and quicker, the more the user will learn. Learning styles are highly variable in individuals. Some learners believe only one learning style works for them. For others a combination of styles. To me, learning styles is difficult integrate as an educator when learners have so much variability in these styles. To combat the issue, using a combination of styles should work for everyone. We won’t get into styles any more than this since there is abundant information in the literature on the topic already.

Approval
The process of approval of use of a new technology can be the biggest obstacle. Consider the key stakeholders in the new technology from the learner to the person holding the “purse strings”. Who needs to approve this technology in your organization? Organizations differ widely from just asking administration for the money to formal proposals and presentations. The project type of approval and who approves depends upon who will be affected by this new learning. The name of the approval game is usually: can it be done quickly and have quick results. Having a good evaluation of the technology process completed will help the approval process.

Development
The development of the new technology to fit into your organization may take only minor energy up to and include a whole system requiring lot of energy for change. Questions that need to be asked include: If the educator can do all the development, will someone else be able to use the technology if they are no longer in the organization (get hit by a truck scenario)? If the project will involve more than one person, will the additional people have time to commit to the project? Will this technology take the place of an older technology? If so, do learners need time to transition or need an orientation to the new technology? How much and who should do the orientation? How involved will the new orientation become?

Instructor
The educator should be an expert in the content. Deciding on a technology without the expertise will result in a very shallow project with little meaning. Become an expert in the nursing area you want to teach then use of correct technology will enhance learning. I can’t tell you how many nurses have come to me and want to start something new but haven’t become an expert in the area first. That is my first recommendation. The educator who is developing the new technology must have perseverance, commitment, and a little obsessive nature to bring the new technology into teaching and learning. An educator can also partner with a technologist but this requires close communication and both learning each other’s roles and content expertise.


Cost/benefit
 
The analysis of cost to benefit can be difficult for educators who have no experience in this area. Use your resources in the organization to assist you in this area. There are resource allocation courses such as the one we have in masters program that teach this content. Without the criteria of cost/benefit analysis completed a proposal for approval of the project will not likely pass. In many organizations it is the development department that can assist you in starting the project. They know the financial resources and have the expertise of project development.


Resources (you may wish to start by searching "evaluation" on our Diigo Group site.  I will include a few to give you examples of links available.):

This is a great site that explains web 2.0 paradigm and also has a format so that a teacher can decide what and how they want their students to learn.  It helps you choose the right technology for your objectives. WebTools4u2use.http://webtools4u2use.wikispaces.com/Finding+the+Right+Tool#Finding%20the%20Right%20Tool-By%20Stage%20of%20Inquiry

A two day workshop I attended called Understanding by Design.  Takes a design framework and starts backwards.  A different approach. https://diigo.com/01hjj7

This Diigo link I found asks basic questions about the program you may wish to design using new technology.  https://diigo.com/01hjjf

Example of evaluating a specific technology - simulation. Dr. Margie Porter is has used this site for students in the teaching methods course. There are teaching modules available including evaluating with simulation.  https://diigo.com/01hjjl

Example of evaluation - OSCEs. This is another good reference showing how they tested reliability and validity of the OSCE. Used Cronbach alpha for reliability testing and discriminant regression analysis for validity.  https://diigo.com/01hjk5

Here is a specific evaluation method for web 2.0 content.  Other information can also be found on this site.  https://diigo.com/01hjlc

No comments:

Post a Comment