Evaluating Educational Software for school and home
The Form
Description of Elements of the Form

Before selecting software, teachers and parents need to understand the limits of using software exclusively as a teaching tool. Software, or computer-based teaching, is not able to make connections to what the student knows and deals with in daily life. Teachers, parents, tutors and peers can do this if they are involved with the learner as he or she interacts with the program. Think back to the discussion of technology as a "tool" as mentioned in the first chapter, and imagine that software, especially educational CD-ROMs, can be a valuable tool when teachers and students use them together. The way software or hardware is used and the structure surrounding the activity has significant instructional value, beyond what just the software or hardware includes. If we look at the research on the use of educational software in the classroom, it seems that when kids use software they perform better on basic skills and knowledge types of standardized tests, but no better on assessments that measure application and evaluation of information. Software works best when teachers and students talk about the experiences and material being presented in the programs. Talking about this allows you and the student to develop greater understanding and application of the program’s information.

Many checklists or rubrics for evaluating technology exist that are available across the board. Most attempt to describe the technical aspects of a piece of software. Special education teaching is not new to this process of finding techniques for evaluating teaching methods. All teachers have strategies for determining whether or not to use a technique, just as all professionals have ways of determining if a product or process is right for them or their "client" (in our case the student).
 

  • Teachers use all sorts of strategies in determining if a technology product is right for their student. Researchers from regular and special education have attempted to create models for software selection over the past decade as more and more software programs have been developed and marketed toward education. For example, the following eight sources were identified by Malouf in a study in 1989 as preferred sources of information by special education teachers for selecting software for use with students with disabilities:
        1. talking with other educators
        2. tryout with students
        3. tryout without students
        4. written ratings/descriptions from evaluations or reviews
        5. pictures of sample screens from evaluations or reviews
        6. school system lists of recommended software
        7. documentation or manual included with software
        8. software catalogs/advertisements.
    These all may sound like preferences you would use today when selecting software. In fact, it appears that not much has changed in the last decade with regard to the way special education teachers select software titles for instructional purposes. Researchers from both regular and special education continue to create models for software selection.

    One model for software selection described by Smith & Vokurka (1990) states that teachers must begin the process of evaluating software by first examining relationships between people and procedures. These include:

    These researchers distinguish between the management process and the instructional process for software selection. According to Smith & Vokurka "The selection process must be grounded on the principles of functional utility of the chosen software and not on the creation of ‘window dressing’" (p. 37). They also suggest an effective evaluation instrument ought to be: As the research in the area of software evaluation progressed throughout the decade, the fundamental concerns of teaching and learning foundations have not been disputed. For example, researchers such as Lahm & Nickels (1999) report that it is essential for all teachers to have the knowledge of procedures for determining if a software program or assistive device has potential for a student or class of students. Often school districts have a particular procedure defined for general software evaluation, but the specific learning needs of students in special education may demand additional or adapted procedures. The essence behind this process is the understanding that technology will be matched to the student’s specific needs, with the goal of keeping the student on target within the curriculum and moving toward meeting the student’s IEP.

    Additional research by Moore (1990) identifies a variety of additional recommendations that should be used by those who work with special needs students in the process of evaluating software. Moore’s (1990) list includes:

    Concerning the program’s instructional soundness he states "unless you are an expert, you may not be able to make that kind of an assessment, however, if the program looks good on the evaluation criteria it will score well on other counts as well" (p. 61). His evaluation criteria are fundamental and look at these characteristics: (a) computer compatibility; (b) documentation thorough, clearly written, and well organized; (c) focus on skills (d) supplement or complement work; (e) appropriate for age, grade, & reading level; (f) can it be customized; (g) design features — music, colors; and (h) use of adaptive devices.

    In addition, one of the most important steps in the process of evaluating software is for teachers to test the application by first using it themselves. You may want to sit down with a student who understands his or her instructional needs and carefully go through the process together while you both talk about the software program. This idea of "talking out" while using the software encourages the student to constantly evaluate their own understanding in any academic procedure, but in particular with software this can encourage students to analyze how well the program is helping them. Teachers and students, in the process of evaluating software should sit together and describe what they are doing, with phrases like "Let's see à it looks like we are supposed to do this now", "How does this program help us accomplish our goal". This gives you a good idea of how well the software program really works and how well that student understands the directions and processes of instruction.

    Again, the best use of many educational software programs is when a group of students or a teacher and student work together and discuss the use of the program while they use it. One of the most important elements of the IEPs of many special needs students include skills related to communication, be that about mathematics, the written word, or about the weather. These communication skills are core to a student's ability to function and demonstrate their knowledge throughout their educational program.
     
    The software evaluation form depicted below is designed to be a simple tool for you to use while in the process of evaluating software. This form, also included in the resource packet, is intended to be something you can copy and share with other teachers, or with parents who are willing to evaluate software for you. Using a "universal" set of standards when examining software will make it easier to compare and contrast when selecting programs for use in the special education classroom or by any teacher or parent who would like to use software to support a student's learning or shore up a set of underlying skills and sub-skills. 

    Software Evaluation Form

    1. Name of Software:______________________________________________________

    2. Price:_____________ Publisher:________________________________________
       
    3. Hardware requirements (include RAM and ROM needed): _______________________

    4.  

       
       
       

      _____________________________________________________________________
       

    5. Content area for which the software is designed: ______________________________

    6.  

       
       
       

      _____________________________________________________________________

    7. Type of knowledge program addresses: _____________________________________

    8.  

       
       
       

      _____________________________________________________________________

    9. Process and type of instruction:
    • Discovery
    • Drill and Practice
    • Simulation
    • Other: ______________________________________________________
    Rate items 6-14 with a score (5=excellent, 4=good, 3=fair, 2=poor, 1=unsatisfactory, 0=non-applicable) and a written description.
    1. Ease of Use (clear instructions, installation) independence for students, and interface:

    2.  

       
       
       

      _____________________________________________________________________

      _____________________________________________________________________

    3. Age Appropriateness: ___________________________________________________
    4. Active learning on the part of student (v. passive learning behavior): _______________

    5.  

       
       
       

      _____________________________________________________________________

    6. Degree of open-endedness and flexibility: ___________________________________
    10. Clear documentation and good support: _____________________________________

    11. Follows principles of learning

    • Matched to instructional level of students: ______________________________ 
    • Appropriate vocabulary: ____________________________________________ 
    • Ability to engage students: __________________________________________
    • Expanding complexity: _____________________________________________
    12. Technical Soundness
    • Animation: ______________________________________________________
    • Colors: _________________________________________________________ 
    • Sound: _________________________________________________________
    • Printing: ________________________________________________________
    • Saves student’s work : _____________________________________________
    • Uncluttered, realistic graphics: _______________________________________
    • Consistent operation: ______________________________________________ 
    13. "PC-ness"
    • Mixed gender and role equity:________________________________________
    • People of diverse cultures: __________________________________________
    • Diverse Family Styles: _____________________________________________
    14. Opportunities for transfer: ________________________________________________

    _____________________________________________________________________

    _____________________________________________________________________

     

    1. Name, price and publisher of the software.

    In any evaluation, it is important to not only keep track of the name, publisher and price of the program, but the date that the evaluation was completed. Software programs are available in so many formats, and the same titles are used for very different programs. This basic information is essential to a useful evaluation as a method to obtain the program in the future if it is selected. 2. Hardware requirements of the software (include RAM and ROM needed) From a technical standpoint, the simplest question with regard to choosing technology is "Will it run on my computer?" The hardware requirements of any software program need to be examined before one selects a program to evaluate. Computer hardware will be discussed later in the chapter, but it is important to keep in mind that many of the newest programs require state of the art computers, with a great deal of storage space and a large amount to memory to run at all.

    RAM = Random Access Memory, this is the additional memory installed on your computer, not part of the computers hard drive.

    ROM = Read Only Memory, this is the memory on your hard drive.

    3. Content area for which the software is designed. When selecting a program, the obvious method we all use is to look at the title and packaging to determine what skills and content that program is designed to teach. As with other products, you can’t always judge the product by how it is packaged. As an evaluator it is your responsibility therefore to determine if, in reality, the packaging is correct. In other words, does this program focus on math (or reading or science or a foreign language) and engage the student in effective mathematical thinking and learning?

    Another reason to investigate the content area of the software is because often programs that are designed to teach one area of the curriculum area are useful in other areas as well. For example, Sim City, a popular software package designed (as stated by the manufacturer) to be an experience in city planning and organization (and used in social studies primarily), can be a great tool in a social skills curriculum or in a math classroom. However, in just as many cases, programs marketed to teach math or reading are simply video games, with a few letters or numbers thrown in and no "instructional use" whatsoever.

    4. Type of knowledge addressed within program. In addition to selecting software within the appropriate content area, as an evaluator you must also attempt to assess what type of knowledge the program is designed to address. Beyond the content area of mathematics, what types of mathematical skills does this program address. There are many ways to classify learning outcomes from Gangé's learning hierarchy -- memorization underlies concept learning which underlies principle learning which underlies problem solving -- to the task analytic approach which involves breaking down a complex task into its component or prerequisite skills. Special education teachers have training to analyze an instructional objective and break it down into the sub-skill and strategies that make up the larger instructional goal. When designing instruction, special education teachers assess what sub-skills and strategies the student already has and teaches to the specific point where the student needs assistance. This is known as "precision teaching", and is a common process that most teachers go through without even thinking about it. When selecting an instructional activity for the student, the teacher focuses his or her teaching on the level that the student needs. The same process must be used when selecting a software package.

    Another way to think about type of knowledge is to use Bloom's taxonomy. Most teachers are familiar with this way of breaking down types of skills and it is easily understandable by others. In the cognitive domain of Blooms Taxonomy there are six categories which are considered hierarchical so they are usually referred to as levels of intellectual objective ranging from Knowledge (lowest level, example: recalling information) to Evaluation (most complex level, example: using previous learning to determine the worth or merit of problem). Higher level tasks subsume tasks at the lower levels, so if students are engaged in a synthesis task, the student must also demonstrate all the other tasks below it.

    COGNITIVE domain AFFECTIVE domain PSYCHOMOTOR domain
    1. Knowledge 1. Receiving 1. Reflex movements
    2. Comprehension 2. Responding 2. Basic fundamental movements
    3. Application 3. Valuing 3. Perceptual abilities
    4. Analysis

    5. Synthesis

    4. Organization

    5. Creating

    4. Physical abilities

    5. Skilled movements

    6. Evaluation 6. Characterization by value 6. Non-discursive movements

    5. Process and type of instruction

    Discovery, Drill and Practice, Simulation, etc. (other….)

    Another important piece to consider about any software package is the process and type of instruction. The process and type of instruction of a software package is the method used to engage the user. This can include discovery learning where students actively explore, question and try out their ideas to develop further knowledge and information. It is important that the process and type of instruction match the students needs otherwise the software cannot provide an engaging or learning experience for the student. 6. Ease of Use (clear instructions, installation) independence for students, and interface How easy is it to load the program, start the program, run the program and use the program? Will the student be able to run the program independently? Are instructions clear and available? If a software program does not provide clear instructions for installation or use, or is difficult to use the software program will probably not be able to meet the needs of the student and therefore not be an effective technology tool for the student. Evaluating the software first on your own or with a student will help you identify if the program can be used independently and successfully. 7. Age Appropriateness What age level is this software designed for? Consider the vocabulary used, graphics, layout and design. You want to make sure that the software package is age appropriate to the age and level of your student. This can be a problem for all students. For example, with older students many remedial software programs are targeted toward younger students and so are seen as ‘babyish’ by older students. A program can not be as beneficial or successful toward meeting a students needs if the student is unable to relate or connect to the level of instruction. 8. Active learning on the part of student (v. passive learning behavior) When you are evaluating any software package how are you the user being engaged? Does the program engage you mentally, physically, and emotionally? Or can you simply point and click your way through the program, not having to actively participate and interact with the software? Research shows the more time a student is engaged in active learning time the more knowledge and understanding they develop. 9. Degree of open-endedness and flexibility How much flexibility is there for the student in using the program? Can the student create their own original creation or are their defined templates the student has to use? For example, software applications like Storybook Weaver versus HyperStudio. Both are popular programs that allow students (or any user) to create their own story or presentation. Due to the lack of individuality on the part of storybook weaver, we push students those less and less by means of intervention (fading) the amount of help a student needs to complete a task. 10. Clear documentation and good support
      Is the software documentation clear? Is technical support available that you can contact to take care of problems with the software? How and when is the technical support available? If support or documentation is available, but you can’t locate those materials or you can’t understand the documentation then you can not use the software as an effective learning tool.
    11. Follows principles of learning
      • Matched to instructional level of students:
      Does the level of instruction match the age and level of the student?

      • Appropriate vocabulary
      Is the vocabulary appropriate for the age and level of the student? If the vocabulary is either too difficult or too simple the student may be turned off to engaging or using the program.

      • Ability to engage students
      It’s important that the students are engaged with the software, you don’t want them to get bored working with the software.

      • Expanding complexity
      Are their different levels of difficulty within the program? Once a student has mastered a desired level, can the student be challenged with further application of the subject matter?
       

    12. Technical Soundness Looking at the technical aspects of the software, are the animation, colors, and sound vivid, enticing and appropriate? Is the layout and design consistent, so students are able to find their way around the program? Watch out for graphics or color schemes that are distracting because they may be counter-productive for students — drawing their attention away from the material and away from an active learning environment. Does the program allow the student to save their work, or save their place so they can come back and resume at a later time? Depending on the student and their needs this may be an important feature for some students and not for others. Can you print pages from the program…are they readable? Also look at: • Animation

    • Colors

    • Sound

    • Printing

    • Saves student’s work

    • uncluttered, realistic graphics

    • consistent operation


    13. "PC-ness"

    Mixed gender and role equity — Is the software free of gender or role biases?

    People of diverse cultures — Are there any assumptions being made within the software about what the user would or would not know based on their background or neighborhood? For example, some children may not know what a "subway train" is, is this explained someplace?

    Diverse Family Styles - For example, does the program assume that all children are living in homes with two parents?

    14. Opportunities for transfer

    This standard examines the degree to which the program provides students with an opportunity to apply the skills they have just learned to another setting. A math software program should not just provide skill and drill, it should present how those skills would be used in a more "natural" setting. For example, student should not only be presented with practice on identifying coins, but on how to count back change.