Index: Educational Technology
Assessment Instruments and Design Issues
Please excuse our mess. The information on this page was taken directly from the following document:
Mathematica Policy Research, Inc. (2000). EVALUATING THE TECHNOLOGY PROFICIENCY OF TEACHER PREPARATION PROGRAMSÕ GRADUATES: ASSESSMENT INSTRUMENTS AND DESIGN ISSUES. Washington DC: U.S. Department of Education.
To access the report, CLICK HERE
Parts of the public domain document are used here to generate class discussion.
Page Contents
types of assessments
online exams
portfolio assessments
performance assessments
interview protocols
self-assessments
---
rubrics can be used with combinations
•
· Online Exams usually consist of multiple-choice questions (or questions of similar format) that are electronically administered and evaluated.
· Portfolio Assessments usually consist of a rubric that aligns the contents of a studentÕs portfolio with specific technology competencies.
· Performance Assessments require that a teacher complete several technology tasks and create separate files using application software that will later be reviewed by a trained evaluator.
· Self-Assessments typically provide a list of technology competencies or skills and asks the preservice or K-12 teacher to rate their ability in those skills.
Two instrument types that were used less frequently include:
· Interview Protocol provides questions for discussion about preservice studentsÕ abilities with technological tools.
· Demonstration and Observation methods detail technology tasks to be performed, either as part of a test or in a K-12 setting, that are observed and evaluated by a trained evaluator.
   
3. Competencies
3. Competencies
Most of the assessments are based either on the International Society for Teacher Education (ISTE) standards or on state standards that are based on ISTE standards (see Appendix A for ISTE Standards). ISTE is responsible for recommending standards in education technology to one of the official bodies for accrediting teacher preparation programs, the National Council for Accreditation of Teacher Education (NCATE). The ISTE standards outline the fundamental concepts that all preservice teachers in education programs should master.
The assessments generally evaluate preservice and K-12 teachersÕ technology proficiency in several or all of the following five competencies :
· Basic Technology generally includes basic computer terminology and usage, such as creating files and folders.
· Software includes proficiency with word processors, spreadsheets, databases, presentation software, e-mail, and the Internet.
· Ethics includes concepts such as software copyrights and privacy issues.
· Basic Integration focuses on a K-12 or preservice teacherÕs ability to integrate technology into the K-12 curriculum and are included either as an independent competency that is tested separately or interwoven into questions relating to the previous competencies.
· Advanced Integration includes using appropriate media and technology resources to address differences in studentsÕ learning and performance. Also includes teachersÕ ability to select and create activities that incorporate the use of media and technology and are aligned with curriculum goals, based upon principles of effective teaching and learning, and support active student involvement.
Sample questions
1. Attention can be drawn to specific words within text through the use of _________.
a) the paste special editing feature
b) font formatting features such as bold (correct answer)
c) the page setup command
d) the select all command
1. Idaho Technology Competency Exam
1. Idaho Technology Competency Exam
a. Content, Administration, and Evaluation
Preservice teachers in Idaho can complete one of three evaluation instruments for state certification: the competency exam, the portfolio assessment (section C1.), or the performance assessment (section D1). Originally developed by technology coordinators in one region in Idaho, the competencies were reviewed by teams of state educators and then aligned with ISTE standards. The competencies cover four of the five major categories:
· Basic Technology Competency. The Computing Environment (understanding basic computer hardware and software and working with files)
· Software Competency. Word Processing (creating and editing documents with a word processor), Instructional Software (selecting, evaluating, and using software for the classroom), Telecommunications (using e-mail and the Internet), Presentation Software (using software and hardware to develop presentations), Spreadsheets (manipulating spreadsheets), and Databases (working within a database).
· Ethics Competency. Issues in Information Technology (understanding ethical, legal, and diversity issues as they relate to the use of technology).
· Basic Integration Competency. Issues in Classroom Management (developing technology-integrated curricula and assessing student technology projects).
The assessment does not address advanced integration competency.
Since 1997, more than 12,000 preservice teachers in Idaho have taken the exam. The Idaho State Department of Education has also provided the test to schools in five states (Pennsylvania, Illinois, Hawaii, California, and Michigan), and the international organization FUTUREKIDS.
The technology competency exam is typically delivered online, although it is also available in Scantron format. The test was piloted in 1995 and then implemented in 1997 to meet the state technology assessment requirement. The test is programmed in Java with a "front page" user interface to maintain user security. The Idaho Department of Education has reported few malfunctions and only one instance in which test-takers lost partially completed tests and had to restart the exam. (This occurred in an area in which Internet access was still being fine-tuned.)
The test contains 77 questions and requires one to one and a half hours to complete. Each batch of questions (from which the 77 multiple-choice questions for each exam are drawn) costs about $10,000 to develop, and preservice teachers are charged a $5 lab fee for the examination. The test is administered several times each year in one location (Boise State University), but individual districts may make arrangements to administer the test separately. Trained proctors oversee administration.
The estimated cost for the U.S. Department of Education to use the Idaho Technology Competency Exam varies depending on the detail of the information required. For a minimal report and summary of each group of test-takers, the cost would be $5 per individual. For individual scores for each test-taker, the cost would be an additional $1 per person and for results for each competency area, the cost would be an additional $1 per teacher.
In analyzing the test, the state department of education has performed the following statistical and validation procedures: content validity, construct validity, coefficient of internal consistency, item difficulty index, item index of discrimination, item reliability, and concurrent validity. The examÕs reliability ranges from .82 to .95 for different administrations and the current batch of questions has a reliability of .89.
The tests are graded electronically so there is no training necessary for evaluators and the time required to complete the evaluation is limited.
b. Strengths and Limitations
One of the strengths of this test is its high reliability, achieved because the state of Idaho has invested a great deal of resources. Due to the cost of developing questions for the exam, the Idaho State Department of Education released for review only a sample of 21 questions from different topic areas. The test covers a range of technology skills, including technology integration.
The major limitation of the test is that it is difficult to assess the depth of an individualÕs technology proficiency from multiple choice questions. Knowing the correct answer to select from a multiple-choice list and being able to execute the task in question require different skill levels. For example, a question such as the following is limited in its ability to test whether or not a student can perform the operation:
1. Attention can be drawn to specific words within text through the use of _________.
a) the paste special editing feature
b) font formatting features such as bold (correct answer)
c) the page setup command
d) the select all command
Although all online exams will suffer from this same drawback, other exams, such as the Teacher Universe Curriculum Integration Assessment System (see below) offer question formats that allow for more in-depth questions.
2. Teacher Universe Curriculum Integration Assessment System
2. Teacher Universe Curriculum Integration Assessment System
a. Content, Administration, and Evaluation
Teacher Universe is a firm that provides instructional technology planning, professional development, instructional tools, yearlong curricula, and career and life services to K-12 teachers. It is currently a member of the California Commission on Teacher Credentialing PT3 Catalyst grant consortium, for which it is providing training and assessment resources. Its assessment covers four of the five major competencies:
· Basic Technology Competency. Operating Systems (creating, naming, and saving files and saving and retrieving files on diskettes)
· Software Competency. Databases (opening, designing, and navigating databases and sorting data), Internet (conducting research on the Internet, communicating with students and colleagues electronically, and understanding the legal, ethical, and safety issues associated with Internet use), Presentation Tools (creating, saving, and closing a presentation and layouts and understanding and working in various views). Spreadsheets (writing basic formulas and changing number formats), and Word Processing (formatting documents, using editing tools, and inserting Clip Art).
· Ethics Competency. Technology Awareness (demonstrating confidence in ability to maintain computer systems and use core software applications and knowledge of legal and ethical issues associated with computer use)
· Basic Integration Competency. Curriculum Integration (selecting appropriate software applications for varying classroom contexts and using curriculum-based software) and Technology Usage (using computer technology in the classroom and in personal life).
This assessment does not address advanced integration.
Teacher Universe developed the Curriculum Integration Assessment System as a pretest to determine the appropriate level of technology proficiency for teachers and as a post-test to determine improvement after completing the course. The current web version of the survey, introduced in March 2000, was developed from a disk-based survey completed by more than 1,000 teachers, the overwhelming majority of which were K-12 teachers. The assessment is also being used with preservice teachers at the SCDEs in the PT3 consortium, including the University of California – Riverside and Fresno State University.
The online test asks 54 questions, including multiple choice, multiple response, true/false, fill in the blank, sequencing, and "hot spotting ." For example, a question might show a screen from Microsoft Word and ask the test taker to "click the alignment button a student should use to fully justify the columns in his class newspaper." The questions are administered using a branching structure, so questions become easier or harder depending on how well the respondent is performing in a particular competency.
There are plans to complete reliability and validity tests on the web version this summer, but no current results exist. In addition, the electronic platform on which the test is administered is being upgraded, so Teacher Universe will have the capability to administer portfolio and performance assessments.
The tests are evaluated electronically and teachers receive a score between 100 and 300 in each of the nine competencies. A score between 100 and 199 places the teacher in the entry-level class, 200 to 299 in the intermediate class, and 300 in the advanced class.
As part of its work with the California Commission on Teacher Credentialing, Teacher Universe agreed to share the assessment with other PT3 grantees.
b. Strengths and Limitations
The use of different question formats can measure knowledge and competence better than a straight multiple-choice exam. Hot spotting and sequencing questions, for example, require the test-taker to demonstrate more familiarity with software than do simple multiple-choice items. In addition, the branching system allows questions to more quickly and more accurately pinpoint an individualÕs level of knowledge than a uniform set of questions for all test takers.
The testÕs ability to measure the integration of technology in teaching is limited by the format. For example, the true/false question "You can use a spreadsheet to teach story sequencing skills" shows whether an individual is able to select the appropriate software for a situation but offers little insight into his or her ability to integrate technology into the K-12 curriculum. As with the Idaho exam, this test assesses knowledge that is necessary but not sufficient for developing a technology-rich curriculum.
No data are available on the testÕs reliability and validity (though there are plans to evaluate the test this summer).
3. North Carolina Essential Technology Skills Inventory
3. North Carolina Essential Technology Skills Inventory
a. Content, Administration, and Evaluation
Before mandating a portfolio assessment for licensure, the state of North Carolina experimented with the Essential Technology Skills Inventory, a multiple-choice exam. The 110-minute, 90-question exam covered the stateÕs Basic Technology Competencies (described below) and was similar to a test administered to the stateÕs eighth-grade students. The North Carolina State Board of Education no longer administers the basic technology skills test to teachers.
b. Strengths and Limitations
After several pilot efforts, both the validity and reliability of the instrument were questioned, and it was determined that the cost of maintaining the test was too high. An inability to guarantee equitable access to the necessary technological equipment during the teacher preparation program made it difficult to ensure a fair testing procedure. Finally, the questions focused on basic skills and did not address the issue of integration or learning enhancement.
4. Appropriateness of the Online Exam for a National Evaluation
The online exam offers several advantages as a potential source of data. First, because each question has a correct answer, test competencies may be applied more uniformly to each K-12 or preservice teacher, allowing for consistent comparison among different teachers. Second, because the test evaluation is completed electronically rather than by a trained assessor, time and cost factors for analyzing and evaluating the data obtained are minimized. Thus, the online format reduces the cost of administering the exam and increases the potential sample size.
The drawback to the online exam is that the data may not reflect an accurate picture of the preservice or K-12 teacherÕs ability to teach with technology. This drawback is particularly true of questions designed to measure a teacherÕs ability to integrate technology. For instance, the following from the Idaho Competency Exam can measure a preservice teacherÕs ability to select appropriate software for a particular lesson:
1. Students need to calculate the population of statistics in their community for a report on the local economy and employment. The best tool for this activity would be ___________.
a) a local area network
b) desktop publishing software
c) a spreadsheet (correct answer)
d) a database
That question, however, might not measure his or her ability to develop and implement a technology-specific lesson plan.
Of the three tests discussed in this report, the Idaho Technology Competency Exam and the Teacher Universe Curriculum Integration Assessment System are most appropriate for a national evaluation. Because the Teacher Universe test also allows for different question formats, it provides more flexibility in measuring the competencies than does the Idaho Technology Competency Exam. While the North Carolina Essential Technology Skills Inventory may not be appropriate for a national evaluation, it highlights some of the problems of developing a multiple-choice exam.
C. PORTFOLIO ASSESSMENT
C. PORTFOLIO ASSESSMENT
Preservice studentsÕ technology proficiency is evaluated using a portfolio assessment at IHEs in several states. For this type of assessment, students are required to develop a technology portfolio that is assessed against a rubric based on competencies. These competencies are developed independently at each institution, yet are typically based on the ISTE technology guidelines. Though they vary from institution to institution, a portfolio is usually compiled throughout the studentÕs time at the teacher preparation program and contains lesson plans, completed coursework, and additional materials that demonstrate the preservice studentsÕ proficiency in the competencies. Eight portfolio assessments from the following institutions and states are discussed below:
· Idaho – Technology Portfolio Assessment Standards and Scoring Guide (see Appendix C for sample rubric)
· North Carolina Agricultural and Technological State University – Advanced Technology Competencies Portfolio Rubric (see Appendix C for sample rubric)
· Western Carolina University - Technology Portfolio Summative Evaluation Form
· North Carolina State University – Technology Competencies Verification Form
· University of North Carolina – Pembroke – Portfolio Evaluation and Summary Forms
· Elizabeth City State University – Technology Portfolio Evaluation Form
· University of Illinois – Technology Competencies Database
· North Carolina (Department of Public Instruction)– Technology Competencies for Educators (Basic and Advanced)
Two additional portfolio assessments from the following institutions are also mentioned but not discussed in detail due to limited information:
· University of North Carolina – Asheville
· University of North Carolina – Charlotte
The eight portfolio assessments obtained are used specifically for preservice teachers and administered before graduation, usually during the studentÕs last semester (see Table 5). The total number of competencies and subcompetencies typically addressed in portfolio assessments ranges from 14 to 26, with three of the assessments addressing five main competencies and 21 subcompetencies and one addressing five main competencies and 22 subcompetencies. At two institutions, students are required to pay an assessment fee of $50.
There are several strengths of using portfolio assessments. They provide a much better assessment of studentsÕ technological skills and ability to apply and adapt technology to specific learning situations. This is in contrast to multiple-choice assessments that strictly catalog a studentÕs knowledge and recall of classroom instructional technology (see Table 6). Furthermore, the portfolio assessment allows students to demonstrate a broader range of skills than other types of assessments. Consequently, the portfolio assessment is best suited for measuring studentsÕ level of technology proficiency during the teacher preparation program because it allows students to demonstrate what they have learned throughout their time in the program and how they plan to apply technology in a classroom setting.
1. Idaho – Technology Portfolio Assessment Scoring Rubric
1. Idaho – Technology Portfolio Assessment Scoring Rubric
a. Content, Administration, and Evaluation
The teacher portfolio used by the state of Idaho consists of eight required entries that provide evidence of mastery of 25 accomplishments based on the ISTE standards.
· Entry 1: Use of Word Processing Tools. Use of word processing tools for instructional planning, development of teaching materials, instructional delivery, or integration of technology into the curriculum
· Entry 2: Use of Database Tools. Use of database for instructional planning, development of teaching materials, instructional delivery, or integration of technology into the curriculum
· Entry 3: Use of Spreadsheet Tools. Use of spreadsheet tools for instructional planning, development of teaching materials, instructional delivery, or integration of technology into the curriculum
· Entry 4: Use of Presentation Tools. Use of presentation tools for instructional planning, development of teaching materials, instructional delivery, or integration of technology into the curriculum
· Entry 5: Use of Telecommunication Tools. Use of e-mail and the Internet for instructional planning, development of teaching materials, instructional delivery, or integration of technology into the curriculum
· Entry 6: Student Work Samples. Inclusion of actual samples of K-12 student work for two of the tools featured in entries one through five
· Entry 7: Evaluation of Educational Software. Evaluation of educational software chosen by preservice student
· Entry 8: Checklist of Trouble-Shooting Techniques. Documentation of the ability to use troubleshooting techniques discussed in state standards
Scoring is based on the evidence in the portfolio entries that supports demonstration of the standards. To pass the assessment, students must meet all 25 standards, which address the following major competencies:
· Basic Technology Competency. Basic Computer/Technology Operations and Concepts (focuses on the operation of a computer system, knowledge of terminology, knowledge of trouble shooting techniques, and ability to use imaging devices).
· Software Competency. Personal and Professional Use of Technology (focuses on the ability to use word processing software, database software, spreadsheet software, and presentation software to develop a product; targets studentsÕ ability to use communications software to access and use the Internet and e-mail to develop a product; includes items that address problem solving and data collection and adaptations for students with special needs).
· Ethics, Basic Integration, and Advanced Integration Competencies. Application of Technology to Instruction (focuses on computer and technology resources; equitable, ethical, and legal use of technology; and integration of technology, including a demonstrated use of productivity, presentation, and communication tools to plan instruction, develop instructional materials, deliver instruction, and assess student learning and performance).
The assessment also contains a component in which the software materials used in the portfolio are evaluated on a variety of criteria including: clarity of directions, documentation, ease of operation, error handling, motivation, accuracy of information, evaluation/feedback, K-12 student materials, promoting intended instructional outcome, readability of text, readability of graphics, and appropriateness for student group. Validity tests for this assessment have been completed at this time, however, the results were not available.
The assessment is conducted during the studentÕs final semester in the teacher preparation program and trained assessors use a detailed scoring guide to evaluate the student portfolios for a fee of $50.
b. Strengths and Limitations
In addition to the Strengths and Limitations stated earlier, this assessment goes beyond focusing solely on basic technology competencies. It evaluates teachers on how they demonstrate applying technology to instruction and requires preservice students to apply skills as they would in teaching: selecting software, evaluating its appropriateness for their students, and determining how it will operate in the classroom setting.
2. North Carolina (Department of Public Instruction) – Technology Competencies for Educators (Basic
2. North Carolina (Department of Public Instruction) – Technology Competencies for Educators (Basic and Advanced)
To become a licensed teacher in North Carolina, preservice students must demonstrate their technology proficiency by submitting a "product of learning" (similar to a portfolio) that is assembled throughout their teacher-education program. At the end of the studentsÕ program (usually during the semester they are student teaching), the product of learning is assessed using a rubric developed from the state competencies, which are based on the ISTE/NCATE standards. There are two levels of state competencies, basic and advanced, which taken together cover all five of the major competencies:
· Basic Computer Technologies. Computer Operation Skills (focuses on the preservice studentsÕ ability to start up and shut down a computer system and its peripherals; to save, open, and place documents inside subdirectories/folders; and to work in more than one application at a time); Setup, Maintenance, and Troubleshooting (measures the preservice studentsÕ ability to setup a computer system and its peripheral devices, make backup copies of key applications and documents, and install and upgrade an application); Networking (tests the preservice studentsÕ ability to use a file server and share files with others on the network); Telecommunications (identifies the competency level of preservice students at accessing and using resources on the Internet and World Wide Web, connecting a computer to a modem and telephone line for dial-in access, and effectively using distance learning, desktop video conferencing, and teleteaching technologies); and Media Communications (assesses preservice studentsÕ ability to produce print-based products and electronic slides and overheads and use a digital camera and scanner).
· Software Competency. Word Processing/Introductory Desktop Publishing (identifies studentsÕ competency level at entering and editing text, copying and moving a block of text, checking spelling, grammar, and word usage, and inserting objects into a document); Spreadsheet/Graphing (focuses on the preservice studentsÕ ability to interpret and communicate information in an existing spreadsheet, create a spreadsheet with rows, columns, and headings, and enter data into a spreadsheet); and Database (measures preservice studentsÕ ability to use information from an existing database and create a database with multiple fields and records).
· Ethics Competency. Social, Legal, and Ethical Issues (focuses on how preservice students establish classroom policies and procedures that ensure compliance with copyright law, fair-use guidelines, security, and child protection).
· Basic Integration Competency. Multimedia Integration (focuses on preservice studentsÕ ability to plan, produce, and use both a linear, multimedia presentation and a non-linear, hypermedia presentation); Curriculum (determines to what degree preservice students are using the computer skills curriculum to identify what students should know and be able to do and to determine preservice studentsÕ ability to locate, evaluate, and select the appropriate teaching/learning resources and curriculum materials for their content area and target audience); and Subject-Specific Knowledge (measures the preservice studentsÕ use of technology-based tools that are specific to the discipline, and their ability to use media and technology to present the subject so that it is comprehensible to others).
· Advanced Integration Competency. Design and Management of Learning Environments/Resources (examines the degree to which preservice students effectively use computers and other technologies to communicate information in a variety of formats on student learning to colleagues, parents, and others); and Child Development, Learning, and Diversity (determines preservice studentsÕ ability to use media and technology to support learning for children with special needs or for children whose primary language is not English).
Though each studentÕs product of learning is assessed using the same competencies, the individual IHEs in North Carolina independently interpret these competencies and each develop a rubric based on that interpretation. Each institution is required to have a review panel of members that use the rubric to assess the studentsÕ technology proficiency, including a K-12 teacher and an SCDE faculty member. Brief descriptions of the various rubrics IHEs in North Carolina use are discussed below.
3. North Carolina Agricultural and Technological State University – Advanced Technology Competencies
3. North Carolina Agricultural and Technological State University – Advanced Technology Competencies Portfolio Rubric
a. Content, Administration, and Evaluation
The technology portfolio rubric used at North Carolina Agriculture and Technology (NC A&T) is based on state standards and was developed with input from the portfolio specialist at the Department of Public Instruction. The assessment is completed electronically the semester before graduation and has been used with nearly 175 students. The rubric aligns the contents of the studentÕs portfolio against 14 competencies (basic and advanced). The portfolio contents include "evidences" that both the competencies and sub-competencies. Based on these evidences within the portfolio, students receive one of four proficiency ratings on each competency and on the portfolio overall. The guidelines for these ratings are:
· Level 1. The portfolio displays a lack of proficiency in multiple competencies. The evidences in at least two areas are lacking either substance or originality. At least two competency areas are lacking evidences at the Level 3 or 4 standard.
· Level 2. The portfolio displays a lack of proficiency in at least one competency. The evidences in at least one competency area are lacking either substance or originality. At least one competency area does not have an evidence at the Level 3 or 4 standard. Additionally, each acceptable evidence is student work that holds the characteristics of being original, integrated, correct, appropriate, and correlated.
· Level 3. The portfolio displays proficiency of the technology competencies. The evidences display substantial, original work. At least one evidence for each competency area is at the Level 3 or 4 standard. Additionally, each evidence is student work that holds the characteristics of being original, integrated, correct, appropriate, and correlated.
· Level 4. The portfolio displays a mastery of the technology competencies. Multiple evidences are included for each competency. The evidences display substantial, original work. Each evidence included is at the Level 3 or 4 standard. Additionally, each evidence is student work that holds the characteristics of being original, integrated, correct, appropriate, and correlated.
The cost of completing the assessment is $50 per student. To date, no reliability or validity tests have been conducted on the rubric. Assessors undergo approximately three hours of training to learn how to properly score a studentÕs portfolio, and the actual assessment takes about 30 minutes to complete.
b. Strengths and Limitations
The NC A&T assessment has several strengths. First, the assessment requires students to provide evidence supporting each competency. In doing so, students demonstrate mastery of the competencies through application of skills. Second, the standards used to evaluate students are clearly defined, and criteria are specified for each component of the portfolio, which helps promote consistency in ratings. Moreover, studentsÕ evidence of technology proficiency is rated on four levels, rather than on a binary scale (satisfactory or unsatisfactory). This provides a more nuanced picture of the studentsÕ abilities than an all-or-nothing approach does.
One of the more notable drawbacks is that necessary evaluator training and the length of the assessment makes it more time consuming. Furthermore, even with detailed specifications, it is difficult to standardize results across more subjective evaluations.
4. Western Carolina UniversityCTechnology Portfolio Summative Evaluation Form
and
5. North Carolina State UniversityCTechnology Competencies Verification Form
a. Content, Administration, and Evaluation
Similar to other IHEs in North Carolina, Western Carolina UniversityÕs and North Carolina State UniversityÕs student portfolios are reviewed using an evaluation form that aligns the portfolio content with the state competencies and sub-competencies. The form has been used at Western Carolina with approximately 350 students in the past three years. The semester before graduation, a review panel uses the form to rate students as either superior (exceeds expectations), satisfactory (meets expectations), or unsatisfactory (does not meet expectations) on each of the five major competencies. At Western Carolina, students are also rated on each of the 21 sub-competencies. The review panel is encouraged to discuss the competency together before recording a proficiency rating on the evaluation form.
Based on hardware, software, and personnel hours, the production and maintenance cost of the instrument is estimated at $15,000 to $20,000 (Western Carolina). Content validity tests have been completed for the Western Carolina assessment, but the results of those tests were not available. The time required for evaluators to complete the assessment is estimated at one hour.
b. Strengths and Limitations
One of the strengths of the portfolio assessment at Western Carolina is that it targets the advanced competencies that focus on integration of technology rather than basic technology skills. Furthermore, as with the NC A&T assessment, the three-level rating scale provides a better idea of the studentÕs actual level of technology proficiency, compared with a binary scale on which students either meet the criteria or not.
Portfolio assessments in general take more time to complete than online or self-assessments and the assessment at Western Carolina necessitates even more time by requiring the review panel to discuss each competency area prior to recording a rating. This collaboration may produce greater consistency in ratings, yet also impose an even greater time burden on the reviewers.
5. University of North Carolina – Pembroke – Portfolio Evaluation and Summary Forms
and
7. Elizabeth City State University – Technology Portfolio Evaluation Form
a. Content, Administration, and Evaluation
Before graduation, UNC-Pembroke and Elizabeth City State University (ECSU) teacher preparation students must submit a portfolio with "artifacts" that provide evidence of the mastery for each of the five advanced state competencies. These artifacts receive ratings that are recorded on portfolio evaluation forms. The forms at ECSU have been used with approximately 100 students over the past two years. Each artifact (student lesson plans, classroom activities, or other materials) receives a rating of either satisfactory or unsatisfactory based on the studentsÕ explanation of why the artifact is included in that competency. To receive a satisfactory rating, artifacts must:
· Show originality
· Support the North Carolina Standard Course of Study
· Be accurate in subject and content
· Be technically correct
If an artifact is rated unsatisfactory, reviewers explain the reason for the lower rating.
For this assessment, each competency area is weighted differently in configuring the overall portfolio rating. Curriculum is worth 20 percent; Subject-specific Knowledge, 40 percent; Design and Management of Learning Environment/Resources, 20 percent; Child Development, Learning, and Diversity, 10 percent; and Social, Legal, and Ethical Issues, 10 percent.
Though no tests have been conducted to date, ECSU has plans to conduct both reliability and validity tests in the future. Evaluators are required to undergo training in order to complete the assessment, which takes approximately 45 minutes.
b. Strengths and Limitations
As with the previous assessments, one strength of these instruments is that each assessment concentrates primarily on the advanced competencies that require a higher level of technology proficiency. Moreover, ECSUÕs portfolio assessment weights each of the competencies, emphasizing those that target the studentÕs ability to integrate technology into the curriculum.
Contact Us
Enter feedback, comments, questions, or suggestions:
Email this page
Add or change any text to your message in the text field below:
|
