Index: Research and Evaluation
Links
Page Contents
Evaluation
Teaching-Licensure Exams: A Limited Tool to Identify Capable Teachers
Construct Validation of Mathematics Achievement: Evidence From Interview Procedures
Tensions Between Competing Pedagogical and Accountability Commitments for Exemplary Teachers of Mathematics in Kentucky
Less Truth Than Error? An independent study of the Massachusetts Teacher Test
Early Childhood Program Research and Evaluation
Student Evaluation of Teacher Performance
Online Evaluation Resource Library [OERL]
Why Production Function Analysis is Irrelevant in Policy Deliberations Concerning Educational Funding Equity
Is Water an Input to a Fish? Problems with the Production-Function Model in Education
Students and Educational Productivity
State Actions for Personnel Evaluation: Analysis of Reform Policies, 1983-1992
Early Childhood Evaluation and Policy Analysis: A Communicative Framework for the Next Decade
Academic Freedom, Promotion, Reappointment, Tenure, and the Administrative Use of Student Evaluation of Faculty [pt. II of IV]
Academic Freedom, Tenure, and Student Evaluation of Faculty [part III of IV]
Academic Freedom, Promotion, Reappointment, Tenure, and the Administrative Use of Student Evaluation of Faculty [pt. IV of IV]
Academic Freedom, Tenure, and Student Evaluation of Faculty [part I of IV]
Response to Haskell:
Critical Evaluation for Education Reform
Evaluation Primer: An Overview of Education Evaluation
Understanding Evaluation: The Way to Better Prevention Programs
User-Friendly Handbook for Mixed Method Evaluations
User-Friendly Handbook for Project Evaluation
[Collection] FairTest: Teacher Testing
Prospective Evaluation Methods: The Prospective Evaluation Synthesis.
Designing Evaluations.
An Audit Quality Control System: Essential Elements.
Basic Guide to Program Evaluation
Evaluation Handbook
What Does Research Say About Early Childhood Education?
Testing and Evaluation: Learning from the Projects We Fund
Understanding Evaluation: The Way to Better Prevention Programs
   
Research General
Facing the Consequences: Identifying the Limitations of How We Categorize People in Research and Policy
The Quality of Researchers¹ Searches of the ERIC Database
Action Research and Social Movement: A Challenge for Policy Research
Possible Indicators of Research Quality for Colleges and Universities
Qualitative Research Methods: An Essay Review
Improving Theory and Research on Quality Enhancement in Organizations: Report of a Workshop
Research and Education Reform: Roles for the Office of Educational Research and Improvement
Improving Student Learning: A Strategic Plan for Education Research and Its Utilization
Research Methods Knowledge Base
The Handbook of Research Impact Assessment
Research Program Peer Review: Principles, Practices, Protocols
Strategies for Improving Survey Turnaround
Using Structured Interviewing Techniques.
Longitudinal Surveys of Children
How to Collect Survey Data
More About Mail Surveys
Surveys & Privacy
How to Plan a Survey
What Is a Survey?
[Journal] Educational Action Research
   
asdf
What is Research Design?
Research design can be thought of as the structure of research -- it is the "glue" that holds all of the elements in a research project together. We often describe a design using a concise notation that enables us to summarize a complex design structure efficiently. What are the "elements" that a design includes? They are:
Observations or Measures
These are symbolized by an 'O' in design notation. An O can refer to a single measure (e.g., a measure of body weight), a single instrument with multiple items (e.g., a 10-item self-esteem scale), a complex multi-part instrument (e.g., a survey), or a whole battery of tests or measures given out on one occasion. If you need to distinguish among specific measures, you can use subscripts with the O, as in O1, O2, and so on.
Treatments or Programs
These are symbolized with an 'X' in design notations. The X can refer to a simple intervention (e.g., a one-time surgical technique) or to a complex hodgepodge program (e.g., an employment training program). Usually, a no-treatment control or comparison group has no symbol for the treatment (some researchers use X+ and X- to indicate the treatment and control respectively). As with observations, you can use subscripts to distinguish different programs or program variations.
Groups
Each group in a design is given its own line in the design structure. if the design notation has three lines, there are three groups in the design.
Assignment to Group
Assignment to group is designated by a letter at the beginning of each line (i.e., group) that describes how the group was assigned. The major types of assignment are:
R = random assignment
N = nonequivalent groups
C = assignment by cutoff
Time
Time moves from left to right. Elements that are listed on the left occur before elements that are listed on the right.
Design Notation Examples
It's always easier to explain design notation through examples than it is to describe it in words. The figure shows the design notation for a pretest-posttest (or before-after) treatment versus comparison group randomized experimental design. Let's go through each of the parts of the design. There are two lines in the notation, so you should realize that the study has two groups. There are four Os in the notation, two on each line and two for each group. When the Os are stacked vertically on top of each other it means they are collected at the same time. In the notation you can see that we have two Os that are taken before (i.e., to the left of) any treatment is given -- the pretest -- and two Os taken after the treatment is given -- the posttest. The R at the beginning of each line signifies that the two groups are randomly assigned (making it an experimental design). The design is a treatment versus comparison group one because the top line (treatment group) has an X while the bottom line (control group) does not. You should be able to see why many of my students have called this type of notation the "tic-tac-toe" method of design notation -- there are lots of Xs and Os! Sometimes we have to be more specific in describing the Os or Xs than just using a single letter. In the second figure, we have the identical research design with some subscripting of the Os. What does this mean? Because all of the Os have a subscript of 1, there is some measure or set of measures that is collected for both groups on both occasions. But the design also has two Os with a subscript of 2, both taken at the posttest. This means that there was some measure or set of measures that were collected only at the posttest.
With this simple set of rules for describing a research design in notational form, you can concisely explain even complex design structures. And, using a notation helps to show common design sub-structures across different designs that we might not recognize as easily without the notation.
Copyright ©2002, William M.K. Trochim, All Rights Reserved Purchase a printed copy of the Research Methods Knowledge Base
[more]
   
Contact Us
Enter feedback, comments, questions, or suggestions:
Email this page
Add or change any text to your message in the text field below:
|
