Methods for Improving Measurement Reliability
- 0:06 Reliability
- 2:00 Inconsistency
- 3:27 Improving Reliability
- 5:35 Lesson Summary
Reliability is the consistency of the results of a measurement tool. But what causes a tool to have low reliability? And what can be done to improve reliability? In this lesson, we'll answer both of those questions.
Imagine that you wake up one morning and hop on the scale to see how much that chocolate cake you had last night is going to cost you in terms of weight. You look down at the scale, and it reads 100 lbs. 'That can't be right,' you think, 'I'm normally around 150 lbs.' Just to make sure, you step off the scale and step back on. This time, it says 184 lbs. Confused, you step off and back on, and it reads 137 lbs. What's going on?
Reliability is when a measurement tool consistently gives the same answer. If your scale tells you that you weigh 150 lbs every time you step on it, it is reliable. But if it says 100 lbs one time, 184 lbs the next time, and 137 lbs the time after that, and nothing has changed, it is not reliable.
Why is reliability important? If a measure is not reliable, we cannot trust what it tells us. It's like being a manager and having an employee who never does what he's supposed to do. He comes in late, surfs Facebook instead of working, and doesn't ever tell the truth when you ask him questions. How can you trust and depend on him?
The same thing is true for measurement tools. If we have a survey that's supposed to measure sexism, we want it to consistently measure sexism. But if it tells us that Johnny is sexist when he takes it one day and then it says he's not on the next day and Johnny hasn't had a personality change, then the survey is not reliable. As a result, we can't trust or depend on the results of the survey. Let's look closer at some things that cause measurement tools to be unreliable and some ways to improve the reliability of measures.
Let's go back to our sexism survey for a moment. We give the survey to Johnny, and the level of his sexism is different every time he takes it. One day it says he's sort of sexist, the next day it says that he's not sexist at all, and the day after that it says that he's the most sexist person who ever lived.
Why is the survey inconsistent? There are three main factors that could be making the survey less reliable.
1. Traits of the subject
Johnny himself might be causing the inconsistent results. Maybe he's not feeling well one day when he takes the survey, or he just got in a fight with his girlfriend, or maybe he just doesn't understand what the survey questions mean. All of these things could cause inconsistent results.
2. Testing conditions
If Johnny is asked to take the survey in the middle of a loud strip club, his answers might be different than if he is asked to take the survey in the middle of a library. Likewise, the way the person giving him the survey gives instructions can cause the results to be less reliable.
3. Chance factors
What if, in the middle of taking the survey, a woman runs by and knocks Johnny over? Or what if Johnny just randomly chooses answers, not even bothering to read the questions? Both of these things can have an impact on Johnny's score and, therefore, lower reliability.
But if all of those things can make a survey or other measurement tool less reliable, how can we ever hope to end up with a reliable measure? What can a researcher do to improve the reliability of his or her measurement tool?
There are several specific things that can improve reliability:
1. Sample size
Remember that Johnny might have personal issues or traits that lead to a lower reliability. If Johnny is the only subject that you give the survey to, this is a big deal. But what if Johnny is just one of hundreds of people taking the survey? If that's the case, then traits of the subjects have less impact on the reliability of the measure.
2. Controlled testing conditions
Another common problem with measurement involves testing conditions. Where and when a person is tested can influence the results, as can things like the instructions the researcher gives to the subject. Controlling these conditions means that the researcher makes sure as many of those issues are taken care of as possible. For example, if we administer our survey in a quiet room every time and give the same instructions word-for-word every time, we are doing our best to control the testing conditions so that we will get reliable results.
3. Reliability Analysis
We might control for everything in the environment and we might have a large sample size, but what if we are using a survey that has questions that are badly worded? Maybe we used vague language, or there are only one or two questions. A reliability analysis tests a measurement tool to see how reliable it is.
Many researchers use a measurement tool that has already been developed and tested by other psychologists. For example, maybe we will use a sexism survey that another researcher already tested for reliability. After all, why reinvent the wheel? But if we decide to develop our own survey, we will want to do a reliability analysis to make sure that it is a reliable measure of sexism.
Reliability is the consistency of the results of a measurement tool. There are several things that can cause inconsistency in results, including traits of the subject, testing conditions, and chance factors. Luckily, there are also several things that psychologists can do to improve reliability, including increasing the sample size, controlling the testing conditions, and running a reliability analysis on the measurement tool.
Chapters in Psychology 105: Research Methods in Psychology
- 1. Introduction to Research Methods (10 lessons)
- 2. Principles of Ethical Research (11 lessons)
- 3. Setting Up the Research Study (12 lessons)
- 4. Data Collection Techniques in Psychology (6 lessons)
- 5. Nonexperimental Research (8 lessons)
- 6. Qualitative Research Methods and Design (7 lessons)
- 7. Quasi-Experimental Research (7 lessons)
- 8. Sampling and Generalization (7 lessons)
- 9. Measurement in Research (12 lessons)
- 10. Internal Validity in Research (9 lessons)
- 11. External Validity (7 lessons)
- 12. Experimental Design (15 lessons)
- 13. Descriptive Statistics in Psychology (4 lessons)
- 14. Inferential Statistics in Psychology (7 lessons)
- 15. Evaluating Research Findings (3 lessons)
People are saying…
"This just saved me about $2,000 and 1 year of my life." — Student
"I learned in 20 minutes what it took 3 months to learn in class." — Student