Term
|
Definition
| dependable, repeatable results – necessary but not sufficient for validity |
|
|
Term
| Methods for Estimating a tests Reliability |
|
Definition
1. Test-Retest Method
2. Parallel/Alternate Forms Method |
|
|
Term
| Explain Test-Retest Method |
|
Definition
a. Evaluates temporal stability – consistency over time. Administer test 2 or more times to the same group of individuals.
b. Time 1 + Time 2 = Score |
|
|
Term
| Advantages & Disadvantages of Test-Retest Method |
|
Definition
c. Advantage – same test, same people
d. Disadvantage – characteristic might change, time interval.
e. Carryover Effects – being tested on one occasion can affect second occasion or may create desire to be consistent. Optimal time 2-3 weeks. |
|
|
Term
| Explain Parallel/Alternate Forms Method |
|
Definition
a. Similar to test/re-test, but form of test changes to an alternate form but still measures the same stuff in content.
b. Compare Correlation between test A & B.
c. High Correlation = Related |
|
|
Term
| Define Internal Consistency |
|
Definition
| Extent to which items are interrelated in the cumulative. Do items “hang together” |
|
|
Term
| Tests for measuring internal consistency |
|
Definition
1. Split-Half Reliability
2. Coefficient Alpha (Cronbach) α or jesus fish |
|
|
Term
| How to do split half reliability |
|
Definition
a. Divide test if half and score both parts, do they correlate.
b. Odd/Even Method – sum of #’s 1,3,5,7,9 etc. so order of items is not as important. |
|
|
Term
| Advantages & Disadvantages of Split-Half Reliability |
|
Definition
c. Advantages – use same items at the same time
d. Disadvantage – different results based on how you split the times. |
|
|
Term
| Define Coefficient Alpha (Cronbach) α or |
|
Definition
a. Better measure than split half.
b. Measures internal reliability
c. Has all advantages as split half
d. Depends on
i. Average intercorrelation of items should be high, higher average intercorrelation = higher α
ii. Number of items 0 éitmes = éreliable
e. Desireable Test Results
i. Should exceed .70
ii. Desireable to be above .90
iii. Self-Esteem Scale = .832 |
|
|
Term
|
Definition
|
|
Term
| Classical Test Theory (True Score Theory) |
|
Definition
Observed Score – true score + error (deviation)
Errors are random +/-
s2 =
Ration of true score to true score variability
True Score + Error |
|
|
Term
| Standard Error of Measurement (SEM) |
|
Definition
SEM is standard deviation of observed scores
Exresses the expected error or margin of error “average error” |
|
|
Term
|
Definition
| To get 95% score calc SEM * 1.96 & calc range |
|
|
Term
|
Definition
Used to build reliabilty into the test –analyses of individual items of test.
-Goal – to produce reliable test which makes it more likely will have a have valid test.
-Item Reliability - Does item contribute to the reliability of the test. “Hang Together” with other items.
-Item total Correlation – score of item compared to score on other items. |
|
|
Term
|
Definition
Choosing items that have good psychometric characteristics.
-Items contributionto Reliability (Internal Consistency)
-SPSS – Corrected Item Total Correlation
-SPSS - Cronbach Alpha if deleted |
|
|
Term
|
Definition
| (Individual Difference) - Do responses on that item discriminate between people that score high or low on the test. |
|
|
Term
|
Definition
| chose questions where people differ and there is a great amount of variability. |
|
|
Term
|
Definition
| Item Characteristc Curve figure on p. 78 |
|
|
Term
|
Definition
| The probability of a correct or passing answer or answer that particular item. |
|
|
Term
|
Definition
Property of the test – does the test measure what it is supposed to measure. Measures a construct (idea such as intelligence).
-Property of the test – does the test measure what it is supposed to measure. Measures a construct (idea such as intelligence). |
|
|
Term
| What is most important part of test |
|
Definition
|
|
Term
| 3 Types of Validity Evidence |
|
Definition
1. Content
2. Criterion
3. Construct |
|
|
Term
|
Definition
-The extent to which content adequately samples the universe of interest.
-Does test cover a specific area adequately.
-Reviewed by “expert” in the area “face validity” looks good.
-Expert judgments based on informed understandings. |
|
|
Term
| Criterion Validity Defined |
|
Definition
-The ability of test to predict certain criteria that are directly relevant to the test.
-Test 1 test against another.
-Correlation – between test score & criterion score. 0-1, 1 means you are measuring the same thing.
-Eg score by self compared to peer score – do they correlate
-Eg SAT & grades in college |
|
|
Term
| 2 Types of Criterion Validity |
|
Definition
-Concurrent – does criterion exist at the same point in time. eg test score & interview at the same time
-Predictive – does it have the ability to predict same things at some future point. Eg memory test @ 55 prdicts alzhymers @ 75. |
|
|
Term
| Criterion Validity - Concurrent |
|
Definition
| does criterion exist at the same point in time. eg test score & interview at the same time |
|
|
Term
| Criterion Validity - Predictive |
|
Definition
| does it have the ability to predict same things at some future point. Eg memory test @ 55 prdicts alzhymers @ 75. |
|
|
Term
|
Definition
-1955 paper (Cronbach & Muhl)
-Includes all other validity types and more.
-Relevant when attribute of interest exists as a construct (a theoretical concept/attribute of an individual eg intelligence – abstraction)
-Theory Building & Theory Testing – Develop test to measure extent to which body of evidence supports what its measuring. |
|
|
Term
|
Definition
| -system of laws that relate it to other constructs including some variables that are directly observed. |
|
|
Term
|
Definition
-Correlationwith another independent measure of the same trait.
-Do scores on one test converge with scores on another test. |
|
|
Term
|
Definition
| -Low correlations with another independent measure of a theoretically unrelated traited eg intelligence & friendliness. |
|
|
Term
|
Definition
| 2 or more traits + 2 or more methods |
|
|
Term
|
Definition
| both measures extroversion |
|
|
Term
|
Definition
|
|
Term
| 3 Basic Strategies of test construction |
|
Definition
1. Correspondance or Rational View
2. Instrumental or Empirical View
3. Substantive or Construct View |
|
|
Term
Test Construction
Correspondance or Rational View |
|
Definition
a. Select items for test on basis of assumed relation between item & attribute to be measured.
b. Based on judgment of individual constructing the test.
c. Measures internal state of the individual “face validity”
d. Woodworth Personality Data Sheet – developed to measure psychoneurotic tendencies – used for military. |
|
|
Term
Test Construction
Correspondance or Rational View
DISADVANTAGES |
|
Definition
-Assumption that items have common meaning.
-Assumption that respondants know their “inner states”
-Respondents may not honestly report on their state.
-Assumes that items are related to the trait |
|
|
Term
Test Construction
Instrumental or Empirical View |
|
Definition
-Use statistical criteria for selecting items – only uses data.
b. Contrasted Groups Method
i. Start with large number of items
ii. Give items to 2 groups of people
1. Group 1 – criterion group – these people “have” the trait, eg they are shy.
2. Group 2 – control group – people that “don’t have the trait”
iii. Compare answers, items that differentiate between the groups become part of the test. Eg MMPI was based on this method.
iv. Totally based on how 2 groups responded to the items. |
|
|
Term
Test Construction
Instrumental or Empirical View
DISADVANTAGES |
|
Definition
-Need to be able to reliably specify the “criterion group”
-Items we srtart out with have to be a large number & braod to find a few that work.
-When items are based on extreme groups, there’s no guarantee items will work on groups in the middle.
-Don’t know the source of the difference, but don’t know why. Some other trait could be confounding.
-Chance – possibility that you may commit Type I error or error of false positive. |
|
|
Term
Test Construction
Substantive or Construct View |
|
Definition
-Used when items we are measuring is a psychological construct.
-Substantive Validity
-Does test capture universe of interest
-Does it have adequacey of coverage in proper proportion.
-Is it a good model of the construct.
-Does is have the right number of items for each feature. |
|
|
Term
Test Construction
Substantive or Construct View
DISADVANTAGES |
|
Definition
-2 Problems
-Construct Under Representation
-Have only covered part of the domain. Test items don’t cover the domain of interest. Eg Beck Depression.
-Construct Irrelevant Variance
-Test items cover too much, includes irrelevant material. Test may be too braod. |
|
|
Term
|
Definition
i. The items together demonstrate interal consistency, that is appropriate for that construct. Do items “hang together”, are the intercorrelated, but appropriate for that construct.
ii. Inter-Item Structure/Correlation
1. Chose items that maximize the interla consistency of the test. SPSS “Item Correlation” or “Contribution to Alpha” |
|
|
Term
| Structural Fidelity (trueness) |
|
Definition
| -Do the items intercorrelate in the same way as the construct is structured. Do items “mirror” the construct? |
|
|
Term
|
Definition
Cummulative model
-When item scores are added together to get a total.
-Appropriate when dimensional or continuous (shyness)
Class Model
-Assumes a categorical or qualitative attribute.
-You either are or are not. No degree of membership.
-Configuration of scores / meeting certain criteria. (DSM) |
|
|
Term
|
Definition
-Concerns the relation of the test scores to criteria that we want to predict, aka validity.
-Relation between test score & construct should be the same. |
|
|
Term
| Response Sets & Response Styles |
|
Definition
| How you respond to test item is affected by all of extraneous factors that affect response. |
|
|
Term
| Response Style (defensive, open) |
|
Definition
-Consistent pattern that would affect how the person responds to many different tests.
-An aspect of their personality.
-Need for social approval may affect how they respond to a test. ** |
|
|
Term
| Response Set (self presentation) |
|
Definition
-Strategy on a particular test influenced by the goal person has on the impression they want to make in that situation.
-Malingerers – present impression as sick, in need of help. |
|
|
Term
|
Definition
-Set meets a need in that moment
-Style is how the person generally reacts. |
|
|
Term
|
Definition
-Represent Non-Substantive basis to the test respons.
-Outside of “construct” that is influencing the results. |
|
|
Term
|
Definition
| Tendency to agree with item regardless of what’s being asked “agreement tenency”. |
|
|
Term
| Problems & solutions w/ acquiesence |
|
Definition
-With both, you don’t get an accurate measure of rait.
-Happens when all items are phrased similarly (affirmative or negative).
-Solution – Balanced test has both affirmative and negative |
|
|
Term
|
Definition
| Tendency to endorse items that describe yourself as “socially desireable”. |
|
|
Term
| Solution for social desireability problem |
|
Definition
EE Edwards – designed a test based on MMPI (he says MMPI is tainted by social desireability)
-Developed scale w/ 3 items that had highest “social desireability”. Higher on scale, lower maladjustment??? |
|
|
Term
| Marlowe & Crown Social Desireabilty Scale |
|
Definition
-Look at items we all have and see if people deny having them.
-Ensure that test isn’t highly contaminated by social desireability. |
|
|
Term
| Structured/Objective Tests |
|
Definition
-Relatively unambiguous set of items.
-Can respond with limited number of responses such as
o True/False
o Yes/No
o Likert Scales (1-5) |
|
|
Term
|
Definition
Structure
-Unambiguous
-Direct
Objective
-Can score the test so that scoring is perfectly reliable. |
|
|
Term
|
Definition
1.Surgency (Extroversion) Largest Factor
oTalkative vs. Silent
2.Agreeableness
oGood natured vs. irritable
oCooperative vs. negativistic
3.Conscienciousness
oResponsible vs. Undependable
4.Emotional Stability (Adjustment) Inverse: Neuroticism
oCalm vs. Anxious
5.Culture (Intellectency) (Openness to Experience)
oImaginative vs. Simple
oConventional vs. Creative |
|
|
Term
|
Definition
| Norman’s big 5 factors are an adequate taxonomy of personality – doesn’t cover all aspects of all possioble ways. E.g. doesn’t cover level of spirituality. |
|
|
Term
| Costa & McCrae’s Five Factor Model |
|
Definition
1. Extroversion
2. Agreeableness
3. Conscintiousness
4. Neuroticism
5. Openness to Experience |
|
|
Term
| Interpersonal “Social Aspect” |
|
Definition
-Personality Traits that affect how you deal with and interact with others.
-Subset of 5 factor model that seems to define interpersonal
i. Extroversion
ii.Agreeableness |
|
|
Term
|
Definition
Hs (1) - Hypochondriasis
D (2) - Depression
Hy (3) - Hysteria
Pd (4) - Psychopathic deviate
Mf (5) - Masculinity/Femininity
Pa (6) - Paranoia
Pt (7) - Psychasthenia
Sc (8) - Schizophrenia
Ma (9) - Hypomania
Si (0) - Social |
|
|
Term
|
Definition
| measures test taking attitudes |
|
|
Term
| List the 3 validity scales |
|
Definition
L - Lie - underreport
F - Frequency - overreport
K - Correction - defensiveness |
|
|
Term
|
Definition
-tendency to underreport
-Naïve attempt to create overly positive impresssion of themselves.
-Deny common truths |
|
|
Term
|
Definition
-tendency to overreport
-present oneself in overly negative lgith (sick, malingerer)
-frequently endorses items endorsed by <10% of the group. |
|
|
Term
|
Definition
-To measure defensiveness
-More sophisticated than L.
-Tested by psychiatric patients but endorsed themselves asx normal. False Normals.
-Add .5% of K scale to account for defensiveness |
|
|
Term
|
Definition
1) Passage of time has made some items obsolete
2) Objectionable content – sexist/racist items
3) Small sample not demographically representative. |
|
|
Term
| Steps in Interpreting MMPI |
|
Definition
1) Note test taking behaviors
2) Score & Plot the Profile
3) Evaluate validity of the results
4) Compare profiles against code types
5) Individuate
6) Write the report |
|
|
Term
MMPI Interpretation
How to compare profile types |
|
Definition
-Identify 2 highest scales & assign code type.
-Have to be above 65 (clinical cutoff)
-Within normal range
-Spike = only1 scale above 65 |
|
|
Term
MMPI Interpretation
How to individuate |
|
Definition
-Supplementary scales & other patterns in profile.
-Critical items
-Demographics |
|
|
