Chapter 4

How Does 

Computerized Testing 

Work?

Chapter 3

Is There are Right or 

Chapter 2

Testing important?

Chapter 1

What are 

Psychological

Tests?

Overview of 

Psychological 

Testing

Chapter 4

How Does Computerized Testing Work?

Chapter 3

Is There a Right or Wrong Way to Use Psychological Tests?

Chapter 2

Chapter 1

What are  Psychological Tests?

Overview of 

Psychological 

Testing

Why is Psychological Testing Important?

The Big Picture: A Blueprint of Course Material

Section I: Overview of Psychological Testing

 

CHAPTER 1—WHAT ARE PSYCHOLOGICAL TESTS? In Chapter 1, we discuss what a psychological test is and introduce you to some tests you might never have considered to be psychological tests. After exploring the history of psychological testing, we discuss the three defining characteristics of psychological tests and the assumptions we must make when using psychological tests. After discussing how tests are classified, we distinguish four commonly confused concepts: psychological assessment, psychological tests, psychological measurement, and surveys. We conclude by sharing printed and online resources that are available for locating information about psychological testing and specific tests.

CHAPTER 2—WHY IS PSYCHOLOGICAL TESTING IMPORTANT? In Chapter 2, we discuss why psychological testing is important. We discuss who uses psychological tests and for what reasons. We also discuss some of the concerns society has about the use of psychological tests.

CHAPTER 3—IS THERE A RIGHT OR WRONG WAY TO USE PSYCHOLOGICAL TESTS? In Chapter 3, we introduce you to ethical standards in psychological testing. We discuss what ethics are, the history of ethical standards, and specific ethical standards for psychological testing, including issues of privacy, anonymity, and informed consent. After introducing you to various publications of ethical guidelines, we focus on the responsibilities of test publishers and test users. Finally, we discuss special considerations and requirements necessary for individuals who have impairments that interfere with taking a test. We address various types of impairments or disabilities and provide information on ethical treatment of these special populations of test takers.

CHAPTER 4—HOW DOES COMPUTERIZED TESTING WORK? In Chapter 4, we look in-depth at issues and concepts that are transforming psychological testing. First, we discuss computer-based testing—how computers are used in the psychological testing process. Second, we discuss computerized adaptive testing (CAT), which can decrease the number of test questions and still provide reliable and valid scores. Third, we discuss the advantages and disadvantages of psychological testing using the World Wide Web. Finally, we discuss the implications that web-based testing has for the testing and survey industry in the future.

CHAPTER 1: What are psychological tests?

After completing your study of this chapter, you should be able to do the following:

•	Define what a psychological test is and understand that psychological tests extend beyond personality and intelligence tests.
•	Trace the history of psychological testing from Alfred Binet and intelligence testing to the tests of today.
•	Describe the ways psychological tests can be similar to and different from one another.
•	Describe the three characteristics that are common to all psychological tests, and understand that psychological tests can demonstrate these characteristics to various degrees.
•	Describe the assumptions that must be made when using psychological tests.
•	Describe the different ways that psychological tests can be classified.
•	Describe the differences among four commonly used terms that students often get confused: psychological assessment, psychological tests, psychological measurement, and surveys.
•	Identify and locate print and online resources that are available for locating information about psychological tests.

Psychological Tests Defined  
•	An instrument/process that requires an individual to perform some behavior(s) used to measure a personal attribute, trait, or characteristic thought to be important in describing/ understanding behavior or predicting outcomes
Three Defining Characteristics of Tests
•	Must include a representative sample of behavior
•	Samples must be collected under standardized conditions
•	Must be rules for scoring
Differences Among Tests
•	Behavior the test taker performs
•	Attribute, trait, or characteristic measured or outcome predicted
•	Content, format, and how administered
•	How scored and interpreted
•	Psychometric quality	History of Psychological Testing
•	Serious research efforts on the use and usefulness of tests began during the 20th century
•	The first test of mental ability, the Binet–Simon Scale, was published in 1905, followed by the Stanford–Binet in 1916 and the Wechsler–Bellevue Intelligence Scale during the 1930s
•	The first personality test, the Personal Data Sheet, was published during the 1930s, followed by the Woodworth Psychoneurotic Inventory, the Rorschach Inkblot Test, and the Thematic Apperception Test
•	The first vocational tests were published during the 1940s, beginning with the General Aptitude Test Battery
•	Today, psychological tests are a part of the American culture, and there are tens of thousands of psychological tests available
Assumptions of Psychological Tests 
•	When using psychological tests, we must make important assumptions, including (a) a test measures what it says it measures, (b) an individual’s behavior, and therefore test scores, will remain stable over time, (c) individuals understand test items similarly, (d) individuals can report accurately about themselves, (e) individuals will report their thoughts and feelings honestly, and (f) test score is equal to ability plus some error
•	We can increase our confidence in many of these assumptions by following certain steps during test development
Test Classification Methods
•	Tests can be classified as maximal performance, behavior observation, or self-report; standardized or nonstandardized; objective or projective; by dimension measured; or by subject
Psychological Assessment, Tests, Measurement, and Surveys
•	Psychological assessment involves multiple methods for gathering data
•	Measurement is the assignment of numbers according to rules
•	A test is a measurement when the behavior sample can be expressed as a numerical score
•	Surveys focus on group outcomes, and tests focus on individual differences Survey results are typically reported at the question level, and test results are typically reported as an overall score	

After completing your study of this chapter, you should be able to do the following:

•	Describe different types of decisions that are made using the results of psychological tests.
•	Explain which professionals use psychological tests, in what settings, and for what reasons.
•	Describe some concerns individuals have regarding the use of psychological tests as well as the social and legal implications of psychological testing, especially as it relates to intelligence, achievement, aptitude, and integrity testing.

The Importance of Psychological Testing
•	Individuals use tests to make important decisions
•	Individual decisions are those made by the person who takes a test, and institutional decisions are those that others make as a result of an individual’s performance on a test
•	Comparative decisions involve comparing people’s scores with one another to see who has the best score, and absolute decisions involve seeing who has the minimum score to qualify	Who Uses Psychological Tests and for What Reasons
•	In educational settings, administrators, teachers, school psychologists, and career counselors use tests to make educational decisions, including admissions, grading, and career decisions
•	In clinical settings, clinical psychologists, psychiatrists, social workers, and other health care professionals use tests to make diagnostic decisions, determine interventions, and assess the outcome of treatment programs 
•	In organizational settings, human resources professionals and industrial/organizational psychologists use tests to make decisions such as who to hire for a particular position, what training individuals need, and what performance rating an individual will receive	
The Social and Legal Implications of Psychological Testing
•	Some individuals are concerned that psychological tests are biased and do not result in the correct institutional decisions
•	Much of this controversy is focused on the use of intelligence, aptitude, and integrity tests 
•	This controversy has influenced social movements, laws, and guidelines on how psychological tests should and should not be used 	

CHAPTER 2: why is psychological testing important?

After completing your study of this chapter, you should be able to do the following:
•	Define ethics and discuss ethical issues of concern to testing professionals and consumers.
•	Identify organizations concerned with ethical testing procedures and their ethical standards or codes of ethics.
•	Describe how organizations can and cannot enforce compliance with their ethical standards.
•	Differentiate between ethical standards and ethical guidelines.
•	Define and understand the rights to privacy, anonymity, and informed consent.
•	Explain the ethical responsibilities of test publishers and test users.
•	Explain what test user qualifications are and why they are important.
•	Explain what the Ethical Standards of Psychologists say about testing special populations.
•	Describe what learning disabilities are and strategies for coping with them.

.

Ethical Standards for Psychological Testing
•	Ethics reflect what is considered “right” or “wrong”  of a society or culture
•	Most professional societies have a set of ethical standards that have been discussed and adopted by their members 
•	The American Psychological Association, American Counseling Association, and others have ethical standards that protect the rights of individuals who are the recipients of psychological testing 
•	Most important, test takers have the right to privacy, anonymity, and informed consent	Appropriate Use of Psychological Tests
•	Test publishers should follow professional standards and guidelines by marketing tests truthfully, selling tests only to qualified users, providing evidence of validity, developing a comprehensive test manual for each test, and maintaining test security
•	Test users have the responsibility to ensure that everyone who uses the test has the necessary training and experience to carry out his or her obligations in regard to the test
•	A test user is anyone who participates in purchasing, administering, interpreting, or using the results of a psychological test
•	A test taker is a person who responds to test questions or whose behavior is being measured	
Testing Special Populations
•	Individuals with physical or mental challenges might need special accommodations during testing to ensure that their test scores are accurate  
•	Test users should ensure that the test outcome accurately indicates the intended skill or attribute and that the test score has not been altered because of a disability  
•	Often test users must modify the testing format and the test interpretation process to accurately reflect the skill or attribute the psychological test measures  
•	To compensate for physical/mental impairments and learning disabilities, students can develop learning and test-taking strategies	

CHAPTER 4: how does computerized testing work?

Computerized Testing
•	Computers make it easier to (a) construct valid classroom tests by storing test banks and test development software, allowing test users to administer more sophisticated assessments such as adaptive testing and video-based scenarios
•	Computers make it easier for test users to purchase computerized scoring programs for immediate scoring and interpretation or ask test takers to respond directly on the publisher’s website
•	Advantages of computerized testing include greater efficiency, convenient scheduling, individual administration of standardized tests, and a decrease in administration and scoring errors	Computerized Adaptive Testing
•	The use of adaptive testing dates to Binet at the beginning of the 20th century
•	The advent of computerized adaptive testing (CAT) allowed psychologists to administer important tests, such as the GRE and the Armed Services Vocational Aptitude Battery, using adaptive testing methods
•	CAT has many applications such as the development of computerized adaptive rating scales (CARS)
•	With CAT, test takers start with the same questions and as the test progresses, the software presents each test taker with harder or easier questions, depending on how he or she answered previous questions
•	CAT provides a fuller profile of a test taker in less time because the test taker does not need to spend time on questions that are too easy or too hard	
Web-Based Assessment
•	The rapid expansion of computer and Internet technologies has produced an increase in online assessment tools
•	Enterprise services are offered by companies that offer online standardized tests for large numbers of test takers at multiple sites
•	Companies that provide server space, web design, and maintenance provide hosted services
•	Web-based testing takes place under two conditions: proctored and unproctored
•	Web-based assessment greatly facilitates administration of standardized tests as well as informal classroom assessment; however, web-based assessment presents unique challenges
	Implications for the Future
•	Test developers and users will continue to find better ways to present tests that are realistic and tailored to the individual
•	E-learning and web-based assessment  are likely to be an integral part of education in the future
•	Adaptive web-based testing will provide more accurate and detailed information about student learning to schools across the nation
•	The World Wide Web will provide better and easier access to many mental health care services such as employee assistance programs (EAPs)
•	As the number of certification programs increases, organizations and government agencies are likely to find that web-based CAT is more accurate and efficient than administering paper-and-pencil exams	

After studying this chapter, you should be able to do the following:

•	Describe how computers have enhanced the development, administration, and scoring of psychological tests.
•	Discuss the advantages and disadvantages of computerized testing.
•	Explain the differences between adaptive testing and traditional testing.
•	Discuss the advantages and disadvantages of administering tests using the web.
•	Discuss the implications of web-based technology for testing in the future.

CHAPTER 3: is there a right or wrong way to use tests?

CHAPTER 5—HOW DO TEST USERS INTERPRET TEST SCORES? In Chapter 5, we focus on increasing your understanding of the procedures used to interpret test scores. Because the procedures used to interpret test scores depend on the type of data that a test produces, we begin with a discussion of the four levels of measurement of psychological test data. We then discuss frequency distributions, measures of central tendency, measures of variability, and measures of relationship. After discussing how to convert raw scores into more meaningful units (for example, z scores, T scores), we discuss the role of norms in interpreting test scores.

CHAPTER 6—WHAT IS TEST RELIAB ILITY? In Chapter 6, we describe three methods of estimating a test’s reliability: test–retest or stability over time, internal consistency or homogeneity of the test questions, and scorer reliability or agreement. We discuss how to calculate an index of reliability called the reliability coefficient, an index of error called the standard error of measurement, and an index of agreement called Cohen’s kappa. We discuss classical test theory and the concept of random and systematic error.  Finally, we discuss factors that increase and decrease reliability of test scores.

CHAPTER 7—HOW DO WE GATHER EVIDENVCE OF VALIDITY BASED ON THE CONTENT OF A TEST? In Chapter 7, we introduce you to the different types of evidence that can be gathered to demonstrate the validity of a test. We introduce you to how validity is now defined and discussed in the Standards for Educational and Psychological Testing (the Standards) (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999) and how this newer view corresponds to the more traditional approach. We begin with a brief discussion of validity as defined by the new Standards as well as providing an overview of the three traditional forms of validity—content, criterion-related, and construct validity.  We then focus most of our attention on defining evidence of validity based on a test’s content and discuss methods for establishing it. (Other types of evidence of validity are discussed in Chapters 8 and 9.) We end Chapter 7 with a brief discussion of a (albeit less important) concept of validity, namely face validity.

CHAPTER 8—HOW DO WE GATHER EVIDENCE OF VALIDITY BASED ON A TEST’S RELATIONSHIP WITH EXTERNAL CRITERIA? In Chapter 8, we describe the processes that psychologists use to ensure that tests perform properly when they are used for making predictions and decisions. We begin by discussing the how evidence of validity can be established by evaluating the relationship between test scores and an external criteria (traditionally referred to as criterion-related validity.) We also discuss the importance of selecting a valid criterion measure, the method for evaluating validity coefficients, and the statistical processes that provide evidence that a test can be used for making predictions.

CHAPTER 9—HOW DO WE GATHER EVIDENCE OF VALDITY BASED ON A TEST”S RELATIONSHIP WITH CONSTRUCTS? In Chapter 9, we define and illustrate the terms psychological construct, theory, and nomological network. Because establishing evidence of construct validity involves accumulating and relating all of the psychometric information known about a test, we show how familiar concepts, such as reliability, the other types of evidence of validity are linked to construct validity. In addition, we discuss how convergent evidence of validity and discriminant evidence validity are two strategies for establishing construct validity. Finally, we discuss experimental methods used to establish construct validity for a test, including two procedures: confirmatory factor analysis and exploratory factor analysis.

Chapter 9

Construct Validity?

Behavior?

Chapter 7

What is Validity,

Specifically, 

Content Validity?

Chapter 6

What is Test 

Reliability?

Chapter 5

How Doe Test 

Users Interpret

Test Scores?

Psychometric 

Principles

Chapter 9

Chapter 8

Chapter 7

How do we gather evidence of validity based on the content of a test? 

,

 

Chapter 6

What Is Test 

Reliability?

Chapter 5

How Do Test 

Users Interpret

Test Scores?

Psychometric 

Principles

How do we gather evidence of validity based on a test’s relationships with constructs

How do we gather evidence of validity based on a test’s relations with external criteria?

Section II: Psychometric Principles

CHAPTER 5: How do test users interpret test scores?

After completing your study of this chapter, you should be able to do the following:

•	Describe and identify the different levels of measurement.
•	Summarize test scores using frequency distributions.
•	Describe the characteristics of the normal curve as well as skewed, peaked, and bimodal distributions.
•	Describe the purpose and calculate measures of central tendency, measures of variability, and measures of relationship.
•	Convert raw test scores into more meaningful units.
•	Describe norm-based interpretation and the different types of norms.

Levels of Measurement
•	The claims we can make about the results of psychological tests depend on the properties of these numbers or what we call a test’s level of measurement
•	Test scores can be classified into one of four levels of measurement: nominal, ordinal, equal interval, or ratio scales
•	With nominal scales, we assign numbers to represent categories
•	With ordinal scales, we assign numbers to order or rank things
•	With equal interval scales, we assign numbers to order things and we are able to determine the distance between two numbers
•	With ratio scales, we assign numbers to order things, we are able to determine the distance between two numbers, and there is a true zero point	Procedures for Interpreting Test Scores
•	Because raw test scores tell us very little, we use descriptive statistics to describe or summarize a distribution of test scores
•	Frequency distributions are graphical pictures that help us to understand the shape of a distribution
•	The normal probability distribution is a theoretical distribution that helps us to understand distributions of scores
•	Measures of central tendency are numerical tools that help us to understand the middle of a distribution of scores They include the mean, mode, and median
•	Measures of variability are numerical tools that help us to understand the spread of a distribution of scores They include the range, variance, and standard deviation
•	Measures of relationship are numerical tools that help us to understand how two sets of scores are related The correlation coefficient is a common measure of relationship	
Standard Scores and the Role of Norms
•	Standard scores are universally understood units of measurement in testing and allow us to evaluate how well an individual did on a test in comparison with others
•	We transform raw scores into standard scores to make comparisons
•	The most popular transformations are percentages, z scores, standard deviation units, T scores, and percentiles
•	Because most test scores provide us with relative measures, we often rely on norms—standards for interpreting scores—to describe a person’s performance
•	There are a variety of types of norms, including percentile ranks, age norms, and grade norms
•	Percentile ranks tell us the percentage of the norm group that scored less than or equal to an individual
•	Age and grade norms tell us whether an individual scored below, similar to, or above his or her age or grade level
•	Test users should be careful to select the appropriate norm group, ensure that the norms they use are up-to-date, and ensure that the size of the norm group is large enough to be representative of the population	

Three Types of Reliability
•	The test–retest method compares the scores of the same test takers taking the same test at two different times This method is appropriate when the test takers have not changed on the ability, trait, or attitude the test measures. The alternate forms method compares the scores of the same test takers on two equivalent forms of the test taken at the same time
•	The internal consistency method compares the scores of test takers on two halves of the test taken at the same time (split-halves method). Scores of test takers on all possible halves can be calculated using the coefficient alpha or KR-20 formula when the test questions are homogeneous
•	Scorer reliability is a method of estimating reliability or accuracy of those making judgments when scoring a test. Two types of scorer reliability are important: the comparison of judgments between or among scorers (interscorer) and the comparison of judgments each scorer makes for all tests (intrascorer). We use correlation to calculate the interscorer reliability for interval data such as numerical test scores. We use Cohen’s kappa to calculate interscorer reliability for categorical data such as ratings and decisions. We calculate intrascorer reliability using the formulas for calculating internal consistency	Calculating and Interpreting the Reliability Coefficient
•	A reliability coefficient is calculated by correlating the scores of test takers on two administrations of the same test
•	A reliability coefficient is interpreted by examining its sign (positive or negative) and its proximity to 100 Reliability coefficients should be positive and above .70
•	The standard error of measurement (SEM) provides a measure of how much an individual’s score is likely to differ from the individual’s true score
•	Using the SEM, we can calculate a confidence interval that is likely to contain the true score
•	Cohen’s kappa is a measure of agreement for use with nominal or ordinal data such as ratings and decisions The nearer  kappa is to 100, the stronger the agreement
	
Factors That  Influence Reliability		
•	Errors that increase or decrease individual scores and change the reliability estimate result from four sources: the test itself, test administration, test scoring, and test takers
•	Errors that result from poor test design include “trick”, ambiguous, and poorly worded questions as well as reading levels that are too high for test takers
•	Test administration can generate error when administrators do not follow instructions for test administration or when the testing environment is uncomfortable or distracting
•	Tests must be scored accurately and according to the instructions in the test manual
•	Test takers can contribute to test error by being fatigued or ill, by cheating, or by providing dishonest answers
•	Reliability is related to test length; the longer the test, the more reliable it is likely to be—provided that the questions on the test are homogeneous	

After studying this chapter, you should be able to do the following:

•	Define reliability and describe three methods for estimating the reliability of a psychological test and its scores.
•	Describe how an observed test score is made up of the “true score” and “random error.” Describe the difference between random error and systematic error.
•	Calculate and interpret a reliability coefficient, including adjusting a reliability coefficient obtained using the split-half method.
•	Differentiate between the “KR-20” and coefficient alpha formulas and understand how they are used to estimate internal consistency.
•	Calculate the standard error of measurement and use it to construct a confidence interval around an observed score.
•	Identify four sources of test error and six factors related to these sources of error that are particularly important to consider.
•	Explain the premises of generalizability theory and describe its contribution to estimating reliability

CHAPTER 6: What is test reliability?

Gathering Evidence of Validity
•	When we ask whether a test is valid, we are asking, “Are the inferences that are we are going to make from the scores on the test appropriate?”
•	The 1999 Standards for Educational and Psychological Testing recognize five sources of evidence of validity
•	Traditionally, testing experts have recognized three primary types of evidence of validity: content validity, criterion-related validity, and construct validity
•	Evidence of validity based on test content, (traditionally called content validity), is a validation strategy that involves scrutinizing the content of a test to determine whether the questions/items in the test are representative of the construct being measured by the test
•	Evidence of validity based on a test’s relationships with external criteria (traditionally called criterion-related validity) is a validation strategy that involves determining whether test scores are systematically related to other indicators of success or criteria. There are two methods that can be used to gather this evidence: the predictive method  and the concurrent method
•	Evidence of validity based on a test’s relationships with constructs (traditionally called construct validity) is a validation strategy that involves accumulating evidence that the scores on a test relate to observable behaviors in the ways predicted by the theory underlying the test.  There are two types of evidence that are gathered: convergent evidence and discriminant evidence
•	Validity is a unitary or single concept.  

	Evidence of Validity Based on a Test’s Content 
•	Evidence of validity based on at test’s content is established by the extent to which the questions on a test are representative of the construct, trait, or attribute being measured
•	Content based evidence of validity is similar to other types of validity evidence in that it helps us to answer the question about how adequately a test measures what it says it measures
•	Evidence of validity based on a test’s content differs from evidence of validity based on a test’s relationship with other variables because it involves examining the questions on a test rather than correlating the test scores to an external criterion or another test
•	One method for establishing content based evidence of validity involves performing a series of systematic steps as a test is being developed
•	Another method for establishing content based evidence of validity  involves evaluating the content of a test after it has been developed
•	Although helpful, face validity—whether the test appears to measure what it claims to measure—alone is not an acceptable means for determining the validity of a test	

After completing your study of this chapter, you should be able to do the following:

•	Explain what validity is.
•	Discuss the five sources of validity evidence described in the Standards for Educational and Psychological Testing (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 1999).
•	Describe the general nature of validity evidence based on test content, validity evidence based on the test’s relations with other variables, and how these sources of evidence suggest construct validity 
•	Explain, at a general level, the appropriate use of various validation strategies.
•	Describe methods for generating validity evidence based on the content of a test.
•	Explain the nature and importance of face validity and why it does not provide evidence for interpreting test scores.

CHAPTER 7: How do we gather evidence of validity based on the content of a test?

CHAPTER 8: How do we gather evidence of validity based on a test’s relationships with external criteria?

Demonstrating Evidence of Validity Based on a Test’s Relationships with External Criteria
•	There are two basic methods for showing a relation between a test and independent events or behaviors (the criterion).
•	Predictive evidence—established by correlating test scores taken at one time with scores on a criterion measure obtained at a later date, usually months later This method establishes that the test provides information about events in the future
•	Concurrent evidence—established by correlating test scores and criterion scores obtained at approximately the same time. This method establishes that the test can provide information about independent events or behaviors in the present
	Evaluating Validity Coefficients
•	Tests of significance establish how likely a correlation between the test and the criterion was obtained by chance
•	The coefficient of determination describes how much variance the test and the criterion share
•	Ongoing investigations of validity include cross-validation and meta-analyses
•	Psychologists also inquire whether validity estimates are stable from one situation to another—a question of validity generalization
	
Using Validity Information and Ethical Issues
•	When a relationship can be established between a test and a criterion, the test scores can be used to predict how well individuals are likely to perform on the criterion
•	Linear regression is used to make predictions from scores from one test
•	The slope (the expected change in one unit of y for every change in x) and the intercept (the point where the regression line crosses the y axis) are important for plotting a regression line
•	Multiple regression is used to make predictions about a criteria using two or more different tests
•	Decisions based on test scores have far-reaching consequences, and test users should have evidence of validity to be sure that the test scores and their predictions will be appropriate
•	When test takers are members of minorities, especially those who do not speak standard English as their primary language, test users must be aware of test bias and how it affects test validity

After completing your study of this chapter, you should be able to do the following:

•	Identify validity evidence for a test based on its relationships to external criteria (criterion-related validity), and describe two methods for obtaining this evidence.
•	Read and interpret validity studies.
•	Discuss how restriction of range occurs and its consequences.
•	Describe the differences between evidence of validity based on test content and evidence based on relationships with other variables.
•	Describe the difference between reliability and validity.
•	Define and give examples of objective and subjective criteria, and explain why criteria must be reliable and valid.
•	Calculate a validity coefficient and the coefficient of determination, and conduct a test of significance for the validity coefficient.
•	Explain the concept of regression, calculate and interpret a linear regression formula, and interpret a multiple regression formula.

CHAPTER 9: How do we gather evidence of validity based on at test’s relationship with constructs?

Defining a Construct
•	Construct validity is the extent to which a test measures a theoretical construct
•	Behaviors—activities that are observable and measurable—are concrete constructs
•	Underlying attitudes or attributes are abstract concepts that exist only in our imaginations
•	Definitions of abstract constructs can vary from person to person
•	Construct explication is the basis for establishing a test’s construct validity A construct must be part of a theory that defines the construct in observable and measurable terms
•	A listing of a construct’s relations and nonrelations to other constructs or tests can provide a number of hypotheses about the behaviors that people who have small or large quantities of the construct should display
	Gathering Evidence of Validity Based on a Test’s Relationships with Constructs 
•	We gather theoretical evidence by establishing a nomological network and proposing experimental hypotheses using the test to measure the construct
•	We gather psychometric evidence of construct validity by conducting empirical studies of the following:
•	Reliability—Test developers and researchers should provide evidence of internal consistency or homogeneity. Evidence of test–retest reliability is also appropriate
•	Convergent evidence of validity—A strong correlation between the test and other tests measuring the same or similar constructs is necessary
•	Discriminant evidence of validity—Lack of correlation between the test and unrelated constructs is also valuable
•	Designing a study that yields a multitrait–multimethod matrix of data is an elegant way to demonstrate the strategies of reliability, convergent evidence, and discriminant evidence
•	Experimental interventions in which the test is used as an independent or dependent variable make a substantial contribution to the argument for construct validity
•	Evidence of validity based on content or relationships with external criteria, when available and appropriate, strengthens the argument for construct validity
	
Factor Analysis
•	Researchers use factor analysis to identify underlying variables or factors that contribute to a construct or an overall test score
•	In confirmatory factor analysis, researchers hypothesize the underlying variables and then test to see whether they are there
•	Confirmatory factor analyses that confirm predicted underlying variables provide evidence of construct validity
•	Exploratory factor analyses take a broad look at the test data to determine and name the maximum number of underlying structures
•	The validation of the Brief Multidimensional Student Life Satisfaction Scale (BMSLSS) provides an example of how researchers carry out validation studies

After completing your study of this chapter, you should be able to do the following:

•	Discuss the concept of a construct and give examples of theoretical constructs
•	Explain how the current Standards for Educational and Psychological Testing (American Educational Research Association [AERA], American Psychological Association [APA], & National Council on Measurement in Education [NCME], 1999) treatment of constructs in testing differs from the more traditional usage of the term construct validity.
•	Explain and give examples of the three steps of construct explication.
•	Explain the process of establishing validity of validity based on a test’s relationship with other constructs.
•	Explain how Campbell and Fiske’s Multitrait–Multimethod Matrix provides evidence of validity
•	Discuss the roles of confirmatory and exploratory factor analysis in establishing validity.

After completing your study of this chapter, you should be able to do the following:

•	Describe the six methods for acquiring knowledge.
•	Explain how the scientific method differs from other methods for acquiring information, and describe the five steps associated with the scientific method.
•	Apply the five-phase scientific approach to constructing and administering surveys and analyzing their data.
•	Describe survey reliability and validity and associated concepts.

Chapter 12

How Do You Know

When a Test Works?

Chapter 11

How Do You 

Develop a Test?

Chapter 10

How Do We 

Construct,

Administer, and

Use Survey

Data?

Developing and

Piloting Surveys

And

Psychological

Tests

Chapter 12

How Do We Assess the Psychometric Quality of a Test?

Chapter 11

How Do We 

Develop a Test?

Chapter 10

How Do We 

Construct,

Administer, and

Use Survey

Data?

Developing and

Piloting Surveys

and

Psychological

Tests

CHAPTER 10—HOW DO WE CONSTRUCT, ADMINISTER, AND USE SURVEY DATA? In Chapter 10, we begin by defining what a survey is and discussing how surveys are popular research tools. Knowing that how we collect information affects its quality, we present an overview of the different ways we collect and acquire data. We focus on one approach, the scientific method, which is the very essence of collecting accurate survey data. After briefly reviewing the scientific method, we focus on a five-phase scientific approach to constructing, administering, and analyzing surveys. Finally, we discuss survey reliability and validity.

CHAPTER 11—HOW DO WE DEVELOP A TEST? In Chapter 11, we  discuss the steps for developing psychological tests. This chapter looks at the process of constructing a test plan, various formats for writing questions (for example, multiple choice, true/false) that include their strengths and weaknesses, and how test takers’ perceptions and preconceived notions can influence test scores. Finally, we present guidelines on how to write test questions and discuss the importance of the instructions that accompany a test.

CHAPTER 12—HOW DO WE ASSESS THE PSYCHOMETRIC QUALITY OF A TEST? In Chapter 12, we continue describing the test development process by discussing piloting the test and analyzing the items in terms of their difficulty, their ability to discriminate among respondents, and their likelihood of introducing error into the test results. We also describe the process of revising the test and gathering evidence of reliability and validity. Finally, we briefly discuss the contents of the test manual.

Surveys, Knowledge Acquisition, and the Scientific Method
•	Surveys are research tools that collect information that enable us to describe and compare people’s attitudes, knowledge, and behaviors
•	To determine whether information is accurate, we must look at how the information was obtained
•	According to Helmstadter (1970), there are six methods for obtaining information: tenacity, intuition, authority, rationalism, empiricism, and the scientific method
•	Information obtained using the scientific method is more accurate and reliable than information obtained in other ways
•	The scientific method is a process or method for generating a body of knowledge
•	This process includes identifying a problem or an issue and forming a hypothesis, designing a study to explore the problem/issue and hypothesis, conducting the study, analyzing and interpreting the results, and communicating the research results
•	The scientific method can be followed using both experimental and descriptive research techniques
•	Descriptive research techniques often involve the use of surveys—research tools that collect information to facilitate the description and comparison of people’s attitudes, knowledge, and behaviors
	The Survey Research Method
•	Surveys are one of various research methods you can use to collect information
•	Constructing a reliable and valid survey takes time, and it is as much a science as it is an art
•	Using a scientific approach to survey research will increase the reliability and validity of a survey
•	A scientific approach to surveys involves five general phases: (a) identifying survey objectives and forming a hypothesis, (b) constructing the survey, (c) administering the survey, (d) entering and analyzing the data, and (e) presenting the findings
•	During the first phase, researchers identify the survey objectives, define the objectives operationally, and construct a plan for completing the survey
•	During the second phase, researchers select the type of survey, write the survey questions, prepare the survey for distribution, and pretest the survey
•	During the third phase, researchers select the appropriate respondents, determine the sample size, and distribute the survey
•	During the fourth phase, researchers code the survey questions, enter and verify the data, and conduct the analyses
•	During the fifth phase, researchers prepare and present an oral or written report	
Survey Reliability and Validity	
•	Good surveys are reliable and valid
•	A reliable survey is free of measurement error and provides you with similar information each time you use it
•	As with psychological tests, we can determine the reliability of a survey using a number of methods, including test–retest reliability, alternate forms reliability, and split-half reliability
•	A valid survey is one that measures what it claims to measure
•	As with psychological tests, we can gather evidence of the validity of a survey using a number of methods, including content validity, criterion-related validity, and construct validity

CHAPTER 10: How do we construct, administer and use survey data?

Section III: Developing and Piloting Surveys and Psychological Tests

Pilot Test and Item Analysis
•	The pilot test provides a thorough examination of how the new test works and the results that it yields
•	After the test has been administered to a sample of participants, the test developers review and analyze the pilot test data
•	Quantitative item analysis examines how well each test item performs
•	The difficulty of each item is calculated by finding the percentage of test takers who responded to the item correctly. Most developers seek a range of difficulty levels that average approximately .5
•	Test developers also create a discrimination index. This statistic compares the performance on each item of those who made very high test scores with the performance on each item of those who made very low test scores
•	Average interitem correlation and item–total correlation provide information for increasing the test’s internal consistency. Each item should be correlated with every other item measuring the same construct
•	Item-total correlations also provide an index of discrimination for each item
•	In item response theory, the performance of each item is related to the test taker’s ability on the construct being measured. The resulting item characteristic curve (ICC) is a graph of the probability of answering an item correctly given a certain level of ability. It combines item difficulty and discrimination
•	Item bias refers to an item being easier for one group than for another group. ICCs provide information on item bias by comparing group scores
•	Test developers use questionnaires and expert panels for a qualitative analysis of test items

Revising the Test
•	Test items are dropped based on their consistency, difficulty, discrimination, and bias until a final form of the test is reached
•	If enough items were piloted, no items will be rewritten. Items that are rewritten must be piloted to be sure they now meet criteria for retention in the test
•	Other test components, such as the instructions, should be revised based on the results of a qualitative analysis in the form of questionnaires or interviews with test takers, subject matter experts, and testing experts

Validation, Norms, Cut Scores, and the Test Manual
•	Standards for designing the validation study are similar to those for designing the pilot study, including using a representative sample of the target audience that is large enough to conduct the desired statistical tests
•	Test developers should follow the American Psychological Association (APA) code of ethics. Using more than one test site will provide evidence that the results generalize from site to site
•	The scores resulting from the validation study should not be used for decision making or evaluation of individuals
•	When the test user will make predictions from test results, a cross-validation study is important. Test developers expect the resulting validity coefficient to be lower, and the difference between the coefficients in the validation study and those in the cross-validation study is called shrinkage
•	Differential validity results when tests have different validity coefficients for different groups. Single group validity means that a test is valid for only a specific group
•	Norms and cut scores can be developed from the validation data to provide test users with information for interpreting test scores
•	Cut scores can be determined empirically using the correlation between the test and an outside criterion or by a panel of expert judges
•	Finally, the developers compile the test manual, which has been in the process of development along with the test. The test manual includes the answer key, instructions for the administrator and test user, information on test development, validation and cross-validation, norms, and cut scores

After completing your study of this chapter, you should be able to do the following:

•	Explain the importance of conducting a pilot test.
•	Describe how a pilot test should be set up and specify the types of information that should be collected.
•	Describe the collection, analyses, and interpretation of data for an item analysis, including item difficulty, item discrimination, interitem correlations, item–criterion correlations, item bias, and item characteristic curves.
•	Describe the collection and interpretation of data for a qualitative item analysis.
•	Identify and explain the criteria for retaining and dropping items to revise a test.
•	Describe the processes of validation and cross-validation.
•	Explain the concepts of differential validity, single-group validity, and unfair test discrimination.
•	Explain the purpose of a cut score and describe two methods for identifying a cut score.
•	Tell what information should be included in a test manual and why.

CHAPTER 12: how do we assess the psychometric quality of a test?

CHAPTER 11: How do we develop a test?

Steps in Test Construction
•	The first step in test development involves defining the test universe, the target audience, and the purpose of the test
•	The next step in test development is to write out a test plan, which includes the construct(s) definition, the test format, the administration method, and the scoring method
•	After a review of the literature, the test developer writes a concise definition of the test construct(s) and operationalizes each construct in terms of observable and measurable behaviors
•	The test developer chooses an objective or subjective format and the type of test question (for example, multiple choice, true/false, open-ended, essay)
•	Then the test developer specifies how the test will be administered and scored. Three models for scoring are the cumulative model, the categorical model, and the ipsative model. The scoring model determines the type of data (nominal, ordinal, or interval) the test will yield
•	After completing the test plan, the test developer is ready to begin writing the actual test questions and administration instructions
•	After writing the test, the developer conducts a pilot test followed by other studies that provide the necessary data for validation and norming
	Composing Test Items
•	The test developer chooses the item format based on information in the test plan such as the target audience, the method of administration, and the requirements for scoring
•	Objective formats include multiple choice, true/false, and forced choice items
•	Subject formats include essay and interview questions
•	Projective tests are another type of subjective format that uses a highly ambiguous stimulus to elicit an unstructured response from test takers. Projective stimuli include pictures and written or spoken words. Test takers may respond verbally or by drawing pictures
•	Some people have response sets—patterns of responding that result in false or misleading information—such as social desirability, acquiescence, random responding, and faking
•	Although there is no set of rules that guarantees items will perform as expected, the test development literature contains a number of suggestions for writing successful items
•	Objective items provide ample sampling of the testing universe, but they are more time-consuming to develop. Scoring of objective items is easier and likely to be more accurate and reliable
•	Subjective items are easier to construct and revise. Some experts suggest that essay and interview questions are better suited to testing higher order skills such as creativity and organization. Scoring of subjective items is more difficult, requiring independent scoring by two experts to increase reliability and accuracy
	
Writing the Instructions for the New Test
•	The instructions for the test administrator should cover group or individual administration, specific requirements for the location, required equipment, time limitations or approximate time for completion of the test, script for the administrator to read to test takers, and required training for the test administrator
•	The instructions on the test need to be simple, concise, and written at a low reading level. Complicated instructions for responding are likely to lead to confused test takers and an increased probability of response errors
•	The scoring instructions and test key ensure that each person who scores the test will follow the same process. The scoring instructions should explain how the test scores relate to the construct the test measures

After completing your study of this chapter, you should be able to do the following:

•	Describe how to define the test domain, the target audience, and the purpose of the test.
•	Develop a test plan, including defining the construct or constructs the test measures, choosing the test format, and specifying how to score the test.
•	Differentiate between objective and subjective test questions and describe the strengths and weaknesses of each.
•	Describe the important issues in writing instructions for the test administrator and the test taker.

Chapter 15

How Are Tests

Used in 

Clinical and 

Settings

?

Chapter 14

How Do 

Organizations

Use Tests

?

Chapter 13

How Are Tests

Used in Educational

Settings?

Using Tests

In Different

Settings

Chapter 15

How Are Test Used in Clinical and Counseling Settings?

Chapter 14

How are Tests Used in Organizational Settings?

Organizations

Chapter 13

How Are Tests

Used in Educational

Settings?

Using Tests

In Different

Settings

How  Are Tests Used In Clinical and Counseling Settings?

After completing your study of this chapter, you should be able to do the following:

•	Describe the types of decisions educators make based on the results of psychological tests.
•	Explain why educators are test users and why they need to follow the test user guidelines discussed in Chapter 3.
•	Describe how teachers use psychological tests in the classroom before, during, and after instruction.
•	Describe how psychological tests can benefit student motivation, retention and transfer of learning, self-assessment, and instructional effectiveness.
•	Describe how educational institutions use tests to make selection and placement decisions, counseling and guidance decisions, program and curriculum decisions, and administrative policy decisions.
•	Explain the differences between norm-referenced and criterion-referenced tests.
•	Describe authentic assessment and the perceived advantages and disadvantages of authentic assessment.

Decision Making in the Educational Setting and Educators as Test Users
•	Psychological tests play an important role in educational settings, and their role will likely continue to increase
•	Teachers, testing specialists, and educational administrators use psychological tests to make a variety of important decisions
•	Teachers use psychological tests to make instructional, grading, and diagnostic decisions
•	Testing specialists and educational administrators use psychological tests to make selection, placement, counseling and guidance, program and curriculum, and administrative policy decisions
•	As test users, teachers, testing specialists, and educational administrators should be properly trained on the appropriate use of psychological tests
•	Educators without proper testing knowledge can make some improper decisions and cause themselves embarrassment or worse	Psychological Test Use in Educational Settings
•	Teachers use psychological tests as placement assessments, formative assessments, and summative assessments
•	As placement assessments, teachers use tests to determine whether students have the skills or knowledge necessary to understand new material and to determine how much information students already know about new material
•	As formative assessments, teachers use tests to determine what information students are and are not learning
•	As summative assessments, teachers use tests to determine whether students have achieved what they intended them to achieve and to assign grades 
•	Tests can also help to motivate students, help students to retain and apply information, and help students to understand themselves
•	Tests can also help teachers to understand the effectiveness of their teaching methods
•	Administrators and testing specialists often use standardized tests of achievement, aptitude, and intelligence to help make selection and placement decisions
•	Although they have been around for many years, portfolios are becoming a more popular alternative for making admissions decisions and evaluating performance and potential
•	Career counselors and other testing professionals use psychological tests, along with other information, to help students understand their interests, strengths, abilities, and preferences and to translate this information into career guidance decisions
•	Educational administrators use psychological tests to make program, curriculum, and administrative policy decisions
	
Norm-Referenced, Criterion-Referenced, and Authentic Assessment of Achievement	
•	Most psychological tests used in educational settings can be classified as norm-referenced or criterion-referenced tests
•	Educators use norm-referenced tests to compare a student’s performance with the performance of other students
•	Educators use criterion-referenced tests to compare a student’s performance with an objectively stated standard of achievement such as being able to multiply numbers
•	Some individuals believe that norm-referenced and criterion-referenced tests are too structured and instead promote the use of authentic assessment
•	Authentic assessment focuses on a student’s ability to apply what he or she has learned in real-world settings

CHAPTER 13—HOW ARE TESTS USED IN EDUCATIONAL SETTINGS? In Chapter 13, we focus on how psychological tests are used in educational settings. We begin with an overview of the types of decisions that are made in education based on the results of psychological tests. We discuss educators as test users and provide a few examples of situations where educators did not use tests properly. Following a detailed discussion of exactly how tests are used in the classroom, we highlight how tests are used in educational settings to make selection and placement decisions, counseling and guidance decisions, and curriculum/administrative policy decisions. We end with a discussion of norm- and criterion-referenced tests and authentic assessment in educational settings.

CHAPTER 14—HOW ARE TESTS USED IN CLINICAL AND COUNSELING SETTINGS? In Chapter 14, we focus on how psychological tests are used in clinical and counseling settings. We begin with an overview of three models clinicians and counselors use in psychological testing. We discuss the types of tests used for diagnosis and intervention, including clinical interviews and various projective techniques. After discussing neuropsychology and neuropsychological tests, we conclude with a discussion of specialized tests for clinical disorders.

CHAPTER 15—HOW ARE TESTS USED IN ORGANIZATIONAL SETTINGS? In Chapter 15, we focus on how psychological tests are used in organizations. We begin with a brief history of the role psychological assessment has played in organizations. We examine various types of tests that are used for hiring employees such as interviews and tests of performance, personality, and integrity. We consider legal constraints on employment testing legislated by Congress and interpreted by the executive branch and the federal court system. Finally, we describe how organizations use psychological assessment to evaluate employee performance.

CHAPTER 13: How are tests used in educational settings?

Section IV: USING TESTS IN DIFFERENT SETTINGS

After completing your study of this chapter, you should be able to do the following:
•	Identify and explain three models of psychological assessment that clinicians and counselors use.
•	Explain the concept of clinical diagnosis, and describe how the clinical interview and structured personality tests are used to make diagnoses.
•	Describe three types of projective techniques and how practitioners use them for diagnosis and treatment.
•	Describe the field of neuropsychology, and explain how neuropsychologists use electrophysiological techniques, neurobehavioral assessments, and temperament or personality measures to diagnose and treat clients of various ages.
•	Describe how practitioners use paper-and-pencil tests to diagnose and treat depression, anxiety, anger, and attention deficit hyperactivity disorder.

Models of Clinical Assessment
•	The information-gathering model represents the use of standardized tests to make diagnoses
•	The therapeutic model represents the use of tests as an intervention that provides new information for the client to use for self-discovery and growth
•	The differential treatment model represents the use of tests for conducting research or evaluating program outcomes
•	The roles of counselors and clinical psychologists differ, but they also overlap
•	Counselors primarily provide services for those with normal developmental problems, and clinical psychologists and psychiatrists generally treat those with abnormal behavior, mental disorders, or emotional disabilities
Making Diagnoses Using the Interview and Structured Personality Tests
•	A diagnosis is the definition of a client’s problem or disorder, and screening is the process of conducting a psychological assessment to arrive at a diagnosis
•	The clinical interview in which the practitioner observes and gathers information about the client is a primary diagnostic tool
•	There are three types of interviews: (a) the structured clinical interview, which has a predetermined set of questions and yields a score; (b) the nondirective clinical interview, in which the practitioner’s questions follow up on the client’s report of symptoms or problems; and (c) the semistructured interview, in which some questions are predetermined but the practitioner also asks follow-up questions based on the client’s responses
•	The practitioner who uses the nondirective approach risks three sources of bias: hypothesis confirmation bias, the self-fulfilling prophecy, and ethnocentrism
•	The nondirective interview may be more useful for an intervention (the therapeutic model) than for a diagnosis
•	Practitioners also use standardized personality tests, such as the Minnesota Multiphasic Personality Inventory-2 (MMPI-2), the NEO Personality Inventory, the California Personality Inventory (CPI), and the 16 Personality Factor Questionnaire (16PF), to make diagnoses
Projective Techniques
•	Projective techniques ask test takers to give meaning to ambiguous stimuli
•	Projective storytelling requires test takers to tell a story about some visual stimuli such as pictures
•	In projective drawing, test takers draw and interpret their own pictures
•	In sentence completion tests, the assessor administers partial sentences, either verbally or on paper, and asks test takers to respond by completing each sentence
•	A major weakness of most projective tests is a lack of evidence of traditional psychometric soundness such as reliability and validity Therefore, the value of projective tests may be in their usefulness as interventions rather than as diagnostic or research instruments
Neuropsychological Tests
•	Neuropsychology is a branch of psychology that focuses on the relation between brain functions and behavior
•	Neuropsychologists use electrophysiological techniques such as the electroencephalogram (EEG), a continuous written record of brainwave activity, and event-related potential (ERP), a record of the brain’s electrical response to the occurrence of a specific event
•	Developmental applications involve assessments that determine whether the client is developing normally
•	Two developmental tests for infants are biochemical assessment, which is an analysis of blood gases to determine the concentration of oxygen and carbon dioxide at the tissue level, and electrophysiological assessment, which involves monitoring vital signs such as heart rate and spontaneous electrical activity of the brain
•	Neuropsychologists conduct neurobehavioral assessments such as eliciting various reflexes to assess the functioning and maturity of an infant’s central nervous system
•	Neuropsychologists use tests that measure temperament or personality as well as intelligence, academic achievement, language, perception, and attention tests to assess social–emotional functioning in children
•	When treating the elderly, neuropsychologists take into account the normal changes in brain function that result from aging, including the fact that aging can exaggerate the psychological impact of medical conditions
•	Three major areas of psychopathological disorders—anxiety, depression, and schizophrenia—have been researched extensively by neuropsychologists
Specialized Tests for Clinical Disorders
•	Psychologists also use traditional paper-and-pencil tests to identify mental disorders such as anger, anxiety, and depression
•	The Beck Depression Inventory and the Beck Anxiety Inventory are examples of tests that assess single constructs or disorders
•	Another dimension of measurement is the assessment of whether an attribute is temporary (state) or an ongoing part of the personality (trait)
•	When clients such as children cannot complete self-report measures, trained professionals use observational scales to rate observed behaviors

CHAPTER 14: how are tests used in clinical and counseling settings?

After completing your study of this chapter, you should be able to do the following:

•	Discuss the history of employment testing in the United States during the 20th century.
•	Report the strengths and weaknesses of the traditional interview and the structured interview for assessing job candidates.
•	Describe the characteristics of a performance test and discuss two types of performance tests used by organizations.
•	Describe the five-factor model of personality and name two tests that are based on the five-factor model.
•	Discuss two types of integrity tests and describe the criticism these tests have received.
•	Discuss performance appraisal instruments, give examples of three types of rating scales, and describe four types of rating errors.

History of Employment Testing
•	In 1915, Scott published “The Scientific Selection of Salesmen,” in which he proposed that employers use group tests for personnel selection
•	Proposals from psychologists led to the development of the US Army’s Alpha and Beta tests during World War I
•	Following World War I, psychologists continued to investigate employment testing and began to study measuring job performance and placing workers in jobs based on their skills and qualifications
•	Two consulting firms that specialized in using tests in organizations emerged. One was founded by Scott, and the other was the Psychological Corporation organized by Cattell
•	Use of tests by organizations expanded greatly during the latter half of the 20th century
•	Title VII of the Civil Rights Act of 1964 stimulated interest in fair employment practices, and psychologists played a key role in developing the federal government’s Uniform Guidelines on Employee Selection Procedures in 1978
•	Today, business organizations use various types of psychological tests to meet their business needs
	Preemployment Testing
•	Psychological assessment provides the basis for hiring employees in most organizations
•	The most popular method of assessment is the employment interview
•	Types of interviews vary from the traditional interview, in which the interviewer pursues different areas of inquiry with each job candidate, to highly structured interviews that are standardized with the same questions asked of each job candidate
•	Many shortcomings of the traditional interview, including its low reliability and validity, can be overcome by structuring the interview and the interviewing procedure
•	Performance tests require the test taker to perform one or more job tasks
•	Personality inventories measure ongoing cognitive constructs, such as conscientiousness, extraversion, and agreeableness, that may influence individuals to perform certain behaviors
•	One widely accepted personality theory is the five-factor model, which proposes five central personality dimensions: surgency, emotional stability, agreeableness, conscientiousness, and intellect or openness to experience
•	Integrity tests that predict an employee’s predisposal to be dishonest fall into two general categories: physiological measures and paper-and-pencil tests
•	When a job analysis indicates that cognitive skills are important for high performance, cognitive tests are useful for inclusion in a preemployment assessment
•	The Uniform Guidelines on Employee Selection Procedures suggest procedures for organizations to follow that enhance the fairness and legal defensibility of their employment practices
•	Preemployment tests should be validated to ensure that the inferences that will be drawn about the suitability of job candidates for employment are appropriate.  
•	Generalization of validity evidence, such as using a validity transportability approach, may be appropriate when it is not practical to perform a local validity study.	
Performance Appraisal
•	Most organizations carry out formal evaluations of employees’ job performance called performance appraisals
•	The major types of performance appraisal systems are ranking employees and rating employees on various types of scales
•	The most popular method of rating employee performance uses the graphic rating scale
•	Two other methods are the behaviorally anchored rating scales (BARS) and the behavioral checklist
•	Raters need to be trained so that they do not make rating errors such as leniency, severity, central tendency, and the halo effect
•	Supervisors and managers usually rate their subordinates
•	In 360 feedback, individuals are rated by their managers, peers, subordinates, customers, and selves

CHAPTER 15: How are tests used in organizational settings?