Philosophy and Theory in Research Science

Inherited knowledge passed down through a specific culture; these are the things that “everybody knows.”

Knowledge gained by listening to experts and social or professional figures with high status (e.g., doctors, teachers, parents).

Common sense

Widely shared beliefs based on tradition, authority, or consensus within a society.

This includes information from newspapers, television, the Internet, etc.

An approach to inquiry that attempts to safeguard errors commonly made in casual human inquiry. 

1.      Viewing all knowledge as provisional (i.e., serving for the time being) and subject to refutation.

3.      Pursuing objectivity in observation.

Evidence-based practice

The ultimate expression of scientific inquiry in social work research is evidence-based practice.

Using the best scientific evidence available in deciding how to intervene with individuals, families, groups, or communities.

Positivism

A paradigm introduced by August Comte that held that social behavior could be studied and understood in a rational, scientific manner – in contrast to explanations based in religion or superstition (i.e., a modern rather than pre-modern interpretation).

Asserts that only verifiable claims based directly on experience could be considered genuine knowledge.

Interpretivism

Postmodernism

Recognizes that discretionary judgment is unavoidable in science, that proving causality with certainty in explaining social phenomena is problematic, that knowledge is embedded in historically specific paradigms and is therefore relative rather than absolute, and that all methods are imperfect, so multiple methods, both qualitative and quantitative, are needed to generate and test theory, improve understanding over time of how the world operates, and support informed policymaking and social program decision making.

A systematic set of interrelated statements intended to explain some aspect of social life.

The logical model in which general principles are developed from specific observations.

Types of social welfare research:

Descriptive

Explanatory

EvaluativeExploratory

Descriptive

Descriptive research involves measuring, counting, or describing a phenomenon.

Explanatory research attempts to find correlational or causal relationships – what effect does one factor have on another?

Evaluative

Evaluative research is often considered a subtype of explanatory research.

Evaluative research attempts to study the effects of a specific intervention or program.

Exploratory research attempts to better understand a problem, population, or phenomenon when no or very little information exists.

are a type of exploratory research.

Conceptualization and Operationalization

together represent the process of moving from vague ideas about what you want to study to being able to recognize and measure what you want to study.

The process of refining and specifying abstract concepts.

The process through which we specify precisely what we will mean when we use particular terms.

Specific concepts or theoretical constructs under investigation.

Comprises more than one attribute, or value, and thus is capable of varying.

If a variable is not capable of varying, it is a constant.

A variable must also be translatable into observable terms.

The concepts that make up a broader concept or variable.

A variable postulated to explain another variable. 

A third variable that effects or explains the relationship between independent and dependent variables

Also known as a confounding variable.

Extraneous variable that is controlled, or adjusted, for in our research design

Controlling for a variable means taking each level of that control variable separately and looking at how the original relationship between the independent and dependent variable changes or remains the same within each level of the control variable

Hypothesis – Statement that postulates the relationship between the independent and dependent variables.

Hypotheses are tentative statements that predict what we expect to find about the way our variables co-vary together.

Positive – As independent variable increase (or decreases), dependent variable also increases (or decreases).

Negative – Independent and dependent variable move in opposite direction  

Curvilinear – Relationship changes at certain levels of the variables.

Non-directional – Direction of relationship is not declared. 

Null – There is no relationship between the variables.

Statement that postulates the relationship between the independent and dependent variables.

Hypotheses are tentative statements that predict what we expect to find about the way our variables co-vary together.

As independent variable increase (or decreases), dependent variable also increases (or decreases).

Independent and dependent variable move in opposite directions.

Types of relationships

Direction of relationship is not declared.

There is no relationship between the variables.

Type of relationship

Developing specific research procedures that will result in empirical observations of constructs that represent these concepts in the real world.

In operationalization, concrete empirical procedures that will result in measurements of variables are specified.

Translating a variable into observable terms.

The operations, or indicators, we will use to determine the attribute we observe about a particular concept.

A type of composite or cumulative measure composed of several items that have a logical or empirical structure among them.

Measurement

Validity

Refers to whether the observed scores accurately reflect the true scores of the construct or variable under investigation. 

Does the instrument measure what it intends to measure?

Non-empirically-based validity

Content validity

Refers to whether an instrument appears to measure what it intends to measure.

Think of “face value.”

The researcher constructing the instrument and experts with knowledge of the construct are the best judges of face validity.

Refers to whether an instrument measures all possible dimensions of the construct or variable under investigation

The degree to which a measure covers the range of meanings included within a concept.

Content validity is also assessed by professionals and experts with knowledge of the construct.

Empirically- or evidence-based validity:

Criterion-based validity Example:

Concurrent validity

Predictive validity:

Construct validity

Construct validity

Discriminant validity

Reliability

Refers to whether an instrument can predict concurrent criterion (concurrent validity) or future criterion (predictive validity).

Example: Concurrent validity: A written driving test that predicts how well a person will drive soon after passing the written test demonstrates concurrent validity. 

Example: Predictive validity: A GRE score that predicts how well a student will do in future graduate studies demonstrates predictive validity.

Concurrent and predictive validity are different forms of criterion-based validity.

Construct validity – Refers to whether an instrument has high correlation with existing instruments that measure the same construct (convergent validity) or has low correlation with instruments that measure other constructs (discriminant validity).

A written driving test that predicts how well a person will drive soon after passing the written test demonstrates concurrent validity.

Refers to whether an instrument can predict concurrent criterion (concurrent validity) or future criterion (predictive validity).

This is a type of Empirically- or evidence-based validity:

: A GRE score that predicts how well a student will do in future graduate studies demonstrates predictive validity.

Concurrent and predictive validity are different forms of criterion-based validity.

Construct validity – Refers

is assessed by examining the relationship between the construct measured by the new instrument and the same construct as measured by an existing instrument.

That quality of measurement method that suggests that the same data would have

been collected each time in repeated observations of the same phenomenon.

It refers to whether a particular measure or test applied repeatedly to the same subjects or objects would yield the same results each time.

An instrument can be reliable, but not valid.

Interrater reliability – The extent of agreement, or consistency, between or among observers.

Test-retest reliability – A method for testing reliability of written instruments that indicates how stable a measure is over time.

Data collection methods: Test, wait, retest (same subjects, same test, two occasions).

Parallel-forms reliability – Comparing the results from two instruments designed to measure the same construct.

Data collection methods: Give test 1; give test 2 (same subjects, two tests, one occasion).

Internal consistency reliability – A practical and commonly used approach to assessing reliability that examines the homogeneity of a measurement instrument by dividing the instrument into equivalent halves and then calculating the correlation of the scores of the two halves (Cronbach’s alpha).

Data collection methods: Give one test (same subjects, one occasion).

The extent of agreement, or consistency, between or among observers.

A method for testing reliability of written instruments that indicates how stable a measure is over time.

Comparing the results from two instruments designed to measure the same construct.

Data collection methods: Give test 1; give test 2 (same subjects, two tests, one occasion).

A practical and commonly used approach to assessing reliability that examines the homogeneity of a measurement instrument by dividing the instrument into equivalent halves and then calculating the correlation of the scores of the two halves (Cronbach’s alpha).

Data collection methods: Give one test (same subjects, one occasion).

Validity has to do with the amount of systematic error in a measurement

explain how errors occur and type

That quality of a measurement device that tends to result in a misrepresentation of what is being measured in a particular direction

– Agreeing or disagreeing with most or all statements regardless of content.

– A measurement error that has no consistent pattern and that reduces the reliability of measurement.

The more reliable the measure, the less random error in it.

Sampling Methods

define

Sampling is the process of selecting observations.

-If a sample of individuals from a population is to provide useful descriptions of the total population, then it must contain essentially the same variations that exist in the population.

The quality of a sample having the same distribution of characteristics as the population from which it was selected.

– The quality of a research finding that justifies the inference that it

-To infer that the findings of a particular study represent causal processes or apply to settings or populations beyond the study conditions.

That unit about which information is collected and that provides the basis for

analysis.

– The list or quasi-list of units that compose a population from which a

sample is selected.

Probability sampling

Random selection

A probability sampling procedure in which each element has an equal chance of selection independent of any other event in the selection process.

-Probability theory is a branch of mathematics that provides the tools researchers need to devise sampling techniques that produce representative samples. 

Probability sampling techniques:

– A type of probability sample in which units that compose a population are assigned numbers. 

-A set of random numbers is then generated, and the units having those numbers are included in the sample.

– A type of probability sample in which every kth unit in a list is selected for inclusion in the sample. 

The standard distance between elements selected in the sample.

There is one danger involved in systematic sampling – the arrangement of elements in the list.

Systematic sampling is usually superior to SRS, in convenience, if nothing else.

Stratified sampling (define)

&

The ultimate function of stratification

– A probability sampling procedure that uses stratification to ensure that appropriate numbers of elements are drawn from homogeneous subsets of that population.

– A sampling method aimed at ensuring that enough cases of certain minority groups are selected to allow for subgroup comparisons within each of these minority groups.

– A multi-stage sampling procedure in which natural groups (clusters) are sampled initially, with the members of each selected group being subsampled afterward.

This refers to a type of multi-stage cluster sample in which clusters are selected not with equal probabilities, but with probabilities proportionate to their sizes – as measured by the number of units to be subsampled.

1-Availability sampling/convenience sampling. 

3-Snowball sampling –

In practice, there is greater likelihood that a probability sample will be representative of the population from which it is drawn than will a nonprobability sample.

– Also referred to as convenience sampling. 

 – Also referred to as judgmental sampling. 

Selecting a sample of observations or elements that the researcher believes will yield the most comprehensive understanding of the study, based on the researcher’s intuitive feel for the subject that comes from extended observation and reflection.

– Asking each person interviewed to suggest additional people for interviewing.

-Snowball sampling is appropriate when the members of a special population are difficult to locate.

A type of sampling in which units are selected into the sample on the basis of prespecified characteristics so that the total sample will have the same distribution of characteristics as are assumed to exist in the population being studied.

Sample size

In determining sample size, you must take into consideration

-the number of comparison or control groups, the anticipated response rate, and the anticipated attrition rate.

-Today, survey research is perhaps the most frequently used mode of observation in the social sciences.

Mailed, delivered, or given at a set location.

Interviewers can probe respondents for answers, as well as explain complicated or unclear questions. 

Interview surveys are much more expensive than any other type of survey format.

Big advantages are safety to the interviewer, time, and money.

Problems with telephone surveys can be unlisted telephone numbers, answering machines, caller id, and losing respondents mid-interview.

The biggest problem with online surveys is representativeness of respondents.

Advantages of a self-administered questionnaire over an interview, whether by mail or email, are

1.      Economy 2.      Speed  3.      Lack of interviewer bias 4.      Safety of interviewers 5.      The possibility of anonymity and privacy to encourage more candid responses on sensitive issues

The advantages of an interview survey over a self-administered questionnaire are:

1.      Fewer incomplete questionnaires

2.      Fewer misunderstood questions

3.      Generally higher response rates

4.      Greater flexibility in terms of sampling and special observations

4-Survey research is generally strong on reliability. 

Weaknesses of Survey Research

1-Survey research can seldom deal with the context of social life. They can be superficial.

Client satisfaction surveys:

-One big problem with client satisfaction surveys is

-Most professionals in the human or social services, if told to design a program evaluation, would likely begin thinking about surveying clients to determine what they thought of the services received.

– Inferring or attributing a change in your dependent variable as “caused” by a change in your independent variable.

precede the change in the dependent variable). [Infering Change in DV "cause change in IV]

Basic AB design

Basic AB design

Multiple-baseline designs (A type of single-case evaluation)

Basic AB design

  -The simplest single-case evaluation design that includes one baseline phase (A) and one intervention phase (B).   

-This is a popular design among practitioners and researchers because it involves only one baseline and therefore poses the least conflict with service delivery priorities.   

-It has less control for history, however, than most alternative single-case evaluation designs.

  -When the baseline trend signifies improvement, it may be advisable to continue collecting baseline measures until the improving trend levels off.

 -Baselines ideally should be extended until a stable trend in the data is evident. 

-How many measures or data points are adequate? The more measures the better. 

-You must have a minimum of three data points. Most authors recommend 5-10 data points to establish a baseline.

Basic AB design

A single-case evaluation design that adds a second baseline phase (A) and a second intervention phase (B). 

Multiple-baseline designs

Although each baseline starts simultaneously, the intervention is introduced at a different point for each.