PSY 3044 - LECTURES > Learning, Part III; Operant Conditioning

BASICS of OPERANT CONDITIONING

Thorndike’s Puzzle Box and the Law of Effect

Thorndike (1898, 1911) - first to systematically study how non-reflexive behaviors can be modified by experience

IV: Experience in puzzle box (# of trials)

DV: Escape latency: the amount of time it took for subject to escape box on each trial

- Thorndike attributed gradual improvement over trials as the strengthening of S-R associations

Law of Effect: if a response in the presence of a stimulus is followed by a ‘satisfying’ event (positive reinforcer), the association btwn stimulus & response is strengthened; if the response is followed by an ‘annoying’ event, the association is weakened

Shaping or Successive Approximations

Variability and flexibility in innate behaviors allows for the formation of completely new behaviors

Shaping the Pigeon Key Peck

- Biological and species-specific constraints must be considered

- "Clunk" of grain dispenser; habituate it OR make it a...

conditioned reinforcer: a previously neutral stimulus that can strengthen responses after being paired
with a primary reinforcer (food, water, sex, etc.)

- Shaping, in conjunction with Law of Effect, can use variability in subject’s behavior to create totally new behaviors

e.g., "Pigeon Pong"

B.F. SKINNER

Discrete trial procedure -vs- Free operant procedures

Discrete trial procedure
- trial begins with subject’s placement in puzzle box
- subject makes one response on each trial
- subject removed from box and replaced for next trial

Free operant procedures
- operant response (key peck, bar press, etc.) can occur at any time
- operant response can occur repeatedly as long as the subject is in the chamber

Skinner used response rate (usually responses/min.) as a DV

The "Skinner Box" (operant chamber) sacrifices realism for control

From Lab to "Real World": General Principles of Behavior

1) can a characteristic of operant behavior be demonstrated across species (even in an artificial, lab setting)?

2) are there similarities btwn behaviors produced in the lab and in natural situations?

3) can the operant principles be used to reliably modify behavior? (e.g., animal training, human behavior therapy)

generalized reinforcers: conditioned reinforcers that are associated with a large # of different primary reinforcers

(e.g., money: exchanged for food, clothing, interesting stimuli)

REINFORCEMENT SCHEDULES

Do different reinforcement schedules affect behavioral responses?

Reinforcement schedule: a rule stating under what circumstances a reinforcer will be delivered

Continuous reinforcement (CRF): every response = reinforcement

The Basic Schedules

Types: ratios (response), intervals (time), fixed or variable

Fixed Ratio (FR) - reinforcer is delivered after every n responses;

n is the size of the ratio

e.g., FR2 = every other response reinforced; FR50 = every 50th

- once established, a typical pattern emerges with constant, rapid responses interrupted by a pause (postreinforcement pause)

Variable Ratio (VR) - reinforcer is delivered on average once for every n responses; exact # of responses required at a given

moment fluctuates in the short term

VR15 - responses 1, 5, 30, 25, 30, 4, 10 (avg=15)

- high, rapid & steady responses, short postreinforcement pauses

Why?...

- after each reinforcement, there is a possibility only a few more responses will bring more reinforcement

e.g., slot machines, video games, haunted houses

- chance & surprise can cause anxiety OR be fun (or addicting...)

Fixed Interval (FI) - the first response after a specified time interval has elapsed is reinforced

FI 30 (V = key peck response, + = food reinforcer) - Notice that Subject #1 is making more repsonses, but gets exactly the same reinforcment as Subject #2

Subj#1 ...V +...V ...V V .....V ....V V ........V +..V V ....V ..........V .....V +......

Subj#2 ...V +...V ..........................V +...V .....................V+....................

- FI responding starts slowly after reinforcement, becomes more rapid as interval approaches; response followed by postreinforcement pause

e.g., waiting for a bus, studying for a class

- A class with daily quizzes = steady rates of studying (like CRF);

exams every 3 weeks = bursts of studying near exam (like FI)

Variable Interval (VI): similar to FI, but time interval required to pass before reinforcement varies unpredictably btwn reinforcements

VI15 (V = key peck response, + = food reinforcer) - Note that Subject#1 and #2 get the same amount of food (reinforcement), but Subject #2 is working (pecking) much more

Subj#1 ...V +...V ...V ...V ....V +....V ..V ..V +.V .....V .. ....V. .V .....V +......

- VI schedules produce steady, moderate response rates

Why lower response rates on VI compared to VR?

Subj#2 ...V +..V V .V .V ..V ....V +..V ..V . V V +..V ..V V V V ..V ..V V V ....V +......

e.g., email; checking graduate student voice mail system

- response: pushing all the necessary buttons, reinforcement is message from one of my students

- similar to VI schedules:

1) unpredictable

2) only one response required if the interval has passed to get reinforcement

3) if the necessary, unpredictable interval has not passed, no amount of responses will lead to reinforcement

EXTINCTION and REINFORCEMENT SCHEDULES

resistance to extinction varies with the reinforcement schedule

What is the most rapidly extinguished?

CRF...the partial reinforcement effect violates the concept of frequency in conditioning (partial vs constant reinforcement)

Why would responses only sometimes reinforced be harder to extinguish?

e.g., Broken vending machines (CRF) gather no coins;

Slot machines which don’t pay off (VR) still make money