Emergence of Instrumental & Operant Learning Theory
Summary
Strategies based on classical conditioning have also been used in the treatment of depression, somatoform disorders, dissociative disorders, substance abuse, sexual difficulties, medical problems, and a variety of other disorders. In general, these approaches represent some of the most effective of the therapeutic interventions. As is the case with other types of behavioral strategies, they rest on a solid foundation of basic empirical work, much of it with nonhuman animals, and on the creative adaptation of those basic principles to human populations.
Emergence of Instrumental & Operant Learning Theory
As a graduate student at Columbia University, Edward Thorndike began a series of experiments that set a new course in the study of processes underlying behavior change and learning. He placed a cat in an enclosed chamber and attached a vertical pole in the center of the compartment to a rope that passed over several pulleys. When the cat bumped against the pole, the pole would tilt, causing the rope to open the door. The cat could then leave the compartment and drink milk from a nearby bowl outside the cage. At first, the cat seemed to move about unpredictably each time it was returned to the compartment. The time required for the cat to tilt the pole grew shorter on successive repetitions of the task, and the cat’s method for opening the door on each trial became progressively similar to the method used on the preceding trial. The trial-by-trial record of time to escape from what Thorndike called his “puzzle box” was the first instrumental learning curve published in a scientific journal. Eventually, each cat quickly approached the pole—seemingly purposively—and tilted it to one side, opening the door. Thorndike described this as an instrumental conditioning process because the pole tilting was instrumental in releasing the cat from the chamber and permitting access to a reward. Thorndike’s method differed from Pavlov’s classical conditioning because no specific response was elicited by a conditioned stimulus. The form of each cat’s behavior that tilted the pole was idiosyncratic and variable. There was nothing fixed about the behavior, as was typical of classically conditioned behavior. Thorndike’s Law of Effect described the necessary and sufficient conditions for instrumental learning to occur.
Skinner & Operant Behavior
Whereas Thorndike studied the process of behavior change, three decades later, B. F. Skinner, a graduate student at Harvard University, was interested in discovering a method for identifying the functional components of sequences of behavior. Skinner was drawn to the writings of the physiologists Charles Sherrington and Ernst Magnus. Skinner was particularly taken with Sherrington’s notion of the reflex arc. Skinner believed that psychologists had gotten seriously off on the wrong track by focusing on unobservable phenomenological events, which no amount of experimentation could verify, rather than following the example of physiology in studying observable events. Skinner wondered whether Thorndike’s Law of Effect might explain how a single component could be isolated from the continuously free flowing activities of an organism, so that the component could be studied scientifically, much as Sherrington had done. Using a method very similar to Thorndike’s, Skinner placed a rat in an enclosed chamber, and each time the rat depressed a telegraph key protruding through the wall of the chamber, a pellet of food dropped into a receptacle near the rat. The lever-pressing methods each rat used varied: most pressed with their paws, some pushed with their muzzles, others held the telegraph key between their teeth and pulled down. All methods produced the same result—delivery of a pellet of food that the hungry rat seized and ate. Skinner said that the rat “operated” on its environment to produce reinforcing consequences, and the type of behavior was correspondingly called operant behavior.
In operant behavior, typically no stimulus was presented before an operant response that “caused” the behavior to occur (ie, there was no conditioned stimulus). When Skinner analyzed the sequence of the rat’s activities in an operant chamber, he found that after many repetitions of the rat approaching the lever, depressing it, and hearing the device click, which had been followed by food pellet presentation, the click sound produced by the lever press began to be rewarding without food pellet presentation. If a light were illuminated above the lever (indicating periods when food would be available), alternating with periods when the light was off (indicating lever presses would not produce food), soon the rat pressed nearly exclusively when the light was illuminated. The rat’s behavior continued to be variable, changing from moment to moment even when the light was illuminated, unlike a classically conditioned reflex. Skinner called the food pellet a reinforcer and the light that signaled that operant responding would lead to reinforcer presentation, a discriminative stimulus. Skinner spelled out in surprisingly accurate detail laws of operant conditioning that have stood the test of time. Immediacy, magnitude, and intermittence of reinforcement affected the pattern of behavior maintained and also determined the persistence of behavior in the absence of reinforcement.
Skinner also observed that a stimulus repeatedly paired with food presentation (eg, the “click” sound of the food pellet dispenser) came to serve as a reinforcer in its own right and would maintain considerable amounts of behavior over extended periods of time in the absence of primary reinforcement. Such previously neutral stimuli that took on reinforcing properties because of their pairing with primary reinforcers were called conditioned reinforcers or secondary reinforcers. Skinner recognized that in most developed parts of the world, relatively limited aspects of human conduct seem to be directed toward seeking food or shelter. Instead, most human conduct seems to be governed by parent or teacher approval, threat of loss of affection, or symbols of recognition from employers or peers (eg, paychecks, awards). Skinner reasoned that these reinforcers had developed their reinforcing properties (usually very early in an individual’s life) from their repeated pairing with primary reinforcers. In short, they were powerful conditioned reinforcers. This observation led later educators, drug abuse counselors, psychologists, and psychiatrists working in various applied settings to develop treatment methods based on conditioned reinforcers such as social approval or concrete objects paired with other reinforcers (eg, check marks, stars, tokens, money).
