Ken Ramirez on the Benefits of Animal Training

I'm still catching up after my East Coast book tour, so I thought I'd link you to another great post by animal expert Mary Hunter, who blogs at www.stalecheerios.com.  (I've linked to her posts before.)

In this one, Mary describes a free one-hour youtube video from the Chicago Humanities Festival by biologist Ken Ramirez, a highly-respected expert in positive reinforcement-based training.  I'm fortunate to have met Ken several times now at conferences.  In The Science of Consequences, I have this to say about him:  "Talking about the move to positives, Ken Ramirez, vice president of Chicago’s Shedd Aquarium, won’t even let his trainers say 'No.' If they use reprimands, he says, they will eventually overdo it."

This one-hour video covers how training helps animals--including some benefits you probably haven't thought of--and what it's told us about animal capabilities and intelligence.  As Mary comments in her review, one of many important points Ken makes is that training is actually essential for the quality of life of most of our pets, as well as for many zoo animals.  And it helps in conservation as well (as I mention in my book).

Here's the link to Mary's blog post.  And here's the link to Ken's youtube video.  Enjoy!

Positive Reinforcement for Animals

I just got back from keynoting a unique conference on a unique venue.  (Yes, "on.")  ClickerExpo is all about positive reinforcement-based animal training, for animals of all sorts, but with a focus on dogs.  It was founded by Karen Pryor, whose wonderful books I refer to--let's see--a dozen times in The Science of Consequences.  This year, the West Coast edition of Expo took place on the Queen Mary, the historic ocean liner now permanently docked at Long Beach.  Zowee!  Clicks, by the way, are "marker" sounds, part of the communication system for learning a new behavior.

In honor of this event, I want to highlight one of the links on my website:  http://www.clickertraining.com/library.   This library offers lots of useful information about positive reinforcement for animals, and in an entertaining style.  My "Links" page also connects to several other resources for animal training and enrichment, and there are dozens more excellent ones out there.  Animal lovers, enjoy!

Bird Migration, Ultralights, Magnetic Fields, and Consequences

Bird migration is on my mind, and not just because of the geese I’ve been hearing overhead.  I’ve been fine-tuning a book-related talk for the Central Valley Birding Symposium here in California later this month:  "No Birdbrains Here: The Latest on Bird Learning, Instinct, and Intelligence."  Should be fun.

This birding festival is scheduled at a time when many thousands of birds descend upon the Central Valley for the winter, particularly ducks and geese.  An additional winter highlight is the spectacular, nearly five-foot-tall "greater" subspecies of the sandhill crane (along with the smaller "lesser" subspecies).  The crane in the photo is an adult, as you can tell by its red crown.

While learning appears to play at least a small role in the migration of many bird species, it clearly plays a major role for cranes and most waterfowl.  The cranes, for example, migrate in their family groups.  Without adults to guide them south, young birds born that year are clueless.

Attempts to re-establish the endangered whooping crane in some of its former eastern US range have relied on ultralight aircraft leading the way (see this link).  Encouragingly, the first group of young birds managed to return to their breeding grounds on their own in the spring.  Researchers have found, however, that the cranes continue learning, fine-tuning their migration routes over many years (and this long and detailed blog post by GrrlScientist provides a nice summary).

As always, the interactions between "instincts" and learning from consequences are fascinating--and there’s still much we don’t know when it comes to migration.  What exactly is the role of the earth's magnetic field, for example?  Some migrant birds are clearly influenced by it, but decades of research have shown that the story is complex and still mysterious.  One of my colleagues, Michael Davison (University of Auckland, New Zealand), coauthored a study on this topic in the prestigious journal Nature.  Using a choice setup--with rewarding consequences, of course--the researchers demonstrated that homing pigeons could detect changes in the intensity and inclination of a magnetic field.  Pretty impressive.

For really impressive, though, visit your local park or wildlife refuge and marvel at the feats of long-distance migrants, frequently returning over thousands of  miles to the same breeding and wintering areas year after year.  If you can't get away, I recommend Scott Weidensaul's Living on the Wind, a great read about bird migration in the Americas. 

Agility Training--with Mice?

I haven't posted on positive animal training in a while, so here's a link to a youtube video from a few years ago that stuck in my memory:  "Trickmousing."  It's a bit less than 3 minutes long, and "cute" is a word that inevitably will come to mind.

Most people have heard of dog agility by now; there are local and national competitions in which the dogs eagerly leap hurdles, dash through tunnels, and handle a series of other obstacles as directed by their humans, who run alongside.  On the Links page of my website is a video of a lamb surmounting the same challenges.  Why not?  Even so, how cool that mice can do this sort of thing too--and more.  On the video, the mice leap little hurdles, retrieve marbles, dunk a basketball, and even skateboard.  (Hey, remember that dogs can learn to drive a car, no foolin'.)

The accomplishments of these mice show what positive reinforcement and clicker training can do.  As many of my readers know, the clicker sound is immediate and obvious (you can hear it in the video), and serves as a reinforcer in itself as well as a "bridge" to a more powerful reward like the food that the mice get. Specific information about these mice and their training is available through the links at the youtube site.  (For general information, see the links on my website or the references in my book.)  I appreciated that the trainer recognized how social these animals are and took pains to avoid isolating them.  That's why more than one mouse is frequently in view, although only one is showing off its stuff at any given time.

Enjoy!

Who's Got Rhythm?

Animals do!  People used to think that our own species might be unique in following a musical beat.  Wrong.

In my book, I cited a 20th century anecdote from author and animal lover Gerald Durrell:  "A hand-raised pigeon loved music and would snuggle close to the speaker of an old-fashioned record player. What’s more, the bird performed distinctive dances to marches and waltzes."  That would seem to indicate some sense of rhythm, along with possible intrinsic reinforcement value for moving in time to a beat.  I'm sure there must be many such anecdotes.  (Do share them if you know of any!)  In the end note accompanying Durrell's story, I mentioned a sulphur-crested cockatoo that appeared to be following the rhythm of rock music, made famous on a youtube video that went viral.  Researchers in the journal Current Biology concluded that indeed the bird was.

The next step:  Check it out with a mammalian species--and one that's not especially good at vocal flexibility in response to consequences.  In a recent issue of the Journal of Comparative Psychology, Peter Cook and colleagues showed that Ronan the female sea lion could learn to bob her head to different rhythms just fine, even from complex music.  Thirty weekend training sessions was all it took.  Standard positive reinforcement methods were used, and if you want to enjoy watching Ronan in action, check out "Beat keeping in a CA Sea Lion" on youtube.
 From lab studies, we've known for many years that animals can learn to time events quite accurately.  How cool that scientists are now building on this research to look at something that appears to come naturally to many people.  I would say everyone, but I've danced with some who could have used a few lessons!

Most people, like Durrell's pigeon, find that moving in rhythm with the beat of music is rewarding.  Where does that come from?  Would Ronan eventually enjoy "dancing" and do it spontaneously, like Durrell's bird?  Stay tuned for further research.

Discriminating Pigeons

We know pigeons can categorize classical music, apparently in similar ways that we do.  We also know they can distinguish a Monet painting from a Picasso.  These and many other pigeon feats are in my book.  But who would possibly have thought that pigeons could learn to categorize children's "good" and "bad" art? 

One of the neat things about being on a book tour is learning from members of my audiences--stories from their lives or their reading.  In Minnesota, one audience member told me about this recent study in Animal Cognition by the same researcher (Shigeru Watanabe) who did the Monet-Picasso study.  The "good" and "bad" art was judged by people, of course, 10 adults and an art teacher--but the pigeons readily learned similar standards.  Then they generalized to novel examples of children's art.

What is it that makes for "bad" art?  It was messier and harder to identify objects, for one.  But defining the basis used for categorizing wasn't easy even for the art teacher.

Using clever tests, Watanabe showed that the birds were using color and pattern as a basis for their choices.  We do too, of course.  Does that mean that the birds would enjoy viewing the "good" art?  It's not an outlandish question.  We know that a species of sparrow prefers melodic music over dissonant sounds, after all (also courtesy of Dr. Watanabe).  And pigeons have great vision, unlike our closer companions, dogs.

Kind of makes us look at these common street birds a bit differently!

Dance of the Balloons

Balloons created and clutched by little dance flies, that is . . . as described in my book.  Over time, different species in this family of insects developed variations on the basic dance that forms their courtship ritual--"as close to behavior fossils as we're going to get."

Dance flies with "balloon."  © Ken R. Schneider

I've never seen this spectacle, but now my brother has!  He was even able to get this photo.  You can't really see it, but attached to the balloon is an even tinier edible insect for the female to nosh on during mating.

For Presidents' Day (US): Sustainability, Consequences, and the "Founding Fish"

As a youngster, I was concerned about where all the garbage and sewage went.  I lived in Chicago, so it was evident there was a lot of it.  Wouldn’t we be drowning ourselves in trash?  OK, in some places that’s a reality . . .

This was clearly not something most people worried much about then (before the environmental movement of the 1970s), a worry in itself.  Perhaps that realization helped contribute to my lifelong concern about "sustainability":  long-term viability for people and planet. I discuss consequences and sustainability in the last chapter of my book.

The problem, of course, is that desirable long-term consequences like sustainability often mean sacrifices now.  Yet, both as individuals and societally, we do sometimes make rational long-term choices, foregoing more immediate rewards. 

In honor of Presidents' Day in the US, I thought it might be appropriate to provide an example from our early history.  In his book The Founding Fish, John McPhee comments on one little-known example of how consequences eventually brought about conservation--but it took some disasters along the way.  The American Shad was a major source of protein for Native Americans and European settlers alike.  While Native Americans had set up some fish-catching weirs, their harvest was sustainable.  Europeans set up so many of them that downstream weirs began taking too many of the returning fish, so that upstream weirs caught very little.  As a consequence, in the early 1700s, the colonial Pennsylvania legislature passed laws forbidding these obstructions entirely, and requiring the removal of existing ones.  These laws were largely ignored, leading to a full-scale battle in 1738 in which lives were lost.  These fish were that important.  Later, additional sustainability laws were passed.  For example, only a few years before the Declaration of Independence, fishing in the Schuylkill River was banned from Saturday after sunset to sunrise on Monday (1771).  

 It was a start.

Link to Interview by Dr. Sophia Yin

I referred to Dr. Sophia Yin's blog back in August in my post on scrub jays.  I got to meet Dr. Yin, a well-known veterinarian and animal behavior expert, when I gave a talk at the downtown Davis, CA bookstore last weekend.  We chatted about everything from my take on B. F. Skinner's views about nature-and-nurture to intrinsic reinforcement in kids and in dogs--great fun!  She's doing a series of blog posts based on interviews with me, and here's a link to the first post in the series:  click here

Fish Stories

In my book, I describe fish that make choices like we do, learn to follow gestures, enjoy roller coaster-like sensations, and handle delays to rewards (and more).

Naturally, aquarium fish can learn to do things like swim through a hoop, and you can view cute videos on both of the animal training/enrichment websites in my "Links" page:  Karen Pryor's www.clickertraining.com/library (also check out the blogs) and Mary Hunter's www.stalecheerios.com/blog. On youtube, you might want to take a look at "Phish’s Target Training" and "Limbo Perch clicker training."  Yup, perch can do the limbo.

What about fish in the wild?  One way that wild fish got to show off provided a great example of signal learning in a lab many years ago.  What actually happens in their natural habitat may involve much of the whole nature-nurture system. 

For generations, people knew that salmon fry hatched in freshwater, swam to the sea, grew large, and then returned to freshwater streams to spawn.  But which streams?  Early experiments in which the young salmon were distinctively marked or tagged showed that they returned to the same streams where they started.  Yet there they were in adulthood, hundreds or even thousands of miles from their homes.  How could they find their way back?  Many suggestions were offered, but I don’t believe anyone guessed part of the answer:  When they get close, they smell their way to their birth stream.

Step back in time.  One 1950s research project rewarded salmon for learning to tell the difference between 14 kinds of aquatic plants by smell.  They did just fine.  Given this clue, follow-ups confirmed that salmon could smell the difference between water from different streams (and had preferences).  Why do they wait for years to return?  As always, "it's a system," and there clearly are unlearned, "instinctive"-type components in this case--the sort of interaction described in my book.

Now that so many salmon runs have been driven to extinction or near-extinction, it’s especially important to try to understand how salmon homing works.  In a recent research article in Transactions of the American Fisheries Society, Andrew Dittman and his colleagues noted that chinook salmon reared in hatcheries sometimes homed as expected when the time came, but all too frequently returned to spawn miles away from where they had carefully been released--and that's a problem for restoration efforts.  Does this modifiability reflect any learning from consequences?  We don’t know for sure, but as Dittman mentioned, the full story probably includes many elements of the nature-nurture system.  That's just what we’ve come to expect.  

Looking the Other Way: Consequences and the "Ostrich Effect"

Sam Kean wrote recently about his unanticipated response when he had his genome sequenced for a book he was writing.  His grandfather had died of Parkinson’s disease, and when push came to shove, Sam realized that he didn’t want to know if he had inherited a genetic predisposition for it.  "I blacked that information out."  (See this link for the article in The Scientist magazine.)

Sam’s not alone in not wanting to risk bad news even when knowing it might be helpful in preparing for it.  As Sam noted, Nobel laureate James Watson had a similar reaction when he had his genome read.  Less serious, everyday examples aren’t hard to come by either.  I know I sometimes turn off the radio when bad news is broadcast.  The consequences of staying tuned in are just too discouraging.  In my book, I refer to this as the "ostrich effect." 

It may not be all that surprising that animals respond similarly to the prospect of bad news.  Pigeons, for example, learned to peck for a signal indicating the schedule of reinforcement that they could currently work on.  This schedule switched back and forth unpredictably, so the best way to optimize their efforts was to peck periodically for the signal during lean times when few or no rewards could be earned.  Eventually, a more encouraging signal would tell them good times had returned.  Unfortunately, the "bad news" signal stopped most of the birds in their tracks (and thus acted as a punisher)--an effect that's been demonstrated in a number of species now, even fish.  And as we’ve seen, for all our sophisticated intellects and fancy delay-bridging rules and rationalizations, people frequently behave the same way, both in the lab and in real life.  That's the case even when the bad news could be really important.  One of the examples in my book is the large Centers for Disease Control estimate of the number of HIV-positive people in the United States who have avoided taking the simple, inexpensive test to check their status.  Some of them may unknowingly be spreading the disease.

Sam Kean eventually worked up the courage to confront his fears about his possible genetic predisposition to Parkinson’s, and get the facts.  Just as ostriches don’t really bury their heads in the sand, we can learn to face potentially bad news.  What we know about the science of consequences can help us do so.  It still isn’t easy, though.  Anyone want to share examples of how they coped with this all-too-common challenge?

Positive Reinforcement Frontiers with Zoo Animals

For this post, I take you to a recent entry on one of the blogs in my Links page:  Mary Hunter's at stalecheerios.com.   Mary covers positive reinforcement training, and she helped advise an artist on the development of a poster featuring "twelve great examples of wild animals who have been trained to do extraordinary tasks without any use of force or punishment."  The first one certainly grabbed my attention:  training a whale to pee in a cup.  Way cool!  Some feats are covered in my own chapter on animal applications, such as birds that spy and animals that willingly weigh themselves on scales, or allow blood samples to be taken.  Here's the link.  It's a great poster!  (Note:  I have no connection with the artist.)  Don't miss the fish that plays soccer.

Risky Choices

Risky, that is, in the sense of passing up a sure thing in favor of a much larger reward--maybe.  In The Science of Consequences, I note that people and animals alike often prefer variable schedules of reinforcement over fixed ones even when they don't pay off as well.  What about when it’s the amount of the reward that's unpredictable?  Do we still take a chance on the riskier variable choice?

(Note:  I regularly post descriptions of new or classic research that are a bit more technical than my regular posts.  This is a "research post" that I hope is of general interest.)  

In a new study, Carla Lagorio and Tim Hackenberg report that past research results have been inconsistent.  They also note that this type of research has been viewed as a way to approach the study of gambling.  Obviously, not every gamble pays off:  Success is variable and (often) unpredictable in amount.  Problem gambling is a large and expensive problem worldwide, and laboratory analogs that help us understand some of its contributing factors are valuable. 

The researchers looked at pigeons working for "token" symbols:  lights on a panel.  Each light could be exchanged for a short period of access to food, but only during signaled "exchange" periods a short while after each choice.  This approach helped make this research setup more analogous to real-life human gambling:  People also frequently get tokens like chips that can be exchanged only later for money (which itself is an exchangeable token, of course!).

In this particular study, seven pigeons pecked to initiate a trial, then pecked again to make their choice.  The fixed choice payoff stayed constant at either 2, 4, 6, or 8 tokens.  The variable payoff could be anything from 0 to 12 tokens, offered on one of a number of different distributions ("rectangular" or "exponential" for those of you who are mathematically inclined).  A bird might earn 9 tokens after a "variable" choice, then only 2 after another "variable" choice.

This was a thorough "parametric study" in which different combinations of fixed vs variable schedules were run.  Once an individual's choice pattern was stable for one combo, that bird would be switched to a different one, and so on.  The outcome?--a strong preference for variable rewards rather than fixed ones, similar to the results for variable vs fixed schedules of reinforcement.  And again, that was frequently the case even when the birds lost by their risky choices:  that is, even when switching to the fixed choice would have provided substantially more reward over time. In a way, they're like problem gamblers in this respect.

Just as interesting:  When the token signals were removed and the birds simply worked for direct access to food, these skewed results were less likely; the birds made more rational choices instead.  What's going on?  Stay tuned.

One final finding I have to mention:  Some of my own past research examined the influence of one particular reinforced schedule choice on the next choice, in a process called "sequential analysis."  If you’ve just TV-surfed to a baseball game and happened to catch a home run or a spectacular double play, are you more likely to stay with the game than if you tuned into a batter engaging in boring warm-up swings?  These moment-to-moment influences on our choices make intuitive sense in our daily lives.  That was also the case here:  Pigeons were significantly more likely to go with the "variable" option if they had just enjoyed a handsome payoff for choosing "variable."  If they’d received no tokens for the "variable" choice, they were very likely to switch to fixed for their next choice.  How human of them . . .

Jays in My Backyard

Western Scrub-Jay (©Ken R. Schneider)

Earlier this summer, I had the pleasure of watching two fledgling Western Scrub-Jays learn from consequences.  They were perched on my brush pile underneath my redwood when I spotted them, and clearly had left the nest not long before.  They were waiting impatiently for their parents, who stopped by regularly with food that got eagerly grabbed.  Meanwhile, they explored their surroundings.  One found a small yellow leaf, trapped it under one foot and pecked at it, as if checking whether it was good to eat (no).  It was discarded.  The other youngster poked around a bit too, also without success.  Still, it was a start on the path to independence.

One week later, I watched what was probably one of these youngsters in my yard again, this time foraging more actively, picking up dead grass stalks systematically.  Once again, its efforts did not appear to be crowned with success, but it was trying.

The adults are omnivorous, kind of like feathered bears:  They’ll go after anything that’s edible.  And they readily learn their way around minor obstacles.  Someone in my neighborhood feeds peanuts in the shell, and the birds come to particular flat fencepost stumps in my backyard to hold them down and peck them open with their all-purpose beaks.  I'm sure the youngsters will be up to that--someday.

Update:   I happened to catch a cool scrub jay post on animal expert Sophia Yin's blog (link here).  As she reports, two scientists published a 1999 Nature article on how jays learn from consequences when they store food like nuts in hidden "caches."  The nuts last a long time, but cached foods like waxworms don't.  It turns out that "the birds had to learn that food such as waxworms degrade after long intervals. A separate set of scrub jays whose rotten waxworms were secretly replaced with fresh ones after long intervals, never learned that waxworms go bad."  And they behaved accordingly, very differently from the jays that had learned otherwise.  The flexibility provided by learning--even if it's researcher-assisted and "unnatural"--helps the birds survive.

Fun with Hummingbirds

Male Anna's (©Ken R. Schneider)

One of the delights of living in California is hosting hummingbirds year-round--including one of the most spectacular North American species, the Anna’s Hummingbird.  My brother took this picture of an adult male.

Last year I rescued a young, probably female Anna’s that hit a back window (despite the warning stickers).  She didn’t seem to  be injured, just stunned.  It was cold, so to help her recover--and keep her away from predators--I held her in my hand in the sun, while she stared at me and one wing quivered.  How could I reassure her?  I decided to take the opportunity to play a game that’s described in my book.  It entails rewarding an animal’s eyeblinks by giving unusually long eyeblinks in immediate response--a sort of communication, if you will.  My hummer didn’t blink much at first, but I took advantage of every blink.  Then she caught on, and within 3 minutes, every time I opened my eyes after my own blinks, she immediately blinked!  Great fun, and best of all, after 5 min, she lifted off, apparently back to normal.  What a magical experience. 

Juvenile Anna's (public domain)

Having a hummingbird feeder lets me enjoy behavior-watching as well as beauty.  I get to laugh at hummingbirds learning to find the nectar.  (Sometimes they try in what seems like every possible place before they succeed.)  When I take the feeder in for cleaning and refilling, I see hummers fly to where they’ve learned it should be, casting about fruitlessly before buzzing off.  Talk about well-learned habits!  And I breathlessly witness frequent dominance battles over access to this prime resource.  In most U.S. hummingbirds, females are larger than males, and my own feeder tends to be dominated by one of them.  I still get to enjoy the beautiful magenta iridescence on the throat "gorget," though, because adult female Anna’s have them.  Maybe the hummer I rescued has developed one by now, attracted a mate, and raised little ones of her own.  

Genes, Behavioral Economics, and Dinner

As many are well aware, obesity in the United States has reached "epidemic" proportions (Centers for Disease Control), with about two-thirds of us overweight.  Helpful consequence-based approaches are discussed in my book.  But wouldn’t it be easier just to pop a pill that would reduce appetite? 

The drug rimonabant did just that, first for animals, and then in human clinical trials.  It was on the market as a prescription drug only briefly, though:  As you might have guessed, it was too good to be true, and came with problematic side effects.  But suppose we could better understand how it works?  And how it interacts with some of the relevant genes? 

A particular genetic strain of rats is primed for obesity, and has been used for decades as a model for related human problems, such as diabetes and high blood pressure.  These rats still don’t necessarily become obese, they’re just more likely to.  Erin Rasmussen and colleagues, for example, studied these rats working for sucrose pellets--table sugar.  They showed that when the schedule of reinforcement became lean enough (that is, required a lot more work per sucrose pellet), these rats worked no harder and earned no more rewards than normal rats.  As always, nature-and-nurture systems offer lots of interactions and flexibility.

It’s probably obvious that increasing the amount of work to get a consequence is similar to increasing its price--making the economic demand likely to decrease.  If it’s something we can’t live without, though, like water in a desert, we’ll do whatever we need to:  Demand is "inelastic" in that case.  My own demand level for premium breakfast cereals is seriously elastic:  I don’t buy one unless it’s on sale!  It's simply not a powerful enough reward to overcome a high price.  Mathematical relations let scientists compare demand levels and degree of elasticity in precise ways.

In a follow-up article that appeared this year in the journal Physiology and Behavior, Rasmussen and colleagues checked out the effects of rimonabant on food reward value and elasticity in the normal and "obese" variants of this rat strain.  Sucrose was again the reward, and these nuggets of pure sugar are usually as desirable to rats as they are to us.  The schedule of reinforcement was a "fixed ratio," which means it was work-based:  just 1 lever press per pellet at first, then 15, 30, 50, 90, 150, and 300 (whew).  At the lower prices (lower ratio values), the overweight rats worked harder and earned significantly more sweet rewards than the normal ones, as we might expect.  But, when the going got tough, as in the earlier study, all the rats behaved similarly:  The obese rats were no longer working more and eating more.  Now add the appetite-reducing drug, with its known effects on neurophysiology.  Not surprisingly, all the rats stopped working as hard as they had before:  Sucrose wasn’t as rewarding.  Mathematically, elasticity increased for both groups.

What a great example of interdisciplinary work, including behavioral economics, neuroscience, schedules of reinforcement, reward value, genetics, and more.  Research like this helps us understand how all these factors interact, and may lead to practical applications in preventing and treating obesity. 

Meanwhile, check out the effective consequence-based methods that already exist if you want to work on your weight.  David Freedman's recent articles offer a good, accessible introduction (for example, here).