Positive Reinforcement Frontiers with Zoo Animals

For this post, I take you to a recent entry on one of the blogs in my Links page:  Mary Hunter's at stalecheerios.com.   Mary covers positive reinforcement training, and she helped advise an artist on the development of a poster featuring "twelve great examples of wild animals who have been trained to do extraordinary tasks without any use of force or punishment."  The first one certainly grabbed my attention:  training a whale to pee in a cup.  Way cool!  Some feats are covered in my own chapter on animal applications, such as birds that spy and animals that willingly weigh themselves on scales, or allow blood samples to be taken.  Here's the link.  It's a great poster!  (Note:  I have no connection with the artist.)  Don't miss the fish that plays soccer.

Risky Choices

Risky, that is, in the sense of passing up a sure thing in favor of a much larger reward--maybe.  In The Science of Consequences, I note that people and animals alike often prefer variable schedules of reinforcement over fixed ones even when they don't pay off as well.  What about when it’s the amount of the reward that's unpredictable?  Do we still take a chance on the riskier variable choice?

(Note:  I regularly post descriptions of new or classic research that are a bit more technical than my regular posts.  This is a "research post" that I hope is of general interest.)  

In a new study, Carla Lagorio and Tim Hackenberg report that past research results have been inconsistent.  They also note that this type of research has been viewed as a way to approach the study of gambling.  Obviously, not every gamble pays off:  Success is variable and (often) unpredictable in amount.  Problem gambling is a large and expensive problem worldwide, and laboratory analogs that help us understand some of its contributing factors are valuable. 

The researchers looked at pigeons working for "token" symbols:  lights on a panel.  Each light could be exchanged for a short period of access to food, but only during signaled "exchange" periods a short while after each choice.  This approach helped make this research setup more analogous to real-life human gambling:  People also frequently get tokens like chips that can be exchanged only later for money (which itself is an exchangeable token, of course!).

In this particular study, seven pigeons pecked to initiate a trial, then pecked again to make their choice.  The fixed choice payoff stayed constant at either 2, 4, 6, or 8 tokens.  The variable payoff could be anything from 0 to 12 tokens, offered on one of a number of different distributions ("rectangular" or "exponential" for those of you who are mathematically inclined).  A bird might earn 9 tokens after a "variable" choice, then only 2 after another "variable" choice.

This was a thorough "parametric study" in which different combinations of fixed vs variable schedules were run.  Once an individual's choice pattern was stable for one combo, that bird would be switched to a different one, and so on.  The outcome?--a strong preference for variable rewards rather than fixed ones, similar to the results for variable vs fixed schedules of reinforcement.  And again, that was frequently the case even when the birds lost by their risky choices:  that is, even when switching to the fixed choice would have provided substantially more reward over time. In a way, they're like problem gamblers in this respect.

Just as interesting:  When the token signals were removed and the birds simply worked for direct access to food, these skewed results were less likely; the birds made more rational choices instead.  What's going on?  Stay tuned.

One final finding I have to mention:  Some of my own past research examined the influence of one particular reinforced schedule choice on the next choice, in a process called "sequential analysis."  If you’ve just TV-surfed to a baseball game and happened to catch a home run or a spectacular double play, are you more likely to stay with the game than if you tuned into a batter engaging in boring warm-up swings?  These moment-to-moment influences on our choices make intuitive sense in our daily lives.  That was also the case here:  Pigeons were significantly more likely to go with the "variable" option if they had just enjoyed a handsome payoff for choosing "variable."  If they’d received no tokens for the "variable" choice, they were very likely to switch to fixed for their next choice.  How human of them . . .

An Odd Consequence Value Effect (Or, Why Do I Mop the Kitchen Floor When I Get Good News?)

There's no doubt about it:  I can go for weeks without mopping the kitchen floor.  (Sweeping is so much easier.)  It weighs on my mind.  A little.

What nudges its consequence value up enough to tip it over the threshold of it-can-wait to let's-get-it-over-with?  It's often getting good news.  Suddenly, chores become easy as their aversiveness mysteriously declines. 

My kitchen floor

  Social psychologists verified years ago what our own
  experience tells us:  "Be happy to be good."  When is
  a charity more likely to successfully appeal to you (in
  other words, when does giving hard-earned cash become
  sufficiently rewarding)?--after you've had a horrible day
  at work, or a great one?  How does this effect fit in with the
  principles of consequences?  Well, we already know
  that the relationship between emotions and consequences
  is complex.  This is another way in which they interact.


I'm sure our past experiences--our history of consequences--and the rules we've learned are involved. As someone particularly aware of the motivators in my life (teaching this science and then writing this book), I can see the value of consequences vary with my mood.  As always, the situation is complex:  For household chores, I've also noticed that the other extreme can work too.  That is, when very little in my life is reinforcing, again, it can be easier than usual to do chores.  After all, they provide some reward value, if only by escaping that annoying feeling that I ought to do them, and, of course, escaping the unpleasantness of seeing dust balls in the corner. I couldn't find any research about this in a quick check, but I'm sure there's some out there.  (Let me know.)

Most effective of all, when I get a visitor, wow, see me reach for that mop.  Serious consequences (both positive and negative).

Am I just weird, or does this phenomenon apply to you too?

Book Launch at the Fair!

The Miami Book Fair International, that is--one of the largest in the United States!  I'm pleased to announce that I will be launching my book there over the weekend of November 17-18 (the weekend before Thanksgiving).  When I get around to it, I'll be opening an Events webpage that will include the specifics (which I don't know yet myself).

I'm especially pleased because I spent five happy years in Miami as a research faculty member at Florida International University, working on nature-nurture relations and the mathematical modeling of behavior.  A behavioral enrichment project from those years is described in my book.