Reinforcement history Archives - Teaching My Old Dog New Tricks

Category: Reinforcement history

Clara’s Novice Masters Trick Dog Title: More Tricks, More Lessons Learned

Posted on November 5, 2021November 5, 2021 by Eileen Anderson

It’s an oxymoron, as my friend Carol pointed out, but Clara and I earned her Novice Masters Trick Dog title recently. I haven’t had time to post about it until now.

Things We Learned

As usual, the balance between stuff she already knew (get in a crate), tricks we could adapt quickly from stuff she already knew (get in a cardboard box), and completely new stuff (roll out the carpet) was…interesting.

Practicing the cookie balancing trick by starting with kibble on the top of Clara’s head

Here are some highlights.

Stand: We finally have a nice stand on cue after our long running debacle. Her stance is a bit crouchy (no idea where that came from), but it’s fixable. What I’ve got is a moderately calm dog with four on the floor and I’m happy! I’m using a hand signal and have no plans for a verbal one. No, no, no way!

Find hidden treats: This should have gone quickly because of all the food searching games we play, but we got haunted with a bit of previous reinforcement history. Clara is so experienced with searching for food that she can find a few pieces of kibble in a very large backyard. For the trick, all she had to do was find three hides of stinky lamb lung in a small area in my bedroom. However, in that room she is patterned to head for the bed and her snuffle mat area where she eats breakfast and often searches. She spent some time over there, off-camera, rather than finding all the easy hides I set.

Rolling out the carpet: I was proud that we finally won our wrestling match with the matching law. In the video, you can see her pause, then flip over the last little corner of the rug. To me, that was better than rolling the whole thing out in one swoop. I could tell she was getting the criteria. It made me very happy.

Platform jump: The directions for this trick say it must be clear that the dog is jumping from platform to platform, not just stepping. We nailed that one!

Tricks We Re-Recorded

There were several tricks I had to tweak and record again when I saw we might not have met some nuance of the requirements. They included the muffin tin game (what could we do wrong with that?), the bang game (whack a board), and eye contact, the last because I had the wrong camera angle.

Finally, we had to re-record the cookie on the nose game. This is one we put some work into, although she already had a good foundation with lots of different Zen/leave-it practice. I taught her to hold still twice as long as the description required, but after we had recorded it, I re-read the rules. It said that the dog would hold the cookie on her nose until the handler reached and took it (or the dog could flip the cookie and eat it for an intermediate trick). We were doing neither. When the time was up, I reached toward her and she dumped it off her head! So I taught her to keep her head still even as I reached for the cookie, and we re-recorded in case the former version was a fail.

I’m pretty proud of that trick, though. My goal was that it would be fun for her. I’ve seen so many miserable-looking dogs with treats piled on their bodies. I was curious, so I finally searched to see how the force-based trainers do this trick, since it is a very common “show-off” trick. The trainer I found, not even one of the worst, held the dog’s chin and scolded whenever they tried to move their head, so the dog learned through physical molding and threats to keep still.

By the way, all that re-reading of the instructions I did? It’s partly because I have considerable ego involvement in this. I don’t plan on screwing up a trick because I misread the directions. But also, re-checking is a habit from my grant writing days. Check the directions, and check them again, and again, and then once more before you turn the damn thing in. Old habits can pay off!

Sloppy Training

Maybe you’ve noticed that I’m not one of those precision trainers (understatement). I am sloppy. I work to get my mechanics decent enough that I am not doing too much of a disservice to my dog, but I hardly ever go that final 20%. But man—watching these videos is an incentive to clean up my act.

Any regular viewer now knows that I keep my treats in my left pocket, because that’s where Clara looks or heads after any behavior. I mention below that I fixed that for our Peekaboo behavior (not shown), so I do know how.

I made the treat orientation worse when working on these tricks because I was often using some kibble from Clara’s breakfast. I was too lazy to take the different kibble out of my pocket so I could pocket her breakfast kibble. So for several tricks I have kibble already in my hand, which I’m waving around in front of her. If you want to see her hand and pocket orientation at its finest, check out the stand behavior about one minute into the video and the hand signals near the end. One of our best tricks is “Look at Eileen’s left hand.” There are so many ways to fix that; I really should!

In the front behavior, I do the classic novice move of taking a step backward and luring her toward me with my two hands together. I made sure this was allowed. I taught Summer a pretty front without that move, since it is disallowed in some levels of obedience. But I’ve never needed formal obedience moves with Clara. The move is included in the official demo video, so I took that shortcut. That’s a lure even without food in the hand, but upon a closer look, I was holding food in that take! I was unintentionally luring her with food.

Many of you have noticed by now that Clara has a superstitious foot lift when she sits. She’s had it forever. A paw lift can be a stress-prompted behavior, but Clara wasn’t a stressy puppy when at home with the other dogs and me, and I suspect that this longstanding addition to her sits is just something I accidentally clicked too much when she was very young. You can see it (barely because of the angle, but I remark on it) in the very first behavior in the video, getting in a box. She holds her paw up for at least three seconds, and I cleverly treat her in that position. Brava, Eileen!

These things I have accidentally trained are annoying for me to look at. It’s my job to remember that they are there because I reinforced them. If I want them to go away, I need to load more reinforcement onto the version I want, not punish or frustrate my dog. There’s a strong human tendency to use punishment to solve problems, but in this case and so many others, it’s completely unfair and uncalled for.

Our Title Video

Things We’re Still Working On

We’ve started our intermediate tricks, but I am keeping my eyes on the prize here, which is to do some great training, not only earn pieces of paper. So we are still working on some of the novice tricks.

Puppy pushups, how about that most basic trick that “everybody” does in obedience schools and many other venues? Hanging my head in shame. We are up to sit, down, sit, down on one treat, but we need to get up to six behaviors rather than four. (She can also do down, sit, down, sit.) I truly do ask my dogs for more than one behavior for a reinforcer at times. But the stand behavior still haunts us, the one that I practiced with her in an aroused state approximately one billion times. And as I extend the number of behaviors on one reinforcer, it usually pops up. “Wait, did you mean this?”

I worked on having her target the long target stick with the enticing ball on the end enough that I was getting a higher percentage of touches than bites, but I haven’t worked the bites down to an acceptable percentage. We still work on it. She’s fine with most anything else I ask her to target.

Our peekaboo during the “whip the head around” phase

One of the novice tricks we both enjoyed was peekaboo, where the dog comes up from behind as you are standing and puts their head between your legs. I didn’t include it for the title, because it required three seconds of duration and we didn’t have that. We kept practicing, though, because I wanted to teach her this behavior for when we are out and about (prompted by the Denise Fenzi “squish” behavior). We got our duration, then I noticed she was poking her head through and immediately whipping it toward my treat hand. I started switching hands, so then I got a dog who poked her head through, then immediately whipped her head back and forth, looking for the treat. C’mon, Eileen, think! I could have predicted that if I’d thought about it for half a second. So I thought about it.

I didn’t want to spit treats like I learned to do in competitive obedience. Clara doesn’t like to catch them, and the only dog treats I’m willing to put in my mouth are pieces of mozzarella cheese. Then I remembered Marge Rogers’ method for treating for a straight front behavior. Before treating the dog, she would bring both hands up to her chin, then bring a hand straight down with the treat. So the treat was not only presented in the center position, it was in the center position for a noticeable amount of time before it came to the dog. That did the trick nicely, and now Clara is getting into position, staying there, and looking forward or straight up at my face. Progress!

That Damn “Roll Out the Carpet” Trick

Posted on October 14, 2021 by Eileen Anderson

I picked the “Roll out the carpet” trick from the novice trick list from Do More with Your Dog because it looked fun and more trick-like than a lot of the other behaviors. We had been doing things like sits and downs and walking on leash and targeting. This was more like a real trick. It would be new, but still looked like a fairly straightforward one because Clara knows how to push things with her nose.

The definition of the trick is:

Dog will use his nose to unroll a rolled-up carpet. Carpet can be a yoga mat, rug or towel and should be roughly 5 feet/~2 meters in length.
DMWYD Novice Trick List

I have rolled food up in towels for Clara before as enrichment, so that seemed like an obvious way to practice. So I took a 5-foot rubber-backed rug and rolled it up with treats inside, and she promptly unrolled it to get the food. I had Clara do this for a couple of days. Easy Peasy.

But that was for practice. Luring is allowed, but I’m not sure about luring-and-eating-as-you-go-along. Even if it’s allowed—the rules for Novice tricks are pretty loose—to me, it’s not in the spirit of the trick. So the next time we practiced, I rolled up the carpet with no treats. Guess what happened? See the photo above?

Clara gave the rolled-up carpet a good sniffing all over, then sat on the little strip that wasn’t rolled up and looked at me. There was obviously no food in there, so why should she bother? Maybe starting with a loaded-up carpet wasn’t the best idea after all!

I had thought the original discriminative stimulus (cue) to get her to unroll the carpet was the rolled-up carpet. But it was the rolled-up carpet with treats in it. I had annihilated a giant lure (perhaps 20 treats) in one blow. Why should she bother with an empty carpet?

Back to Square 1. I realized I was going to have to actually teach the trick instead of coasting in on previous behaviors.

First Teaching Attempt: Get Clara’s Nose in the Right Spot without Treats

I started rolling up an empty carpet and shaped a nose touch in the correct area to push the carpet. This wasn’t hard. She would sniff when she approached the rolled carpet, anyway. So I turned that sniff into a little nudge. And I was thoughtful about my treat delivery, aiming for the little crack under the roll of the mat so I would direct her nose right back to the correct area when she went for the treat.

Tan dog with a black muzzle and ears is putting her nose under part of a rolled up carpet and receiving a treat — **Crappy photo of my glorious treat placement. She always scrunched herself up to stand on the mat, because guess why?**

However, I had two problems. One was that she has an enormous reinforcement history (there’s that problem again!) for lying down on mats or anything matlike. Possibly the most reinforced behavior in her life. So even though I kept my rate of reinforcement high for the nose touches, whenever there was even a momentary lull, her first choice was to lie down on the mat.

Tan dog with a black muzzle and ears is lying down on a maroon carpet. The very end of the carpet is turned over, showing the white backing. — **This is what we do on mats: lie down.**

The second problem was yet another behavior that was stronger than the nose push: a foot target. She would sometimes hit the unrolled part of the carpet with her foot or stand on it.

Tan dog with a black tail and muzzle has her foot on a rolled up carpet — **Or else we put our foot on this part that looks like a target.**

Standing on it was incompatible with unrolling it for sure! And once she would start these other two highly reinforced behaviors, it was not likely she would find her way back to the nose touch. So I didn’t just leave her to figure it out. That would have been too frustrating. I would interrupt, ask for a nose touch to my hand or simply toss a treat, then start us over again.

I did succeed in shaping the gentle sniff under the rolled part of the rug into a nudge, then a push. Sometimes she would give a big push and the whole thing would unroll! I reinforced well for that, but again, I didn’t feel like it was in the spirit of the trick. It happened frequently when I used a yoga mat instead of the rubber-backed carpet runner, so we stuck with the latter.

I was getting the nudge, but I had a problem. I needed to thin my reinforcement schedule and get enough pushes from Clara to unroll the carpet completely before I reinforced. But I had these two other behaviors lurking, ready to pop out the minute Clara didn’t get reinforced for a nose touch. I knew if I tried to thin my schedule now, the first time I didn’t reinforce a nose push (because I wanted a second one), she would try one of the other behaviors instead.

Extinction and Thinning the Ratio Schedule

I’ve made it no secret that I generally pay my dogs for every behavior. You can see my article on it here and another by Dr. Eduardo Fernandez here. You can also look up Nevin’s work outlining the arguments for rich reinforcement schedules creating behaviors that are resistant to extinction (Nevin, Mandell, & Atak, 1983).

I do have a few exceptions to using a 1:1 ratio schedule with my dogs. For loose leash walking, I have extended the number of steps between reinforcers. I probably reinforce on a VR15 (steps) or so. I have also trained stationary duration behaviors where the reinforcers get fairly spaced out. For instance, there can be time periods between reinforcers measured in minutes when I am reinforcing Clara for staying on a mat while I work in the kitchen. I have at least one behavior chain (retrieve) where I generally only reinforce the terminal behavior. Finally, just living with my dogs, sometimes I randomly don’t reinforce for everyday behaviors. But I probably reinforce daily behaviors far more than most people. For instance, I still reinforce 10-year-old Clara with food virtually every time she pees or poops in the yard.

What I haven’t asked for from Clara, since back when I was working on the Training Levels, is multiple iterations of the same behavior for one reinforcer. What Sue Ailsby calls “twofers.” I found this out the hard way early in our trick training endeavor. Clara could not do puppy pushups unless I reinforced every behavior, or at least every other one. Doing six iterations, as is required for the trick (sit, down, sit, down, sit, down), was not possible for us. On the third cue or so, if I failed to reinforce for a sit or a down, she “assumed” she was wrong and started hopping around and throwing behaviors, usually a stand or a hop. I got an extinction burst. How humbling. I hadn’t worked hard enough on cue recognition.

We had an even worse situation with unrolling the carpet, because my goal was to cue her to unroll the carpet, which meant nudging it up to five or six times before it was all the way unrolled, then reinforce. Multiple nudges for one terminal reinforcer! I knew the nudging was still weak enough that I needed to reinforce every single one for a while. Because as soon as I would space out the reinforcement, in would pop into the foot targets and lying down. And I don’t want to put her through extinction without a really clear idea of what she can do for reinforcement.

Then I realized what I should do.

Backchain It!

I don’t think I’ve ever written about backchaining here. I don’t teach many chains. Backchaining means you start with the last behavior of a behavior chain first and work backward. There are several benefits. One is that you load a lot of reinforcement onto the final behavior (stay tuned to see the result in the video below). Another is that because of this, the dog is working toward the more familiar part of the chain that has gotten more reinforcement.

I can think of three behaviors I backchained. First, I backchained a retrieve with Summer and Clara. I also backchained Clara to drop a ball into a bowl using this video as a model. That’s a good video that shows how backchaining can work, if you are curious. It can be almost magical. I also backchained stopped contacts in agility.

Here is how I I used backchaining to get out of the rug trap.

I folded over only the very end of the rug. Clara had enough practice with pushing at the rug that she happily unrolled the little end I had folded over. I didn’t load it with treats, but she had enough experience by then that she would do it without seeing the money on the front end. We did many reinforced repetitions of opening one fold. Many. Then I folded it over twice. Oops, too soon! Got a down and a foot target. Went back to the beginning with just one fold, worked up again to two, and voilà! She pushed it hard enough (or pushed twice) to unroll both folds! Lots of reps of that, too. So we continued, working backward, with me rolling the rug more and more. I sometimes gave interim treats. She was giving multiple pushes rather than one constant one, which was fine with me. I didn’t want her to go from feast to famine, but I wanted her to gradually learn I would pay well if she performed the nose push multiple times to get to the end criterion: unfolded rug. That was backchaining.

While working on the trick, I also remembered she knows how to get food out of a rolling food toy, so I got out the Tricky Treat Ball and fed her some of her breakfast in there. It seemed like a good idea to build some more repetitions of nose pushes however I could get them.

As we got close to success with the backchaining, I added a cue, “Push,” and started using a conditioned reinforcer, “Good girl,” instead of the intermittent treats to let her know she was on the right track. As for that verbal cue: I was cheating a little. It didn’t matter what I said. Clara didn’t instantly learn the specific meaning of “Push.” If I were to say “Push” to her when she was lounging on the couch, for example, she wouldn’t start hunting for a rolled-up rug to nudge. It’s contextual. “Lady says something in a certain tone while I’m standing on the rug, so I will do the thing I just did.” But hey, it worked.

A tan dog with a black face and tail pushes an orange ball filled with food with her nose — **I always think of the Tricky Treat Ball as “Summer’s food toy,” but Clara gained some fluency at it after she aged out of trying to eat the toy itself.**

Progress Video

The video shows the steps we took and our victory a couple of days ago. For such a simple-seeming trick, this feels like quite an accomplishment. But I know exactly why it was a challenge for us, and I’m pretty pleased I could thread my way through all those heavily reinforced but “wrong” behaviors to tease out the right one.https://www.youtube.com/embed/HST5O2Km2pU?version=3&rel=1&showsearch=0&showinfo=1&iv_load_policy=1&fs=1&hl=en-US&autohide=2&wmode=transparent

I love the last iteration of the trick on the video. She pushes the rug several times and ends up with a small flap of it still folded over at the end. She looks at me, she looks at the rug. I am holding my breath, waiting for the reinforcement history to burst into the picture. But the folded over flap, because of the backchaining, became a pretty good discriminative stimulus for “push with your nose to unfold that.” She pushed both sides of it to open the rug flat all the way!

She never did one constant nose push, but it doesn’t appear to be a requirement, so I don’t think we’ll bother. We have already learned a lot by working on this trick!

Speaking of learning a lot, shout out to Marge Rogers for not saying, “I told you so!” She’s been trying to get me to train more tricks for years!

This post was also published on my main Eileenanddogs blog.

References

Nevin, J. A., Mandell, C., & Atak, J. R. (1983). The analysis of behavioral momentum. Journal of the Experimental analysis of behavior, 39(1), 49-59.

Clara’s Stand Disaster and Why She Still Hops

Posted on September 24, 2021September 24, 2021 by Eileen Anderson

Yes, this is part of the trick training series! Stand is one of the tricks we are working on. We just happened to touch on this long-standing can of worms created by my training!

I considered titling this post “Eileen’s Stand Disaster,” but I thought that might be too confusing. Clara was the one standing, but the disaster part was definitely on me.

Thousands of people worldwide have used Susan’ Garrett’s fun method for teaching the stand and gotten fabulous results. I wasn’t one of them, but I blame myself, not the method.

The method is to have the dog in heel position in a sit, and to use a hand target above the dog’s head to get them to leap into the air, hit the target with their nose, then land on four feet. It’s a fun, flashy behavior. But the activity got Clara over-aroused, and I didn’t know how to handle that.

When I casually mentioned on social media that Clara and I had bombed using Susan’ Garrett’s method, a group of angry fans came for me. But wait! I am a Susan Garrett fan, too! I took part in the very first Recaller’s Class, and that was probably the time I saw all three of my dogs at their happiest. I respect her work and have used lots of her methods, both in and out of agility settings.

My failure with that stand method was just that: my failure. Between Clara’s temperament and my weaknesses as a trainer, we had a wreck that kept on wrecking. Not every method works for every trainer’s skill level with every dog.

What Did the Disaster Look Like?

I messed around with the method for a couple of years, alternating between trying to make it work and trying to reteach stand another way. Here’s a video from that time. It’s pretty embarrassing, but things had been even worse earlier. You can see (and hear from my yelps) that Clara’s arousal level was a tremendous problem. I know better now how I helped ramp her up. I literally fed into it by feeding too rapidly and never adding duration. Shark creation.

And no, I don’t know why the half-squat position with her back legs crept into her stand!?

I do kind of wish I had that “leapfrog” behavior at 0:05 on cue, but I know better than to work on it! Another thing to notice is at 0:50 when, after trying unsuccessfully multiple times to get her to stand with a forward hand target, I give up and say “OK.” That, unfortunately, became the verbal cue for the behavior. What to me was a release cue, to her meant “stand.” You’ll see it in subsequent videos.

Clara is not always ramped up. She can melt into a mat and relax, and has successfully relaxed through every session of Dr. Karen Overall’s Relaxation Protocol in several locations. But jumpiness became part of the stand sessions.

Susan says in her video: “Even a six-year-old can do it.” Sigh. But I couldn’t, not with this dog.

Teaching Stand Successfully with Previous Dogs

I taught standing on cue to two other dogs with no problems. I taught Summer while training for competitive obedience and rally. I believe I used targeting but forward, not high in the air as in Susan’s method. Later, I taught Zani an adorable kickback stand using Sue Ailsby’s clever luring method: you lure them backward so they don’t step forward. Stand was the first and maybe only behavior I taught Zani by luring with food. I had to persuade her to follow the lure, since we had done so much leave-it practice.

This video of Zani is from 2018, after her spinal cord concussion. She couldn’t do a tight sit because of the effects of the spinal cord injury on her right rear leg, but she could still do the kickback.

Thanks for indulging me. Yeah, I posted this video partly to calm my ego, but also, damn, Zani was so cute!

What Were the Problems with Teaching Clara to Stand?

Clara gets aroused when she moves. A common problem, but I had not experienced it with any dogs before Clara. For instance, when we practice loose leash walking—ouch! She gets excited and super chompy when she gets treats while in motion. So when I taught her to leap from a sit or down and touch my hand in the air, then jammed food in her face when she hit the ground, I got her all worked up.

I was trying to capture the stand before she moved, so I shoved the treat into her mouth reallyreallyfast. And, predictably, got shark behavior in return.

Also, when Clara got confused about what we were doing, she would land in a sit or a down and stay there. I responded by giving my release cue to get her to move. She would stand, and I would reinforce. So the release cue (“OK”) became the cue for stand. By the time I added a real cue (“Brace”), I had already accidentally but firmly taught her to listen for “OK.” And it was unhandy that Clara popped into a stand whenever I tried to release her from anything.

I kept starting over with the process, so I got no duration.

I had an over-aroused, grabby dog who would bounce around nervously whenever I tried to train stand. Whatever I did with my reinforcement mechanics left her manically continuing to try other behaviors.

How I Solved the Problems

Clara already knows how to stand, of course, as does any dog with typical mobility. She stands dozens of times every day, meaning she performs both the motion of standing up from a sit or down, and the duration behavior of standing around. Standing in everyday life doesn’t arouse her; I created the arousal in our training.

A few months ago, I started over one more time. I took a page from my previous training and looked up my video on capturing stands. I thought I had made the video with Zani, but there was Clara, in 2015, with a nice little start on a fairly calm kickback stand! LOL, that’s what can happen when you write lots of blog posts and make lots of videos over the years! I think I had done that before I got her all overwrought with the jumping and targeting method! In the video, you’ll see her roach her back a couple of times, but her rear legs are in a much better position than the ultra-squatty stuff reinforced during the Bad Times.

So this year, I copied the steps from my own video.

First, I captured quiet stands for a while (not the behavior of standing up, just standing). I changed the picture from the method that didn’t work for us and stopped using the heel position setup. After sessions of capturing, I set her up facing me and waited. She offered movement into a stand quickly (but not nervously!). I was careful not to feed too fast, and I added duration as soon as I could. (One of my tragic flaws as a trainer is neglecting duration.) And I used a new cue—a flicking hand signal. No more “Brace” or “OK.”

This video from April 2021 through today shows our progress. She was still crouching somewhat with her hind legs in April, but that is mostly gone now.

Reinforcement History: Ghosts of Behaviors Past

But my “sort-of “victory isn’t the point of this post. I did some decent problem-solving. I was very patient. I got back a pretty nice stand behavior that we can continue to work on. You’ll see it in our next trick training video as well. But the nervous behaviors I reinforced during the stand practice still pop up frequently in training sessions of all sorts.

Behaviors rarely diminish all the way to zero. Clara unfortunately has a big reinforcement history for the debacle-stand. Here are a few examples of the attendant behaviors reappearing. And at the end, she does a perfect stand from a down (in response to my release cue) while I turn my back to go turn off the camera!

The concept of reinforcement history comes straight from Thorndike’s Law of Effect.

Of several responses made to the same situation, those which are accompanied or closely followed by satisfaction to the animal will, other things being equal, be more firmly connected with the situation, so that, when it recurs, they will be more likely to recur; those which are accompanied or closely followed by discomfort to the animal will, other things being equal, have their connections with that situation weakened, so that, when it recurs, they will be less likely to occur. The greater the satisfaction or discomfort, the greater the strengthening or weakening of the bond.
Thorndike, 1911, p. 244

The matching law quantifies the Law of Effect. The percentage of the time a behavior is reinforced will be reflected in how often the animal performs the behavior. So it’s a numbers game. If I stop reinforcing Clara for popping into a stand after I give the release cue (the last behavior in the above video), the stand will gradually decrease in that context. But the leapfrogging around in training sessions will be hard to eliminate because it got a ton of reinforcement in the past and gets chained into other behaviors I reinforce. You can see me reinforcing these chains throughout the video. The chain problem is not impossible to fix, but I need to sit down and think about whether I (with my own limitations) can carry out a plan without frustrating Clara, whether it’s worth it to try.

In the meantime, I’m pleased I am getting a semi-normal stand.

Reference

Thorndike, Edward L (1911) Animal intelligence: Experimental Studies. Macmillan.

Tan dog with black muzzle and tail sits in "front" position and gazes up at a woman standing — **Clara thinks “front” is a great game. Stay tuned for more tricks!**

P.S. Although I didn’t categorize it that way, this post was born of the trick training we are working on. We aren’t through with tricks by a long shot!