Technical Clicker/operant Conditioning Question

[Staranais]

So, I was reading or possibly watching something about dog training the other day, and the author made a point that really confused me. Hopefully one of the trainers here can shed some light on it.

The author said that when you use a marker/clicker/bridge to capture a behaviour, not only is the behavior you mark being reinforced by the primary reinforcer, but he reckoned that what the dog was doing at the moment you actually deliver the primary reinforcer was also being reinforced. So if your dog gives you a lovely heel position, and you mark/click that heel position, and the then dog jumps ahead of you to get his reward, and you feed the dog, then you're actually reinforcing both the heel position you clicked and the forging that the dog was doing afterwards.

Now, that's not something I've ever really thought through before. As far as I've learned, the whole idea behind using a marker is that it captures the behaviour the dog was doing at the exact time they heard the click. That's the main advantage of using a marker, since you can mark far faster than you can throw a piece of food or a toy at your dog, and to a clicker literate dog, that's supposed to be "the same" as if you'd managed to get the food to them immediately.

Yet this guy was saying that although yes, the behaviour you mark is being reinforced by your reward, the behaviour the dog is performing when the primary reinforcer is delivered is also being reinforced.

Does anyone have any insight as to whether this is actually correct? I have lots to think about if he is right.

[kchogan]

i am no dog trainer, however do use a clicker.

lets say you want your dog to heel. you should lure him into the heel position with treat in hand the same hand you lure him with, and clicker in other hand so as soon as your dog heels you can immediately give the treat. in the end you want to be able to get rid of the treat so he will do what you want and know that he can have his fave treat at the end of session or day.

if you get him in the heel position, click and then he moves do not give him treat. move him back into the heel position and try again if he stays then make a big fuss as our furry friends love attention

[Kavik]

In the agility instructor's seminar I attended last year this was brought up as correct - that the behaviour between the marker and teh reinforcer was also rewarded. Would explain one of the problems I am having with my contact training - after I mark the behaviour he tends to turn his head and look at me (even when I throw the reward) - so it has been incorporated into his performance. Trying to fix that to looking forward or at the target instead is proving a bit tricky

[Aidan]

It's possible for ANYTHING from click to reward to be reinforced.

What is IMPORTANT is what is being increased or maintained. If the thing you clicked is being increased or maintained, you got what you wanted.

If, between click and treat, your dog decides to bite a little old lady then hump your leg, probably best not to give the treat...

[Staranais]

Hmmm, lots to think about, thanks guys.

Thinking further about it, I've never had any trouble teaching a nice heel by rewarding a few seconds after the marker/release cue, no matter what the dog was doing at the time of reward. I always figured that it didn't really matter, seeing as the dog fully understand what click meant. However, now I'm wondering if the training would have progressed even quicker if I'd somehow managed to mark then reward in heel position. Any thoughts?

Sigh! I've been doing this for ages, and still learning about such basic stuff!

p.s, Welcome to the forum, Kchogan!

[Aidan]

Thinking further about it, I've never had any trouble teaching a nice heel by rewarding a few seconds after the marker/release cue, no matter what the dog was doing at the time of reward. I always figured that it didn't really matter, seeing as the dog fully understand what click meant. However, now I'm wondering if the training would have progressed even quicker if I'd somehow managed to mark then reward in heel position. Any thoughts?

If you were getting more of what you were clicking, then it wouldn't have been any faster to reward in position.

[Staranais]

Thinking further about it, I've never had any trouble teaching a nice heel by rewarding a few seconds after the marker/release cue, no matter what the dog was doing at the time of reward. I always figured that it didn't really matter, seeing as the dog fully understand what click meant. However, now I'm wondering if the training would have progressed even quicker if I'd somehow managed to mark then reward in heel position. Any thoughts?

If you were getting more of what you were clicking, then it wouldn't have been any faster to reward in position.

How do you know? (Genuine question!)

I definitely was getting more of what I was clicking. But perhaps I would have got more even faster if I'd rewarded in position?

Like, perhaps I was taking two steps forward and one step back, when I could have been going forward the whole time?

Does that even make sense? I think my brain's in holiday mode.

[pipsqueak]

i was watching a Bob Bailey DVD recently, and he discussed this concept. His attitude is to "click for movement and feed for position", meaning that ideally you should feed/reward the animal whilst still in the position you want, or in the case with his chickens, feeding to help them move in the desired direction.

[raineth]

Hmmm, lots to think about, thanks guys.

Thinking further about it, I've never had any trouble teaching a nice heel by rewarding a few seconds after the marker/release cue, no matter what the dog was doing at the time of reward. I always figured that it didn't really matter, seeing as the dog fully understand what click meant. However, now I'm wondering if the training would have progressed even quicker if I'd somehow managed to mark then reward in heel position. Any thoughts?

Sigh! I've been doing this for ages, and still learning about such basic stuff!

p.s, Welcome to the forum, Kchogan!

Hi Staranais,

Something I have learnt with clicker training is that it is better to click when the dog is looking at you, and preferable not to click if the dog is staring at the treat.

I have found with my new dog this works very well and he is still concentrating on either my hand signal or my face (as well as doing what I'm asking him to do!)

So maybe heeling would have been learnt quicker if you treated while he maintained that position. But I would say that if you do it that way, don't fall into the trap of having him focused more on the treat than on you (assuming that in order to treat in heel position you will have the treat in hand )

[megan_]

i am no dog trainer, however do use a clicker.

lets say you want your dog to heel. you should lure him into the heel position with treat in hand the same hand you lure him with, and clicker in other hand so as soon as your dog heels you can immediately give the treat. in the end you want to be able to get rid of the treat so he will do what you want and know that he can have his fave treat at the end of session or day.

if you get him in the heel position, click and then he moves do not give him treat. move him back into the heel position and try again if he stays then make a big fuss as our furry friends love attention

This is one method, but some trainers (eg Uta Bindels) don't advocate luring when using the clicker. Rather, they prefer free shaping. Also, I believe that, with the clicker training methodology, you must ALWAYS give the dog a treat after you have clicked - even if you have clicked in error. A click is a promise that you will treat.

To answer the OP's question, I could be wrong here but Uta believes that the click is also a release, so the exercise is over the second you have clicked - so the "move to get the treat" is not being re-inforced - the dog is free to move where it likes.

I hope someone from teh seminar comes along and explains it better than me! She is a 5 times World Champion so must know what she's talking about.

[Staranais]

Hmmm, lots to think about, thanks guys.

Thinking further about it, I've never had any trouble teaching a nice heel by rewarding a few seconds after the marker/release cue, no matter what the dog was doing at the time of reward. I always figured that it didn't really matter, seeing as the dog fully understand what click meant. However, now I'm wondering if the training would have progressed even quicker if I'd somehow managed to mark then reward in heel position. Any thoughts?

Sigh! I've been doing this for ages, and still learning about such basic stuff!

p.s, Welcome to the forum, Kchogan!

Hi Staranais,

Something I have learnt with clicker training is that it is better to click when the dog is looking at you, and preferable not to click if the dog is staring at the treat.

I have found with my new dog this works very well and he is still concentrating on either my hand signal or my face (as well as doing what I'm asking him to do!)

So maybe heeling would have been learnt quicker if you treated while he maintained that position. But I would say that if you do it that way, don't fall into the trap of having him focused more on the treat than on you (assuming that in order to treat in heel position you will have the treat in hand )

Heel position for me includes the dog looking at my face, so it wouldn't be heel position if she was looking at the treat (unless it was balanced on my nose or something!)

To be honest though, it's not actually at all practical for me to reward in heel position. Competition heel is a pretty precise behaviour, and the chances of me delivering a reward between me clicking and the dog jumping out of position are pretty remote - which is exactly why I use a marker to train it in the first place. So it was a pretty bad example for me to pick, sorry!

I should have picked an example where it was actually possible for me to reward with the dog in position. I guess a more appropriate example might be something like a send away to an obstacle, like Kavik suggested, where you can deliver the reward to the dog when he's still doing the behaviour. Is it better for me to throw the reward to the dog when he's still out there, or does it not make any appreciable difference if I just mark him getting there and give the reward when he dashes back in to me?

[Kavik]

I guess it depends on whether you want the dog running to you after he has done the behaviour? In my case, I want the dog to negotiate the obstacle, hit his 'pounce' contact position and then stay there until released by verbal command. So I reward in position by placing or throwing the food as close as I can to him when he is in position after I mark and also when I release him (throw it forward as I want him to go forward and I want him to focus forward)

[Aidan]

Thinking further about it, I've never had any trouble teaching a nice heel by rewarding a few seconds after the marker/release cue, no matter what the dog was doing at the time of reward. I always figured that it didn't really matter, seeing as the dog fully understand what click meant. However, now I'm wondering if the training would have progressed even quicker if I'd somehow managed to mark then reward in heel position. Any thoughts?

If you were getting more of what you were clicking, then it wouldn't have been any faster to reward in position.

How do you know? (Genuine question!)

I definitely was getting more of what I was clicking. But perhaps I would have got more even faster if I'd rewarded in position?

Haha, I know what you're getting at and I only KNOW the answer IF it is true that you were getting more of what you were clicking and not "more of moving to where the food was coming from" instead.

I suppose there is some slight advantage to having the dog stay in position after the click, but only in that it reduces the time taken to get into the correct position. If you add it up and calculate how long it took you to get the finished product you probably could have got a dozen or so more clicked responses in there for the same amount of time. Not really a big deal.

[corvus]

i was watching a Bob Bailey DVD recently, and he discussed this concept. His attitude is to "click for movement and feed for position", meaning that ideally you should feed/reward the animal whilst still in the position you want, or in the case with his chickens, feeding to help them move in the desired direction.

I very much like this attitude and try to follow it wherever practical. Where it's not practical and I'm shaping duration I use a different bridge "good boy" and a release word to signal when he can break the position to collect his reward.

IME most of the time you mark and your animal will wait for several seconds doing nothing but watching your hand (or face) as you reach (or fumble) for a treat. However, I have had moments where one of the dogs has done something else as I was feeding the treat and then that becomes a behaviour chain. It usually goes away in time if you keep marking the right behaviour, but IME it goes away a good deal quicker if you feed the position. So now I just feed the position rather than waiting for something to crop up that needs fixing. It's just easier that way.

[Staranais]

Haha, I know what you're getting at and I only KNOW the answer IF it is true that you were getting more of what you were clicking and not "more of moving to where the food was coming from" instead.

Ah, I get it. Yes, I guess I would see sign tracking if the marker wasn't doing its job. So even though the primary reinforcer might be reinforcing reward orienting behaviour, the fact that I'm not seeing any movement towards the reward before I click shows that the marker is far more "powerful" than the primary reinforcer in capturing behaviour - for want of better teminology! I'm sure there's a more elegant way of saying that.

I suppose there is some slight advantage to having the dog stay in position after the click, but only in that it reduces the time taken to get into the correct position. If you add it up and calculate how long it took you to get the finished product you probably could have got a dozen or so more clicked responses in there for the same amount of time. Not really a big deal.

Yes, and in my experience a happy, enthusiastic, clear release actually becomes very rewarding to a dog, so I guess that muddying that clarity by trying to keep the dog in position while I'm fumbing round for a treat would probably be counterproductive in most cases.

I suppose I merely have far too much time to think about these things in the holidays! Thanks everyone for their thoughts.

[Agility Dogs]

I guess it depends on whether you want the dog running to you after he has done the behaviour? In my case, I want the dog to negotiate the obstacle, hit his 'pounce' contact position and then stay there until released by verbal command. So I reward in position by placing or throwing the food as close as I can to him when he is in position after I mark and also when I release him (throw it forward as I want him to go forward and I want him to focus forward)

In this case though I don't think you should be using a clicker OR you should be leaving him in the pounce position (effectively a stay) for a greater period of time and then using the toy as the reward that follows the click. Sounds to me like you are rewarding the entire chain of behaviour and not just the end position.

IMO a click means - you have done your job and are now free to do what you will. Chances are they will come to you becuase they know a treat is coming.

[Kelpie-i]

I think different trainers have different ways of using the clicker and it can get down to semantics as to whether the click is the release and/or the marker.

I prefer to use the clicker as a pure marker rather than to cue the end. My dog's eyes light up like light bulbs at the sound of the click and awaits his reward so this has worked for me.

i was watching a Bob Bailey DVD recently, and he discussed this concept. His attitude is to "click for movement and feed for position", meaning that ideally you should feed/reward the animal whilst still in the position you want, or in the case with his chickens, feeding to help them move in the desired direction

This makes perfect sense to me. Remaining in position should be rewarding, more rewarding than the release. Feeding for position works very well and eliminates the dog breaking through anticipation of the release.

[Zug Zug]

This seems to be a live debate amongst clicker trainers.

Morgan Spector's book says again and again 'click ends the behaviour' - i.e. don't worry about what the dog does after you've clicked.

Susan Garrett's book says 'click does not end the behaviour - your dog is always learning'

My dog loves to break and run to the treats (in a container on the ground somewhere nearby) and wait for me to get there to reward. But we don't always do this and she needs me to release her, which is a separate thing from the click.

When heeling, we sometimes click, then run to the treats, and sometimes I click and treat her in a halt or something. So we mix it up. Otherwise she gets bored. Or maybe I should say - otherwise, I get bored/boring.

[Kavik]

I guess it depends on whether you want the dog running to you after he has done the behaviour? In my case, I want the dog to negotiate the obstacle, hit his 'pounce' contact position and then stay there until released by verbal command. So I reward in position by placing or throwing the food as close as I can to him when he is in position after I mark and also when I release him (throw it forward as I want him to go forward and I want him to focus forward)

In this case though I don't think you should be using a clicker OR you should be leaving him in the pounce position (effectively a stay) for a greater period of time and then using the toy as the reward that follows the click. Sounds to me like you are rewarding the entire chain of behaviour and not just the end position.

IMO a click means - you have done your job and are now free to do what you will. Chances are they will come to you becuase they know a treat is coming.

I think it comes down to placement of reward. If you always click and give the treat from your hand your dog will look to your hand when you click. If you always throw the treat away from you when you click, the dog will go forward in the direction you throw. I have moved away from using an actual clicker with this exercise and am using a verbal marker instead.

[Kelpie-i]

With my own dogs, I came away from the traditional, command-release-reward method as I found the dog's anticipated the release too much and would sometimes break prematurely. The release was cued by a release word such as "free" followed by the reward. I switched to the clicker because it was more precise and I liked the idea of marking/shaping the behaviours I wanted, as well as rewarding the dog when it was in position. I still use the release cue but it means nothing and the dog merely gets a good boy/small pat and he is on his own time.

For adding distance and duration to an exercise, I am able to mark/reward at increasing intervals and/or distance yet keep the dog in position. I have also ensured that I taught a pause between click and treat. I believe that if you've conditioned the marker correctly, then you shouldn't have problems with creating secondary behaviours due to the delivery of the primary reinforcer.

To me, using a clicker as the end of the exericse is no different to the traditional C_R_R, just that the dog is conditioned to the sound of the click as the release, rather than the word. Not a very powerful way of shaping IMO but hey..I'm relatively new to clicker (well only a few years ).

Edited January 21, 2010 by Kelpie-i