Extinction Vs -p

[Erny]

I've read that too, K-i. Only thing I figured was that for extinction to occur, the dog would have to receive zilch reinforcement (self-reinforcement inclusive) from the exhibition of the "unwanted behaviour". One would have to be very careful about increasing reinforcement unless they were aware of what reinforcement the dog was receiving that brought about the unwanted behaviour in the first place.

[Aidan]

Has anyone tried this?

Yes, but Erny makes some good points. If you start off with some functional analysis you can figure out whether it is a safe strategy to use or not.

Keep in mind that problem behaviours are being reinforced on some sort of schedule, often continuous, sometimes variable. If they are already on a continuous schedule and you have the opportunity to use an extinction procedure then you don't need to do anything else. If they are on a variable schedule you can make an attempt to make this a continuous schedule so that the dog is fully expecting reinforcement every single time before abruptly using an extinction procedure.

Something else to be careful not to do if you do have the opportunity to attempt this strategy - do not introduce a different reinforcer! E.g dog jumps on person for attention, which is given intermittently. Trainer instructs person to toss a ball every single time for 1 week every time dog jumps up, before abruptly ceasing ball toss the following week. OK, this was a strange example, but you can probably guess what would happen - the dog would simply resume jumping up for attention.

Edited September 9, 2009 by Aidan

[Aidan]

No wonder it's confusing. Oh well, I think I've got it, now, and next time someone tries to confuse me with another example of extinction instead of P- I won't be so confused.

I think in this sort of example there isn't much harm and it is a common mistake. It is more of a problem when you are using a Differential Reinforcement procedure, or using functional analysis to get to the root of a problem.

[Kelpie-i]

Only thing I figured was that for extinction to occur, the dog would have to receive zilch reinforcement (self-reinforcement inclusive) from the exhibition of the "unwanted behaviour". One would have to be very careful about increasing reinforcement unless they were aware of what reinforcement the dog was receiving that brought about the unwanted behaviour in the first place.

Hey Erny, yes I figured this when I stated that one would have to take other environmental factors into account. I was referring to the established reinforcer. It would be much easier just to jump straight into an extinction procedure without all the hoo-haa, especially for Joe Public and his pet pooch.

[Erny]

Agree, Kelpie-i.

Perhaps I'm missing something, but the way I figure it is simple enough.

When I read Aidan's post/s (which are very well and thoughtfully articulated, I might add) I got the impression that he was suggesting that "extinction" was some other 'learning principal' that perhaps somehow should have been included in the four quadrant matrix. I don't know if that's what you meant, Aidan, and somehow I don't think you do, but I certainly don't. (ETA : Re-reading earlier posts in this thread .... perhaps I got that impression from posts in the other thread where it all began ???)

The human 'trains for' extinction.

For the dog, extinction 'occurs'.

So extinction is a 'process' (I think Aidan might have said that).

For extinction to occur, the dog must not be able to reach its goal (which is of course the reinforcer), whatever that goal might be. Of course the trick is to know what the goal is and to ensure that it cannot be reached.

As for the P- -vs- extinction discussion ..... Aiden, I really think that's something that unless you can literally be inside the dog's head and know precisely its thoughts from one moment to the next (wouldn't that be lovely ..... our jobs would be soooooo easy ), you could debate until the cows come home. Given I don't have any cows, that presents to me as being a long, long time .

I think whether the behaviour is extinguished through punishment (P+ or P-) depends much on whether it outweighs the reinforcer or not. So, in some cases the punishment (P+ or P-) might not remove the original reinforcer, but it might be perceived as sufficient enough to weaken the unwanted behaviour. I don't believe that in the technical sense (the way I've been taught it, anyway) that this is "extinction training".

• Punishment (+ or -)of course refers to consequences that make the behaviour less likely to occur again.
  
• Extinction training on its own doesn't make behaviour less likely to occur again as we know that "extinction" is prone to spontaneous recovery in the presence of the conditioned or unconditioned reinforcer.
  
• The above is often why extinction training is required to be paired with punishment (+ or -).
  

We also know that "extinction training" is prone to 'extinction bursts'.

So - using the example that was given in a previous post (Aidan or Kelpie-i) of a dog who has taken to rolling over when commanded to drop : If the treat is withheld (P-) and the dog therefore stops rolling out of the drop then I can't help but assume that the dog's goal/reinforcer is the food treat. But I'm also inclined to assume that it is not so much that extinction has occurred either ... there's been no extinction burst. But further thought/investigation as to whether "extinction" had actually occurred in this situation would be required (if you really wanted to know). For example, you'd need to contemplate/know what the reinforcer was for the dog to roll over and out of the drop was in the first place and whether the dog's goal for that reinforcer was ever met. (Are any of those cows coming home yet .... ?)

As the subject title is "Extinction -vs- Negative Punishment" .... I confess to not being completely sure what discussion you're really aiming for (my apologies). Are you saying is one better than the other or equally the same? My answer to that would be "that depends on the dog, its goal/s, what the reinforcer is and whether the punishment outweighs the reinforcement.". Or are you suggesting that extinction (training) is the same thing as negative punishment? My answer to that is again "that depends on the dog, its goal/s, what the reinforcer is and whether the punishment outweighs the reinforcement." To me in theory they are different, but to the dog in practicality? That's where I guess I would need to be able to be in his head ......

Pardon me if I have completely missed the direction of your thread Aidan and whilst your posts are extremely well written and thought provoking, I think this has been complicated somewhat beyond what it really needs to be.

Just my 2c worth and hope I have managed to make sense. I can see where we could each spend many hours splitting hairs over it all and whilst this would make for interesting round table discussion (provided we had a few bottles or ten of wine to absorb while we were at it) I think perhaps we are making it more complicated than necessary.

In fact, when I have thought about it, it has reminded me of the mirror trick - the one where one mirror shows an image that turns out only to be an image that is in another mirror which is showing an image from another mirror and so on .

ETA: I hope you have read my post more than once, because I've edited, re-edited and re-edited in an attempt to tidy it up and make the words flow in such a way as to having any chance of being understood.

Edited September 9, 2009 by Erny

[corvus]

For extinction to occur, the dog must not be able to reach its goal (which is of course the reinforcer), whatever that goal might be. Of course the trick is to know what the goal is and to ensure that it cannot be reached.

Or the dog needs to not be reinforced by anything. It might be there are several potential reinforcers involved. Right? You could accidentally reinforce something with anything rewarding without having the dog actually reach its goal.

As for the P- -vs- extinction discussion ..... Aiden, I really think that's something that unless you can literally be inside the dog's head and know precisely its thoughts from one moment to the next (wouldn't that be lovely ..... our jobs would be soooooo easy ), you could debate until the cows come home. Given I don't have any cows, that presents to me as being a long, long time .

But isn't the nature of working with animals that you will never really know for certain? You can make some very educated guesses, though, and if you didn't make those you may as well not bother.

So - using the example that was given in a previous post (Aidan or Kelpie-i) of a dog who has taken to rolling over when commanded to drop : If the treat is withheld (P-) and the dog therefore stops rolling out of the drop then I can't help but assume that the dog's goal/reinforcer is the food treat. But I'm also inclined to assume that it is not so much that extinction has occurred either ... there's been no extinction burst. But further thought/investigation as to whether "extinction" had actually occurred in this situation would be required (if you really wanted to know). For example, you'd need to contemplate/know what the reinforcer was for the dog to roll over and out of the drop was in the first place and whether the dog's goal for that reinforcer was ever met. (Are any of those cows coming home yet .... ?)

Okay, I thought that withholding a reward is not P-, because how can you "take away" something that the dog doesn't even have yet? To me, just because I intend to give the dog a reward doesn't mean the dog already has it before I give it to them. Say I'm shaping and hold off rewarding for a second while my dog looks away and then reward when the dog looks back. The dog doesn't go through an extinction burst of looking away. Looking away just drops out because it wasn't rewarded.

Have I got this right?

I may be quite off the mark, but I see where Aidan is coming from in that extinction vs P- is a harmless mistake in the scheme of things. But still, I don't know about anyone else, but I like to have things right rather than wrong. It ultimately doesn't matter for me as a layman whether my method is working because I'm using P- or setting up an extinction process, but it matters to me as a scientist interested in animal behaviour! I imagine it might matter to me if I were trying to nail a behaviour quickly with a dog by teaching it what gets rewarded and what results in punishment, seeing as taking something away to decrease behaviour is a form of punishment. If I set up an extinction process instead of the P-, the result might be the same in the end, but it might be slower as I haven't taught the dog what is behaviour that is a bad idea so much as what is behaviour that won't be rewarded.

[Erny]

Or the dog needs to not be reinforced by anything.

Yes. That's right.

But isn't the nature of working with animals that you will never really know for certain?

Sometimes you can know - especially from results. But yes .... because we aren't actually mind readers, it is about analysis of what we can see.

Okay, I thought that withholding a reward is not P-, because how can you "take away" something that the dog doesn't even have yet?

The dog has to have prior learning and hence expectations of the reward.

I may be quite off the mark, but I see where Aidan is coming from in that extinction vs P- is a harmless mistake ...

I don't know what you mean by this.

But still, I don't know about anyone else, but I like to have things right rather than wrong.

Of course that would be the desire of 99.9% of the population I expect.

If I set up an extinction process instead of the P-, the result might be the same in the end, but it might be slower as I haven't taught the dog what is behaviour that is a bad idea so much as what is behaviour that won't be rewarded.

And you may find that the behaviour very quickly returns due to spontaneous recovery. You also might find that P- isn't sufficient enough (even if you did apply it in your methodology). Much would depend on how much of an aversive the dog perceived your P- as a punishment.

Edited September 9, 2009 by Erny

[corvus]

Okay, I thought that withholding a reward is not P-, because how can you "take away" something that the dog doesn't even have yet?

The dog has to have prior learning and hence expectations of the reward.

So are you saying the expectation of a reward is the same thing as a reward? To me, expectation is more related to an establishing operation (a term I learnt yesterday - woo!). Maybe I'm wrong, though! I can see how you could think of failing to live up to an expectation as P-. My main beef with the idea is that I feel like you have to take something away for it to be considered P-. I'm not convinced that taking away a high likelihood of reward is actually taking something away. Or taking away something that predicts rewards is the same as taking away something rewarding. Although... Okay, hopefully Aidan can clear that up.

I may be quite off the mark, but I see where Aidan is coming from in that extinction vs P- is a harmless mistake ...

I don't know what you mean by this.

I mean it doesn't make much difference in the scheme of things whether you call it extinction or P-.

If I set up an extinction process instead of the P-, the result might be the same in the end, but it might be slower as I haven't taught the dog what is behaviour that is a bad idea so much as what is behaviour that won't be rewarded.

And you may find that the behaviour very quickly returns due to spontaneous recovery. You also might find that P- isn't sufficient enough (even if you did apply it in your methodology). Much would depend on how much of an aversive the dog perceived your P- as a punishment.

True. But if it wasn't aversive enough to see a decrease in the behaviour, can we really call it a punishment? Maybe that's a different discussion.

[Aidan]

the impression that he was suggesting that "extinction" was some other 'learning principal' that perhaps somehow should have been included in the four quadrant matrix

Extinction certainly is a learning process and some academics are suggesting that it should be called the "5th quadrant" - which apart from being an oxymoron, I feel is erroneous. Why? Operant Extinction is not a consequence, it is when the environment fails to add or withdraw something from the environment where previously it had through either positive or negative reinforcement.

Does that make sense?

As I will detail later, also an operant extinction procedure does not have to produce a change in behaviour.

As for the P- -vs- extinction discussion ..... Aiden, I really think that's something that unless you can literally be inside the dog's head and know precisely its thoughts from one moment to the next

Both have very clear definitions, this article is worth a read:

http://www.animalbehavioranswers.com/id117.html

If the two are confused (and they are similar and hence, frequently confused), then I can see how it appears to be a debatable point, but in fact there are clear differences between the two. In fact, if you were inside the dog's head I think it would be a lot harder to tell the difference! They don't care for the names we have given these natural processes of learning

Extinction training on its own doesn't make behaviour less likely to occur again as we know that "extinction" is prone to spontaneous recovery in the presence of the conditioned or unconditioned reinforcer.

The above is often why extinction training is required to be paired with punishment (+ or -).

Punishment is not immune to recovery, especially in the absence of the punisher or conditioned punisher.

Differential Reinforcement procedures (which rely on operant extinction) are very effective at suppressing behaviours in the long term without using punishment.

So - using the example that was given in a previous post (Aidan or Kelpie-i) of a dog who has taken to rolling over when commanded to drop : If the treat is withheld (P-) and the dog therefore stops rolling out of the drop then I can't help but assume that the dog's goal/reinforcer is the food treat. But I'm also inclined to assume that it is not so much that extinction has occurred either ... there's been no extinction burst.

In this example withholding the treat is not -P. Nothing was taken away, the dog didn't have the treat. You might take away a conditioned reinforcer, which may be a punishing consequence, but that was not described in this example.

Not all extinction procedures involve extinctions bursts, spontaneous recovery, or for that matter even a reduction in the behaviour! It is an extinction procedure for the simple fact that a response which has previously resulted in a reinforcer is no longer producing that reinforcer. (This is another reason why extinction does not belong in the quadrants, in my opinion).

However, we can't say that a response has been "extinguished" until it disappears. THAT we could debate until the cows come home!

As the subject title is "Extinction -vs- Negative Punishment" .... I confess to not being completely sure what discussion you're really aiming for (my apologies). Are you saying is one better than the other or equally the same?

No, there is no value judgement to be made. A point that I had made two threads back had been misinterpreted because the difference between extinction and -P was not understood. It was suggested that I start a new thread to discuss this as it was off-topic elsewhere.

I am merely seeking to point out that there is a difference between operant extinction and -P. For most dog owners and even dog trainers this difference is not particularly important, but for those of us who utilise functional analysis, Differential Reinforcement procedures, and particularly for those of us who want to change behaviours which have been maintained with negative reinforcement (anyone who works with reactive dogs) it can be very important.

Pardon me if I have completely missed the direction of your thread Aidan and whilst your posts are extremely well written and thought provoking, I think this has been complicated somewhat beyond what it really needs to be.

Possibly! We always seek to improve, sometimes the areas we can gain benefit from are unknown until we need to call upon that information.

Just my 2c worth and hope I have managed to make sense. I can see where we could each spend many hours splitting hairs over it all and whilst this would make for interesting round table discussion (provided we had a few bottles or ten of wine to absorb while we were at it)

A good suggestion!

ETA: I hope you have read my post more than once, because I've edited, re-edited and re-edited in an attempt to tidy it up and make the words flow in such a way as to having any chance of being understood.

Yes, I only caught up with it after you included this ETA. Thanks for your comments and questions

[corvus]

So - using the example that was given in a previous post (Aidan or Kelpie-i) of a dog who has taken to rolling over when commanded to drop : If the treat is withheld (P-) and the dog therefore stops rolling out of the drop then I can't help but assume that the dog's goal/reinforcer is the food treat. But I'm also inclined to assume that it is not so much that extinction has occurred either ... there's been no extinction burst.

In this example withholding the treat is not -P. Nothing was taken away, the dog didn't have the treat. You might take away a conditioned reinforcer, which may be a punishing consequence, but that was not described in this example.

Not all extinction procedures involve extinctions bursts, spontaneous recovery, or for that matter even a reduction in the behaviour! It is an extinction procedure for the simple fact that a response which has previously resulted in a reinforcer is no longer producing that reinforcer. (This is another reason why extinction does not belong in the quadrants, in my opinion).

Ah, I see.

A question, though: If you took away a conditioned reinforcer - something that predicted rewards - before any reinforcement was given, and you saw a resultant decrease in the behaviour usually displayed before taking away the conditioned reinforcer, how would you know if you were seeing an extinction process or if taking away the conditioned reinforcer was punishing?

[Aidan]

A question, though: If you took away a conditioned reinforcer - something that predicted rewards - before any reinforcement was given, and you saw a resultant decrease in the behaviour usually displayed before taking away the conditioned reinforcer, how would you know if you were seeing an extinction process or if taking away the conditioned reinforcer was punishing?

You answered your own question - you took it away contingent upon a response, and that response was suppressed.

In hindsight I may have been wrong about your crate example from earlier, provided that approaching the crate was a conditioned reinforcer (which seems reasonable now that I have thought about it some more).