Variable reinforcement schedule

[The Spotted Devil]

My pleasure entirely. Thanks for being so open to suggestions. 

[corvus]

A response cost is a penalising consequence that is in some way costly to the individual's goals, and usually refers to taking something away that has already been gained. Generally the hope is it will be punishing. A good example might be penalty yards in loose leash walking training. When the dog pulls, you walk backwards rather than just stopping. The idea is there is a reward for walking with a loose leash, and that is going forwards, but there is a cost to a tight leash, and that is not only halting the forward motion, but taking it away by going backwards. I don't usually use it in this context because most dogs are quite responsive enough to a halt IME, and you always risk frustration with a response cost. Frustration can make things worse and get you bitten if you're not careful. I think most of the time if you work on the dog's arousal and emotional state to ensure that is right and then ensure your reinforcement is sufficiently valuable and you have controlled the environment to the extent where the correct response is easy for the dog, you don't need to infuriate the heck out of the dog by imposing a response cost. 

 

I occasionally find myself working with a dog that has hit on a behaviour that is easy and self-reinforcing and despite clearly understanding how to earn reinforcement, they persist in this other behaviour regularly. Usually it's energetically cheap, like barking or a quick jump up. One of my dogs can be very impulsive. I find that occasionally he indulges his impulsiveness and it's quite hard to head it off. A good example is his love of jumping onto my shoulders. Unfortunately, he likes it so well that sometimes he decides I just gave him the cue, particularly if I'm not looking at him, so he does it. It seems to make very little impact on him if I just nudge him off and refuse to reward it. He continues to sneak one in when I never asked for it. Apparently he's never heard of stimulus control despite my efforts to help him discriminate between a cue and no cue. 

[KobiD]

That makes perfect sense. 

 

The arousal and emotional state particularly makes a big difference for sure, and if I understand correctly that is part of the proving process. Increasing levels of distraction and varying environments while seeking certain behaviors. There is no quick fix for this other than putting in the time and building the focus gradually. 

 

I can certainly see that during the span of a day the dogs emotional states change greatly, from first thing in the morning when she is a bundle of energy, both physically and mentally, and also hungry which makes her very food driven. This is when I like to actively train, with food and toy rewards. I should probably get up earlier on work days to give her a good go of it. During the middle of the day it's too hot and the dog is very relaxed in nature, happy to laze around, stick in the shaddows and just generally tag along wherever I go (if not curled up asleep somewhere). Afternoon energy changes again as the kids get home, the schedule gets busy, dinner time, etc. I try to give her a good play as it cools off, working on leave it, drop, fetch, etc. And then once she's burnt off some zoomie energy do some training with the children and dog, depending on all of their emotional states (mine included). 

 

Have been steadily trying to vary the rewarding and making sure I'm not predictable, although I try and keep some extra tasty treats for strong recalls. 

 

Has anyone experimented with using certain rewards to improve responses? Ie, a slow sit only gets a piece of kibble. A better one maybe some watermelon?, and the fastest sits get some fresh meat? 

 

 

[The Spotted Devil]

I might use praise for a slower response or if I have to help - and then absolutely ramp it up with a retrieve, cheese and wild game of tug. 

[KobiD]

Over a month on and I can see our little girl maturing day by day, week by week. She's still highly motivated by treats, but we continue to mix it up. Sometimes we don't have treats and just use toys or praise. Sometimes we have treats and still only use praise. Sometimes she gets lots of tasty treats.

 

Have been busy involving the whole family including the children in the process and it's really positive seeing things progress. Lately have been working on her recall at feeding times, simply by moving behind her to various distances and then calling from there in a very positive tone. She scampers back, bounces off before returning to her starting point to eat. Have been building impulse control at dinner since day one, so obviously built up to here, both with time, distance, and naturally occuring distractions.  

 

All the threads I've posted and all the feedback I've received has helped greatly! Thanks everyone!

 

 

Edited March 24, 2017 by KobiD

[karen15]

be careful with the recall as you have described it. You may be inadvertently teaching her to come and then leave again, as you say she is returning to her food after coming when called. I made the same mistake teaching my horse to go into a crush. He'd walk in, get a treat then walk out. All fine and dandy till I asked him to stand in there. I'd inadvertently taught him that he was to come in, stand 30 seconds and then back out, because that's what we did when desensitising with initial training. I had to reteach him to stand and stay. Your teaching your pup to come when you call but then it's ok to run off. Fine at the moment, but you may want her to come and stay, which she may not understand if she's allowed to run off after.

 

On your initial posts, my animals are always allowed to say no. I think that is important, particularly when they are injured. All of their lives they know that if they say no but I then say, well i really want you to do it, then I expect obedience. That really helps when they are hurt and don't want you touching something. They can say no it hurts, but I've always found them to be accepting if I say, I know but I have to do it. I think animals that aren't allowed to express their feelings are the ones that can snap. Before I got my first horse the vet had never seen him with all four feet on the ground. After a few years with me he was a different animal, so much so the vet (who was very blunt) commented that before I bought him she would not have been upset to hear he'd had a fatal paddock accident. He was dangerous and I think that was because he wasn't allowed to say he was worried.

 

 

[KobiD]

Thanks Karen,

 

Understand all you are saying! 

 

We're already on top of the recall scenario. I've only just started the feeding recall and it's only a single aspect which also includes her impulse control. In the last couple days it's building from a bounce into a collar touch and release, and it will build into a sit (follow cues) and release shortly. She would probably do it right now, but I'm working up to it. In all her other training she comes in and sits, and will sit and happily have her collar clipped on/off; to the point I don't really need to ask or cue. The dinner routine is more aimed at building a strong/faster recall - stronger/super mega high value reward (in her eyes).

 

I'm understanding your second part as well. I don't really have high expectations of what she is to do/not to do. Happy for her to express herself, particularly if she is injured/in pain. As she is maturing more and learning the cues and proofing in various situations and with greater distractions she's getting it and I'm not feeling like she's as purely interested in only listening when treats are on offer. 

 

 

[KobiD]

Another month on, another update.

 

This has become more of a training/obedience thread than a puppy problem one, but at 6 months I guess she is still a puppy, heading towards adolescence.

 

I've been very happy with her progress, however I'm still struggling with the variable reinforcement to a certain extent. I continue to try and mix up where the treats come from, how often they come, what they are and their value. But clearly I'm not mixing it up enough. I continue to ask for more from here, stringing behaviours together, increasing duration, and adding distractions once she's got things to a certain point.. The rewards get built into what we are doing. If we're playing the toys become the reward and release, if walking sometimes going to where she wants to sniff etc is the reward, other times it's just praise and affection, and sometimes it's food. 

 

BUT!! she's a clever dog who is very self confident and a little hard, and quite independent.. AND she knows before you even ask her to do anything if there is anything in it for her. She's not overly driven by praise or pats, and I'm a male with quite a deep monotone voice who is pretty calm in nature (not easily excitable) which probably plays a bit into it too. She is extremely food driven!! but she knows there is food on hand. I don't flash the food first or lure her to perform using it (unless learning something new, which quickly turns to a hand signal, followed by cue)... I get frustrated that she looks at me, and then looks at my hand, looks at me, looks at hand. If I have treats somewhere else to do "lets get a treat".. she'll run back to the treats. Essentially I feel like she's only working for the food, and not so much for me...

 

Perhaps I just need to change my perspective and be happy that she is so driven during training sessions, but I'd like to see her give similar levels of focus and attentiveness when she knows that there is no food/toys on hand. That she'll still listen while we're out in the yard. And to a certain degree she does. I think it just shows/proves that she is still a puppy and still developing. She is a bit aloof if there is no food, she'll follow me around, and play still, but with much less focus, and may/may not listen.. I don't reward her ignoring me by then bribing, and I don't really let her self reward by ignoring her either. I don't continue to ask. If she won;t play I give her a negative mark word/sound and usually cease trying to ask for anything, or follow with a firmer posture and strong hand signal for her to follow, and then praise. 

 

Also, I've been working hard to build her impulse control with other dogs setting up the relationship between other animals and treats coming. She is learing to settle and not pull and retain calmer, while looking back at me for direction which are great, but when she plays off leash she is still very hard headed too.. Doesn't really respect other dogs space and doesn't read their signals well at all. She has been put in her place by my parents cattle dog, and she just bounces back up and continues like it's all a game... she does get it, but it takes a while... and then they play along really well for the rest of the day. I think it's just part of her breed/temperament in general. I found the same with her as a puppy.. 

 

Happy to hear any advice or suggestions, or just feedback in general. 

[Tassie]

Sounds like you're making lots of progress, and I really like the way you're analysing what's going on.   I guess my main comment would be .. at 6 months, you're quite right .. she is still a puppy, and heading into that delightful adolescent period when in some cases, biddability and learned behaviours can go out the window.   So you may find you will actually be needing to take several steps back and "teaching" stuff all over again .. so building more value into the behaviours you want, even when you think she "knows" them and should do them without food rewards.

 

What you probably could do, rather than removing food from the picture altogether, is have several stashes of food (with lids on) around the place, and some concealed around your pockets.   So then you ask for a simple behaviour, that she's likely to get right without thinking, mark it (clicker or marker word) and then race her to one of the stashes to reward her.   So it can become a real game, which will hopefully involve her more in doing the behaviour ready for the race, rather than just looking to you for the reward.

 

I personally would not at this stage be letting her self-reinforce by inappropriate behaviour round off leash dogs.   This will only increase the impulse control problems, IMO.  The problem is that in that situation, you are not in control of the rewards that she finds valuable .. so not a good learning situation. I can't remember if the Control Unleashed book and DVD have been recommended already, but I think they'd be worth a look.  And there are some really good alternatives in The Glasgow Dog Trainer YouTube channel (and the plus about those for you  is that the trainer is a man .. and you're right, it does make a difference..

[KobiD]

Thanks for chiming in Barb!

 

She only really gets to go off leash with my parents dog. Never with any unknowns. The advantage is that their dog will both put her in place when she needs it, but she will also break away and return on command, so while my puppy is a bit out of control the whole situation isn't entirely. While this happens I go right back to basics. HIGH value treats, and train everything like it's a new behaviour. Mark and reward for minimal eye contact. Mark and reward when calm/passive/submissive. Mark and reward for a 'lets go' the moment she turns. Mark and reward 'leave it' if she's getting into it a bit rough. They do play quite well together and I feel it teaches her things I can't, while also being a big out for her. We also don't just unclip and let them at it. It's structured and we close the distance as she's calm, but the initial greeting always has some kind of stoush between them and play starts on the older dogs say so.

 

I've been working on things as you say too. Having stashes of treats rather than always out of the pocket or hand. She still expects it a bit so more work needed for sure!

 

I'll see if I can source those resources you've recommended and expand my mind/knowledge a bit more too! I really need to get a pic of her as well

[Tassie]

Oh sorry .. I misunderstood.   Yes, the controlled off lead with your parent's dog should be fine, as you're describing.  And I'd say you're definitely on the right track with your other training.   I guess you're adding in new bits and pieces, including 'silly' tricks .. at this age they're lovely little sponges, so it's a great time to introduce new things, within their physical capabilities .. but just as a game .. a thing to do, not expecting perfection.

 

And yes ... we totally need photos  .

[KobiD]

A pic from today, and one from last month.. I've still got no idea what she is.. 6 months, 16 kg so medium sized. I'm thinking some staffy maybe, but also has a long snout. Sometimes looks a bit lab/staff.. Nice girl though.

Edited April 28, 2017 by KobiD

[Tassie]

Whatever she is, she's lovely.    Thanks for sharing her.

 

[The Spotted Devil]

Looking good!

 

I'm TOTALLY ok with the dogs knowing where the treats come from. Gee but it comes in handy when they are running or swimming at top speed after a rabbit or a duck!!! Where I want to TRANSFER value is when I've got a dog that loves running, or hunting, or flirting (yes I have one of each  ) ...that's when it's my release to go run, hunt or flirt that becomes important. I also want to transfer the value to agility equipment but that's something where I don't reward on my body....I throw the toy for them to retrieve to me for a treat OR a game of tug OR I simply throw a big cheese cube. The real trick is to control their access to rewards but at the same time give them the perception of CHOICE....add distractions and show them "this is what it is and this is what it is NOT". 

[KobiD]

I guess the problem I have is not so much with her knowing where the treats come from.. but seeing the shift in her drive from the treats flowing to the hand running empty. Maybe poor management on my behalf for letting that happen, but then again the reality is that I won't always have a hand/pocket/bowl full of treats on me, nor do I really want it. I think it will just be a long period of getting through the puppy stages and proofing the behaviours until they become natural to her at which stage I imagine the rate of reinforcement can probably drop off. Also as she learns to control her impulses it will be less of a challenge and as such the rate/value of reward will probably drop down too.

 

We've had my sister in laws maltese shih tzu here for the best part of a week now. They are separated by a screen door most times, and went from barking with excitement wanting to play to the point where I can sit with the little one on my lap and my dog laying at my feet. All just time, patience, and making it known what the expectations are.. and then being consistent.  

[The Spotted Devil]

@KobiD what does your dog love? Apart from food. You need to start building permissions to play, run, drink, go to bed and in my Dally's case pee and flirt lol

[KobiD]

She loves food!!! food is her everything.

She loves new and exciting. Be it smells, animals, or people, or toys.

She likes to play, but it depends on her energy level. She's typically very laid back during the day. The afternoon she get's a bit zoomie so not ideal for training other than quick short bursts. She's full of energy in the mornings and depending on the time I have we'll either try and go for a walk to work on impulse control with people/animals.. or we'll work obedience with her tug toys/rope.

 

She isn't a Velcro dog and while she'll follow me in a general sense, she doesn't feel the need to be in contact and I don't think she values physical affection much (other than a tummy rub or lap cuddle when she's in a calm state).

 

Been having trouble getting a decent heel out of her when walking too. Take three steps nicely and reward and then she's 3 steps infront. She doesn't hold with or naturally drop into a sit. She tends to find the end of the leash and then back off, so not particularly pulling but leading and investigating. I continue to reward when she looks back to check in, or when I call her back (get back), or stops for me to make up ground (wait).

 

On the other side, using the TOT principals for dinner I can place the bowl and walk off saying let's go.. she'll heel on my left leg as far as I want. Place her in a sit stay, down, stay, come, stop, come, spin, sit, paw, other paw, etc etc etc. OK and she'll go and eat. She will work for food like there is no tomorrow.

[corvus]

I nearly always have food on me. If it's not dog treats, I can usually find something. Bread, chips, cat food... The times I don't happen to have food on me don't matter much to the dogs because I nearly always have food on me. They stopped caring about whether I had food or not and just trusted I would have food. The times I don't, they get some happy praise or play or some life reward, and the worst that happens is they go "That was weird there was no food." Because they are so conditioned to work for food wherever we go, they will work for anything that vaguely resembles food. I once did some training with one of them using fruit from my fruit punch at a BBQ. They will work for literally anything edible at any time, and they assume it's coming and it will be good before they know what it is. That's what a continuous reinforcement schedule does for you. 

[KobiD]

So you're more of an advocate for continuous reinforcement rather than variable? Does that mean you reward everything your dog does? Or as the behaviours become proved you simply keep increasing the requirements before rewarding?

[Scottsmum]

36 minutes ago, KobiD said:

So you're more of an advocate for continuous reinforcement rather than variable? Does that mean you reward everything your dog does? Or as the behaviours become proved you simply keep increasing the requirements before rewarding?

I couldn't be further from an expert on the matter, I might call my self an amateur if I was being kind.... but yes.  I always rewarded my dog with a treat or a bit pet or big over the top cuddle & fuss  - which was *just* higher value than a treat.  He would work for nothing - except toward the end - I'm not sure if it was achy joints or a declining brain or a combo of both or possibly just him being a stubborn old man LOL....  But in his healthier days he always appreciated a reward.  And who can blame him. I don't do to work for nothing and I love being acknowledged. So why expect them to work for nothing?