My suggestion is not to let him off the lead at all, Some dogs no matter how good the recall is seem to become 'deaf' when water is involved. I had a Golden Retriever that did exactly that and he would have been happy to keep swimming and swimming, one time when he was younger he wouldn't come back in when we were at a lake, he'd come just out of arms reach and head out again untill eventually we had to jump in and get him. Especially if you have children to watch and look after at the same time, the last thing you need is to worry about banjo not coming back.
Practice recalling and take him out prior to the whimpering, usually 5 minutes of swimming, then recall then out again is a good way to do things. I personally prefer to just swim/walk with my dogswaist height where they can't touch the bottom good exercise for them and me :p