When would it be the best time of the season to seal them off?

I doubt the holes are big enough for wasps, maybe not even for bees. But if so: Bees are likely to stay, wasps might move on in autumn (?) So, without harm, it depends on who came to live there but maybe without harm isn’t an option. You could wait until it seems abandoned and fill the holes then.

He never saw them, so they don’t exist.