In 'The Lodger', Smith dropped his towel, prompting rumours that he had exposed himself.
However, a spokesperson later refuted the speculation and Smith has now also denied the claims.
Smith told PA: "They give you these modesty trunks, they call them, or something. They are flesh-coloured trunks and I'm afraid I was wearing them. [Fans] definitely didn't see more. It goes out at 6.45 on a Saturday. It wouldn't be proper."
However, Smith added that he does not mind stripping off for the show.
"Oh, it's fine," he said. "I've done it before and I've just done it again afterwards. It's absolutely fine. It's always a bit odd because you've got sort of 40 people looking at you in your modesty shorts. But I think part of acting is about making yourself feel vulnerable and that certainly does it so, you know, maybe it's doing something right."
Doctor Who continues on Saturday at 6.40pm on BBC One.