"You gave it your best shot in 2002. You felt like you had to do it. You had to go and give back to the country, and you tried and they didn't take you, so let's move on."

4 of 5 of Romney's sons thought he should not run again:

And here are the sons talking about Romney's sense of humor — such as saying "the trees are the right height" and using the word "guffaw":
The full interview is on "Rock Center with Brian Williams," Thursday, July 19 at 10pm/9c on NBC.

ADDED: Here's an interpretation of the quote in the post title, which may or may not be the way the Romney family thinks about things like this. Romney explained what he was doing in 2008 in terms of service and duty — that he had reached the point in his life where he had something to offer, or so he believed, and that entailed a duty to offer it. When the offer was turned down, the duty was fulfilled. To offer a second time is to do something else, something that has more to do with ambition or vanity — something other than altruism. This kind of thinking could also have a religious dimension. Perhaps Romney felt called to serve as President. He came forward, but he was not selected. Thus concludes the calling. Move on.