The witness codenamed Person 18 gave evidence in the Federal Court on Wednesday about a meeting in Australia his squadron had soon after the 2010 battle of Tizak in Afghanistan.
Mr Roberts-Smith gave his fellow soldiers a printout with a citation detailing alleged actions on that mission that led to him receiving the medal.
Person 18 said he told Mr Roberts-Smith words to the effect: "The award and you are making me feel uncomfortable."
He said the "rumours were running wild," of mistakes the decorated soldier allegedly made, and that another comrade codenamed Person Four was said to be more deserving of the award.
Person 18 said he was told Mr Roberts-Smith had not properly pulled his grenade pin before throwing it during the battle, and that his weapon had a stoppage after entering one building.
He said the decision to award Mr Roberts-Smith the highest military honour was misguided under such a heavy cloud of rumours.
"If something was surrounded by that many rumours ... probably wasn't deserving, unfortunately."
Mr Roberts-Smith, 43, is suing The Age, The Sydney Morning Herald and The Canberra Times for defamation over reports that he committed war crimes and murders in Afghanistan between 2006 and 2012.
One of a handful of Australian recipients of the VC since 1970, he has suggested some claims stem from jealous associates spiteful of his medallic achievements.
On Tuesday Person 18 said he explained to Mr Roberts-Smith that he didn't want to be in the limelight with the war veteran who expressed desires to go on speaking tours and other public ventures.
The court has previously heard testimony Person Four who fought side-by-side with Mr Roberts-Smith at Tizak, and said they "did as much as each other that day".
Person Four was hurt he was awarded a medal of gallantry two years after his achievements, he said.
While still deployed in Afghanistan, Person 18 said he overheard Mr Roberts-Smith in a "heated and panicked" conversation following an alleged execution during a 2009 mission to a compound dubbed Whiskey 108.
Person Five allegedly told him "you've just done this," underneath a surveillance drone.
"We need to find out whether (it) was still recording," Person 18 alleges Mr Roberts-Smith responded.
Person Five then contacted the troop radio and asked whether the ISR, or the Intelligence Surveillance Reconnaissance device, was recording during the assault, Person 18 said.
The response was that the ISR had been "pushed off before we made entry".
Mr Roberts-Smith is accused by the newspapers he is suing of throwing the prisoner with a prosthetic leg on the ground and executing him.
Person 18 also gave evidence of receiving a threatening letter in the mail days after giving evidence at a secret inquiry.
"You and others have colluded to tell lies to the media and before the inquiry," Person 18 said the letter read.
"It also stated that I had till the end of the month to change my statement, otherwise I'd go down," he said, the sign-off being "a friend of the regiment".
On Tuesday he said he only disclosed the contents to a superior he handed the letter to, but it somehow was leaked to Nine media ending up in an article days later.
He denies ever speaking to journalists.
The trial continues.
Lifeline 13 11 14
Open Arms 1800 011 046
9°C