We discussed this recently, so I'm going to sound like a broken record, but there were plenty of Silver Age stories that saw Superman lose ALL his powers under a red sun, even on ancient Earth. (Yep, they used to claim that in Earth's infancy, its sun was red.)
If the loss of yellow solar energy deprived him of all his powers, that threw the whole "gravity/mass" concept out the window.
If he was actually made of "more sturdy stuff" than a human (due to Krypton having had higher gravity), much of his durability, speed, and strength should have remained.
Silver Age claims were that his powers came from both. Silver Age red sun stories said his powers were solar only.
I like the solar-only theory. Not only does it allow him to lose his powers completely, it also means when he was growing up, he wasn't super-heavy, and still won't be when he loses his solar powers.
It's really the only thing that makes sense, especially now with the idea that his powers weren't very pronounced when he was younger. (No breaking beds or chairs...or Jonathan's and Martha's backs...as a child, please.)
I actually would rather seem him as being physical as strong/fast/durable as say Spiderman naturally, and the exposure to Sun allows him to have his super senses and have all his natural powers magnified thousands of times over!