Not cast out, but deliberately decided that fame was not to their liking.
The more famous at the time they turned their back the better
This might be a short list...
Cat Stevens
sandy koufax
sharebarry sanders
shareKurt Cobain
Anthony Bourdain
Chester Bennington
Chris Cornell
Phoebe Cates is another one that comes to mind.
shareJ.D. Salinger
shareCat Stevens has actually gone back to performing under his original name because he knows he'll make more money that way. He also did plenty of interviews during his break from music, especially on the subject of islam.
shareI still think he qualifies. He dropped out of the rock star game when his career was still going strong.
shareJosh Hartnett
shareTeresa Graves became a Jehovah's Witness
share