Too good looking
I read somewhere that actors shouldn't be too good-looking because it can be both distracting and can alienate audiences. While I think Toby is a pretty decent actor, I think that his looks could potentially hinder his career.
Other actors who suffer from this too-beautiful handicap are Eva Green and Angelina Jolie.
"I can't keep doing this on my own with these... people."